perishable data FEATURE

Sensors are allowing us to capture more data than ever. There are good reasons why much of it should stay where it is.

It’s a little-known fact that much of the massive amounts of data coming off the Internet of Things (IoT) may never go anywhere. Its value will be lost if it is not processed immediately, and there’s just not enough bandwidth for it all to be transmitted, anyway.

“I think you’ll end up processing most data at the edge and throwing it away,” predicts Alexander Hill, co-founder of Senseye, a British company that develops Internet of Things (IoT)-based machinery failure forecasting for industry.

One area Senseye is working on involves reducing failure rates for auto manufacturing equipment. Vibration sensors can help to tell when a machine is about to break. But systems would get clogged transmitting the huge amounts of data this can generate.

Sending vibration data to a central warehouse is inefficient and you could lose interesting data in the noise, particularly if the information has to travel over a congested wireless connection or a low power WAN such as those being promoted by the LoRa Alliance or SIGFOX.

And the consequences of a catastrophic failure could be significant. “If one robot goes down you have a whole factory line that could be stopped, costing millions of dollars,” Hill says.

This is precisely the kind of data that is best processed in real time or near-real time, at the network edge.

Such ‘perishable’ or time-sensitive data “is only actionable within a short amount of time after being collected,” says Dima Tokar, co-founder and chief technology officer of the IoT analyst firm MachNation.

“It plays a key role in certain IoT solutions that rely on sensor data to generate rapid actionable insight,” he notes. “For IoT solutions focused on safety and automation, ensuring that data is acquired and processed with minimal delay is of paramount importance.”

Senseye hopes its improved monitoring techniques could help to cut automotive manufacturing machinery’s already extremely low fault rates by a further 50 percent or so. Elsewhere, Tokar cites the value of perishable data in accident avoidance at construction sites.

“The ability to receive and process data in near real-time is what makes the solution work,” he says. “Without processing data close to the source, the solution couldn’t provide actionable insight quickly enough to allow decisions to be made in time."

Helping to reduce the risk of human injury or cut the cost of machinery failures are two obvious reasons why companies should embrace perishable data.

But, says Tokar: “The lowest-hanging-fruit applications for edge processing are scenarios where connectivity is unreliable or limited. Edge processing can increase automation and operational efficiency in environments typically considered too remote and complex to operate in.”  

Because of this, it is hardly surprising that sectors such as oil and gas, which have extremely high-value operations in remote, poorly connected sites, are already taking perishable data seriously.

More widely, 37 percent of Cisco customers believe most of their IoT data will be processed at the edge of their networks within three years. That should help increase the value of the data overall since, according to IDC, less than 1 percent of it is being analyzed at present.

Furthermore, the proportion of data that gets processed at the edge is likely to grow as the IoT expands. Cisco predicts 50 billion things will be connected to the IoT by 2020.

Trying to centrally process the data from each one would put an enormous strain on networks and computing systems. “Our vibration sensors produce tens of thousands of data points a second,” says Hill. “You don’t want to push that back up the 3G data link.”

Instead, intelligent network edge devices could process the bulk of data and then just a small portion could be sent on for further analysis within a data warehouse. 

“The most exciting thing about data processing at the network edge is that it opens up a world of new possibilities that simply couldn’t be achieved if all of the raw data had to make a trip to the cloud,” says Tokar.


The contents or opinions in this feature are independent and may not necessarily represent the views of Cisco. They are offered in an effort to encourage continuing conversations on a broad range of innovative technology subjects. We welcome your comments and engagement.

We welcome the re-use, republication, and distribution of "The Network" content. Please credit us with the following information: Used with the permission of

Share this article:


About Jason Deign @DeigninSpain

Jason Deign is a Barcelona-based business writer, journalist and author.