A Story of Synergy – Big Data and the Internet of Things

Dresner Advisory Services’ “2016 The Internet of Things and Business Intelligence Market Study” presented an extremely detailed view of these growing technology areas. Among the insights was the revelation that the ability of organisations to manage big data / analytics and to manage a successful IoT initiative were highly correlated.

• The study showed that when compared with the wider sample of respondents, 3 times as many advocates of IoT believed that big data / analytics was “critically important” and 2 times as many found it to be “very important”.
• These advocates of IoT saw the implementation of big data / analytics as being driven primarily by the need to make their IoT initiatives work.
• IoT was seen as among the top 3 use-cases for big data / analytics according to the study.

There seems to be little doubt that these big data and the internet of things make a powerful case when brought together. Let’s take a closer look at this combined case.

The Internet of Things is a construct created by many multitudes of sensor-laden devices that are connected to the internet – 50 billion by 2020 according to Cisco. A lot has been written about how these devices can be controlled or induced to take some action by virtue of being thus connected. The other side of this equation, though is that the sensors on these devices are constantly collecting data on all manner of parameters critical to the effective functioning of the devices. This data is available for analysing, drawing intelligent insights and for driving timely actions. When you consider the massive volume of data coming in from several thousand such devices, the case for big data and analytics becomes glaringly obvious.

So, how do these two technologies come together? Well, consider that the mandate is to be able to initiate independent action in the devices, and by extension concerted action across a large number such devices. If this is so, then the greatest value is when an appropriate action, from among a range of possible actions, is initiated in response to certain sets of parameters being achieved. This is where analytics comes into play – looking at the masses of data and drawing the right conclusions before communicating to the devices, the details of the action to be taken. Think of a large mobile network with sensors located across its length and breadth. The data coming from the sensors on the network it is both comprehensive and timely. Analytics looks at this real-time traffic and device-specific data streaming in from the sensors, draws the relevant conclusions about service quality and potential issues, and initiates corrective actions like signal re-routing. EMC had released a report on “The Digital Universe” that showed that in 2013, 22% of the digital information available was “analyzable”, but with the emergence of the IoT and embedded systems, this was slated to grow to over 35% of all available data by 2020. That’s a lot of data, and a lot of analytics.

Somehow the most popular examples of companies that have successfully leveraged big data and analytics are mostly about issues related to better customer targeting and marketing based on the analysis of data provided by the customers themselves. Customers generate this data through their interactions with the various touchpoints of the organization in the physical, online and social world. The data generated from the IoT world is in many ways a perfect complement to this, customer-generated data. Writing in Forbes, Howard Baldwin says it well, “Once the Internet of things gets rolling, stand back. We’re going to have data spewing at us from all directions – from appliances, from machinery, from train tracks, from shipping containers, from power stations. If that doesn’t get you thinking how to handle real-time data feeds, nothing will.” This is the data that comes directly from the devices at the edge points of the organisation as well as embedded across operational junctions within the organization. The data generated by these devices has its own structure and syntax and it is key to understand that while setting out to apply the analytics filter to it.

Statistician and pollster, Nate Silver, has famously spoken of the need to focus on the signal amidst all the noise – the thread of relevance among the mass of distractions. It’s fair to say that to be able to make an IoT initiative work, it is mandatory to have a functioning big data and analytics program in place. In the absence of that, it will just not be possible to make sense of the tremendous flood of data being generated.