How Does IoT Affect Big Data?

In the words of Andrew McAfee, the world is one big data problem. Some statistics culled from various sources and surveys tell the story:

• More than 150 zettabytes (150 trillion gigabytes) of data will need analysis by 2025, and…
• 87% of companies surveyed believe that data is an asset for their organization. But…
• Nearly 36% of companies don’t use all the data they possess, and…
• 75% said they don’t feel prepared to make use of data.

Ever since the term “big data” was first used back in 2005, the idea of massive data sets just waiting to be mined for insights represented a huge opportunity for companies—an opportunity that has also proven difficult to unlock. And now, with the amount of information generated by the Internet of Things (IoT), big data has gotten even bigger, exacerbating the challenges even more.

So what is IoT data, and how does it fit into the big data picture? We’ll answer those questions in this article, and take a look at how some companies have been successful in using the IoT and big data projects to achieve business goals.

What are your business goals? Tell us and we can help you achieve them with the IoT. Schedule a call today.

The IoT & Big Data: What’s the difference?

Even before the IoT existed, companies have always collected data—and lots of it. Sales data, operations data, financial data, HR data, consumer data, and more were collected from a variety of sources, usually with the intention of being analyzed for the purpose of gaining specific insights. This type of data is typically produced in large volumes at a high velocity, and comes from a variety of sources, for example, social media, financial markets, and sales transactions (volume, velocity, and variety are known as the three V’s of big data).

It’s no secret that analyzing big data could produce significant insights about a business and the way it operates; the problems revolve around contending with the sheer volume of it:

• Much of the data is siloed, isolated from other types of data and inaccessible for the purposes of bigger-picture analysis. Financial data, for example, cannot easily be aggregated with consumer data for more complex insights about the impact of certain customer behavior on the company’s financial performance.
• It’s difficult to process big data fast enough for insights to be useful. The value of most types of data is short-lived; what consumers are doing today will be different tomorrow, and the day after that. To gain the greatest benefit, businesses need insights they can act on quickly, but most traditional database systems aren’t capable of processing data at the necessary speed.
• Much of the data collected is wasted. Business analysts tasked with finding “answers” to business questions in the sea of data must filter out what is perceived to be irrelevant data, and pinpoint certain data sets where answers are likely to lie. As a result, it is estimated that between 60% and 73% of data goes unanalyzed.

68% of respondents said their data analytics efforts are hampered by data siloing.

I’ve seen firsthand the challenges of wrangling big data; imagine a large utility serving millions of customers trying to forecast the top 100 hours of peak demand during an upcoming summer. Collecting and aggregating the necessary data, and then forecasting and modeling it, was a nearly impossible task, with data sets so large in size they were overwhelming. (Add to that 5-minute-interval IoT sensor data for individual customers and you can clearly see the struggle.)

Today, another major source of data is contributing to the tidal wave—IoT data. In many ways IoT exacerbates the data problem, but it also offers a solution.

What is IoT data?

The system of networked objects known as the Internet of Things has exploded in recent years. Any physical object can be turned into a part of the IoT simply by attaching a sensor to it; by 2021, there will be an estimated 35 billion IoT devices worldwide. Data generated by IoT devices is forecasted to generate 79.4 zettabytes of data by 2025. (In case you’re wondering, it would take 1,000 data centers to hold one zettabyte!) Clearly the data generated by IoT devices is compounding the problem—or is it?

By 2021, there will be an estimated 35 billion IoT devices worldwide (@SecurToday). Data generated by IoT devices is forecasted to generate 79.4 zettabytes of data by 2025 (@cxociety). Click To Tweet

There are three important ways in which IoT data collection is impacting big data as we knew it:

1. The data generated by IoT devices is, in many ways, richer than other types of data. Because sensors can be attached to any physical device, IoT data is diverse and granular, which means businesses can get more information about their business operations than ever before. For example, smart buildings can collect data related to:
   • Environmental conditions, such as air quality, temperature/humidity, and luminosity, so you can understand what needs to be altered for human safety or comfort
   • Energy usage patterns, so you understand how and when your building uses energy and can take steps to optimize energy efficiency
   • Occupants’ behavior, activity levels, and behavior patterns, so you understand how they impact energy efficiency measures
   • Water use, so you can minimize waste
   • Your building’s equipment, so you can determine your building’s power factor and use the information for predictive maintenance
2. Less data is wasted because it is aggregated and analyzed automatically. Many IoT platforms use machine learning to gather a variety of relevant data streams, and then catalog and analyze the data sets together. For example, sensor data measuring the vibration level of high-speed rotating equipment combined with the established vibration signature of the equipment can help detect anomalies and, over time, predict problems before serious issues arise. Having the capability to use all the data produces more actionable insights—and greater ROI as a result.
3. IoT platforms collect and analyze data in real-time. Big data is gathered for analysis after it’s collected; many IoT data platforms simultaneously analyze and collect data in real time so insights can be gained faster. For example, on a hot summer day, a building manager can get insight as to what a building’s conditions are like in the moment, and what actions can be taken immediately to reduce energy consumption.

How can companies use big data generated by the IoT?

The point of IoT data collection and analysis is to apply the insights in a way that derives value for a company. Some of the ways companies are using big data to realize gains are:

• Engineering group Sandvik, which focuses on mining and rock excavation, metal-cutting, and materials technology, was looking for a way to solve the mining industry’s under-productivity problem by keeping equipment up and running (the main driver behind under-productivity). Sandvik worked with IBM to jointly develop an analytics and predictive maintenance solution that would better predict when an equipment breakdown was imminent. As a result of implementing this solution, it saw a productivity improvement in some areas of between 25% and 30%.
• Hershey used the IoT in its manufacturing facilities to regulate temperature and other factors of its production process, ensuring its candies are within legal sizing guidelines. For example, 22 sensors on a cooking vat assess the temperature every second the licorice is in the tank, helping to predict the end weight of the product. Each 1% change in sizing of a licorice piece resulted in savings of $500,000. (You can read more about their recommendations for leveraging data here.)
• New York City has equipped several city fleet vehicles with air quality sensors to learn about pollution levels within its neighborhoods. Other cities have taken similar initiatives, including Chicago (which attached sensors to lampposts), Denver, and Barcelona.

Ready to get started with the IoT? Get Big Data You Can Use

If you’re looking for a simple way to get started collecting and analyzing big data, talk to us at Iota. Our goal is to help businesses and municipalities derive valuable insights from IoT data collection.

We’ll work with you to identify your goals, discuss opportunities where real-time data can be valuable, and set you up with a remote monitoring system that’s both cost-effective and easy to use. And our data experts will be with you every step of the way, offering guidance on ways to optimize your data and your operations.

Big data doesn’t have to be a big challenge. Reach out to us today and let’s get started!