I saw Big Data

Like the lone intrepid hikers who make it their business to track down the ever-elusive and folkloric Bigfoot creature of the West Coast, the IoT (Internet of Things) market is equally hot on the trail for something rather shadowy – “Big Data”.

Clearly the big money is in who can best provide the best data analysis tools to sort and analyze all sorts of unstructured machine data into digital gold.

So yes, while the market focuses on all the fun-to-talk-about mobile devices, sensor machines, and wearable tech, there is an even greater opportunity to develop a new breed of business intelligence for IoT. It starts by coding new applications that are more front-end, user-friendly and offer the best insights in the shortest amount of time.

This intelligence will open new “Service” doors for enterprises to take a more preventative approach to problem areas, compared to the “we’ll fix it when it breaks” mode. Thus, saving bottom lines and any downtime.

Consider the volume of data expected from IoT, and the most efficient way to deploy and service that intelligence is in the cloud. Whether that be public, private or hybrid is another story, for another day. But this also extends the opportunities to all the infrastructure enablers, such as Amazon. It recently launched its Kinesis IoT infrastructure service for high-speed streaming data processing, connected to its S3, DynamoDB and RedShift for storing and analyzing data. Other cloud providers are also in the hunt, of course.

So therein lies the rub. What kind of hardware and software is best suited for this? The IT systems of the past 20 years are sorely not going to keep up. So any new breed of software should be matched with a new breed of hardware, à la Symkloud MS2900 cloud platform from Kontron.

This is when I saw Big Data – the real thing. In constructing a “Big data” demo with ISV media transcoding partner, Vantrix, we showcased how a remotely located cluster of eight Symklouds (a total of 16RU) loaded with 144 Intel® Core™ i7 processors and 69 TB of SSD Storage could crunch in mere seconds more than 2 TB of archived data consisting of network mobile user experiences (CDRs). Interestingly, on paper, this outperforms a stack of 16 Cisco C240M3 (a total of 32RU) in a 1 TB TeraGen/TeraSort performance test by more than two to five times. But I digress.

The other half to this demo story is the software stack, which consists of open source software tools that work in a symbiotic, Software defined way with the modular, high density design of Symkloud. Whether to analyze mobile network traffic or IoT unstructured data, the ideal tool we used comes from Splunk and, by extension, its HUNK analytics tool design specifically for Hadoop data structures that spread data processing across multiple shared servers (i.e.: cloud environment). Underneath this layer is OpenStack to take full advantage of orchestrating the required compute instances.

The data analytics front-end interface was slick and extremely easy to use. More importantly, one could drill down extremely fast to any myriad of details to find that elusive golden nugget of info.

Research firm Gartner states that the pace of new unstructured data is doubling that of structured, essentially 80 percent of all enterprise data. With “data” becoming the new currency, do you have the next up and coming IoT data analytics application? Does it reach its full performance potential? Why not find out from our new Kontron SYMLAB built for remote App Dev testing. I can see Big Data. Do you?

Thank you!

Your comment was submitted.

An error occured on subscribing!:

{{comment.date.format('MMMM DD, YYYY')}}


There are no comments yet.

Stay connected