Humpback Whale and Big Data

I’ve got Big Data on my mind.

I’m in Singapore now, just a day ahead of the Strata + Hadoop world conference to co-present with one of my clients and share two stories on how we’re Acting on Insight with IoT in NZ.

During my flight I had Big Data on my mind too. In between movies on my flight and while unable to sleep, I found myself listening to the hum of the Dreamliner 787’s engines and staring at the in-flight map. I was thinking about how the sound reached my ears. How through the air, the fuselage’s carbon fibre and fire/sound/atmosphere insulation, the pressurised cabin, wiring and plastic I still heard the engines but, it was white noise and very much in the background.

I started to think about how sound travels through different mediums at different speeds. I also started to think about the upcoming conference. About proper use cases for Big Data and how it relates to traditional analytics, data warehousing, BI and data management.

On the in-flight map I saw that we were still over water in an area where Humpback whales migrate. I thought about the whales and how they can communicate over vast distances through water to find mates, organise hunts, or give ease to new born calves.

I found myself comparing the whale’s ability to exchange messages across the ocean with the hum of the jet engines. It made me think about the difference between conducting analytics using traditional data management methods and tools, and Big Data.

Here’s the metaphor: In water, sound travels faster and farther than it does in air. The same occurs when you do analytics with Big Data versus traditional data management methods and tools.

I’ve witnessed this effect myself with my teams building Big Data platforms and then ingesting and wrangling gobs and gobs of seemingly unrelated data in close collaboration with my clients’ data scientists – who now can rapidly develop, train and tune their analytic models with feedback to the data supply chain.

In the traditional environment, the data scientists used both vendor and open source analytic products on top of either relational or proprietary data stores and delivered insights fast using analytics. Fast relative to how others were doing so using the same tools and methods. Now, with the core Hadoop and Cloudera enhancements as a data platform, they have been able to share models and work collaboratively – and process the results of their many models even faster than before.

In one example, a client’s team produced a result from analysis in less than two weeks, enabling a business decision with the potential to earn double-digit millions within a year.

I’m approaching the Singapore Strata + Hadoop conference in search of similar stories which I hope that the contingent from NZ will join me in sharing with the cynics and fanatics in the peak of the NZ summer early in the new year. For those of you not from Middle Earth, start planning a trip down and join us for our Big Data conference on 18 January 2017.

Oh yes, here are the two numbers you’re thinking about – 343.2 m/s (air) and 1,484 m/s (water).

Devin’s speaker bio

Act on Insight with IoT


Share This Post:

Devin Deen

As Director, Data & Analytics at enterprise IT, Devin ensures customers get to navigate, much more easily, the complexities of implementing data warehouse, data management, business intelligence and business analytic tools and solutions. Outside of e-IT he is active in the NZ tech-startup community and directly involved with successful SaaS company,

Related Posts