When I think about Big Data , I think in 3 dimensions:

1. What is Big Data and why does it matter?
2. The science & art of understanding Big Data = Data Science
3. Analytics and Big Data

It is for this reason that I very much enjoyed the Harvard Business Review, HBR October issue.…a similar way of looking at Big Data.

1. Data has grown in volume, distribution, types and diversity. Gone are the days where data were apriori formatted to fit into a nice SQL (Structured Query Language). Today we are dealing with Twitter, Facebook, LinkedIn, YouTube and many other types of data including NoSQL. So, as data has evolved, we needed a new name to signify today’s complex data environment, which is appropriately called Big Data. Like all emerging elements, initially there is confusion around what a term means. In my work with large enterprises, Big Data refers to the ‘data’ and not the analytics. Yet, I have noticed that at times people want to define Big Data with analytics included. Big Data still has the same issues of access, meta data definition and knowing what I need to look at in order to be an asset in the art of hugely distributed data. So, to me, Big Data does not inherently include analytics. See (3) below. A key element of Big Data revolves around the lack of data structure, which has limited us in the past. (More regarding this in blogs to come.)

2. Awesome!!! I have been involved in the science of analyzing and interpreting data for decades and nothing makes me happier than having an official term for what I am so incredibly passionate about. Data Science: the Science of understanding Big Data. Furthermore, it has bothered me immensely that large organizations used to defer the ‘understanding and analysis’ of data to only a handful of statisticians, while the many more who totally understand the ‘value’ of the interpretation for business purposes had to wait and get the analysis. I think we should be careful NOT TO segment today’s Data Scientists, as we previously did with statisticians. The science of understanding data belongs to, and must be shared by many across all business functions, including and not limited to engineering, sales, marketing, production, support and all other groups. The more and better we can understand ‘data’ the better we can be in making any/all business decisions. Knowledge is power and no sense in limiting this to a few. While there will always be complex analysis needing higher end specific professionals, we will do better not making this a ‘niche’ sport! I see this happening already and I caution folks to watch out. With today’s analytics tools, we can analyze data easily from remote devices and make better business decisions. Let’s not silo this activity. In a future blog I will share more of my thoughts regarding data science.

3. And any time we have data, be it Big Data or other, we need to have analytics. That is, if you agree with me that Big Data does not inherently include analytics. This means that we need analytic tools and engines to help us interpret the data. So many of the old school analytics tools such as SAS and others are rapidly evolving to support big data and more excitingly, new tools are rapidly emerging which were designed from the ground up with big data in mind such as Tableau. The latter category were designed to be usable by large and varied groups within the company allowing many to access, review and interpret the data.

A key element of big data analytics is the real-time nature of our interaction with data. To get a sense of this, look at your interactions with social media today. You want to know everything that is going on at any point in time, ‘instantaneously’! While this is not to say that large enterprises are going to have immediate turn around on data interpretation, but we are rapidly (and I am extremely excited about this) moving towards, rapid access to data and interpretation of data…tools like Tableau allow you  to connect ‘on the fly’ with your data sources, review your data via various graphical interpretation tools and hopefully make sound and timely decisions based on access to data.

So, what excites me my friends, is the combination of (1)+(2)+(3) above. Where more people in a small or large company can access and make wise decisions by obtaining information from data. Big Data to me has unleashed data from the guarded vaults owned by a few to many people’s minds and hands allowing us to be much smarter about data. Wow- that is fantastic!

What do you think?