What’s the difference between Business Intelligence and Big Data?

THE-DIFFERENCEI’m often asked the following question:

What is the difference between Business Intelligence and Big Data?

Before getting into my approach to answering that question, let’s be clear on what we’re talking about. When most people say “Business Intelligence” they’re talking about the class of products that have been implemented in most organizations, not the actual information or knowledge that is derived from the use of these systems. When it comes to Big Data, most people aren’t quite sure what they are talking about…some are talking about the size of the data, some are referring to the approach to analysis and others are talking about the process as a whole.

One of the problems that exists today is ensuring everyone understands what big data is and isn’t. While that is a discussion for another time, in this post I’ll simply say that Big Data isn’t something you buy, implement, configure and start using like you would do with Business Intelligence systems. It’s much more complicated than that.

Back to the original question: What is the difference between BI and Big Data?

I’ve never really been able to answer that question as fully as I’ve wanted to because most people aren’t willing to sit down and listen to me walk through the history of business intelligence and big data and explain their differences.

After many attempts at finding a succinct way to describe the differences, I finally figured out that most people don’t care about the technical differences or the history. Most people just want a ‘sound bite’ answer so I came up with this response:

Business Intelligence helps find answers to questions you know. Big Data helps you find the questions you don’t know you want to ask.

When I answer the question this way, I tend to get the nod of the head and and response similar to “…well that sounds really complicated!”.

It is complicated. That’s the difference between Business Intelligence and Big Data. You don’t have easy, well defined reports and answers with big data like you do with BI. You don’t have a single system to implement and manage with big data like you do with BI.  Don’t get me wrong…BI systems aren’t “simple” and the thought and planning that needs to go into BI systems and planning is very detailed, but BI and Big Data are completely different.

Business Intelligence systems have their place in business. They deliver neat, well-designed answers to neat, well-designed questions. Nothing wrong with that…but most businesses don’t have neat, well-designed questions these days. In fact, most organizations don’t really know what questions they need to be asking.

That’s the difference between BI and Big Data.

Data Discovery, BI and the SMB

1159615_30768144A CIO I spoke with last week mentioned that she and her team are struggling with their BI solution today. The solution works perfectly for analyzing their ‘standard’ data and does its job as a basic analysis and reporting tool but it doesn’t do much for their needs in the ‘big data’ realm. Their current BI tool doesn’t handle unstructured data or large datasets.

The particular BI tool owned by this organization isn’t a top tier solution in the BI space, but it was the right platform for the organization at the time they selected it. This CIO is now in the market for a new solution to solve their ‘big data’ visualization and analytics problem. In her search, she’s started looking at her current BI vendor to see if they can provide any assistance but she’s also looking at new vendors who might be able to provide a complete BI and Big Data solutions.

When I spoke to the CIO, I asked her the following question: Are you more interested in Reports or Analysis? Her answer came quick – she wanted to be able to analyze data not just report on data. She continued with a nice little diatribe on the power of analyzing data and how, by asking the right questions, an analyst can find real gems of knowledge in large data sets.

In order to ask the right questions, the CIO and her team needs to be able to dig into the data to understand what they have. Their BI platform doesn’t really allow them to do that because it doesn’t allow them to interact with the data in the necessary manner.

Data Discovery vs Business Intelligence

There’s a difference between BI and Data Discovery (DD), at least from the ‘standard’ BI platforms and vendors. Think of the difference as the same as the difference between reporting and analysis. BI seeks to use structured data in repositories to report and monitor. You can analyze data with BI tools in many different ways but the structured nature of the data and the fact that the data has been ‘scrubbed’ makes its analysis very straightforward.

Compare the above with the idea of unstructured data. There are gigabytes and terabytes of data within organizations that has been sitting around without any real means of analyzing or using it. This is where the ‘big data’ approach comes in. This is where Data Discovery comes into play. Using data discovery methods, this unstructured data can be collected, sifted and analyzed to determine if there’s any useful information sitting in it. Traditional BI tools aren’t great at this type of approach.

The answer to the question of ‘report or analyze’ is the answer for any business looking at analytics / reporting tools. If you want to just report on the data you have and have no interest in sifting through large data sets, a traditional BI tool might be perfect. If you want to dig into your data, you might want to make sure whatever platform you choose has analytics tools as well as visualization tools.

So where does that leave the SMB that has invested money into BI tools? Do they need to replace their BI tool with something else that has a more modern approach to BI and Analysis (including the ability to work with large data sets)? Or does the SMB leave their BI tool alone and bring in another platform specifically for Big Data work?

The CIO that I spoke with took the second option. She left her BI platform in place and kicked off a project to understand her organization’s needs for data analysis tools and identify a platform that would allow them to truly analyze unstructured data and start ‘doing’ big data.

This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I’ve been compensated to contribute to this program, but the opinions expressed in this post are my own and don’t necessarily represent IBM’s positions, strategies or opinions.

hit counter

Links for May 4 2009

We have a huge budget and too much time to complete our business intelligence (BI) project by Martin Proulx on Analytical Mind

9 tactics to effectively communicate your vision by George Ambler on The Practice of Leadership

Pros and Cons of Software-as-a-Service models by Laura Brandau on Bridging the Gap between IT and Business

Why Bother Looking At Social Media For Use In Projects by Bas de Baar on Project Shrink

Thought Leadership Alone Is Not Enough by Dana VanDen Heuvel on Marketing Profs Daily Fix

Proving the ROI of Community Based Customer Service by Bob Warfield on The Helpstream Blog

Lessons on Lessons Learned by Doug Bedinger on UCSC Extension in Silicon Valley

Cluetrain Plus 10: Hollow Corp Speak by CC Chapman

Rehire Every Employee, Every Day by Jurgen Appelo on NOOP.NL

Women in Business Are Risk Takers by Lela Davidson on Business Pundit

10 lessons from a failed startup by Mark Goldenson on VentureBeat

10 Faulty Beliefs That Can Doom IT Leaders by Ilya Bogoradon TechRepublic (hat tip to JourneyX)

Enhanced by Zemanta