Data Discovery, BI and the SMB

A CIO I spoke with last week mentioned that she and her team are struggling with their BI solution today. The solution works perfectly for analyzing their ‘standard’ data and does its job as a basic analysis and reporting tool but it doesn’t do much for their needs in the ‘big data’ realm. Their current BI tool doesn’t handle unstructured data or large datasets.

The particular BI tool owned by this organization isn’t a top tier solution in the BI space, but it was the right platform for the organization at the time they selected it. This CIO is now in the market for a new solution to solve their ‘big data’ visualization and analytics problem. In her search, she’s started looking at her current BI vendor to see if they can provide any assistance but she’s also looking at new vendors who might be able to provide a complete BI and Big Data solutions.

When I spoke to the CIO, I asked her the following question: Are you more interested in Reports or Analysis? Her answer came quick – she wanted to be able to analyze data not just report on data. She continued with a nice little diatribe on the power of analyzing data and how, by asking the right questions, an analyst can find real gems of knowledge in large data sets.

In order to ask the right questions, the CIO and her team needs to be able to dig into the data to understand what they have. Their BI platform doesn’t really allow them to do that because it doesn’t allow them to interact with the data in the necessary manner.

Data Discovery vs Business Intelligence

There’s a difference between BI and Data Discovery (DD), at least from the ‘standard’ BI platforms and vendors. Think of the difference as the same as the difference between reporting and analysis. BI seeks to use structured data in repositories to report and monitor. You can analyze data with BI tools in many different ways but the structured nature of the data and the fact that the data has been ‘scrubbed’ makes its analysis very straightforward.

Compare the above with the idea of unstructured data. There are gigabytes and terabytes of data within organizations that has been sitting around without any real means of analyzing or using it. This is where the ‘big data’ approach comes in. This is where Data Discovery comes into play. Using data discovery methods, this unstructured data can be collected, sifted and analyzed to determine if there’s any useful information sitting in it. Traditional BI tools aren’t great at this type of approach.

The answer to the question of ‘report or analyze’ is the answer for any business looking at analytics / reporting tools. If you want to just report on the data you have and have no interest in sifting through large data sets, a traditional BI tool might be perfect. If you want to dig into your data, you might want to make sure whatever platform you choose has analytics tools as well as visualization tools.

So where does that leave the SMB that has invested money into BI tools? Do they need to replace their BI tool with something else that has a more modern approach to BI and Analysis (including the ability to work with large data sets)? Or does the SMB leave their BI tool alone and bring in another platform specifically for Big Data work?

The CIO that I spoke with took the second option. She left her BI platform in place and kicked off a project to understand her organization’s needs for data analysis tools and identify a platform that would allow them to truly analyze unstructured data and start ‘doing’ big data.

This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I’ve been compensated to contribute to this program, but the opinions expressed in this post are my own and don’t necessarily represent IBM’s positions, strategies or opinions.