IBM’s Four Ways to Innovate using Big Data

Keyboard with Big Data Button.I just read through “Four Ways to Innovate Using Big Data and Analytics” over on Forbes. It’s a good read…you should jump over and read it yourself.

If you don’t have the time (or just don’t want to), I give my thoughts on these ‘four ways” below.

The ‘four ways’ are:

  1. The payback on big data investments is happening quickly
  2. Businesses are increasingly using big data to solve operational challenges
  3. Organizations are reinventing business processes using digital tech
  4. Velocity, not volume, is driving the impact of big data

Before I dive into my thoughts, I have to point out that these ‘four ways’ are really only three as #1 above is really just informational.  I could also argue that #4 is mostly another informational tidbit (and one could argue that #2 and #3 are really the same thing) but I won’t get that picky here.

The fact that many companies are seeing payback on big data investments is wonderful. The article reports on an IBM survey that shows payback is happening very quickly with “some 63 percent of companies surveyed are seeing a return within a year, and 26 percent are getting a payback in six month.” That’s impressive, considering many organizations I’ve worked with and spoke to have no clear idea on how to calculate ROI or payback on big data initiatives when they first begin researching those initiatives.

I do love the fact that companies are using big data to solve operational challenges. That’s one of the real areas of value that you can easily point at and say “we saved X dollars or Y percent due to operational changes due to big data and analytics.”  One of the examples given in the article for this innovative approach is Wellpoint’s use of big data to “make more effective decisions about approving medical procedures and getting patients the care they need more quickly.” Wellpoint’s new system can reportedly ‘provide responses to requests for urgent pre-authorization in seconds instead of 72 hours.’ That’s pretty impressive.

I also love the fact that companies are reinventing business processes using digital tech, big data and analytics.  Companies are using all sorts of data collected from many different locations (e.g., social, mobile, cloud, etc) and then using that data to cut costs, create new services and products and increase revenue.  Solving operational challenges is basically the same thing as reinventing business processes. That said, there’s value in using data and analytics and applying them to your entire business to see if there is any improvements you can find.

The last item isn’t exactly an innovative ‘way’ to use big data but it is an extremely important thing for every organization to consider. Companies not only need to think about the ‘volume’ of data that they’re analyzing…they also need to think about the velocity of the data as it is collected and analyzed. The article again points to an IBM survey that claims “nearly three quarters of respondents say demand for data-driven insights will accelerate during the next 12 to 18 months.” Not only are we collecting more data today then ever before, but we are collecting and analyzing more data at a much faster rate then ever before.

Data analysis is no longer just about the about the size or type of data but about how fast that data can be converted from bits and bytes into useful information for the business. If your organization can quickly and efficiently convert your data into useful, informative and innovative knowledge you’ll be ahead of the big data game.

Note to Self – Don’t say “Data Driven” Anymore

dataJim Harris just wrote a nice piece titled “It’s not about being Data Driven” over on his wonderful Obsessive-Compulsive Data Quality Blog.  If you don’t have Jim’s blog on your radar, you should…he does great work over there.

In this recent post, Jim writes about the difference between being ‘data-driven’ and using data to make better decisions.  Jim writes:

In the era of big data, it’s not about being data-driven—because your organization has always been data-driven. It’s about what data your organization is being driven by—and whether that data is driving your organization to make better decisions.

I’ve been guilty of writing about companies needing to be ‘data-driven’ without making the very important distinction that Jim points out there. Success doesn’t come about because a company is data-driven…success comes from what a company does with their data and how they use that data to inform their decisions.

Jim is correct that many companies have been ‘data-driven’ for years. Most businesses would argue that they’ve been data-driven since inception. Most managers love to look at data to help them make decisions but I’d argue that many managers have historically looked at data in the wrong way. They looked at data as their ‘truth’ of how their team was doing. They looked at their data as a way to understand how their business was doing.   Many managers even look at their data as a way to improve their businesses.

The push for ‘being data-driven’ today can often make many of these managers angry, and rightfully so.  These managers aren’t idiots…they know data is important. They’ve always used data.

So…let’s stop imploring these managers and companies to be data-driven and start asking them to look at the data they’re using. Are they using all the data available to them? Is the quality of their data at question? Can they point to a full lifecycle of data management for their data? Can they ensure security, quality and governance of that data?

If they can’t answer these types of questions in a positive manner, its time for them to visit their data management and data quality processes and systems. Perhaps they’ve always been a data-driven company but they may have been using bad data or maybe they’ve just been using the wrong data.

Once an organization’s data quality and management practices are understood and new processes/systems implemented (if needed), the next question has to be about how that company uses their data. Do they use it to make decisions? Do they dive deep into the data to look for new ideas and problems to solve? Or do they just use that data as a way to point to how ‘great’ their business is?

There’s different ways of being data-driven, but like Jim said…the only way to be successful at using data is to use it to make better decisions.  Your organization can be data-driven and still be very unsuccessful. Find the data and data systems that work for your business and use them to make great decisions to make your company better.

Building a Data Culture

virt_1_346x214Many companies want to ‘do’ big data today. They’re spending money on systems, software, consulting, training and other services to be able to capture, process, analyze and use data. Those are all things that need to be done to be up their data science capabilities and skills. Companies need the right platforms, the right systems, the right people and skills to be able to properly analyze and use their data.

There’s one area that many organizations fail to address when building up their data analytics programs and skills. That area involves the corporate culture. Specifically, it involves the culture around listening, curiosity, investigation and willingness to try and fail.

Corporate culture can play a huge role in the success or failure of data analytics programs. If your company’s culture doesn’t like hearing new data that may provide conflicting information, your big data initiatives may be set up for failure from the very beginning.

In my experiences, the ability to listen and act on new data is one of the most important aspects of corporate culture that leads to success with data analytics and big data. If you don’t have a corporate culture and leadership team willing to listen to new information. For example, if your CEO doesn’t listen to data or arguments that go against her beliefs, you may be in for a very difficult time if your data analysis shows a reality different than the one that she expects or wants.

While listening and accepting competing arguments and data is the top cultural issue that can make or break big data, the other cultural aspects are important as well. For example, if the people who are working with your data aren’t curious about the data and willing to spend plenty of time investigating that data then you may be wasting money giving those people the proper skills to become a data scientist. You may be training them to act as your data scientists, but if they aren’t interested in finding out more about your data and investigation new avenues of analysis, you may not get the move value from them or your big data initiatives.

Lastly, your corporate culture should be willing to accept failure. Now, I’m not saying you should embrace or excuse failure, but many times in the data analysis world you end up finding analyses that don’t match with your expectations. Much of the time spent by data scientists is spent in small analysis projects looking for new ways to look at data. Many of these small projects end in failure with nothing of measurable value to show for the time spent on that project. Even though it may seem like wasted time, these types of projects are what make great data scientists as it allows them to continuously improve on their skills.

Successfully implementing big data initiatives is much more than just buying some software or systems. Successful big data initiatives require working on soft skills as well as organizational culture to ensure that the big data mindset is ingrained throughout the organization.

This post is brought to you by SAS.

Drowning in Data, Starved for Information

Drowning in Data, Starved for informationIn his 1982 book Megatrends, John Naisbitt wrote “We are drowning in information but starved for knowledge.” While written over 30 years ago, that line is as very true today…but I might change it a bit to match the current state of affairs. Today, we are drowning in data and starved for information.

Every organization has a great deal of data and more data is being collected every day. In addition to the already large data-sets that exist today, many organizations are looking for ways to collect exponentially more data with the Internet of Things (IoT). They want sensors to collect data from all aspects of the business including how their clients interact and use their products and services.

Anyone can collect data. Its easy. All you need to do is turn on a collection system and store data somewhere.  IDC reported in 2012 (pdf) that by 2015, we’d see data stores grow to roughly 8 Zetabytes (ZB) within organizations worldwide.

That’s a lot of data…but how much of that data will actually be useful?  They have a lot of data…but do they have any actionable intelligence?

Data is useless unless you can convert it to information and ultimately into knowledge.  In recent years, big data has been what organizations use to describe their attempts to converting all of their data into useful information.

I’m a fan of big data. I really am. I’ve said for a while that big data is more than a buzzword. Done right, big data can bring a great deal of value to a business but done poorly, big data is nothing more than bits and bytes flying around an organization. Done poorly, big data is just adding more layers of data to make it easier to drown.

When I speak to organizations about their big data initiatives, I find many that understand how important it is to convert their data into useful information. These companies understand that the work they are undertaking is much more than data analysis. They know that data is worthless unless it can be analyzed in a way that produces useful and actionable information. They understand that their big data initiatives are actually big information initiatives.

But…there are still many who don’t understand the importance of the output of data analysis. Sure, most people and organizations understand that data needs to be analyzed but many don’t understand how best to analyze that data. They implement systems and processes for data analysis but never stop to think about how best to use those systems to get the most from their data.

Big data initiatives are worthless unless their end-goal is to deliver information to an organization. That information must then be converted into knowledge to ultimately be worthwhile to the business.  Maybe its time we stop talking about big data and start talking about big information…or even better…big knowledge.

Winning Championships with Big Data?

Toronto_Maple_Leafs_logo.svgWhen I talk to people about big data and data analytics I try to tailor the message to their experience level. For example, if I’m talking to data scientists I’m generally talking at a much different level than if I’m talking to people who have no clue what the term ‘big data’ means.

For the latter group, I try to find popular examples of data analytics to help them grasp the concepts behind analytics and big data. One of the examples I use often is the movie Moneyball. Most people I talk to have seen the movie and with a little explanation, they can see how the movie and storyline evolves from the use of ‘old’ methods of instinct and ‘rules of thumb’ to analyze baseball players to the ‘new’ methods of data analysis.

Using the example of Moneyball helps people grasp the concepts of data analysis and understand how data can be used to drive decision making capabilities (ot at least assist with decisions).

I like using examples like Moneyball. It’s not quite an example of big data in action, but it is a great example of how organizations can use data to make better decisions.

This morning, I read another example of a sports team using data analytics and big data to build a better team and organization. The Toronto Maple Leafs have recently began using big data methods and systems to analyze data from across the National Hockey League (NHL) to assist in driving new decisions about players, lineups and the overall operations of the team.They are following the Moneyball example and replacing their hunches with data to attempt to make better decisions.

Time will tell if the Maple Leafs win a championship in the next few years. While that challenge is quite daunting, a major challenge for the Maple Leafs (and any other organization) is to apply data-driven decision making to their organization while also keeping those ‘hunches’ around to help guide overall decisions.

Data is wonderful but data works best when you find ways to combine that data with instinct and experience. That’s how you win championships and grow revenue using big data.


Hadoop and Big Data

lix6W5tstQNIQWhen I talk to people and companies who are just starting out in big data, I usually hear something about Hadoop. I’ll hear things like “If we are going to get into big data, we’ll need to implement Hadoop” or “we don’t have any Hadoop experience so we’ll need to gain that skill before we get into big data.”

At some point in the recent past, Hadoop has become synonymous with big data, which is a bit disconcerting. Hadoop is not a requirement for big data nor is it a required skill for anyone trying to break into big data. Sure, Hadoop is very helpful in combining and analyzing large data sets but it isn’t something that you must have to ‘do’ big data.

When I hear people talk about the need to learn or implement Hadoop before they can do anything related to big data, I always tell them to ignore Hadoop – for now. Take on some small data analysis projects before implementing new systems. Make sure your strategy is sound first, then worry about how to implement that strategy

Now, you may think that I dislike Hadoop. I’m actually a huge fan of the Hadoop platform and believe that it should be at the top of the list of platforms for every organization to consider. Hadoop is a major component of most big data initiatives, which is one of the drivers behind people automatically thinking of Hadoop when they think of big data.

Hadoop has become so popular because it provides an effective and efficient architecture to store all types of data, scales very easily and allows queries and analysis to be performed on that data. Hadoop provides solutions to many of the problems that face organizations when working with large data sets. Hadoop provides functionality to address, data integration, data visualization, in-memory analytics, interactive analytics and in-database queries.

Hadoop gives an organization a great platform to build big data processes and analytical approaches off of. There’s a great deal of value to be found in using Hadoop. In fact, there’s so much value that companies like SAS have built functionality to take advantage of Hadoop in-memory analytics capabilities to make use of the data and infrastructure that many organizations already have in place.

While you don’t need Hadoop for big data, it’s a great fit for big data. If you want to get into very large data sets and use cutting edge platforms and systems to analyze your data, Hadoop will give you the underlying platform for your big data initiatives.

This post is brought to you by SAS.