Big Data Roadmap – A roadmap for success with big data

Big Data RoadmapI’m regularly asked about how to get started with big data. My response is always the same: I give them my big data roadmap for success.  Most organizations want to jump in a do something ‘cool’ with big data. They want to do a project that brings in new revenue or adds some new / cool service or product, but I always point them to this roadmap and say ‘start here’.

The big data roadmap for success looks starts with the following initiatives:

  • Data Quality / Data Management systems (if you don’t have these in place, that should be the absolute first thing you do)
  • Build a data lake (and utilize it)
  • Create self-service reporting and analytical systems / processes.
  • Bring your data into the line-of-business.

These are fairly broad types of initiatives, but they are general enough for any organization to be able to find some value.

Data Management / Data Quality / Data Governance

First of all, if you don’t have proper data management / data quality / data governance, fix that. Don’t do anything else until you can say with absolute certainty that you know where your data has been, who has touched your data and where that data is today. Without this first step, you are playing with fire when it comes to your data. If you aren’t sure how good your data is, there’s no way to really understand how good the output is of whatever data initiative(s) you undertake.

Build a data lake (and utilize it)

I cringe anytime I (or anyone else) says/writes data lake because it reminds me too much of the data warehouse craze that took CIO’s and IT departments by storm a number of years ago. That said, data lakes are valuable (just like data warehouses where/are valuable) but it isn’t enough to just build a data lake…you need to utilize it. Rather than just being a large data store, a data lake should store data and give your team(s) the ability to find and use the data in the lake.

Create self-service reporting and analytical systems / processes.

Combined with the below initiative or implemented separately, developing self-service access and reporting to your data is something that can free up your IT and analytics staff. Your organization will be much more efficient if any member of the team can build and run a report rather than waiting for a custom report to be created and executed for them. This type of project might feel a bit like ‘dashboards’ but it should be much more than that – your people should be able to get into the data, see the data and manipulate the data and then build a report or visualization based on those manipulations. Of course, you need a good data governance process in place to ensure that the right people can see the right data.

Bring your data into the Line of Business

This particular initiative can be (and probably should be) combined with the previous one (self-service), but by itself it still makes sense to focus on by itself. By bringing your data into the line of business, you are getting it closer to the people that best understand the data and the context of the data. By bringing data into the line of business (and providing the ability to easily access and utilize said data), you are exponentially growing the data analytical capabilities of your organization.

Big Data Roadmap – a guarantee?

There’s no guarantee’s in life, but I can tell you that if you follow this roadmap you will have a much better chance at success than if you don’t.  The key here is to ensure that your ‘data in’ isn’t garbage (hence the data governance and data lake aspects) and that you get as much data as you can in the hands of the people that understand the context of that data.

This big data roadmap won’t guarantee success, but it will get you further up the road toward success then you would have been without it.


Five things the CEO wants to know about Big Data

Five things a CEO needs to know about big dataI spend a lot of time talking to companies about big data and data science. Many conversations are with people at the CxO level (CEO’s, COO’s, CFO’s, etc etc) and usually revolve around basic discussions of big data and data analytics.   One of the things that has surprised me a little from these discussions is that these CxO level people have the same basic questions about big data.

Those of us who are consultants and practitioners within the big data space like to wax poetic about big data and data science like to think that ‘this time is different’ and that big data is really going to change things for the better for any company.   While that may be the case, there are still some very basic questions that need to be answered within every organization before any major investment is made. The questions that I hear most from CxO level people can be categorized into the following types of questions:

  1. What is it?
  2. Why do we care?
  3. How is this different than {insert name of previous approach here}?
  4. What is this going to cost?
  5. Who is going to manage this?

All valid questions and all questions that should be expected when any major initiatives are being discussed. Additionally, these questions shouldn’t come as any surprise to anyone that’s been around CxO level folks before…but they often come as a surprise to many technical people because many think that big data ‘just makes sense’ and should be implemented immediately. The problem with this line of thinking is that it is the exact same type of thinking that has led organizations down many other non-fruitful paths in the past.

For example, I can think back to my early days in telecom and remember my very first job out of college. I was a software tester working on a new hardware platform that was being designed / built to offload data traffic from the public telephone network (PSTN) onto an ATM network. This was cutting edge stuff at the time during the late 1990’s when getting online meant to connect your modem to the PSTN.   The market research had been performed to show that a need existed for this and many discussions where held with technical people at many different telecom service companies. Everything looked great for this particular company until the time came to sell the product.  The CxO level people at these telecom companies were basically asking the questions I’ve listed above…and the answers weren’t compelling enough to warrant an investment in this new, unproven technology.  Sadly, the company I worked for shut down this particular product line after finding no real interest in the product.

Some of you may be thinking that my example is quite different than big data, Sure, there are proven examples of big data initiatives bringing fantastic rewards for organizations – but there are also many other examples of big data initiative failures so it makes sense that companies are cautious when it comes to new technology /initiatives.

When it comes to your big data initiatives, can you answer the above five questions for your organization?

Innovation and the CIO – Survey Results

Michael Krigsman just published a piece titled “What do business people think of their CIO?” that shares some output of a research report put out by Tech Pro Research regarding the perception of the CIO within the business,  including some results related to innovation and the CIO.

I’ll leave you to go read Michael’s write-up on the subject…he does a good job pointing out the highlights.   I did want to point out one of the results from the report that discusses innovation and the CIO within organizations.  From the report:

…there is a sizeable gap of 32 percent between CIOs and non-CIOs regarding whether the role has a strong impact on technological innovation and creativity. Nearly one-third of non-CIOs feel their CIO has little or no impact, but only 6 percent of the CIOs feel they have little impact, and none feel they have no impact. The low percentages of those reporting little or no impact across both groups show a solid footprint on the part of the CIO.

This particular passage has the following chart to accompany it:

Innovation and the CIO

From the chart, you can see that 47% of all respondents believe the CIO/CTO has ‘some impact’ and 30% believe they have a strong impact. That said, if you pull out the CIO/CTO responses, its a 49% and 20% split. Still, that’s a majority of people that believe the CIO/CTO have some impact on innovation and creativity when it comes to using technology.

Based on these responses, it is clear that most respondents believe the CIO / CTO role is vital to innovation within the organization. That’s a good sign for the CIO/CTO role.  Jump over and read the rest of the results from Tech Pro Research and/or  Michael’s “What do business people think of their CIO?” article.

Digital Transformation – Are CEO’s and CIO’s aligned?

Digital TransformationI just read “Digital transformation will shape 2016” over on and was a a passage caught my eye.

Before I give you the passage, let me put the sentence into context.  The CIO article is discussing a recent IDC report titled ‘IDC FutureScape: Worldside CIO Agenda 2016 Predictions.’ This report provides some predictions on what the CIO and IT  will be focused on for the coming year(s).

The passage that caught my eye was this one:

According to IDC, the biggest issues in IT leadership will center on business needs, capabilities and availability related to digital transformation. The data shows that two-thirds of CEOs plan to focus on digital transformation strategies for 2016 and that CIOs will be major players in leading every department through this shift. In terms of capabilities, only 25 percent of CIOs report feeling confident in how they are driving new digital revenue streams.

Emphasis mine.

When I read that paragraph, I was a bit perplexed as to how two-thirds of CEO’s will be focused on digital transformation and the CIO will a ‘major player’ in leading these initiatives while at the same time only one-quarter of CIO’s felt confident in how they and their teams are driving new digital initiatives and revenue streams.

Now – before anyone skewers me, I do realize that digital transformation is about much more than finding new revenue streams. Digital transformation covers all aspects of the business from finding new revenue streams to reducing costs throughout the business via technology.  CIO’s know how to do the latter…but as we see from the survey, not many of them are confident they can do the former.

If CEO’s are going to be focused on digital transformation in 2016 and CIO’s are going to be one of the leaders of those initiatives, one would think that those CIO’s would be much more confident in their capabilities (and abilities) to drive revenue. Right?

Sure, CIO’s can help to drive digital transformation without ‘knowing’ how they are doing with digital revenue streams, but if I were a CEO, I’d want to know my CIO had a real handle on all things digital, including how digital is driving revenue.  Alternatively, if I were a CIO and I knew the CEO was focusing on digital transformation, I’d be doing everything in my power to make sure I (and my team) were fully up to speed on what we were doing, planning to do and could potentially do in the coming year.

Based on the responses in this survey, I worry that the the CEO and CIO aren’t quite on the same page when it comes to digital transformation going into 2016.

Does IT Agility Equate to Business Survival?

IT Agility Key to Survival?I’ve been writing about agility within the data center and IT group recently. I truly believe the future of IT (and the business) requires that those of us in IT rethink how and what we deliver to our organizations.

While reading a white paper titled “Agile IT Empowers Business Success”, I ran across an intriguing sentence:  “If the business can’t adapt, it may not survive.“ I’d actually change that to say that, in most instances, if a business can’t adapt, it will not survive.

To enable a business to adapt, a business must take a look at a all aspects of their business. They need to look at systems, technology, processes, people and the culture of the business. While the entire business needs to be analyzed and transformed, one of the focal points for any organization should be their IT group.

In order to drive this adaptability, organization’s require both an agile business and an agile IT group, which is something I wrote a bit about last week.  In that previous post, I argue that the IT group must transform itself into an efficient and agile group in order to help the business become more agile.

I’ll go a step further and say that the IT group must be one of the leaders of the transformative change throughout the business.  Adapting to the fast-changing business environment today requires a great deal of technological agility so it makes sense to have the IT group be at the forefront of this change.

This type of transformation isn’t an easy undertaking. It requires an organization take a look at all aspects of their business including processes, technology, systems, people and culture.  While I’ll leave the ‘people’ and ‘culture’ topics for others to work through, the processes, technology and systems fall squarely in the wheelhouse of the IT group.

One of the areas that can be reviewed fairly easily for agility (or lack thereof) is the data center.  The data center is the backbone of the modern organization and, as such, plays a large role in determining how agile and adaptable the business can be. If that data center cannot provide the agility needed for today’s fast-paced environment, the business will suffer.

Some organizations are hesitant to undertake a transformative project like this due to the costs. While there can be large initial costs to replace legacy systems or build out new capabilities, these costs can usually be recouped in a short time from lower operational and management costs throughout the following years.

In order for companies to survive and thrive in the future, their data centers and their IT groups will need to be operating from an agile mindset. No longer can the CIO and IT group be comfortable in their old legacy systems and thinking – they must develop and maintain an agile approach to the business to allow that business to survive.

This post is brought to you by Symantec and The Transition To The Agile Data Center.

The IT Group as Cost Center – Or Innovation Center?

Cost CenterMark Thiele published a post recently titled “Why we Value IT Incorrectly – Innovation vs Cost Center.”  You should go read it.

If you don’t follow Mark’s writing on his blog or twitter, you should…he has some great insights into IT , IT leadership, the data center and many other aspects of the IT group..

While Mark’s entire post is great, I wanted to point out one particular paragraph from Mark’s post and make a few comments about it. Mark writes that:

Why isn’t IT expected to live up to a higher standard in demonstrating the benefits of projects and being more innovative in applying IT to the task of creating business value? I don’t think IT is expected to live to a higher standard because again, the average C-Suite doesn’t expect anything other than cost and risk management.

I’ve been asking that question for years. I’ve asked it at every company I’ve worked at and asked it of every CIO, CEO, COO and VP I could find.  The answer usually tended to be something along the lines of “because IT is a cost center”.

What poppycock! (I’ve been waiting a long time to use ‘poppycock).  IT isn’t a cost center! Or at least it shouldn’t be.

Maybe IT could be called a cost center in the past when IT was just a ‘cost of doing business’ but not today.Technology IS the backbone of business today and is the driving force behind most of the innovations that have happened over the last few years.

With this in mind, why do companies still view IT as a cost center?  Is it because companies still don’t really understand technology? Or…is it because the IT group hasn’t done enough to make the case for no longer being a cost center? Or is it because it is easier to remain a cost center and fly ‘under the radar’?

For most organizations and CIO’s, I’d argue its that middle option I provided above. Most CIO’s have done a poor job of ‘selling’ the value of their organization.  Either they don’t know how to sell the value of their team or don’t want to. Regardless of the answer, many CIO’s are comfortable remaining a cost center.

That comfort is why many CIO’s are being sidelined today. Those sidelined CIO’s are complaining about the fact that they don’t feel ‘included’ but most have put themselves on the sidelines by not driving change within their teams and within their organizations.

Mark ended his post with a very good comment. He wrote:

Plant the seeds of innovation, hire the right people, accept some level of risk, don’t be afraid to make tough decisions in real time, and maximize the opportunity that is locked up in information technology. This doesn’t mean you do everything internally, and it doesn’t mean the opposite. What it means is that your teams learn to accept the notion that innovative thinking can come from anywhere; how you leverage it is what is really important.

Well said Mark.

If you'd like to receive updates when new posts are published, signup for my mailing list. I won't sell or share your email.