Big Data Roadmap – A roadmap for success with big data

Big Data RoadmapI’m regularly asked about how to get started with big data. My response is always the same: I give them my big data roadmap for success.  Most organizations want to jump in a do something ‘cool’ with big data. They want to do a project that brings in new revenue or adds some new / cool service or product, but I always point them to this roadmap and say ‘start here’.

The big data roadmap for success looks starts with the following initiatives:

  • Data Quality / Data Management systems (if you don’t have these in place, that should be the absolute first thing you do)
  • Build a data lake (and utilize it)
  • Create self-service reporting and analytical systems / processes.
  • Bring your data into the line-of-business.

These are fairly broad types of initiatives, but they are general enough for any organization to be able to find some value.

Data Management / Data Quality / Data Governance

First of all, if you don’t have proper data management / data quality / data governance, fix that. Don’t do anything else until you can say with absolute certainty that you know where your data has been, who has touched your data and where that data is today. Without this first step, you are playing with fire when it comes to your data. If you aren’t sure how good your data is, there’s no way to really understand how good the output is of whatever data initiative(s) you undertake.

Build a data lake (and utilize it)

I cringe anytime I (or anyone else) says/writes data lake because it reminds me too much of the data warehouse craze that took CIO’s and IT departments by storm a number of years ago. That said, data lakes are valuable (just like data warehouses where/are valuable) but it isn’t enough to just build a data lake…you need to utilize it. Rather than just being a large data store, a data lake should store data and give your team(s) the ability to find and use the data in the lake.

Create self-service reporting and analytical systems / processes.

Combined with the below initiative or implemented separately, developing self-service access and reporting to your data is something that can free up your IT and analytics staff. Your organization will be much more efficient if any member of the team can build and run a report rather than waiting for a custom report to be created and executed for them. This type of project might feel a bit like ‘dashboards’ but it should be much more than that – your people should be able to get into the data, see the data and manipulate the data and then build a report or visualization based on those manipulations. Of course, you need a good data governance process in place to ensure that the right people can see the right data.

Bring your data into the Line of Business

This particular initiative can be (and probably should be) combined with the previous one (self-service), but by itself it still makes sense to focus on by itself. By bringing your data into the line of business, you are getting it closer to the people that best understand the data and the context of the data. By bringing data into the line of business (and providing the ability to easily access and utilize said data), you are exponentially growing the data analytical capabilities of your organization.

Big Data Roadmap – a guarantee?

There’s no guarantee’s in life, but I can tell you that if you follow this roadmap you will have a much better chance at success than if you don’t.  The key here is to ensure that your ‘data in’ isn’t garbage (hence the data governance and data lake aspects) and that you get as much data as you can in the hands of the people that understand the context of that data.

This big data roadmap won’t guarantee success, but it will get you further up the road toward success then you would have been without it.


The New CIO – Leader first, technologist second

bigstock-Think-About-Leadership-39370759I’ve been writing about the “New CIO”  for almost 5 years and I finally feel like there’s some real change coming to the CIO role and subsequently, to the world of the IT Pro.

For example, Dion Hinchcliffe recently outlined his thoughts on The New CIO Mandate.  In that post, he describes his ‘Five Elements of the New CIO’ as:

  • Lead the business from the front.
  • Re-invent the business for next-generation digital.
  • Create new and highly engaging digital workplace, customer, and business partner experiences.
  • Enable emergent, decentralized tech change in the organization.
  • Don’t constrain IT, fundamentally empower.

All good elements for CIO’s to focus on.

You’ll notice that there’s nothing in there about technology specifically. There’s nothing in there about ‘buzzwords’ and ‘trends’.  There’s nothing there about ‘process’ or ‘cloud’ or ‘big data’.

These five elements are focused on leadership.  Pure and simple.  The New CIO is a leader. Not a technical leader….just a leader.  Obviously, a good CIO will need to understand the high-level aspects of their team and the systems and processes they use, but the focus of the New CIO has to be on the business and delivering business value.

I’m happy to see other folks out there talking about the ‘new CIO’ and the need for the CIO to move into a real, proper leadership role.  Now…its time for CIO’s to heed the advice and step up and lead.

A CIO and A CMO walk into a bar…

direction signs By emreterok on flickrA CIO and A CMO walk into a bar…

…except…they aren’t in the same bar.

They made plans to meet at Good Time Charlies on Monday.   You know – the CMO and CIO are best friends now so its time to go have a few drinks together, right?

They agree on a time and day and schedule the happy hour together in their calendar.

On Monday, the CIO leaves the office and drives over to Good Time Charlies on the east side of town.

The CMO leaves a few minutes after the CIO and heads off to to Good Time Charlies as well. Except…she goes to the new Good Time Charlies on the West side of town.

Kind of sounds like most organizations today, no?  Lots of talk about where they are going, but no real planning or clear communications about the actual destination.

The CIO and CMO want to be friends. They want to work together. They want to do the right thing.

But…are they on the same ‘page’ when it comes what needs to be done?   If not, you’ll end up just like our CIO and CMO above – in different places waiting for the other.

Are the goals of the CIO and CMO well articulated and understood?  Sure…each person understands their own goals, but does the CIO understand the goals of the CMO and vice versa?

Do the CIO and CMO communicate regularly? Do they meet regularly one-on-one?

CIO’s – do you know what the goals of the CMO and marketing group are?  Do you understand them? Do you understand how the IT group can help with those goals?

CMO’s – do you understand the goals of the CIO and IT group?  Do you understand how your team can work together with IT go ensure your goals and their goals are met?

Its very easy to say that the CIO and CMO are working closer together and will be doing so for years to come…but without clear goals and an agreed upon strategy, they may not actually be doing the best work possible.

Forget whether the work is the best possible work…without clear goals, strategy and regular communications, the CIO and CMO may end up at completely different destinations.

Image Credit: direction signs By emreterok on flickr

The Asshole CIO

The No Asshole Rule

This is based on a true story…its long but worth it…I hope.

As long as Little Johnny could remember, he wanted to be ‘the boss’.

He came from a long line of bosses.  His Dad was an important boss.  His grandfather was too.  In fact, his grandfather was fairly well known in some circles as a boss.

Little Johnny went to school and got a degree.  He started his career like most did back in those days…working for one of the blue chip companies.  He was going to be a company man like his father and grandfather before him.  And he was going to be ‘the boss’.

Little Johnny, who I guess I should call John now since he isn’t so little anymore, spent a few years working for the Fortune 500 company working within the information systems group.  He learned his lessons and was promoted to a managerial role within a few years.

John was feeling great…he was now ‘A’ boss…but not ‘THE’ boss.  He still had a long way to go.

From ‘A’ boss to “THE’ boss

John was happy to be a manager but he had his sights set on being THE boss.  His goal was to first become a VP and then move from there into the role he was striving for – the role of the CIO.

To John, the CIO was the ideal role for him and was his goal.  The role would allow him to be THE boss in IT and would, perhaps, allow him to stretch into a CEO role in the future.  But for now, he was focused on the CIO role.

To get to the CIO role, he knew he’d have to have a lot of experience in different areas of IT and business.  He also knew he’d have to be tough. He’d have to be  a tough negotiator, tough on his people and tough with vendors.   Toughness goes hand-in-hand with being ‘the boss’.

With that in mind, John set out to claw his way up to the CIO role.

John set out some goals for himself.  In 5 years, he planned to be a VP. In 10 years, CIO of a mid-sized organization.  In 15 years, he planned to be the CIO of a large company, perhaps the very blue chip company where he worked.

Climbing the ladder

With this plan and goals in mind, John worked hard.  He politicked.  He cajoled.  He threatened.  He did anything & everything he could to climb the ladder to success.

John became known as tough guy.   He never accepted no for an answer and rarely accepted yes too…if you told him ‘yes’, his expectations were that whatever you said ‘yes’ to would be done quickly and cheaply.

He became a hard nosed task master to those people that worked for him.  He didn’t necessarily care if the tasks were done right…he just wanted them done.  And he wanted them done as cheaply as possible.

John drove his team mercilessly and he became known as someone that could get anything done.  If you needed something done in IT, you went to John because you knew he’d do whatever it took to accomplish it.  He’d stab someone in the back (figuratively, of course). He’d lie anytime it suited his needs.  He’d cheat too.  Whatever it took to climb the ladder.

To get a lot of his work done quickly, John had to turn to outsourced workers and vendors.   He hated this though…he hated vendors with a passion.  Hated them. He even boasted about it claiming to the worlds worst customer.

He hated vendors and told them that on a regular basis…but yet many vendors would do anything to get John’s business.

John became despised by his team and most of the organization, but he was tolerated because he got the job done.  Never-mind the fact that most things weren’t completed correctly…they were just completed.  And John’s bosses couldn’t tell that he was getting things ‘done’ rather ‘done right’.

John quickly moved up the ranks.  His goal of becoming a VP in 5 years was achieved.  He’d been quickly promoted up the ladder into a VP role because of his uncanny ability to get anything done under budget and within schedule.

The politicking and ‘asshole-ish-ness’ (a new word I think) continued for a few more years.  John decided it was time to move on and start working towards moving into that CIO role that he coveted.

Finally….he gets to be THE boss.

Caution Raised Assholes Ahead By Prehensile Eye on flickr

Caution Raised Assholes Ahead By Prehensile Eye on flickr

John put his feelers out to many of his vendors and colleagues.  Secretly, these people disliked (perhaps even hated) John, but they thought he’d be a good CIO…not because he’d be a good CIO…but because they could make money from him once in the role.

John quickly found himself with a few offers for CIO roles in companies large and small.  He studied the opportunities briefly and quickly chose the one offering the larger salary and benefits package.

After accepting the offer, he was ecstatic.  He was now the CIO of a large organization….a company much larger than the one he expected to have landed at for his first CIO gig.  He’d blown past his 2nd goal and was well onto the 3rd goal.

John was set now. He was going to be rolling in the dough now.  AND…he’d finally get to be THE boss.

John’s first year on the job was a productive one.  He accomplished a ton of projects.  The CEO of the company told him he’d done more in that one year than the previous CIO had been able to get done in his five years on the job.

John was loving life. He had a team of 300 IT professionals working for him spread around the world.  He had more than 100 vendors, consultants and contracting companies  lined up to do project work for him.  He was set.

Never-mind that none of those 300 IT professionals were happy. Never-mind that the vendors and consultants were barely marking a profit on the project work they did. Never-mind that John’s “get it done at all costs” attitude was driving the IT shop and people into the ground.  Few in IT were happy.  Heck…few outside of IT were happy to have to deal with anyone in IT either.

Neither John nor the leadership team cared to hear about these issues….at least during the good times.  Sure the IT team had a revolving door and it was rare to have someone new stay on board for more than 2 years. John didn’t really care…those people that couldn’t take the pressure could leave.  He had his core team of people who’d been there 10 years or more and they weren’t leaving anytime soon. What he didn’t know (or perhaps knew but didn’t care about) was that the folks that were long-term employees were basically stuck due to retirement plans and their inability to keep their skills sharp and up-to-date.

While John was kicking off his biggest project to date, the economy started tanking.  And it tanked hard.

Being THE boss…in bad times

The economic troubles hit John hard.  He was forced to layoff some staff.  His vendors were laying off staff…if not closing up shop completely.

It dawned on John that with the economy in such a bad state, he could push the remaining staff, vendors and consultants as hard (or harder) than before because they needed his projects more than ever.

John pushed and pushed.  He forced vendors to take projects for far less money than they used to. He forced staff to work harder and for longer hours.

But…John’s projects weren’t getting done and people were noticing.  The organization started noticing that their IT projects were falling behind.  They were also noticing it was getting more difficult to get anything out of the IT group.

John started feeling the pressure from the other senior leaders in the company.  He argued that his vendors were letting him down. He denigrated his staff.   He placed blame everywhere he could, except for himself.

John responded by pushing his team harder.  He began to call daily meetings for all his project teams.  He would scream at his project managers.  He’d scream at his IT managers.  He’d scream at everyone he could find.

John’s vendors and consultants were getting fed up.  They tried talking to John to see if he would understand that they couldn’t work under John’s current approach.  After months of being treated poorly, many started walking out the door.

John screamed. He threatened. He cajoled.  And he failed.

After 6 months of not getting anything done, John was asked to resign as the CIO.

No boss at all

John was devastated.  He couldn’t understand what happened…he was doing his job as CIO.  He was doing everything that he’d always done.  He’d always been successful by being tough…but it didn’t work this time.

After leaving, John started emailing and calling his old colleagues and vendors asking for referrals and leads for openings.  How many responses did he receive to his more than 200 emails / phone calls?  Zero.

He networked. He called. He emailed.  He was relentless in his pursuit of leads and referalls for a new CIO role but he got nowhere.

One morning, he heard that the CIO he’d worked for at the Blue Chip was retiring.  He knew he’d have a chance to replace him…so he set up a time to meet with the CEO of the Blue Chip to discuss the role.

The day of the meeting saw John sitting in a conference room with the CEO, COO, CFO and a few senior VP’s from IT.  He was excited.  This was it…he was going to get the CIO role that he always wanted.  The previous CIO experience was just a roadbump on the way to ultimate success.

The meeting started with handshakes and coffee.  Then John decided to jump right into it.

He started selling himself.  He talked for an hour about his experience and his successes.  He talked about all the things that he was able to accomplish for the Blue Chip and for the other organization.

John…the asshole

When John was done talking, the CEO spoke up.

‘He proceeded to destroy John’s hopes.

He told John that all the ‘successes’ that he’d just spent an hour talking about had to be reworked over the last few years.  The systems built by John and his team had major flaws throughout them and had been rebuilt.  He told him that the IT staff that worked for John had celebrated when he left.

The CEO then surprised John when he told him that he’d talked to a few folks who’d worked with him as CIO in his previous company.  To a person, they’d all said John had been a disaster. Every project undertaken by John had to be reworked there as well.

John was incredulous.  He couldn’t believe what he was hearing.  John was pissed.

He struck out at the folks in the room.  He placed blame on others.  He placed blame on everyone who’d ever worked for or with him. He  placed blame on everyone in the room.  He ranted and raved.

The CEO interrupted John’s tirade and said something that John couldn’t really believe. He said:

John…you’re an asshole.  Always have been and always will be.  It was a mistake to promote you here and obviously a mistake that you got that CIO role.  You’re an asshole and until you realize that and confront it, you’ll never have another good job again.

John was an asshole.  Everyone that worked for him learned this instantly.  Most people that worked with him realized it quickly.  And finally, John started to realize it too.

John made it to the CIO role. He’d met his goals. But he’d been an asshole doing it.

Don’t be an asshole CIO…or an asshole in general.  Pick up Bob Sutton’s The No Asshole Rule and learn more about becoming less of an asshole yourself.

Oh…and about John.  He’s now working temporary project work as a project manager.  He’s done OK for himself but rumor has it that he hasn’t learned much from his mistakes.  Last I saw him, he was the same asshole he was when I worked for him.

Which IT project comes first – Legacy or Sexy?

If you’re organization is like most, you’ve got a lot of legacy IT systems.   Most of those legacy systems require a lot of maintenance. Most of those legacy systems need updates.

In addition, you’re organization has a ton of ‘sexy’ projects that need to get done. Projects like integration of social platforms.  Collaboration tools.  New marketing technology platforms. Etc. Etc.

But…you’ve got a limited budget,  limited workforce and, at times, limited visibility by senior leadership into what projects need to get done.

In addition, you’ve got a lot of buzzwords out there driving a lot of the thinking within the leadership of the organization.  Social.  Enterprise 2.0.  Web 3.0. Cloud.  Digital this…and digital that.

So…which project comes first?  Legacy or Sexy?

The Legacy Projects

Legacy systems can almost make a case for themselves.  I mean…their called ‘legacy’ for a reason, right?

Legacy systems are those that have been around for a while….maybe a year or maybe 20 years.  Perhaps you’ve got a mainframe system running your Financial system that needs an update.  Or…maybe you’ve got a server farm running a poorly architected data system.

Whatever your definition of ‘legacy system’ is,  you’ve got them and many of them need to be updated and/or replaced.

The Sexy Projects

Sexy systems are those that are new and in the forefront of your top management teams’ minds.  Their the Enterprise 2.0 platforms.  The centralized content management system.  Social Network systems.

These systems are those that are driven, at first, by the buzzwords of the day.  Of course, a good CIO and/or IT team will push the buzzwords away and make these sexy systems fit into, and drive value for, the organization.

Whatever your definition of ‘sexy’ is you’ve got many of these types projects on your plate and many more coming down the road.

How do you decide?

Well…unfortunately, you probably don’t decide…at least not alone. Most organizations have a lot of people involved in project selections for IT.  Marketing has a ton of technology projects as does HR, Finance and Sales.

Most organizations have a different approach to selecting those projects that will be funded. In many organizations, a selection process is followed to make sure that the right projects get funded. That selection process varies by company but at its core, the process selects (or should select) those projects that fit with the organization’s strategic goals for the coming year(s).

It doesn’t matter how your organization selects project to fund; just understand what that process is.

Once you understand that process, you can help the people involved in that process to understand what the important projects are.  You can help them understand the legacy vs sexy debate.

But first…you’ve got to understand that debate yourself. Many times…its the legacy project that brings the most long term value to the organization, but its the sexy projects that get the funding.

If the legacy projects are the most important to your business, make sure everyone knows why.  And vice versa for the sexy projects.

What’s more important to your company for the next year?  The sexy project…or the legacy project? Or both?

Solving today’s problem; creating more for tomorrow?

fixing problems?Over on InfoBOOM yesterday I wrote a post titled The Cloud and The Silo where I discussed one of the current challenge of CIO’s and IT groups today – that of non-IT groups going to the cloud for their IT services/applications.

I’ve mentioned the topic before in posts about Shadow IT and the diminishing role of the CIO. I have also argued for a broader role for non-IT groups in managing their own technology initiatives – as long as IT has a role in those initiatives.

I’m a big fan of the concepts behind Scott Brinker’s Chief Marketing Technologist concept and would love to see that approach taken within a few companies that I’ve worked with/for…I think it makes a lot of sense.

I’ve heard many IT professionals  lament that these non-IT groups are ‘going around IT’ and aren’t ‘following the processes’ setup by the IT group and the CIO. I’ve also heard many non-IT people talk about ‘going around IT’  because IT is ‘just too slow’ and “they can’t get anything done”.

While there are a lot of issues here that must be addressed by the organization (and the CIO). There’s a leadership issue at play here…one that squarely lands on the IT and CIO’s shoulders.  If the IT group can’t get something done, then you can’t really blame someone trying to find someone / something that will get the job done, can you?

But….I’m also concerned that there’s a larger organizational issue at play here.  Allowing IT to be circumvented, regardless of the reason, shows a disrespect that would drive anyone crazy.  Would the finance team be happy if someone went around them to run their own books?

OK…so I am rambling a bit here…but the basic question I have is this: By ‘going around IT’, are we building a bunch of silos around the organization that will come back to bite us in the ass in the future?

I’ve heard proponents of ‘going around IT’ say that the cloud makes them more agile.  The cloud allows them to rollout new products, offerings, initiatives much faster then when waiting for IT.  All true.

By ‘going around IT’ and using the cloud to build applications and/or roll-out services, are organizations building silos that will never be integrated? Are we building these silos without a thought to what happens to the data?

What happens to the customer data ‘in the cloud’? What happens when you change platforms?  Will the data ever be pulled back into the enterprise for use by other parts of the organization?  Does it need to be?

All valid questions that many people ignore and.or dismiss as irrelevant.  I’ve even heard a few people say ‘to hell with the data…it doesn’t matter'(!!!!!).  Of course it matters…data is the lifeblood of the organization.  If it doesn’t matter to you, you’re missing the point of whatever project you’re working on.

Now…don’t get me wrong here…there are times when going to the cloud is the right way to go.  There are even times when going around IT is the only way to get something done…heck…I’ve done that myself.

But…before you (or I)  ‘go around IT’ to get something done….take a few minutes to think about the long term implications.

Are we solving our problem today while at the same time creating more problems for tomorrow?

If you'd like to receive updates when new posts are published, signup for my mailing list. I won't sell or share your email.