Is Big Data too big for small business?

6621199759Big Data.  Two small words with huge meaning.

Do a search for “big data” on Google and you’ll find over 23 million results. Do a book search on amazon for ‘big data’ (affiliate link) and you’ll get over 5,000 results (I’m actually surprised the number is that low).

Big Data has a few different connotations.   There are some contexts where it points to the data that we (organizations, society, etc) are collecting.  In other contexts, it is used to describe the approach and technology to analyzing the large data-sets found within organizations and society.

Needless to say, Big Data has become a Big Topic over the last few years.   In 2011, McKinsey claimed Big Data was the “next frontier for innovation, competition and productivity“.   Big organizations have jumped on the Big Data  bandwagon over the years and brought in all sorts of consultants and software to slice and dice their data and find new ways to run their business and compete in their markets.

One of the areas that seems to be overlooked – at least in my experience – is the use of data analysis tools and data-setsfor use by small businesses.   Is there a place for small organizations in the world of big data? Can small businesses take advantage of this ‘big data’ stuff?  Are small business’ data-sets even large enough to be considered ‘big data’?

The answers are complex…because every question has multiple answers.

Is Big Data too big for small business?

The  book search on amazon for ‘big data’ on amazon returned over 5000 results.  Do the same search for ‘big data small business’ in the book section of amazon and you’ll get 16 results…with only 3 of the 16 books being targeted ‘big data’ books for small business.

Any organization can take the concepts behind ‘big data’ and apply them to their business.   Anyone can collect data. Anyone can analyze data.  But…the key to “big data” is having not only the right data, but also having the right technology and the right skill sets. And, for most small businesses, those last two points – technology and skills – are the real roadblocks.

Roadblock #1: Big DataTechnology

That said, technology isn’t as big a roadblock as most people think. You don’t need to spend a lot of money on technology to ‘do’ analysis of your data.

My preferred setup for analysis is made up of 100% open source software that cost me nothing to obtain.  I use MySQL and/or MongoDB to store data and Python to collect and analyze data with some other open source software thrown in for other pieces of the puzzle (Hadoop, Apache, etc).The only money I’ve spent on any ‘technology’ is for hardware – and that was only because i wanted a dedicated server to collect/store data on.   At no point have I spent money to ‘buy’ any other piece of technology or software to ‘do’ big data.

Using open source platforms, the technology argument is one that can be overcome for small businesses (or any sized business).

So…one roadblock down…or at least a new map to get around the roadblack…one more to go.

Roadblock #2: Big Data Skills

Data analysis is a skill that can be learned. In fact, its a skill set that can be learned by anyone willing to study and learn the concepts. That said, there’s more to data analysis than just spreadsheets and numbers. The hardest part of analyzing data…is doing the stuff before you get to the analysis itself.  This includes determining what data to really look at. It also includes thinking about how best to capture / collect data.  Additionally, it includes the often ignored skill of understanding what data to ignore.

The bigger issue that hits small businesses isn’t the skills themselves but the lack of resources to allow someone to devote to learning data analysis methods and tools.  Then, after learning said skills, they’ve got to devote time to actually applying these skills.

Finding this person (or persons) to learn a new skill set and spend time analyzing data is a hard sell for most small and medium organizations. There aren’t a lot of people sitting around doing nothing in a small business.

That said, there are other approaches then just hiring people to do this type of work.   There are tools being built now specifically for those folks that don’t have the technology or skill sets to ‘do’ big data.  An example is Tableau (no affiliation…I’ve just used the tool before), a company focusing on building analytics tools for ‘everyone’.   While I’ve not used Tableau in depth, I have spent some time with the tool and it seems to be a nice platform that could easily be learned and used by any business.

Big Data and Small Business

The two main roadblocks for small businesses to ‘do’ big data aren’t really roadblocks at all. Yes, they are challenges that must be overcome but they aren’t unmovable roadblocks unless the small business continues to make them so.

The key for any small business who wants to ‘do’ big data is to invest the time and resources to take advantage of the data that they have. That ‘investment’ doesn’t necessarily mean going out and hiring someone to do the work nor does it meaning outsourcing the work.  But…it does mean finding someone that can help with analysis and/or teach you / your team the necessary methods for analysis.   It could be as simple as asking a few folks around the office that might have an interest in data and analysis methods. It could be as simple as finding someone at a university (or even a high school) that wants to take on a project to learn new skills.

As its defined, big data might be too big for small business, but the concepts behind big data – identifying, collecting, analyzing and using data – aren’t too big. Anyone can use do four steps regardless of business size and technical accumen.

To ‘do’ big data like the ‘big boys’ might be out of reach for most small businesses but the ability to collect, identify, analyze and use data is available to anyone.  Tools are available via open source for the technically incline to build their own data analysis platforms.For those that aren’t technically inclined, there are analytics platforms available for use, but you’ll still need to identify and collect data to analyze.

Actually…its not that simple…but I’ll touch more on that in later posts.

Image Credit: Visualization of Twitter bios in the big data community by metaroll on flickr

7 responses to “Is Big Data too big for small business?”

  1. WJVonAlt Avatar

    Good morning, Eric! Disclaimer before we get started – I am a Splunk employee and shareholder.

    While many points of your article are spot on, I’m happy to say that not all is lost for small business being able to leverage the power of big data and extract operational intelligence from it!

    I encourage you to examine Splunk (SPLK) @ Our software aims to help consumers of data from all levels of experience and backgrounds be able to organize their unstructured data, and search and report on it without needing a PhD in Dat Science or millions of dollars of high-end equipment.

    Prior to being an employee for the last four months, I was a customer for years. While no product is a panacea, we have successfully blown past the two roadblocks cited here, and many others that customarily prevent data consumers from achieving their big data goals.

    1. Eric D. Brown Avatar

      Thanks for the link to Splunk…I’ll take a look at it.

  2. […] A few weeks ago I wrote about Big Data and Small Business. […]

  3. […] month I wrote a post titled “Is Big Data to Big for Small Business?” where I asked the […]

  4. […] you into thinking big data is more than it is. You and your organization do need to have the right technologies and the right skills in place to do big data correctly, but you don’t need to hire heavy-duty consultants ore spend […]

  5. Nadkalpur Manjunath Avatar

    An insightful post. On Roadblock #2: Big Data Skills, there are a few free alternatives to Tableau these days. Qlik Sense and Microsoft BI offer free desktop versions should SMBs need to do some data analysis. Other than that this article is full of useful tips and sets the expectations right for data analytics for small businesses.

    1. Eric D. Brown, D.Sc. Avatar

      Thanks! I haven’t looked at Qlik Sense yet…I’ll take a look at that now.