The enormous scale of big data will keep many businesses from realizing the tremendous value embedded in it – a task that will require bringing big data down to a manageable size.
At the dawn of the big data era, business leaders are being bombarded with a virtual shock and awe campaign. The 1.8 zettabytes — that's 1.8 trillion gigabytes — of information generated in 2011 will grow by a factor of nine over the next five years, according to IDC. Gartner predicts that the big data market, now valued at $5 billion in revenues annually, will explode to $53 billion by 2016.
The initial reaction, and rightfully so, is how in the world are we going to deal with all this? But while much of the attention has been on the three Vs of big data — volume, velocity, and variety — the most important aspect has been on the back burner: the actual value to the business.
What will separate the winners from the losers will be the ability to sift through the mounds of new and emerging data to uncover the few precious nuggets with significant business value. And that will require something that few, if any, companies have today: the ability to make big data seem small.
As we all know, there is more — and more valuable — data available to the enterprise than ever before. But it's all over the place and in all kinds of formats. Figuring out how to efficiently capture, process, and analyze all that information is daunting and, at the moment, virtually impossible — which is why companies shouldn't even try.
Instead, we're going to see sophisticated companies creating methods for discerning which types of incoming data are likely to have business value. They won't form a dozen executive committees, hold endless meetings, and develop five-year plans. They will instead create agile but relevant and actionable data blueprints that, overlaid onto their business objectives, will clarify their big data analytics initiatives. If a priority is to enhance customer experience, for example, they will focus only on information that can improve the supply chain or time to market or customer service as appropriate. This data blueprint will be mapped to a subset of new IT capabilities that quickly deliver value to the business.
Big data focus will be different for every enterprise. But one thing is clear: Without that blueprint and frequent iterations to test for business value, there will be some big big data disasters. Enterprises that continue to be absorbed by the enormity of the task will overspend on data warehousing and capacity. The computing power required to keep pace with the explosion of data will skyrocket. Meanwhile, these companies will have to eat the opportunity costs associated with spinning their wheels on the wrong priorities. Big data spending may not only consume the IT budget but also become one of the business's biggest costs. For a look at what's ahead for those that refuse to focus their efforts and iterate rapidly, just look back at the multimillion-dollar ERP failures, when companies overspent without a value compass.
The distinction between wasted investment and effective transformation — the ability to convert this big data opportunity into value — will be focus and agility. That's what will make big data small. This doesn't mean that a tight, central group will oversee an enterprise's big data strategy and operations. To the contrary, the owners and stakeholders of big data efforts will expand and change over time — the CIO or CMO one day, the COO or line of business leader the next. Companies will collect, process, and analyze big data in different places throughout the organization. But in each place, they will maintain a single-minded focus on business alignment and value so that even the largest amounts of information can be made relevant.
Big data success won't be a big bang for the enterprise. Those who succeed will take an incremental, iterative approach to unlocking its value over time. Not every company will transform itself into the next Amazon or Google overnight or ever – nor should they.
But a health insurance company will be able to reduce fraudulent claims. A pharmaceutical company will be able to improve drug efficacy and safety. Manufacturers will create predictive supply chains. Financial service firms will manage risk more effectively. Telecom companies will reduce customer churn. Retailers will master real-time inventory and pricing.
Big data can effect big transformation, but one focused step at a time.