April 20, 2012


Big Data and Data Management – is there any Relation ?

According to Industry experts, only close to 0-10% of Information created is structured and remaining 0-90% is multi data sets which are both structured and un-structured in nature and growing at an exponential pace that will surpass the growth of only transactional data. One of the classical example is the location detail in mobiles gets more quickly updated than the number of times one goes to bank ATM and do a withdrawal from the bank.

Around 19th century, thinking of 1 Exabyte or Petabyte of data storage or intelligence data analysis was an imagination people would not have acknowledged. 10 years post to that industries recognized the need for the analysis of Gigabyte and slowly move towards Terabyte and today in 21st century we are all talking how we could handle, store, search, organize, archive, de-duplicate and do intelligence analysis of Petabyte of data – to increase the top line and bottom-line of the organization.

Suddenly with the exponential “Data Explosion” across Applications landscape, companies are doing a due-diligence how to use the “Data Deluge” to drive the profitability and revenue growth. As we move ahead towards 2020, Organization needs to shift their gears from SAAS- software as a service, PAAS – process as a service, and IAAS – infrastructure as a service to “DAAS – Data as a Service”. The growing volume of Data inform of “Structured Data – Transactional and Master data”, “Unstructured Data – in form of files, letter, videos, images etc etc..” and Social Media Data – in form of public websites, corporate social websites, multichannel ” has resulted in coining a term called as “Big Data”

Big Data as the name suggests is an amalgamation of large, complex, dynamic data sets which in un-manageable under the existing or current technological landscape both from storage and intelligence insights overview. According to Gartner, many IT leaders are attempting to manage "big data" challenges by focusing on the high volumes of information to the exclusion of the many other dimensions of information management, leaving massive challenges to be addressed later. Big data is a popular term used to acknowledge the exponential growth, availability and use of information in the data-rich landscape of tomorrow.

"Big Data" is not one more mechanism with Data-warehousing capabilities which stores neither large volume of data or data types nor one ERP package. Its objective is towards realizing the Business outcomes by tapping all then Enterprise data assets and discovering unknown relationships between customers, products, locations or suppliers. To get the best out of the “Big Data” Organizations needs to build the physical infrastructure to capture and handle the extreme volumes of data with respective data types. It can then be consolidated and integrated with proper Data Governance to get the “Single source of Truth” – which can be feed for Data mining and Analytics and result in more Acquisitions, Analysis and be an enabler for increasing the top line and bottom-line of the organization.


Today as the world going through lot of uncertainty and crisis situations, organization look forward to implement “Data as a Strategic Asset”. Data Harnessing and Enrichment is like Renewable energy. Like organizations harness raw energy to be converted into usable energy, today companies feel the raw data to be converted into usable data and then enrich the same to get the maximum benefits from the insights of Data. Companies are looking for re-engineering the “Big Data” as a Strategic Initiative and using technology to implement the same and monetize it.