At HCLTech, we work with business leaders from all aspects of corporate life – finance, accounting, and human resources to engineering, information technology, and customer support – representing the full range of business activities – from small startups to global mega-corporations, from agriculture to manufacturing to robotics, and from services that range from insurance to education, to health care.
The common thread in our discussions and data science practice is business leaders’ goals to use data to make better decisions, to reduce costs, to increase efficiencies, to improve their product and service offerings, to increase sales, to grow their customer base, and to improve customer retention. These goals are timeless. Most famously, Frederick Taylor (Reference 1) and Frank Gilbreth (Reference 2) documented the theory and practices of industrial organization and decision theory in the early years of the modern industrial era (1911-1912).
Modern data science represents the natural evolution and symbiosis of these Principles of Scientific Management with modern statistics, computational algorithms, and scalable computing platforms like Windows Azure and the Microsoft Cortana Intelligence Suite. By systematically collecting and curating data with intention, we can use data science to extract insights from data.
The corporate journey towards a data driven culture is, like many journeys, a bumpy and indirect path, filled with pleasant and unpleasant surprises, advances and setbacks, and unknown savings and costs. Intentional design is required in order to change organizational culture to be data driven, to be able to derive and take advantage of insights that the data may yield. It requires a long term strategy that begins with planting the seeds of data driven practices throughout the entire organization and the sustained cultivation of data driven goals, so that one day there will be a bountiful and recurring harvest. Unfortunately, it is all too common that past decisions and infrastructure and process investments need to be revisited with further investment to support a data driven culture, sometimes without a clear understanding of the exact ROI and that the entire organization will need examples and incentives and support to make the changes needed.
For example, a business that offers a data collection service and online portal to access the data had a product policy in which their customers may view and obtain reports for historical data for the prior 24-month period of time. This policy or decision arose from the earliest days of the business when stability, scalability, and performance were business challenges. The product policy became codified as a 24-month data retention organizational goal, and organizational metrics were built around this goal. This tactical retention goal led to the “disappearance” of older, legacy data. This loss of legacy data was due to a variety of reasons, including intentional deletion, changes in storage architecture that did not include a strategy for legacy data, databases that were not replicated, and server failures that were not salvaged. This loss of data led to a significant delay in understanding customer needs, the ability to offer competitive new features and products, and the ability to improve efficiencies in production.
For example, a business organization with a well-developed work flow had invested heavily in computerized systems to support the work flow. However, there were exceptions, errors, and escalations that were managed by a manual process. This policy arose at the time the computerized system was developed, where the features needed to support exception handling did not meet the bar for funding. While the process was effective in handling exceptions, to systematically improve the process though scientific principles, rather than an anecdotal and ad hoc process, the process itself, including the types of errors and exceptions, the time to resolution and all factors that may contribute to the errors and exceptions, needs to be systematically captured through data. The needed data was never collected, perhaps due to a functional spec error or an intentional omission because as nice it might have been to have the data, the cost and effort to do this did not make the cut and the work was placed indefinitely in the backlog of work items. This loss of data led to a significant delay in understanding and optimizing the work flow, in understanding customer needs, and the ability to improve efficiencies.
A data driven culture must incorporate the data science perspective into both the tactical and strategic goals and metrics of an organization. It is never too late to make change, and it is easier now to do so than it has ever been in the past. Change starts now by working with data scientists in the intentional design of organizational goals and cultivation of data driven practices, so that we can quickly derive the insights from data that are needed to drive forward business goals and enable nimble and competitive decision making.
- Taylor, Frederick Winslow. The Principles of Scientific Management, New York, NY, USA and London, UK: Harper & Brothers, (1911),
- Gilbreth, Frank Bunker. Primer of Scientific Management. D. Van Nostrand Company, 1912.
