Data Analytics – Self Service BI Journey | HCL Blogs

Data Analytics – Self Service BI Journey

Data Analytics – Self Service BI Journey
November 28, 2018

Utilities have Big data and it is a huge thing in utilities to analyze those data sets and patterns to really understand there customer behavior pattern, changes in business due to weather, enhance their customer satisfaction ratios, enhance energy production/efficiency reliability. The data is the key information set for analyzing patterns/estimates and analytics is all about discovery, interpretation, and communication of important and meaningful patterns in data.

BI platform needs tool sets which are easy to deploy, incur less maintenance cost, no or minimal IT requirements and less user training to scale up the platform.

The fundamental difference in “analysis” and “analytics” per definition is that “analysis” is focused on understanding the past; as what happened vs. analytics which focus on why it happened and what will happen next, based on data patterns and predictive analysis

This was always a need in utilities as they move toward the journey of analytics. Business intelligence (BI), as the name suggests, gives critical insights around the data and enables the intelligence in the organization data which empower decision-makers for efficient and in-time decision-making. A “self-service” BI will help organizations deliver cost-effective ways to deploy decision support system which is presented in a manner easily accessible across the globe, in different locations or departments, presented in efficient and with less overhead. The key benefits of having an enabled BI user group for utilities, where the data volumes are significantly more and the data needs keep changing where we cannot have IT-built stories and answers using standard IT setup of built/test/deploy. These are ad hoc needs which are sometimes not worth just an answer by and executive responding back to some audit/hearing from a regulatory body. Hence, enabling a self-service BI platform will help businesses perform their own deep-dive study and pattern analysis by empowering them with tools and data elements at their disposal. This also eliminates IT intervention and manually arranging the data sets for this odd request, which is time-consuming for IT processes. Self-service BI system allows the user to get the right numbers, to identify the trends, and then drill down into the details to get the root causes of problems or to isolate specific opportunities. It also shifts your culture from being reactive to proactive as data is readily available.

Enabling the Self-Service BI Journey for Utilities:

The technologies/toolsets to understand the data and report are changing rapidly with time. BI users are deep diving into the data according to their needs and analyzing and preparing without knowing too much with data exploration. The correct decision is enabling a self-service BI platform and is the fundamental foundation stone needed for enabling the culture. The key decision of empowering a self-service BI setup is really helping identify your users/analyst requirements and categorizing the needs in various segments to understand and enable a platform which can cater to the data requirements. Utilities have Big Data sets which need exploration and analytics to enable a decision-making. For example:

  • Rate case studies – Need to massage huge billing data sets to understand the rate pattern on-peak and off-peak analysis to understand customer behavior.
  • Demand analysis/production – The key for utilities is to understand their customer usage patterns/needs to cater and meet there production.
  • Theft/Fraud detection – Understanding customer patterns to understand frauds and prevent losses especially in industrial customer sets.
  • Aging asset analysis – A classic example to reduce the loss in transmission if the transformers are old, they will have low efficiency. By replacing these old assets, we save energy.

The typical data democratization model should like below:


The right decision-making is only possible if we have the right platform infrastructure which enables this self-service BI exploration journey. The following are the key factors which outline various fundamental analysis needed to set up a truly performing self-service BI system:

Understand the users – It sounds simple but this is very critical set in determining a truly working self-service BI system, without knowing how many powers users, analysts, data consumers, data explorers, data scientist, and statisticians are there in the organization, we cannot scale up and prepare a fully efficient self-service BI system for users. It is important to classify them and have their count to scale up/size a platform which caters to the needs.

Understand there usage patterns: Each user type uses data in various capacity/flavor, for instance, a regulatory commission audit response may demand an analyst to understand the estimation accuracy of the utilities and a data scientist is exploring patterns of theft cases by finding patterns. Hence, in order to determine a self-service BI platform, it is important to understand the usage patterns to determine the correct tools for exploration and details around the data sets.

Focus on data governance strategies for effective self-service BI: Without having a data governance strategy in place and a metadata repository, the benefit of having a self-service BI is defeated as a customer can be called Business Partner in the CRM system, but can be a recipient in the order tracking system. So, defining data governance and enabling a common data dictionary definition prevent you from data swamp.

Based on the responses and analysis of the above key decision drivers to enable a self-service BI platform, the next step toward enablement is setting up the right platform. Most of the self-service BI platform needs toolsets which are easy to deploy, less maintenance cost, no or minimal IT requirements and less user training to scale up the platform.

The self-service journey starts with the following steps:

  1. Acquire data
  2. Data preparation
  3. Visualizations – features
  4. Exploration
  5. How to share and collaborate.

Case study: Tools sets used for one of the major utilities in North America and their journey to self-service BI

The following is a case study enabling their end users to the journey of self-service BI

The architecture of the self-service platform constitutes of below elements/tools sets.

Acquire data:

This utilies uses various tools like SAP Businss objects data services, TIBCO, ESB integatrion

Data preparation:

This is done with tool like SAP Native HANA model, Stored procedure in MemSQL(In memory cost effective database) , SAP Business Objects Universe layer


This is done with Tableau, R studio, SAP Business Objects


Accomplished by R, Python, SAP Business objects explorer

Share and Collaborate:

SharePoint, Tableau Workbooks, data dashboards, reports


With various tool sets and enabled self-service BI, the pattern seen is to enable the user with a free hand to drive the requirement and enable them to get the answers they need and this can be accomplished by providing them with a data lake where they are empowered to jump in as and when data is needed with their choice of toolset from the self-service enable BI toolkits to explore.