How Insight Optimization Helps Data Scientists | HCL Blogs

Insight optimization – how data science is changing the way information is utilized

Insight optimization – how data science is changing the way information is utilized
January 17, 2017

A number of concerns have emerged regarding the traditional BI environment, as voiced by stakeholders:

  • Lack of data integrity, including quality and governance
  • Absence of data integration, as silos are available but an enterprise view is restricted
  • Delays in availability of data (time to market)
  • Absence of all data, as only data from reporting requirements are captured
  • Inability to mix and match with external data
  • Extensive time to market from ideation to production

As data obtained from source applications is uploaded to the data lake in its entirety, business users are eager to fully explore the potential of raw data.

However, it’s essential to impart a certain level of training for tools like R/SAS/SPSS/Infinite Insights and/or statistical background, possibly unavailable to business users.

Consequently, data science is now an important area of study and research. Data science is now being incorporated even into business transformation teams/plans.

Data scientists strive to deliver insights on the available data. In order to leverage the insights thus obtained, it is essential to simplify the insight-delivery framework:

  1. Data scientists usually associate themselves with “predictive” work/models
  2. Business process comprehension for data scientists is somewhat akin to business users through association and knowledge sharing
  3. Data scientists connect source application data to a function or sub-function and then evaluate the insights obtained
  4. These insights could impact a use case positively or negatively. If negative, the organization will rescind that use case and vice-versa
  5. Data scientists check data quality as they profile it for quality

When the business transformation team synergizes workflows with data scientists, insights gained can be shared with stakeholders, ensuring a more efficient adoption of use cases and thereby insight optimization.