Data Profiling - A Quick Primer on the What and the Why of Data Integration
Databases in most companies have evolved in an ad-hoc manner, which has resulted in information silos. Companies, therefore, do not have a unified view of their customers, resulting in missed business opportunities or increased cost of operations. Data integration addresses those issues, but poses data verification challenges, since the source data are in diverse databases. Most data integration and migration projects overshoot their time and cost estimates because of the effort expended to understand the source data. Data profiling automates the identification of problematic source data, inconsistencies, redundancies, and inaccuracies. Data profiling also provides a factual foundation, based on which data can be cleansed and then consolidated before integration.
Excerpts from the Paper
A company’s database contains information that touches most aspects of its business activity market data, customer information, accounting information, production details, sales records, billing details, collection details, personnel records, salary records, and so on. This data is utilized by the company for various business decisions, and it is therefore imperative that the data in the database be consistent, accurate and reliable. Since the costs of poor data quality are high, increasingly companies are “profiling data” to check its quality and suitability for business. Data profiling uses “analytical techniques to discover the true content, structure, and quality of data.”