The quality of any data is affected by its entry, storage and management. The process of data quality assurance (DQA) verifies the effectiveness and reliability of data. Managing data quality requires frequent digging into the data and scrubbing it. The process involves the update, standardization, and de-duplication of records creating a singular view of all the data, whether or not it is stored in disparate multiple systems. Data quality aspects include: completeness, accuracy, relevance, update status, reliability, consistency, accessibility, and appropriate presentation.
Data governance is the continuous process of data quality improvement and is embraced at all organizational levels of data quality management. The governance process requires filtering unnecessary information through the definition and enforcement of approvals and policies.
Phases in the Data Quality Management Life Cycle
- 1.Quality Assessment: The data quality manager determines the data source quality. The first step involves loading the source data, which may be stored in diverse sources. Data and metadata can be imported from sources such as Oracle. Once source data is loaded, data profiling is done to assess its quality. Profiling data uncovers anomalies, redundancies, and inconsistencies. The data discovery and analysis techniques are the basis for monitoring data.
- 2.Quality Design: Processes design is an essential stage of the lifecycle. The legal data is specified within a legal relationship or data object based on data rules. Data is also corrected and augmented using quality operators. Transformations that ensure quality of the data are also designed at this phase based on mappings that the data quality manager creates.
- 3.Quality Transformation: This phase involves correction mappings that are used in correcting the source data.
- 4.Quality Monitoring: The data monitoring is the data quality management phase that includes frequent examining of data over time. Data monitoring alerts the manager when a violation of any set business rules occurs.
Data Quality Manager Firewall
Data forms a strategic asset for organizations and should be handled as such. Similar to other organizational assets, the data that is stored in the information systems of an organization has financial value. Entering data that is inaccurate into the mastering systems or data warehouse makes data quality management difficult. This is because it not only damages viable data but complicates access to precise business insights and information.
A virtual firewall can detect and block harmful data at the entry point. The firewall acts proactively at preventing bad data from contaminating information sources of the enterprise. A comprehensive solution for managing data quality is building a data quality firewall that dynamically identifies corrupt and invalid data. It detects the data once it is generated or while it streams from external sources. The firewall should be founded on business rules that are previously defined by the enterprise. Firewalls will ensure continuous data quality improvement and integrity of information.