Business Basics: Unscrubbed Data Is Poisonous Data

  • Written By:
  • Published:


A manufacturing company experienced continually increasing overhead costs in its purchasing and materials departments. It also began to see increased inventory levels and production line delays. A consulting firm was brought in to analyze the company's materials management processes and the level of competence of its materials and procurement staff.

After mapping the entire process, from the creation of a bill of materials through post-sales service and repair, the consultants determined that the major process weakness was in the purchasing system, and more precisely, in the lack of an expediting system. In response, a more streamlined process was designed and training programs were recommended to improve the skills of all materials and procurement staff.

Lacking current knowledge of state-of-the-art manufacturing systems, the company engaged another consulting firm to guide the company through the process of selecting an ERP system that would address the apparent problem and to provide a platform for business growth. After only a few weeks of study, a short list of potential suppliers was isolated and provided with a request for proposal, then the selection consultant turned to project risk analysis.

Note: This note first appeared in a column by James F. Dowling in Mid-Range Computing. Look for other previously published Mid-Range Computing columns by Mr. Dowling at this site or visit Midrange Showcase at

Project Risk Analysis

Most business software system changes falter--if not fail--because of only a few root causes, among which poor data quality is high on the list. In this case, not only was data quality seen to be a troublesome issue for implementation of a new system, it was apparent that it was at the root of the material management problems that had been plaguing the company for more than a year.

There were several telltale operational symptoms: 150 purchase orders and 115 change orders per month; piles of packages in the receiving department prior to their shipping date; 12 percent of the manufacturing staff chasing parts; 75 percent of each shipment meeting spent addressing parts availability; and piles of parts in each work cell marked with customer names on them.

A focus group discussion regarding materials scheduling revealed the crucial data quality issues:

  • Purchase lead times were missing or not maintained
  • Bills of material did not reflect the true assembly of the product
  • Shop floor routings had missing or erroneous lead times
  • 30 percent of items scheduled as independent demand and bill of material quantities were set to zero

The existing management resource planning (MRP) system was capable of scheduling and tracking parts as long as it had been provided with correct data. However, organization changes to the company now had bills of materials managed by the engineering department; the item master files were now managed by Inventory Control; shop floor routings were now managed in Manufacturing Operations; and no one was responsible for anything except getting the goods out the door.

As a result, problems abounded. The purchasing department ordered everything on the bill of materials as soon as a sales order was received. MRP netting was totally ignored. Manufacturing teams hoarded parts and expedited just about everything even before it was needed. Salespeople pulled material through the receiving and inspection departments. Everyone met twice a week, stacks of paper in hand, and provided each other with only as much information as requested.

Automating the Problem

An information system did not cause this problem, but it did automate it and make it felt by everyone. When business rules and algorithms within the MRP system operated on inaccurate data, completely accurate and totally inappropriate actions were suggested. Eventually, everyone came to know this and developed workaround processes and compensating behaviors in order to complete customer orders.

Project risk was significant, and immediate actions were suggested to correct the data before attempting a migration. There were two choices: Correct the data in the current system and then migrate, or correct the data in the new system before bringing it online. Either action would require considerable work and demand changes to business processes and behaviors to ensure ongoing manufacturing productivity.

Data Quality Control

Every business application should be periodically assessed for data quality. It should be a quarterly effort to assure reliable processing and to maintain the value of business decisions made from data. Software tools such as Carleton's Passport, Prism Software Solutions' Warehouse Manager, QDB Solutions' Analyze, and Vality Technology's Integrity can be purchased, or homemade tools can be built to examine data fields and relationships.

Data should be managed as a corporate asset that appreciates in value over time. Historical data must be addressed with as much care as current database content. To accurately reflect trends, history files and archives should be considered whenever data validity and business rules are adjusted. Consideration should also be given to bringing history files in sync with current edit rules. By doing so, analytical processing on such data will be enhanced.

IT and business managers are jointly responsible and ultimately accountable for data quality. Every IT manager must provide the tools necessary both to ensure that only high-quality data can get into application systems and to identify exceptional conditions that might creep into their systems. By doing so, business managers can be held accountable for data cleansing and quality management. The cost of high data quality is low, and the short- and long-term benefits are great.

comments powered by Disqus