1.5 Awareness Levels

Almost everyone is aware that data from time to time causes a visible problem. However, visibility to the magnitude of the problems and to the impact on the corporation is generally low. There are several reasons for this.

Correction activities, rework, order reprocessing, handling returns, and dealing with customer complaints are all considered a normal part of corporate life. Many of the problems are not associated with information quality, even when that is the problem. The activities tend to grow in size with little fanfare or visibility. Since the people who carry out these activities are generally not isolated within a function, the cost and scope of such problems are generally not appreciated.

When decision makers reject IT data because "they just know it can't be right," they generally do not rush into the CEO's office and demand that the data coming from IT be improved. They usually just depend on their previous methods for making decisions and do not use the information from the databases. Many times data warehouse and decision support systems get built and then become not used for this reason. To make matters worse, decision makers sometimes generate alternative data collection and storage minisystems to use instead of the mainline databases. These often tend to be as bad or worse in quality than the systems they reject.

IT management often does not want to raise a red flag regarding quality, since they know that they will get blamed for it. Their systems are collecting, storing, and disseminating information efficiently, and they are content with not surfacing the fact that the quality of the data flowing through these systems is bad.

Corporate management wants to believe that their IT departments are top notch and that their systems are first rate. They do not want to expose to their board or to the outside world the facts of inefficiencies or lost opportunities caused by inaccurate data.

If a company included in its annual report a statement that their information quality caused a loss equal to 20% of their operating profit, their stock price would plunge overnight. They do not want this information published, they do not want investors to know, and they do not want their competitors to know. The obvious psychology drives them to not want to know (or believe) it themselves.

Companies tend to hide news about information quality problems. You will never see a company voluntarily agree to a magazine article on how they discovered huge data quality problems and invested millions of dollars to fix them. Even though this is a great story for the corporation, and the results may save them many times the money they spent, the story makes them look like they lost control and were just getting back to where they should have been. It smacks of saying that they have been bad executives and managers and had to spend money to correct their inefficient ways.

I had a conversation with a government agency official in which they indicated that disclosure of data accuracy problems in a particular database would generate a political scandal of considerable proportions, even though the root cause of the quality problems had nothing to do with any of the elected officials. Needless to say, they went about fixing the problem as best they could, with no publicity at all about the project or their findings.

Data quality (and more specifically, data accuracy) problems can have liability consequences. As we move more into the Internet age, in which your company's data is used by other corporations to make decisions about purchasing and selling, costs associated with bad data will eventually be the target of litigation. Corporations surely do not want to trumpet any knowledge they have of quality problems in their databases and give ammunition to the legal staff of others.

The time to brag about spending large budgets to get and maintain highly accurate data and highly accurate information products has not yet arrived. However, the tide is turning on awareness. If you go into almost any IT organization, the data management specialists will all tell you that there are considerable problems with the accuracy of data. The business analysts will tell you that they have problems with data and information quality. As you move up the management chain, the willingness to assert the problems diminishes, usually ending with the executive level denying any quality problems at all. Figure 1.3 summarizes reasons for lack of initiative in regard to problems with information quality.