The Beginning (4)

In my experience, those who can do the least about data quality feel most responsible, while those who have quality literally at their fingertips feel no responsibility whatsoever. This creates many difficulties for one who wants business intelligence.

I'm going to collect receptionists, customer support, sales, anybody who enters data into any of your computer information systems under the heading of "the business." We could also include those who collect or transcribe data to/from paper if it eventually winds up in a computer-accessible data file.

Maybe we have done too good a job in convincing the people seated at computer keyboards that they have nothing to fear. In any event, many simply do not pay adequate attention to what they are doing. There are many reasons for this including:

• heavy work volumes causing a pace that is too fast for error recognition or for going back to correct errors
• inadequate training
• inadequate guidance built into the user interface
• inattention/distraction

On the other hand, I have encountered way too many instances in which people were actually aware that they were producing garbage but didn't care enough to do anything about it. Sometimes it's sabotaging the people in the next department. Sometimes it's a statement to the supervisor and sometimes it's "so what."

It is true that some data can be repaired after the fact, but the sad fact is that "cleansing" can never be 100% effective and in some applications cleansing isn't even possible--if it isn't captured correctly the first time, there's no going back. Repairing bad data is very expensive for your business. Experts estimate that, for anyone who receives data as part of their work process, from 30-60% of their work day is spent in rechecking or validating or repairing the data so that they can do their job.

The data architect will have worked with the folks to understand their data needs so if we can prove that they have participated in that process and that the architects and developers have complied with their defined processes, then what we are left with is attitude or training as problem sources. These are issues for management to resolve.

One final area for the business to think about: there is a need to include problem recognition in training and provide safe reporting paths for those who do take note and take action. Those at the front line typically have little awareness of the business that they front-end for. They don't know that someone cares or that someone can actually do something to fix a problem that they struggle with on a daily basis.

You will want to include a module on the data resource in the New Employee Orientation. You'll want to take remediation to department meetings to catch all who missed the new employee offering. It must be simple to report a problem and there can be no "grilling" of callers by the support line triage staff. Take the call and send someone to see the problem in person while the reporting employee works. Use a remote desktop capability to watch the problem happen. There are many options, but if the caller is made somehow to feel guilty or foolish or ignorant, they will never call again.

Next time we'll take on the vendors.

The Beginning (2)

Because we want our intelligence at our fingertips, easily accessible, we collect and store data in computer-based filing systems. In return for the convenience of this and not having to store mountains of virtually useless paper (the bigger the pile, the less useful), we have taken on the responsibility of hiring and working with various kinds of technology-savvy people.

At the foundation is the programmer, also called a developer or a software engineer. These are the people who actually create the scripts for the computer to execute. 20 years ago, programmers were visible to the rest of the organization. Today they are typically segregated and largely invisible except to the CIO.

These are the people who create the screen forms and buttons and functionality that your front line employees actually use to capture and look up information. In larger organizations today, the programmers do not create the filing system for data. Instead, that is designed and built by someone else (the data architect) and the programmer merely connects to it. The power that the programmer holds in terms of our four characteristics of good BI is the power to knowingly or unknowingly subvert the quality of our data.

This is a good time to introduce the concept of the data resource. It is essential that today's business view the data that is captured, modified, stored, retrieved and archived as a resource in the same way that capital is a resource or buildings and property is a resource or employees are a resource. The business must devote the same kind of attention to the data resource as it does to the financial resource of the company. Neglect or failure to do so will render the data resource valueless at best and a liability at worst. In between those two extremes, the business will experience increased costs as your workers struggle to get the quality they need from the data that feeds the processes they work within.

So the programmers who build screens and functionality that allow corruption into the data resource are slowly destroying the business just like termites in the framing of your house. It is vital, in order to end up with the BI we need, that programmers employ processes that are controlled for quality purposes. Employing process standards for programming will not guarantee that our four characteristics will be delivered, but NOT doing so will guarantee that we will never be able to produce the BI that we need.

Before we wrap up this installment, it's good to recognize that there are two kinds of programmers; those who work for you and those who work for someone else from whom you purchase or license the programs (or software or applications or systems or...). No business today is unaffected by programming.

When you are getting control of your processes and instituting quality assurance, you will want to ask the same questions of your software or application vendors. Today, they are often unwilling to talk about this and will use deflectors like "proprietary" to avoid the questions. You need to ask yourself whether you can bet your data resource on "proprietary."

We'll get into this a little deeper next time when we discuss the role of the data architect.