Survival, Error & Technology

I'm going to pass on some wisdom here. It's not very often that we encounter wisdom today, especially where technology is concerned, and it's often the case that we don't recognize or acknowledge wisdom until we're looking back over the wreckage and trying to figure out what we should have done. I'm probably also setting myself up by labeling this as wisdom but I am so weary of seeing the same ads with different acronyms and talking to the same people with different names.

You will never find your way out of the current mess you're in or about to be in by searching for and hiring someone with recent experience on a specific product. To put it another way, a specific product, no matter how much buzz it enjoys, is never the answer.

I will be among the first to acknowledge that the use of absolute language (never, always...) and even the use of unqualified superlatives (best, worst, fastest...) is a habit to be avoided, nevertheless, decades of experience have proven that the absolute statements in the preceding paragraph represent wisdom and that failure to heed this wisdom will produce cost overruns, timeline disasters, confusion, stress, employee turnover and a host of other undesirable outcomes.

In large part, the success of the human race has been due to our ability to recognize exceptions without necessarily understanding the rule. My own take on this is that, with today's reliance on technology, we may have reached the point where the process of natural selection that has honed this skill over countless generations has now produced a liability. "Something's different," is enough to put us on guard and may be enough to launch a complex defensive reaction to preserve the safety of the individual or group.

First of all, while it is still good to recognize exceptions, it is now absolutely essential (that's an absolute absolute) that we develop the ability to recognize the underlying rule. A study of human error (Human Error, Set Phasers on Stun...) shows that leaping to conclusions about the rule is what produces the error condition. In fact, if we can't describe the rule in terms of the logic of the computer (if... then... else...), we can't rely on technology at all.

You might ask, as I did, how we might acquire this ability. The time tested way is known as [survivable] experience. There are a host of cause-and-effect analysis tools and techniques that have the appearance of rigor and reliability and are an improvement over experience, especially when combined with exhaustive testing, but you will find that even these are more productive when used by people with experience in the world being analyzed.

Tools are great and another critical human enabler, but--and this can't be over-emphasized--no tool is so advanced that it runs itself. Every tool, no matter how advanced the technology requires human hands and a human mind to guide it. If you were to be presented with the greatest woodworking tool in the world or the most advanced sewing machine or fishing gear or computer, would you immediately become a master cabinetmaker or designer or fisherman or software developer? You might note that the only immediate change is one of expectation.

An experienced person with rudimentary tools is more likely to produce a quality result than the inexperienced person using the "best" tools. The fact that I used a tool just yesterday says nothing whatsoever about the level of my experience in producing the required outcome. I have made the mistake of looking for help and focusing too narrowly on what amounts to recent experience with the tools in my shop. I have learned (through experience) that I will enjoy better results if I'm learning while interviewing my prospective employee. If I'm talking with someone whose knowledge stops at the tool's user interface, then I had better be prepared to devote myself to directing the employee's work. If I have a staff composed of such employees, then I need to possess all of the requisite experience myself or else be prepared to conduct a project whose principle product is more experienced workers.

The challenge is to find the right mix of experienced people in supervisory or team lead roles and people who possess dexterity but are in need of experience. If I'm in a director or management role, I have to have experience producing a product with that scope. A technology "system" has a complexity that is beyond human comprehension. The only way to design and build it is through a process of identifying smaller and simpler pieces, building those and then assembling them into the final product. You need to look for people who have an appreciation for the amount of effort this takes and the discipline--both personal and organizational--that it takes.

Stop looking for Oracle or CRM or Rational or even "use case" or "data model" experience except as clues about the approach that the candidate might be expected to take. I understand that these things are ideal as targets for a logic rule processor, but the rule ("find resumes that include these terms") is so simple-minded as to be useless. If your only goal is to turn 1000 resumes into 100, then proceed, but if your goal is to find someone who can get you out of the predicament you're in, then you should spend more time on your rules so that the exceptions are more productive.

Business, Information and Technology

Are you in management? Do you have annual/quarterly goals? Will you be held accountable for achieving those goals? Is the accountability expressed in bonus dollars? Is there any possibility of a zero bonus?

Are you still with me?

How will the achievement of your goal(s) be measured? Please note here that "how" has two dimensions: one is related to process and the other to a unit of measure. In all of my vast personal experience, all of the attention has been focused on the unit of measure part (when there has been any attention at all), and the process part has never even been part of the conversation.

Please understand that what follows is not intended to sling mud at any individual or organization. My purpose is to clear the air so that we can talk about how we're really going to achieve our goal(s).

I am going to generalize based on extensive, though anecdotal, experience. In other words, I have not conducted a survey, scientific or otherwise, and cannot produce data to back up anything I'm about to say here, so I'm leaving it up to you, the reader to determine whether it feels like truth or not. Should you feel that this does not ring true, or should you wish to fault me for not being more objective, I would ask that you produce a study or at least a body of experience in support of your position.

Awards of bonus dollars tied to achievement are based solely on whether the holder of the dollars wishes to give them away or not. There is rarely, if ever, any protocol defined for defining metrics, units of measure or measurement process. You will go into an "annual assessment" meeting with your boss and he or she may discuss your level of achievement in very general terms before announcing the amount of your bonus or a recommendation for an increased level of compensation.

Why is this important, you ask? Well, it is important because it's the the way things are done. Despite vigorous protests to the contrary, the business world is set up to run on subjective assessment supporting subjective decisions. What, you say that your decisions are "data-driven" (objective)? I would love to hear the story behind the data that was used to arrive at your most recent decision.

All of this is background for understanding why "data" initiatives so frequently become mired in a swamp of politics and personality. Let's walk back from a data-driven decision.

1. You are able to make the decision because you trust the data.
2. You trust the data because you are familiar with and trust its source.
3. You trust the source because you know that it is reliable.
4. You know that it is reliable because it consistently produces information that can be relied upon.
5. The source has been consistent because it always uses a tried and true methodology (set of processes) to produce its product.
6. The consistency is possible because the methodology includes steps designed to validate the source's inputs.
7. The validation decision returns us to #1.

How do you feel about standards (e.g., standard operating procedure or SOP)? If you don't currently support the creation, implementation auditable use of standards or, at any time in the past have not done so, you have no right to and almost certainly do not have access to reliable information and therefore no claim to data-driven decisions. Just to drive the point home, when your boss decides that you won't be getting that bonus or increase you were counting on, your only acceptable response is to smile and say thank you.

By the way, if you notice that your bank account (or budget) is suddenly much bigger (or smaller) than it was yesterday, what is your responsibility? Who are you accountable to? How much trust can you afford? Now you have some insight into compliance.

The use of technology introduces an additional huge portion of uncertainty into the trust equation. Take another look at the decision walk-back above and note the points where the use of technology means adding additional paths and complexity to the validation process. This is what your data governance people are trying to get their arms around.

To summarize:

• data-driven or intelligence-driven decisions demand trust
• trust demands reliability
• reliability demands consistency
• consistency demands compliance
• compliance demands governance

OK, you can go back to work now.

What Is "Data" Anyway?

If you have any experience with phone support (on either end) you will recognize how easy it is to get deep into a process before realizing that the other person is on a completely different path than you are. RTFM (Read The F---ing Manual) often pops up as the answer to our communication difficulties, but it clearly is not the answer or it would have been universally embraced by now.

I happen to be a proponent of the theory that the answers aren't nearly as important as the questions. As a teacher, I know that learning is happening when the pupil is asking questions--particularly a series of related questions. This has become very important as I have attempted to make headway on [data] governance and [data] quality.

My early attempts assumed that everyone knows what data is--and they do, in the same way that a picture is worth 10,000 words. Each and every person you talk to knows what "data" is and each has a different idea in mind. For most, it's a picture of the last set of values they looked at. This might have been a spreadsheet, a graph, a collection of measurements... The key is that data is a set of values. For some, "data" is a commodity. It is files, stripes on a disk, a percent of capacity, a quantity of bytes measured in "mega-", "tera-" or "peta-". For still others, "data" is represented by a schema, model, definition, or some other abstraction.

Given those varying perceptions or perspectives, is it any wonder that at some point in the quest for "data" anything we find ourselves stuck in the quicksand of confusion. Even when all parties have been saying the same things and have been agreeing on goals, there comes a point when someone will say, "We're not going to do that." or "I don't see why that is necessary." or "But that will change my work flow." This is frequently the point at which everything starts to unravel.

So what I have learned, and what I offer to you now, is that the initial phase of any data initiative must be a carefully constructed education process to insure that the quicksand moment never happens. This must be thought of as risk management. Remember, too, that the really important questions (and answers) initially are not the ones you're hearing. All of the different constituencies are going to be much more comfortable exchanging information (or misinformation) within the tribal group than with "outsiders."

The best way to head off this risk is to carefully choose allies from each constituent tribe and use informal conversation about their pain points and the ways that data figures in the relief of that pain. You will be setting these people up to be the "experts" within their respective tribes. Part of this will be coaching them in how to respond to questions and discussion in which they don't feel themselves to be on firm ground. They need ways to postpone a response until they've had a chance to confer with other experts. This is easy to do by setting up a collaborative model.

Rule one of this model is that I never answer for someone else. Everyone can understand that a situation involves yet another perspective and that it is necessary to involve someone from that tribe in order to get a complete answer. The most common danger here is "We don't have time for that." Everyone must understand that this is an absolute red flag event. It signals that we still have not achieved a universal understanding of objective.

When a red flag event happens, it isn't the same as finding ourselves up to our necks in quicksand. It just means that we need to engage in some risk mitigation. It's a sign saying "Quicksand ahead." We will need to bring this person into the fold--usually through informal and non-threatening discussion with at least one peer or trusted expert.

Your role, should you choose to accept it, is to be a non-judgmental, constant, committed, and helpful presence that can be relied upon to be a neutral mediator and facilitator who feels like a friend in any need. Your motives must be above reproach. You cannot count on and should not hope for recognition. All around you will be better off for your presence.

If you are senior in the organization to this person, you should make sure that you are appreciating their contribution but they will appreciate non-public affirmation since putting them in a spotlight may negatively impact their ability to continue to function in the same way.

"What is data anyway?" is a question that requires many answers initially and one answer eventually. Remember, though, that many people are really only interested in what they have to do differently. "Data" may have no meaning whatsoever in their day-to-day responsibilities even though they may be monitoring real-time run charts with instructions to take a specified action when the line goes above or below a certain point. You can't possibly know where to start or where to stop in defining data for them. That's why you need the tribal expert.

Don't seek "important." "Helpful" will take you much farther more quickly.

Disturbances in the Force

So, if we know how to make things better in terms of data quality and we're motivated to do so, what's stopping us? A word of caution; what you're about to read may be harmful to your health.

Maybe you're old enough to have lived through the Watergate fiasco and can remember the facts coming to light one by one in the press until they eventually began to make a complete picture. Maybe you remember the Hollywood version, All The President's Men, in which the whole picture was produced in two+ hours rather than months, or maybe the whole thing is in the same category as the Crimean War for you and is nothing more than a question on a pop quiz in one of your least favorite subjects.

I'd like to suggest for your consideration that if we want to track down why we are having such a difficult time accomplishing something that we all claim to want, we need look no further than the paragraph above to get all the answers we need.

First, let's imagine that data quality is like truth in government. It's a good thing and we would like to assume that we have it. If, in fact we do not have truth in government (or data quality), who benefits? The answer is that it is in the interests of those who believe they can/will be blamed for the status quo to cover up the problems and subvert efforts to get at the facts that can provide the complete picture. This is especially true if they are responsible for the problems. Even if the only identifiable responsibility is that the person is the supervisor or manager of the function(s) that owns the troubled processes, they still may elect to resist and subvert in order to avoid becoming responsible for the fix.

If we want to avoid this situation, we should absolutely avoid any questions that seem directed at why or who or even how. We should avoid to the extent possible, any investigation into the past. Try to keep all discussions focused on process-based causes that might be producing the effects you are seeing. Do not zoom in on isolated instances but look for trends. Remember, your goal is not prosecution but consistent quality.

In the words of Bob Woodward's source, Deep Throat, "Follow the money." The programmer-champion will struggle against this repeatedly. There is a perception that implementing integrity checking at the point of input represents added cost. Like any other complex process, system development should seek to minimize total cost of ownership rather than any single cost line item. If it takes an extra day of programmer time to ensure that we get 99.99% consistency of integrity in the database and thereby avoid dedicating multiple full-time staff to data clean up, this is a net cost reduction.

Our system design and project management processes may not be mature enough to assign dollar values to this, but it should be easy to determine how much money we are spending on fixing poor quality data every month (or year) and then amending the design and development processes to devote a fraction of that amount to prevention.

The final perspective to be extracted from our example is that a short attention span provides little hope of even recognizing that a problem exists let alone understanding it enough to develop a mitigation strategy. Data quality (and truth in government) requires that everyone be involved. People are capable of recognizing self interest within the corporate interest and enough people will be motivated to act that the ball will be kept moving, but the media of the late 1960's is not the media of 2009. In the 60's there was an interest in the truth that perhaps doesn't exist today. In your corporate environment, you may find it easier to maintain a constant pressure of communication directed at a single theme. The widespread motivation will not be produced by a single appeal surrounded by banners and fanfare and free cake. A communications campaign must be designed for the long haul with continuous refreshing of the message.

You don't even have one percent of your employee workforce today who are ready to grapple with the issue of data quality. You are going to have to break it down in multiple variations and start with the concept of data itself. What is data? You'll be surprised at what you uncover when you go out to talk to people about their data. Stay tuned for some samples.

Programmer as Data Quality Champion

The programmer is the one who takes all the wish lists and turns them into something that a programmable logic device (computer) can execute to fulfill the wishes. Today, several additional roles have attached themselves to this process. The architects, designers, modelers, testers... all play an important part in the final product but it is important to remember that these roles have motivations other than the ability of the product to satisfy wishes. At best they satisfy a different set of wishes that have more to do with the process than the product.


In the not so long ago days when I started in the information systems industry, none of those other roles even existed. It was all about programming.

In talking with a programmer, you will detect a hint of pride and superiority based on the sure knowledge that none of "them" could produce a program that ran without error and produced a useful result. Other than the "end user" or simply "user", there may be no one lower on the respect totem pole than the "data" people. The programmer only needs to know what you want it to "do"; data is just something that you move from one place to another.


In those bygone days before information technology there were organizations known as data processing. I'm leaving out broad segments of programming known as systems programming because at the operating system level, the data really is a commodity consisting of groups of on/off bits known as bytes. In the very act of ignoring this segment of programming we stumble over the origins of our problem. In early computer systems, there really was no data as we think of data today.


A programmer could grant wishes by making a process that took large amounts of "data" values from one file, combined them with large amounts of "data" values from other files and deposited the resulting "data" values in a new file from which address labels or paychecks were printed. The programmer's responsibility was simply to make sure the program didn't unexpectedly halt.


At first, they just told the users what the data values had to look like in order to ensure that the program kept running. When the users proved incapable of guaranteeing the necessary consistency, programmers took matters into their own hands and created scrubbing programs that would for example guarantee that a file contained only values that looked like $nnnnnn.nn where the value of n is from the set (0-9). Now everyone was happy until one day a big order came in for $1,250,000.00 and was thrown out as erroneous. At the same time, someone figured out how to divert the fractional round-off amounts into a private account.


I'm leaving out some reasoning steps in an effort to keep this to an essay length. If you get lost, just drop me a note and I'll be happy to fill in any missing pieces.


Eventually it was realized that we don't have to store data in a form recognizable to humans--the computer could be taught to present a data value in any format that a human might care to see. This leap forward allowed programmers to distance themselves even more from the data. The idea to take away from this is that programmers may not have the same concept of data that you do.


When non-programmers talk about data, they are typically talking about instances rather than types. To a non-programmer, "Walgreens" is an example of a piece of data as is "sea foam green" and "$900 billion." To a programmer, these are all character strings or text values and may be of three different subrange "types". The subrange (store, color, gross revenue) determines how the value should be handled and the value may be acceptable if it fits the pattern defined for the type.


Today, there are many opportunities to enforce patterns on data values and most of them require no programming at all. The problem is that they all produce errors and error messages that the typical user could not hope to comprehend. In effect, they cause the program to terminate unexpectedly. So, despite all the advancements in technology, we are still having to scrub data files. The alternative is for the programmer to think like a human instead of like a programmable controller and the problem with this alternative is that it introduces orders of magnitude increases (x10, x100...) in complexity and corresponding increases in development costs.


So how can programmers become champions of data quality? One relatively simple way would be to avoid accepting text values as program input. This tactic is a favorite because it defers many decisions until a later time when "we know more" and the big problem here is that we never have to go back and change it. An example here might be useful. Imagine that you are programming a system that accepts input from nurses who are taking vital signs (temperature, BP, pulse, respiration, height and weight) in a patient exam room. You take the usual shortcut and implement all the fields on the screen as text.


Everybody is happy because the nurses don't ever have to go back and correct anything and the program runs without apparent error. One day, though, a health insurance company decides to reward its contractual clients by paying a slightly higher rate to those who document that they are doing a consistent job of collecting vitals at every patient visit. Now we're asked to verify that we do an acceptable job of collecting and recording vital signs. Since the values input to a screen go directly to a database, we should have no problem. It is, in fact, no problem to count the records for which there is or is not a value in those fields, however, when we attempt to aggregate those values to show the range of values or the average value, our query fails. the aggregation query must convert the text values in the pulse field to integers and the text values in the temperature field to floating point (real) numbers in order to compute an average.


We finally discover that pulse contains some values like "100.4", "98.5", "SAME"... that cause an error because they can't be converted to an integer value. When we look at this as a nurse or physician, we can see that the mind ignores the labels on the screen and simply produces a picture of the patient based on the values displayed. Our poor computer, though, is unable to continue. The database architect could have made pulse an integer type and the DBMS would have enforced that typing by not allowing these values to be stored in the database. Using a text type allows the DBMS to accept any value for storage. The programmer could enforce a text value that is guaranteed to convert to an integer or could enforce integer types directly but in order to do so he or she must handle resulting errors in a way that is understood and accepted by the nurses.


More often, though, the nurse managers show the incorrect data to the nurses and exhort them to pay more attention. Do you believe the nurses will respond better to blame and exhortation or to assistance from the program? Check out W. E. Deming's Red Bead Experiment to get your answer.


The programmer champion will be suspicious of a discrete valued field whose data type is text. A value that may be used in a computation or any other operation where a conversion must be done must be investigated carefully. Any value that may be used as a tag for identifying rolled-up aggregations, such as store name, must get additional attention if we don't want to see quarterly sales for "Walgreens" and "Walgreen's" and "Wlagreens". The time to catch and repair these data quality errors is the very first time they are captured by a computer program. That makes the programmer responsible. Other roles have a duty to identify situations where these problems might arise, but only the programmer is positioned to do anything about it.


I realize this is asking a lot. A programmer is only human and can't be expected to know everything (right?). This suggests another way in which the programmer can become a champion. Since it isn't possible for one person to know everything that must be known (hard though that may be to swallow), the programmer must develop enthusiasm for consultation and collaboration. Every role in your environment was created for a reason and each has its own goals and responsibilities. The programmer is accustomed to the data people coming with requests. The requests are nearly always framed in terms of something that the programmer should do to make the [modeler's, architect's, steward's...] life easier and improve overall quality.

It's easy to understand how this can get old in a hurry. The solution is for the programmer(s) to sit down with these other roles and get everyone's needs on the table. All of the other roles mentioned have a different view of data than you do and here's the thing--their view is much closer to that of the customer/user than yours is. You need each other.

Accept that you are a key member of a team and as such the team can't succeed without your commitment. The flip side is that you will not be able to enjoy the success you dream of without the commitment, skills and knowledge of the rest of the team. Be a Data Quality Champion--it's within your grasp.

Next we'll take a look at some forces that act to keep the team from being all they could be. Stay tuned for Disturbances in the Force.

The Problem With Quality

1. Meeting the spec[ification]
2. Documented adherence to established [process] norms
3. The product's effects are primarily positive (for example, it tastes good and doesn't make me ill)
4. I'll know it when I see it

Multiple choice: which of the above (choose only one) is the definition you want applied to your new car?

Does the answer change if we apply the definition to your morning cup of coffee?

Last question: which definition do you apply to the next example of "business intelligence" that comes before you?

I have two points in mind. First, defining quality is not an exact science even within a specific context. Second, #4 may be the deciding factor regardless of #s 1-3. In the end, the consumer/customer merely has to say, "That's not what I was looking for" to relegate a product to the trash heap. We all know that it does no good to say, "This is what you asked for" (meeting the spec) or "I did it just like you told me" (followed established procedure) or "One won't hurt you" (tastes good and not sick--yet).

So what is quality and especially, what is data quality? How we obtain data quality is completely dependent on the answer to this question.

I'd like to suggest that we divide the question in order to produce at least one useful answer. If we examine data quality from the perspective of a computer and its logic, we can come up with an answer that will allow us to progress. The second perspective is obviously the consumer/customer or the human perspective.

Recently I received an email with what at first glance seemed like an innocuous statement full of typographic and/or spelling errors, but when I actually looked at it, it was a nearly perfect illustration of a principle I have been talking about for years.

Teh hmuan mnid is cpalbae of miankg snese of amslot atynhnig taht ftis smoe bisac ptratnes.

This is the principle that draws the line between computer logic and human "logic". It is also what makes programmers (an outmoded term, I know, but best suited to the point I'm making) so vitally important. There is only one role in the continuum of roles involved in producing an information system product that must bear the full weight of responsibility for the integrity of the data quality at the boundary between computer and human. That role is best thought of as programmer.

Unless you earn your living as a programmer, some alarms may be going off. In fact, if you are a programmer, some alarms should be going off. I earned a degree in computer science and made a living as a programmer. From there I moved into data modeling, data administration and database administration. Now I'm involved in data quality and governance. In all of that time, I have never come into contact with any training, education, book or even a job description that addressed my accountability for preserving data quality at the man/machine interface.

This may be a poor forum for this, but my intention is to change this situation right here. My next few posts will present some background on how a programmer might live up to this responsibility and some of the forces that will need to be fended off in order to make it a reality.

Next: Programmer as Data Quality Champion

Answers and Questions

We are living in a time in which seemingly everything is driven by the need for answers. You may be saying, "So, what?" as you read this but the impression I have developed as I have moved through this time is that in many, if not most instances, this amounts to nothing more than "trivial pursuits." A quote found online recently sums it up perfectly:
"We are compelled to drive toward total knowledge, right down to the levels of the neuron and the gene. When we have progressed enough to explain ourselves in these mechanistic terms...the result might be hard to accept." [Edward O. Wilson]

Unfortunately, though having answers readily available can get you around the game board ahead of everyone else, it can't necessarily produce the big win for a cooperative venture. I don't know what research exists on this subject, if any, but it seems to me that living in "the age of information" is vastly overrated. If you think it is good to have a library of information at your fingertips then you may want to think about some desireable properties of information, for example:

• Is it true?
• Is it accurate?
• Is it current?
• Is it useful?

A relatively recent recent development has been the creation of a new area of specialization around data quality. This is not a technical post, though, so I leave it up to the reader to go as deep as necessary. The point I would like to make is that the useful property trumps all the others.

The publicity that surrounded recent preseidential elections in the U.S. shows us that "information" need not be true or accurate or current in order to be useful. If nothing else, this should cause us to pause and consider our thirst for information.

The bottom line is that a command of facts is useless unless those facts enable us to accomplish something. This is the function of experience, education, training, practice... In truth, a person who commands an encyclopedia of facts and is fluent in acronymese (see previous post) may or may not be able to accomplish a goal. This is also why we tend to look for experience when we need help.

The catch is that a person who has no experience, training, or applicable knowledge has no way of recognizing experience that will be useful. This leads to insistence on things that may have little or no bearing on eventual achievement. For example, when we need a new roof on our house, we don't ask that the roofer have experience with a certain brand of shingle or a certain kind of fastener. We don't specify how the work will be done. We simply insist on straight lines and no leaks. It's inexplicably strange that we do not take the same approach to technology goals including those relating to information. Read some recent position descriptions or postings on Monster or Dice or CareerBuilder. They are chock full of brand names to the extent that the real goal is obscured.

The proof of my point is the extent to which the information management situation is becoming more complex, with dramatic increases in the amount of work and the specialization of that work. These increases in turn produce increases in costs. It's hard to point to achievements today. Micromanagement, based on a swarm of facts with no understanding produces a lot of activity and few results.