The Beginning (5)

In order to get where we want to be (business intelligence) and have the characteristics we are looking for, we will have to make an alliance with our systems vendors.

Our vendors are grown accustomed to thinking of themselves as "partners" and now it's time to call their bluff. We have learned that the long term success of our business depends on the quality of the information that it possesses. Without reliable information, the most efficient and productive process will fall immediately into non-productive and costly tachycardia. Tachycardia is the disorder in which the heart's productive rhythm is disrupted and it's consistent, productive beat turns into an erratic, non-productive, and potentially life-threatening arrhythmia.

How can our partners help us deal with this? They must be responsive to questions about data models, semantics and relationships. They must show evidence that their internal consistency is at least as good as ours. You may want to audit their process consistency. Companies that have achieved the Malcolm Baldridge Quality Award have suppliers who allowed themselves to become this kind of partner.

You will have a robust Quality Assurance capability and you will want them to be linked with a similar capability in the suppliers' organizations. Quality Assurance has been alluded to in previous posts and the QA involvement is data quality should not be minimized.

Quality Processes Produce Quality Data

Processes that are uncontrolled or out of control are incapable of producing reliable data--whether they are in your organization or in a supplier's organization.

Next time we'll address Management specifically.

The Beginning (4)

In my experience, those who can do the least about data quality feel most responsible, while those who have quality literally at their fingertips feel no responsibility whatsoever. This creates many difficulties for one who wants business intelligence.

I'm going to collect receptionists, customer support, sales, anybody who enters data into any of your computer information systems under the heading of "the business." We could also include those who collect or transcribe data to/from paper if it eventually winds up in a computer-accessible data file.

Maybe we have done too good a job in convincing the people seated at computer keyboards that they have nothing to fear. In any event, many simply do not pay adequate attention to what they are doing. There are many reasons for this including:

• heavy work volumes causing a pace that is too fast for error recognition or for going back to correct errors
• inadequate training
• inadequate guidance built into the user interface
• inattention/distraction

On the other hand, I have encountered way too many instances in which people were actually aware that they were producing garbage but didn't care enough to do anything about it. Sometimes it's sabotaging the people in the next department. Sometimes it's a statement to the supervisor and sometimes it's "so what."

It is true that some data can be repaired after the fact, but the sad fact is that "cleansing" can never be 100% effective and in some applications cleansing isn't even possible--if it isn't captured correctly the first time, there's no going back. Repairing bad data is very expensive for your business. Experts estimate that, for anyone who receives data as part of their work process, from 30-60% of their work day is spent in rechecking or validating or repairing the data so that they can do their job.

The data architect will have worked with the folks to understand their data needs so if we can prove that they have participated in that process and that the architects and developers have complied with their defined processes, then what we are left with is attitude or training as problem sources. These are issues for management to resolve.

One final area for the business to think about: there is a need to include problem recognition in training and provide safe reporting paths for those who do take note and take action. Those at the front line typically have little awareness of the business that they front-end for. They don't know that someone cares or that someone can actually do something to fix a problem that they struggle with on a daily basis.

You will want to include a module on the data resource in the New Employee Orientation. You'll want to take remediation to department meetings to catch all who missed the new employee offering. It must be simple to report a problem and there can be no "grilling" of callers by the support line triage staff. Take the call and send someone to see the problem in person while the reporting employee works. Use a remote desktop capability to watch the problem happen. There are many options, but if the caller is made somehow to feel guilty or foolish or ignorant, they will never call again.

Next time we'll take on the vendors.

The Beginning (3)

Before we explore the data architect role in terms of our overall goal, I should say that I do not intend to go into depth for any of the roles. I mentioned the need for standard processes but didn't specify any of the necessary processes. For one thing, that level of detail is well beyond the scope of this (or any) blog. For another, the processes and standards must be compatible with the organizational culture. I wouldn't go so far as to say that any path will do, but I will say that, if you live in Minneapolis, there is more than one way to get to Miami. The thing to remember is that there are more constraints as you depart and again as you arrive. We ask, "What's the best way out of town if I'm headed south?" And then we ask, "What's the best route to city hall if I'm coming from the north?" But in between we can take the scenic route, the shortest route, the fastest route, the Civil War route or any other route that seems good at the time as long as we keep our eventual goal in mind.

From the standpoint of business intelligence and our four characteristics, we would want to pay special attention to what the programmers are doing or not doing with respect to definitions (semantics). The data architect will have spent considerable effort in researching and compiling information about the data. They will have learned about how various kinds of data relate to each other for different business functions and users and they will have defined quality rules for each kind of data.

The process standards, to be monitored by Quality Assurance and warrantied by Quality Control, will ensure that the programmers have those definitions and rules in a format that they can use and that they do, in fact, use them.

If your programmers work for someone else, the processes and standards will be about acquisition. They will ensure that the data definitions and relationship rules embodied in the application or system are compatible with those of your business. You are going to have to lean hard on your vendors and they will squirm and plead "proprietary." The best advice that I can offer is to walk away from this vendor. Another vendor will be happy you asked because it will allow them to really get close to you and they will be proud of their quality processes. The ones who drag their feet do so because they aren't able to produce the assurance you need. Proprietary is a euphemism for we don't know.

Become interested in these things. Ask questions. Expect answers that you can understand. Don't accept arm-waving and diversionary tactics. You will be well on the way to business intelligence from a high quality, reliable data resource.

Next time: the "business" role

The Beginning (2)

Because we want our intelligence at our fingertips, easily accessible, we collect and store data in computer-based filing systems. In return for the convenience of this and not having to store mountains of virtually useless paper (the bigger the pile, the less useful), we have taken on the responsibility of hiring and working with various kinds of technology-savvy people.

At the foundation is the programmer, also called a developer or a software engineer. These are the people who actually create the scripts for the computer to execute. 20 years ago, programmers were visible to the rest of the organization. Today they are typically segregated and largely invisible except to the CIO.

These are the people who create the screen forms and buttons and functionality that your front line employees actually use to capture and look up information. In larger organizations today, the programmers do not create the filing system for data. Instead, that is designed and built by someone else (the data architect) and the programmer merely connects to it. The power that the programmer holds in terms of our four characteristics of good BI is the power to knowingly or unknowingly subvert the quality of our data.

This is a good time to introduce the concept of the data resource. It is essential that today's business view the data that is captured, modified, stored, retrieved and archived as a resource in the same way that capital is a resource or buildings and property is a resource or employees are a resource. The business must devote the same kind of attention to the data resource as it does to the financial resource of the company. Neglect or failure to do so will render the data resource valueless at best and a liability at worst. In between those two extremes, the business will experience increased costs as your workers struggle to get the quality they need from the data that feeds the processes they work within.

So the programmers who build screens and functionality that allow corruption into the data resource are slowly destroying the business just like termites in the framing of your house. It is vital, in order to end up with the BI we need, that programmers employ processes that are controlled for quality purposes. Employing process standards for programming will not guarantee that our four characteristics will be delivered, but NOT doing so will guarantee that we will never be able to produce the BI that we need.

Before we wrap up this installment, it's good to recognize that there are two kinds of programmers; those who work for you and those who work for someone else from whom you purchase or license the programs (or software or applications or systems or...). No business today is unaffected by programming.

When you are getting control of your processes and instituting quality assurance, you will want to ask the same questions of your software or application vendors. Today, they are often unwilling to talk about this and will use deflectors like "proprietary" to avoid the questions. You need to ask yourself whether you can bet your data resource on "proprietary."

We'll get into this a little deeper next time when we discuss the role of the data architect.

Begin at the Beginning (but with the end in mind)

Business intelligence, data warehouses, data stores, stewardship, governance... Where do we begin?

Let's begin by assuming that we want to have something we can recognize as "business intelligence" when we're done. The most important characteristics of business intelligence will be.

• meaning (semantic) understood
• applicability (use/utility) understood
• currency (timeliness) understood
• source/lineage/pedigree understood

There may be other characteristics that are of specific interest to a business customer, but these will put us squarely in the middle of the ballpark.

Before we go any farther, this is going to take more than one post so let's think in terms of a series of posts addressed to specific roles that, together, create the governance road map.

If business intelligence is the goal, then we must begin with the business and we'll start right at the top with the CEO. Of all those involved in the business, the CEO has the greatest power to influence, for better or for worse, the attitudes and motivations that will eventually produce business intelligence. Others in leadership roles will have similar effects to lesser degrees.

Mr., Mrs. or Ms CEO, begin by absorbing the four characteristics above. Ask yourself what it would take for you to feel good about the intelligence presented to you. Once you feel comfortable about your ownership interest, you will need to become single-minded in your pursuit of these characteristics.

Practice asking for proof, for information about the intelligence (which will be known as meta data by some of your staff) and insisting on answers that give you comfort. You may view this as self protection, which is a very good viewpoint to have. In fact, if you are consistent in this, you will find that Sarbanes-Oxley is a piece of cake for your business.

This is where it will start. The next post will address sources of the information that will eventually emerge as business intelligence.

Fear, Accountability and Approval

Fear is an interesting thing. Fear is an emotion that is at the root of many other emotions. If negative emotions comprise a spectrum, then fear is like the sunlight, which, passing through our situational prism, produces the stress, anxiety, mistrust that we actually feel. It takes a lot of self-examination and hard work to be able to let the lesser emotions go and find the fear and its source.

Of all the ways that fear manifests, perhaps the most destructive for a business is that of controlling behavior. The need for control is based on feelings of inadequacy. Many people feel inadequate and still manage to function well in a cooperative environment. Sometimes, though, a person finds himself in a position that he never dreamed of being in and inadequacy, fueled by the fear of losing it all (by proving that he really is inadequate) creates a desperate need for control.

This person will find a way to insert himself into as many important committees as possible and will create new committees if there seems to be a gap in the information flow. This person can't tolerate subordinates who are successful because they become a threat. They have a dislike for group contexts and prefer to use one-on-one meetings to better control the message.

The single worst thing effect of controlling behavior is that the controller manipulates everything so that he has the key decision. This produces several negative impacts including

• an entire organization is slowed to the pace of one individual
• decisions are arrived at through discussion with peers rather than with knowledgeable subordinates
• information needed by subordinates may be concealed in order to preserve the decision authority
• frantic scrambles to meet deadlines arrived at without benefit of process
• no closure on "projects" because of information hiding and diminished credibility
• frustration among subordinates--although a really talented controller will be able to keep this frustration focused among and between subordinates
• after-meeting meetings among subordinates for the purpose of validating perceptions
• much talk about accountability without any accountability
• lack of standards because control diminishes in an objective environment

Do you have someone like this in your organization? How do you deal with it?

One approach might be to create standards around process and measure compliance. With good, useful and actively used standards accountability can be made real. Without them, the best we can do is approval. Accountability is objective. Approval is subjective. Accountability creates no fear. Approval is all about fear.

Your best people have a set of internal standards to which they hold themselves accountable and they won't stay long in an approval environment once it becomes clear that they will have to compromise their standards. The ones who do stay...

A Hammer in Search of a Nail

Are we all familiar with this metaphor? Abraham Maslow ("Hierarchy of Human Needs") is said to have originated this, but it hardly matters. The concept rings so true that it has probably been known since the first tool.

I have lately participated in some discussions on LinkedIn with the result that I now believe that the vast majority of workers in information technology are in possession of a tool that they are seeking to apply to every problem that they encounter. The really dangerous ones are creating problems to use their tool on. Oh, wait, there's a name for that--it's called marketing.

Notice, too that I said "in possession of" a tool. It is apparently no longer necessary (if it ever was necessary) to actually be skilled in the application of your favorite tool.

I once worked with a guy--a programmer in this case, but I'm not looking to single out programmers--who said something rather like, "We don't support the business. We are the business." The business was rail transportation and it is true that if all of the applications used by this particular company were to suddenly disappear, those "left behind" would no doubt have had to cease operations until they could be reorganized.

That's not the bad part though. the bad part is that this is a really good example of just how far the notion of "where's the next nail?" can take us. When I believe that the world as I know it is held together by this tool that I hold in my hand, I am in the midst of a dissociative process. I don't go home and act this out. When I'm not at work for example, I'm just the guy next door. When I do get to work, though, I'm still the guy next door--it's just a different door. We--most, if not all--live in two separate realities. About the only thing that keeps us from being diagnosed with a dissociative identity disorder (DID) is that we (usually) remember what happened in the other world.

I could go on at length but the bottom line is that, not only is our work life a separate reality from our real life, but there is a completely different reality in the executive suite than there is on the floor, and (for me) most importantly, a separate one for I.T. Leave aside for a moment, the variety of realities we might encounter as we go from networks to servers to DBA, to data architecture, to development to QA--it is absolutely amazing that we get anything at all accomplished.

The poor data architect finds himself stepping into and out of a dozen distinct realities every day. It is certainly a defense mechanism to take refuge in a favorite tool--the "data model." This is the talisman used to shield against the swarm of alternative realities. Unfortunately, the tool was designed for a different purpose, to capture and integrate all the different realities. Nails come in many forms, too.