Business Intelligence and Other Oxymorons

Now

2011-11-13T15:50:00.001-06:00

We don't value now. We talk about it all the time. We use it for emphasis. We mostly use it to separate the past from the future.

Often when we talk about now we have only the fuzziest of notions, the flimsiest of definitions in mind. "Now is the time for men and women of conscience..." The concept of now in this instance could mean anything from a generation to a session of Congress to a particular crisis. Sometimes we use now to mean very soon or as soon as humanly possible (the future) as in, "I need it now!" Occasionally we use it to bound some time in the past as in, "Now in those days..."

Now has power that, for most people, is unrealized because it is unrecognized. For our purposes now is the moment of decision. If we are able to grasp that moment, that now, and use it, we can change our own life and the lives of those around us. We can use now to create a new future.

In the gap between stimulus and response there is a piece of eternity--now--in which we can decide what the future will look like and decide on the response that will launch that future. Please be aware of now and use it as it is intended to be used.

When we are present in our life we are conscious of each now and we use them to create a future that matches our vision. What is your vision, your ideal? What will you use to direct the decisions you make in each of your nows?

All Governance (like Politics) is Local

2011-11-09T08:03:00.000-06:00

Tip O'Neill, the Speaker of the House of Representatives during the Kennedy/Johnson years, is famously said to have offered the advice that "All politics is local." If there is anyone out there who doesn't understand that Data Governance is politics then wait.

If we're to gain any advantage from the former Speaker's wisdom, we going to have to pull it apart and take a look at all the pieces. Clearly he wasn't denying the existence of national and even international politics. He had participated in politics at every possible level so what did he mean and how can we benefit?

First of all the context (which is always eliminated from "sound bites") is that of successful politics. Which of us doesn't dream of successful data governance? If we can accept that DG is political rather than technological or administrative or managerial, then we're ready to make use of political wisdom in our quest for successful data governance.

Successful politics is about getting enough people to come with you so that you can accomplish a vision. Because we're human, we look for shortcuts. We start by assuming that if we can convince the right person then that person will bring everyone else along. So we start with our elevator speech in case we find ourselves confined with an influential person for any period of time.

We also adopt the position that money will equate to support. We pursue funding which requires approval at the executive level. In short, we focus much if not most of our efforts on the critical few in the blind hope that all others are followers.

For sheep this may work. Substantial research has been done on flock or herd behavior in an attempt to understand how humans are influenced to move one way or the other. We have all seen a flock of starlings or sparrows or a school of fish suddenly change direction--apparently with a single will. What magic would get people to act that way? Leaving aside the question of goal or vision, which may or may not involve the common good, if we could master this magical force, think of all the effort that could be put to better use.

I have read some of this research and at the risk of oversimplification the answer lies not in identifying the leader but in identifying the first followers. When one bird or fish or wildebeest, in motion, changes direction it may be for any reason or no reason at all. If no one comes with them, they will very quickly rejoin the mass. If another individual comes along then two going in the same direction exert some "gravitational" attraction that acts to influence others in the vicinity.

In the human world we divide people into leaders and followers. More generally, we try to create leaders by assigning titles or creating org charts. As mentioned above, we intend to leverage leadership by devoting our efforts to affecting their path, trusting that they will bring with them enough followers to makes our effort successful. The problem with all of this is that titles do not confer leadership.

What lesson can we learn then from Tip's advice? My own take is that, rather than search for a leader, we might better be a leader, campaigning locally and helping our neighbors and those in need. When we have one or more others with us because they are benefiting form the relationship we become much more effective in changing the direction of the heard. Tip understood that grand political movements arise from individual voters recognizing common goals. No legislation is effective when the governed choose not to obey. Devote your efforts locally and pay attention to what your neighbors in the next block are saying.

Politics is local.

How Would Jesus Do Data Governance (Part 3)

2011-07-01T10:41:00.002-05:00

Having tried the preaching route, attracting big crowds, name recognition and a following, (are you with me?) Jesus recognized that the results he came to provide were not being realized. People listened, cheered and went home to the same life they were living before. Anyone who has been involved in the data industry for any length of time will immediately recognize this situation.

Data-driven development, Data Administration, Information Engineering, Data Architecture, Data Quality, Data Management, Data Governance... Cheering followed by more of the same.

What now? Even if you are God and are omnipotent, you still have to get humans to change in order to bring them to the new world--one in which they can rely on one another and in which each acts in light of a defining principle and for the greater good. How long have you been a human? I have more than 60 years of experience in being and being with humans. Jesus was obviously smarter than I. He realized after only 30 years that preaching and teaching just wasn't going to work--even though the Ideal was attractive and clearly in the best interests of all. He was getting, "Well, even if..." and "But everyone would have to buy in..." and "That's not how we understand the policies..."

Maybe this is what you're hearing as well. Maybe you are beset on all sides by scribes and pharisees, nit-pickers and policy wonks. Maybe they are constantly trying to trap you in a bit of heresy or a policy violation. Maybe you've thought about giving up because for some reason the people would rather listen to them than to you.

What did Jesus do? He changed his emphasis from preacher/teacher to minister. He modeled the changes he was talking about and he did it consistently with each and every person he met, meeting each one where they were. He showed them that the past was irrelevant and the future was not a given. He gave them the present and they experienced for themselves that their lives were better.

He did not give them rules, instead he gave them hope and someone to come to. He gave them someone who understood them and who showed them how, by subtly changing their perception, they could obtain victory over the troubles that plagued their lives. He never showed them judgment or condemnation. He always cared about their welfare and he planted seeds of change.

He gave himself for the people long before he gave his life for them.

If you believe that better policies, better standards, better rules, better laws will force people to care about data, I'd like to help you and I hope that I may already have helped you by planting this seed. Sitting in an ivory tower and sending out criers to inform the people of the duke's latest whim will never (as in NEVER) be productive. Maybe you'll think about giving up some day and you'll remember this seed.

How Would Jesus Do Data Governance (part 2)

2011-06-21T08:06:00.005-05:00

First of all, as I mentioned last time, I am NOT comparing data governance to the Salvation of humanity--I am simply examining methods in light of the fact that both ideas resist description, can't be marketed and are not for sale.

So, I've got an idea that will make the lives of many easier to get through, make the company more profitable, reduce the level of risk involved in making use of information technology, and improve the ability to communicate. My problem is that the idea is based on something I can't describe. I can provide plenty of examples but that only seems to increase the confusion. So far I'm in good company. We've all been exposed to the parables of Jesus. He used them as everyday examples to shed light on some seriously abstract concepts. His disciples scratched their heads and asked each other, "What did he mean by that?" Two thousand years later, they're still being explained to us on Sunday mornings around the world.

Fortunately my idea has no expiration date. There is no competition to put the idea across--no race to the finish. The pain will continue until a critical mass of enlightenment is achieved. Enlightenment is essential because without it all we'll ever have is a collection of examples and stories that, to the average person, seem completely disconnected. Because people like for things to make sense and to be predictable, this disconnectedness leads to alienation.

When Jesus stuck to his idea, looking for better examples, better stories, he found himself on the outside. He was a different sort, something of a kook, but certainly not threatening. He had his small following who were largely content to be associated with him for the limited notoriety that the association provided. They knew him, liked him, respected him. The powers-that-be were not threatened and left him free to try to get his idea across.

Up to this point we have already made us of his methods and find ourselves in exactly the same position. We are tolerated and even receive minimal support from those who hear that others are doing it and seem to derive some non-quantifiable benefit. Unfortunately, as a band of disciples we lack much. We either lack a common cultural foundation or we ignore it. perhaps Jesus' disciples were fortunate in that they weren't constantly bombarded with "fresh, new takes" on their central idea. They were able (forced to) discuss the examples and stories among themselves. This process no doubt kept them cohesive as a group and, in the end was the springboard that helped them to launch the Idea out into the world.

How Would Jesus do Data Governance?

2011-06-18T08:43:00.003-05:00

Recently I've been led to a reassessment of my approach to this whole blogging thing. Originally I had the idea that people would be attracted to a common sense, git'r done view of data governance. This quaint notion was driven by the fact that, although a lot of money was changing hands under the label "data governance" not much was really getting done to realize the vision.

There have been several incidents of late that have honed this approach by grinding away some remaining misconceptions on my part. What is left is the sharp edge--no assumptions, no vain hopes, no vanity, no illusions or delusions--just a cutting edge that can be applied to any knotty issue in the data governance/data quality landscape. Actually this edge can be applied to any knot whatsoever.

First I had to let go of the idea that anyone would listen to and heed an idea just because it's good. We've come to expect marketing glitz. A wise person once said, "All that glitters is not gold." The best ideas are seeds that must be tended over time but which will, in their own time, produce fruit.

Then I had to abandon the notion that people would rally to an idea. As it turns out, people rally to people--to leaders. Here I would ask the reader to consider leaders in general and some specific leaders. What was the basis of their "leadership?" In most cases it was charisma. Too often their followers realized too late the direction in which they were being led. I'm not charismatic.

Finally, I have understood that people will follow reward (most often money) rather than ideas. The great idea of data governance has been crushed, chopped, sliced and diced in search of marketing leverage and greater monetary reward.

All of this honing has left me with a better appreciation of Jesus of Nazareth, known as the Christ. I'm not comparing his mission with data governance--that would just be ridiculous. I do want to look at his methods, however, since many of the problems are strikingly similar.

Stay tuned here for an exploration of "What Would Jesus Do?" applied to data governance.

Chapter One

2011-02-14T09:48:00.002-06:00

This will be chapter one of the eventual book. Some things have to laid out very explicitly. No one should ever be able to complain that they didn't get their money's worth or that they thought they were getting something different.

Some basic principles that will govern everything else that is said:

Data is part of language
All communication about data is, itself, data
Language consists of denoted meaning (denotation) and connoted meaning (connotation) and on top of these is layered implication and inference which involve human perspective
Nothing about language guarantees communication
Communication requires a minimum of two entities from the following set [human, machine, logical construct (e.g., software)]

If we are to get a grasp on data quality we have the best chance of success if we restrict our discussion to ONLY that data that is part of communication between machines or between machine and logical construct. Of course this is neither exciting nor even very useful. We have many specifications formats (ethernet, etc.) that guarantee that communication on some level will take place between machines and logicals. ASCII or EBCDIC are the most basic of communication specification. It doesn't take very long for the alert observer to notice that unless a human is involved somewhere in the process, it doesn't really matter what the communication is.

"Matter" implies human involvement or at least we can infer human involvement from a statement that something does or does not matter. Matter is a value judgment couched in an emotional context. It's only when we start to peel back the layers, asking why or in what sense something matters that we begin to get to the idea of quality. Our exploration, then, will follow the trail of what matters.

As we follow this trail we're going to encounter the idea that what matters is, in many ways, distinct to the judge. What matters to the reader of a graphical novel (formerly comic book) may not be the same things that matter to a reader of War and Peace or a viewer of The Mona Lisa. What matters to someone watching Wile E. Coyote fail in yet another attempt at catching the Road Runner is not the same as what matters to someone watching Being John Malkovitch or Inception.

How then do we determine whose perspective to assume? Whose view matters?

The answer of course is that, where communication is concerned, everyone's perspective matters. It would be a great feat of communication if we could present a context-free (perspective-free) discussion of data quality. In fact, it would be such a feat that we're not likely to ever see it and it certainly won't happen here. Our intent is to zero in (or home in but NOT hone in) on a very small number of perspectives to see what matters to them and then step back to see if there are any common themes that can be exploited. If we are successfull in that, we may have created a springboard for the one who comes after.

In the next chapter we will nominate some key perspectives and begin to investigate what matters to them.

DQ: More Than Meets The Eye

2011-02-12T12:47:00.002-06:00

As we move forward toward a view of data quality that allows us to create and use a language specific to DQ issues, descriptions and solutions, let’s take a minute here to examine the behavior of data.

Certainly, one of the attributes of quality data is that it is well-behaved. In other words it consistently delivers value according to principles that are applicable because of its type, domain, range, relationships, maturity, purpose(s)…

It is useful at this point to differentiate between static and dynamic properties of data. Any DQL (data quality language) that we might define should work well where static properties are concerned. When we begin to consider dynamic properties, the task becomes much more complex. The greater the number of dynamic properties, the greater will be the complexity.

Our chances of designing a DQL will be significantly greater if we can restrict ourselves to static properties only. Before we can do that, we have to understand the dynamic properties and assess their relative importance. Can we carve them out of the discussion? Will excluding them compromise our DQL’s capabilities?

Looking back at the list in paragraph 2, the first three properties might be thought of as static. These are the focus of our modeling efforts or, if we only pretend to do modeling, of our programming efforts. There is a tangent here that we’ll resist for now, but at some point we have to come back to it. The question of how data is initially defined is huge and the effect of initial definition on the lifetime of a datum and in particular on its quality is not to be underestimated.

For now, though, we’ll put that on the back burner. We expect the individual pieces of data to possess a definition (usually called a description), and our DBMS requires that we say what kind of data it is. Is it variable length text strings, a specified number of characters, integer, floating point, money, date/time, etc. It is surprising how many data are defined to the DBMS as varchar. It shouldn’t be surprising since all of our modeling tools allow us to set a default type and the default for the default is always varchar(n). This is popular because it guarantees that any value supplied will be accepted. Oops, another tangent almost sucked us in.

The final three items in the list are dynamic in the sense that their values can and will change, sometimes rapidly and usually unexpectedly. Let’s take the last first. Purpose, as “fit for…,” will change whenever we’re not paying attention. We hope that our stewards will be on top of this but pragmatically (everyone likes pragmatism), they may be too close to the business itself so that changing business needs or drivers loom so large that defined purpose fades to insignificance.

Maturity is also dynamic. We expect maturity to change over time. When we think of data maturity (if we do) we include stability (of all the other properties), quality metrics that have flattened out, recognition within the enterprise and probably several other aspects.

Finally, we have to face relationships. We’re not very good at relationship management. Some of us wouldn’t recognize a relationship if it sent us a valentine. Others pile all sorts of unwarranted expectations on top of our relationships and then wonder where has the quality gone.

It all starts in the modeling phase. Chen, when he invented a graphical notation for describing data, gave equal weight to entities and relationships. Both had a two dimensional symbol and the opportunity to possess attributes. For many reasons, not least perhaps that tool developers didn’t grasp the importance of relationship, “data modeling” tools eventually turned a multi-dimension, real thing into a single one-dimensional line that is only present all as a clue to the schema generator to copy the identifier from one of the linked entities into the attribute list of the other and label it as a foreign key so that the database engine can build an index.

Although I find examples are often counter-productive in the discussion of data quality, one example may illustrate the role of relationship in completing the semantic of a data set. PATIENT is such a common entity in the health care marketplace that no one even bothers to define it. It is a set of “demographics” by which we mean the attributes and it has relationship with PHYSICIAN or PROVIDER. It probably also has relationship with Visit or Admission, Order, Procedure, Prescription, Specimen and other entities of specific interest to the enterprise such as EDUCATION_SESSION, CLAIM…

It deosn’t take long to figure out that the relationship between patient and physician is more complex than can be accommodated by a single foreign key. A physician can “see” a patient, refer a patient, treat a patient, consult (with) a patient, admit a patient…the list goes on and on. Each of these relationships has real meaning or semantic value and may even be regulated by an outside body. Typically, these are implemented by a single foreign key attribute for each.

Now, imagine a situation in which an in-utero procedure is scheduled on a fetus. You may be aware that transfusions, heart valve repair and a host of other medical procedures are actually performed on the fetus while it is still within the mother’s womb. So, who is a patient? If the facility also terminates pregnancies for any reason you can see the conundrum. Medicine doesn’t allow for terminating the life of a patient (Dr. Kevorkian excepted). At the same time, we would like to sometime treat the fetus as a patient, perhaps for reasons of safety. We also experience the lack of values for attributes that we may have viewed as mandatory, e.g., DOB, SSN.

It is only when we explicitly talk about relationships that these issues emerge. Relationships cast light on the entity from all angles.

Relationships also represent the business processes that inform the purpose of the data. Often, undocumented meaning gets attached to data. Two analysts will get together and agree that for the purpose of this analytic, this combination of attribute values will be included (or excluded). For a given ETL job, we decide that an attribute value that isn’t on the approved list will be replaced with “&”. The adjustments to business processes are constant and usually undocumented and unnoticed. Until we can point to a documented process/relationship, we have no way of capturing and dealing with changes.

What’s the difference between an association and a relationship? Somewhere in there we’ll find clues about dynamic quality properties. One thing leaps out as a property of quality and a property of relationship—expectation. When we claim that something has quality, we establish an environment in which it is permitted to have certain kinds of expectations. The same is true of relationship. When two parties or entities enter into relationship they agree as to the expectations they will have of each other.

In our quest to define quality for data, we will be forced to document expectations and to monitor accountability with respect to those expectations.

Data, Governance and Data Governance

2011-02-07T14:58:00.002-06:00

There have been some great discussion threads on the IAIDQ LinkedIn group recently. One thread that attracted a lot of attention started with a question from a PhD candidate in Data Quality. It simply asked whether there is an accepted definition of data quality. 200 replies later, most people would say, "No." More recently a thread began by bemoaning the fact that there is no accepted definition of Data Governance. A lively discussion followed that continues even now. Yet another refers to an article on five reasons to cleanse downstream instead of preventing upstream.

I am about to shed some light, however feeble, on the subject. Allow me to start by admitting that I am a person who likes to do the analysis necessary to solve a problem. Though my patience is improving with practice, those who know me will back me up when I say that I want to solve a problem ONCE.

Previous posts here have explained the abstract nature of data and all that implies in terms of getting people on board when it comes to doing something about quality. People will listen to or read a horror story about some preventable data issue that cost Company ABC $umpteen million. They will nod sagely and say something like, "They should have seen that coming." They are simply unable to see that their own company is engaged in the exact same practices and completely at risk for the $umpteen million.

Friends, it isn't just our favorite whipping boys, Management. There is no more recognition within IT than there is in the boardroom. Our boxes and wires friends think of data in terms of DASD and Raid configurations or bandwidth and throughput. Our developer pals don't really think of data at all except as the fuel that activates their code. Architects appear to be concerned with the storage and throughput views overlaid with an access management filter. They seem more concerned with making developers and DBAs happy than with the quality of the asset.

Enter Data Governance, which in most instances wants to be about definitions, rules and "enforcement." Often Data Governance tries to heap another thick layer, called meta data, on top of all the data that is already being mismanaged in the organization. It's often the case that Data Governance fails to practice what they preach.

Here's the revelation: Data Governance isn't about data. Data Governance is about process. It is the means to the Data Quality end. I have already said that Data Governance is that part of corporate governance that is dedicated to stewarding the corporation's data asset. It is exactly analogous to the role of Finance/Accounting with respect to the capital asset. Unfortunately, Finance has two things going for it that Data Governance doesn't have: GAAP and audits.

Generally Accepted Accounting Practice is a set of guidelines for money management processes that are accepted as the name implies and USED nationally and internationally. The use of these practices insures that processes will be auditable. The audit process verifies that GAAP was used and if there were exceptions, that they were clearly noted with enough information to allow the results to be brought back into alignment with GAAP. The underlying theme is that if the processes were sound then the result is believable.

Imagine if every company of any size whatsoever were able to devise and use its own bookkeeping structure and process. There could never be a stock market. Equity trading would be too risky for anyone and all business would essentially be sole proprietorships. Moreover, there would be no chance of oversight by outside bodies (Government).

This is a picture of the situation with respect to data today. When will it get better? Data Governance has no power to make the situation better. Without an externally defined data management framework and periodic audits by independent auditors, there will be no improvement. In the meantime, if data quality metrics improve, it's only because some particularly strong and charismatic personality is present.

No one questions the need for accounting nor the rigor of accounting procedures. Actually the same can be said for data governance and data management procedures. The difference being that in the case of money, the lack of question results in compliance while in the case of data it results in apathy or confusion.

Does the data world have something like GAAP that could become the necessary process infrastructure to support data management audits? I don't see it. Data is still too personal, too subjective, too misunderstood to attract the attention of researchers. Data management is a black box to virtually everyone and they like it that way.

People prefer to cleanse downstream data because their customers fell their pain being relieved. Happy customers is the goal after all. The bonus is that cleansing provides an unending source of employment for those doing the cleansing. It's win-win! People aren't going to be highly motivated to change a win-win scenario any time soon.

Whose Job Is It, Anyway?

2010-10-05T11:19:00.006-05:00

There are several reasons why you are reading this blog post. Leaving out all the self-aggrandizing ones, let's focus on those who actually have the title question in their minds. You may have been drawn here by an interest in BI (business intelligence), the latest name for "reporting." It is likely that you have had some bad experiences involving reports or dashboards, or mash-ups or some other information display/access effort.

You spent what seemed like way too much time getting to an understanding of

how this this was to be used
the kind(s) of content that would be useful/acceptable
how the information should be arranged/displayed

If you are part of a really accomplished organization, you may also have had seemingly endless discussions concerning

How "bad" data would be recognized
How "bad" data would be handled
Remediation or cleansing processes to reduce the incidence of "bad" data

And, finally, if your organization is in the six-sigma population

What is "bad" or poor quality data?
Where does it come from?
What does it cost?
Where should we devote our efforts?
What kinds of efforts offer the greatest ROI?

If you follow the various discussion forums concerning data quality, you will find one question popping up with regularity: "Who is responsible for Data Quality?" I asked myself why the question is asked. What prompts the question? It seems not to matter whether the organization has a reputation for quality, nor whether it has a history with data quality, nor whether the questioner is experienced or inexperienced, executive, manager, or front-line production. Why?

Having talked with some of these folks and researched the situations of several others and then simply meditated on this over an extended period of time, I have come up with a few likely scenarios.

The questioner knows the answer but either wants validation or a sufficient number of the "right" answer from people who are likely to be respected.
The questioner has encountered roadblocks from unexpected directions and is dealing with surprise and disappointment.
The questioner is curious about what others are doing.

What I have NOT seen is any evidence that the questioner is sincerely trying to determine how best to attack the problem of poor data quality. It would be easy to assume from these same discussion forums and conversations, that nearly everyone knows what they're doing and has either solved the problem or is well on their way to a solution. When we learn enough about human nature we understand that these people are all feeding their egos (or rather that their egos are feeding themselves since much of this is unconscious) and that they are taking an incidence of limited or small-scale success and projecting it into an eventual enterprise level solution. I have done this myself.

I know this--that data quality, or any other variety of quality for that matter, will not bend to advanced degrees, nor to mastery of data and information design concepts, nor to any product or set of products, nor to any effort by marketers, nor even to the best-designed procedures, methods, governance structures, architectures...

The ONLY way to data quality lies through the hearts and minds of each and every person in your organization. Each of them has the power to subvert any plan, procedure or method; to render ineffective any tool or product; and to humble the greatest of egos.

The bottom line is that it must be everyone's job but of course that isn't a satisfying answer because the very short list of things that everyone believes are important includes things like breathing, eating, elimination of wastes, procreation, maybe community, relationship, acknowledgement... This list of universally agreed-upon important things will never spontaneously include data quality. In fact, if a poll were conducted in the boardroom, it is unlikely that data quality would appear on the list of things important to the company.

Don't misunderstand--the quality, security and accessibility of your data assets is at least as important to the continued health and well-being of your company as that of your capital assets. The problem lies in the fact that data isn't real and tangible. If bad data smelled bad or rusted or developed crumbling holes, or if it resigned and went elsewhere where it was more appreciated or was subjected to audits by outside entities, or showed up on a P&L or balance sheet where it was reviewed by prospective data contributors--THEN it might get some attention.

It is true that anyone can produce an example of data but virtually no one--even those responsible for collecting and storing its instances--will understand that data is something other than what they are holding or pointing to or storing. But I digress.

If we can't accept the answer that data quality is everyone's job then we need to move on to identify the person or corporate function who will be accountable for the quality of the company's data. It's not possible here to put a name to this accountable party. What we can do, though is itemize some of the skills and abilities required to help point the way to your unique name.

First and most important, let's agree that what we are talking about here is cultural change and cultural change, more than any other kind of change, requires leadership. Already we see that a corporate function can never be accountable although you can tape a job title on the person's door when you identify him/her. This leader will be able to move freely across the company and will be able to give everyone the feeling that they have been heard. This DQL (data quality leader) will be conversant with principles and practices of quality improvement. The DQL must be completely comfortable with the nature of data and will not be confused by the display of samples. Attributes of the data asset as a whole will be the focus of all of the DQL's efforts. S/he may well choose to shine the spotlight on a segment of the population and may delegate someone more familiar with that segment to assume the leadership of this effort. That surrogate DQL will also deal only with population attributes (metrics).

The DQL will never fall into the trap of confusing examples with anything else. An example may be representative or it may be an anomaly. Only the population metrics allow us to tell the difference.

Further attempts at guiding your choice may be counter-productive if you begin to feel manipulated or otherwise used.

A final caution concerns those characteristics that will render a person unsuitable. Just as you might wonder about a carpenter who feels compelled to talk interminably about his hammer or his saw, the person who leads with the name of a tool, tool vendor, methodology, author, book, etc., is unlikely to be the one you're looking for. The last thing that the leader/agent of cultural change needs is to divert any attention away from the primary focus. Products and tools may be useful for producing the population metrics discussed above, but beyond that should be well in the background and completely invisible to the majority of those you are attempting to influence.

Those who confuse files or [mega/tera]bytes with data are likewise unsuitable as are those who confuse a spreadsheet, chart or report with data.

I hope I haven't made it seem like an impossibility. Talk with people about this and over time you'll begin to get a feel for what to look for and what to avoid.

Leadership, Management, Governance

2010-09-30T08:32:00.005-05:00

Leadership for change

Management for effectiveness

Governance for stability

It is way too easy to become confused where these three functions are concerned. We like to think of ourselves as leaders when what actually consumes our time and attention is hitting deadlines and deliverables. Leaders emerge when change is in the air.

When the marketplace is shifting; when the economy is deteriorating and taking our profitability down with it; when we're no longer able to keep up--that's when leadership is required and leaders step forward. Anyone can steer a straight course through calm seas with good charts. It takes a master to sense the environmental factors, inspire confidence among the crew and make the continuous changes required to keep the ship from breaking up or running aground until things settle down and we get back into familiar waters. We risk disaster when leadership isn't acknowledged and permitted to assume control. When the crisis is over, we often find that the leader has little or no interest in the day-to-day operations of work rosters, schedules, performance reviews... The leader may make a very poor manager.

The manager is the master of routine. She is the one who keeps the machinery humming and the product going out the door. He makes sure that time boxes are hit, that deliverables are delivered and that budgets are created and followed. When the winds of change begin to blow, the manager who fails to recognize the need for leadership or believes in error that he can handle the leadership role can cause massive and sometimes irrecoverable damage before she agrees to [temporarily] relinquish the helm. Exceptional management defines the team, builds the team and keeps the team vision alive. This management is essential.

Governance is ubiquitous and invisible. Governance is observing, analyzing, formalizing, monitoring, measuring, improving. Governance establishes the standards to which managers hold themselves, each other and their teams accountable. It should be readily apparent that governance is every bit as necessary within a high-performance organization as is management. An exceptional governance function is a combination of historian, engineer and seer. A liberal dose of management is required to insure that governance doesn't degenerate into approval by the de facto expert.

The six sigma governance function will have incorporated leadership into the system of standards. A set of standards and standard processes may be of little use to the leader in the midst of the storm but may have helped to prepare that leader to be able to step forward.

Leadership for change, management for effectiveness, governance for stability is the tag line of WhiteLake Data Management. Data Management will be a microcosm of the enterprise as a whole. Every point of view (perspective) within the enterprise will be represented in Data Management. Virtually every process within the enterprise will be examined by Data Management in an attempt to "get it right." The risk associated with unreliable information (data) can only be assessed in light of the process(es)--and the personalities-- involved. Data Management is not governance but must include governance as an essential component. As its name implies, management is its bread and butter. Because it is frequently considered the homely step child, however, the availability of leadership may well be the key to its success.

Data Governance Is...

2010-07-23T19:17:00.002-05:00

Recently someone on a LinkedIn discussion forum asked for a definition of data governance because he had yet to find one that was universally accepted. Now he has about 30 definitions and all of them are "personal" in the sense that they apparently work for the person who responded and/or that person's company.

In fact the lessons of the past 10 years or so produce the conviction that data governance is anything, everything and nothing.

It is anything we need it to be that serves our immediate purpose.

It is everything in that it spans all corporate functions in order to produce the needed results.

It is nothing because we always have to define the term whenever it is used and no two definitions are the same.

My own input to the discussion was that

Data governance is that part of corporate governance that is concerned with insuring the integrity of the corporate information resource.

The only useful application for this definition is in establishing a context for any initiatives and establishing responsibility or accountability. This definition cannot be used as a strategy or a vision to drive results. It doesn't suggest any metrics. It doesn't help us to isolate key processes nor does it suggest any standards.

Other definitions you may have seen involve "decision rights". The problem with "decision rights" is that there are always individuals who will pop up once one of these rightful decisions is made and insist that the decision in this particular case was rightfully theirs. Quite often this individual will make a good case and the "rightful" decsion will be overturned--often at considerable cost. When this happens, it calls into question the makeup of the existing decision-making bodies and can cast a long shadow over the entire concept.

My definition contains a problem in that it invokes another poorly defined concept--that of corporate governance. As I have discussed in previous posts, corporate governance in any form that would support the needs of those desperate for data governance is as rare as a polar bear in west Texas. It is the most challenging, most demanding and most thankless job imaginable to create a system of governance in the midst of a feudal culture.

This is not to say that results--even valuable and far-reaching results--can't be obtained. Such results are possible for those who are dedicated, courageous, knowledgeable and visionary. If one can keep the vision of data governance as part of a corporate culture and pursue integrity for all information but do it one relationship, one entity--even one attribute--at a time, then real progress can be made.

I have often wondered why we think we need a definition for data governance when it is so obviously subjective. Of course data integrity and data quality and even data itself are equally subjective. None can be discussed without first offering a definition ("What do you mean by that?") and we don't have definitions that we can quote that are meaningful to "this" audience. In fact, we are given to definitions that are nearly meaningless even to our colleagues and serve only to get us all in the same ballpark.

It is possible to take any result and call it data governance (who could argue?). It is possible to take any corporate initiative and use it to promote data governance. Why don't we simply get busy and spend our time discussing results instead of definitions? Show me the data that demonstrates you have brought processes into control for some subset of your company. Let's talk about which processes are most critical and which represent the greatest opportunity. Let's get moving.

Principle Before Practice

2010-06-20T17:20:00.003-05:00

Before I begin, a caution. Do NOT think that I am taking a negative position re: data governance. On the contrary, I firmly believe that the concept is essential. But what is the concept?

As I watch data governance related position descriptions parade by on DICE.com, MONSTER.com, etc., I am struck by the focus on the practice of data governance. They are all about tools and skills and methods and all either assume that everyone knows what the goals are or (worse yet) that goals for this company are the same as those for every other company.

I have learned at least one thing in my 27 year adventure in all things data-related and that is that doing something, no matter how efficiently or effectively, is very often a wasted effort without some forethought about the principles that we are attempting to implement.

Principles provide the glue that links all of our efforts together and the medium that allows us to be productive even when our last effort was a failure.

As with so many initiatives around data, this wave (governance) has crested and achieved the status of "best practice" without ever achieving measurable ROI. We have seen smaller-scale successes--enough to keep hopes high--but not, to date, the enterprise wide success that was used to sell the initiative in the first place.

But maybe I'm getting ahead of myself. What is the principle that data governance represents? Some candidates are:

Make life easier for DBA's by reducing the rate of database schema changes
Make life easier for developers (programmers) by making requirements less ambiguous
Reduce cost due to rework by making the rules for data quality and completeness more accessible at the point of capture
Make the data warehouse more useful by applying the same rules to data everywhere it is captured.
Reduce the number of sales lost because we can't keep track of our customers
Establish process consistency so that we can begin improvement efforts
Stop those sobs from delivering junk and expecting me to fix it
Define and establish (standard) processes to reduce variability in output quality
your favorite here

Which one(s) do you like? I believe the answer is all of the above and then some. What is the principle that unifies all of these goals? I discussed this in a broader context in a prior post. My point though is that if we don't have a firm and commonly held idea of the principle(s) we are attempting to implement, then no matter what we do, which tools we use, or how skilled the workers are, we aren't going to accomplish anything meaningful. To put it another way, "If you don't know where you're going, any route will do."

What Do You Do When Things Aren't Working The Way You'd Hoped?

2010-05-26T19:14:00.003-05:00

Let's pick a context first because a) this problem is pervasive in the world I live in (how about you?) and b) the context will determine our course of action. I'll use my own life as an example.

I have spent my life seeking to understand my environment so that I could have a chance of staying out of hot water by being able to predict outcomes. I actually got pretty good at the predicting part but was never able to translate that into the staying out of hot water part. It turns out that when you see a result coming that is unwelcome to everyone, hot water is the least of your worries.

Of course I could have kept quiet and just let things happen but the problem with that is that almost invariably a minor course change would have prevented the outcome. It always seemed reasonable to attempt that minor change. Just as invariably there were political implications involved in any changes to the published plan. Bottom line: my career is littered with "you were right's" that came three years after I moved on.

So. if you would learn anything from my example, maybe it would be that "being right" carries no value. Maybe it's that you should just keep your head down and wait for the seniority promotion of for retirement. Maybe the lesson is that you do what you can and the rest belongs to someone else.

I will say that over time I have achieved objectives that others considered "impossible" because I was willing to take risks. The problem there of course if that if the objective was considered unachievable then no one is prepared when the find themselves standing inside the walls.

I think that this is also the story of data management (to include what has come to be known as data governance). Organizations have been talking about data management for nearly thirty years now and there are hundreds if not thousand of experts who will tell you exactly what you should to enjoy the benefits of good data management practice. What none of them will tell you because a) you don't want to hear it and b) you wouldn't hire them is that there is no proven methodology--no set of practices and tools, skills and technology--that will guarantee results.

Why should this be? You would think that in 30 years someone would have stumbled across something that will deliver predictable results. The answer lies in the subject matter. "Data" is a concept understood by everyone. Everyone in the boardroom has their favorite data. The issue at the root of all problems is that "everyone" is seeing data "as through a glass, darkly."

The inability to communicate about data and reach a consensus is what is keeping us from our objective. To this add the cult of personality that defines the management--let's call it governance--of the corporation. The decision makers understand nothing of the underwater portion of the data iceberg, seeing only the table, graph or dashboard that's in front of them. What must be managed is the abstraction that is data and not the values that are only the visible portion. When we try to do anything with the abstract, we find that there are side effects on the visible portion that cause VIP personalities to convulsively respond in exactly the least useful way.

You can get useful results if your objective is modest. For example, it is possible to get two business functions who are exchanging data or three or more that have a symbiotic relationship based on data to take consensus action to stop what is often a great deal of daily pain. The intractability is encountered when we attempt to broaden the scope to cross departmental or divisional boundaries. The goals and methods of data management are counterintuitive to those raised in the power politics of corporate "success."

We usually find ourselves managing data as a commodity, "how much", "how many", "what is the cost", "who produces", "who consumes", "spoilage rate", "how fast"... While these all have an attraction in that the answers can be easily captured in one of those tables, graphs, dashboards, none deal with the underlying problem of managing the abstraction. Data is the most complex thing that a corporation attempts to manage. It is more complext even than money.

The pity is that we treat data as if putting it into a "piggy bank" solves all our problems. You heard it here first:

Technology is no answer--technology can help us sort different kinds of values into different piggy banks, no more.
Technical skills (modeling, DBA, quality...) are no answer. The cashier makes use of such skills to keep his/her drawer in order and reconciled.
People skills by themselves can't achieve any result except perhaps building meaningless consensus.

This is enough clues. If you call, don't bother to tell me what DBMS or CRP system or BI tools you're using. None of those things are of interest until the final stages of a solution. I don't expect any calls because too much credibility is wrapped up in the current initiative--whatever it is. When it fails to produce results, a new personality will step in and you'll start the cycle anew. Someone, someday may actually be willing to take a risk to stop the pain. I'll be retired or deceased by that time but maybe you'll have learned from this what you should be searching for.

Data Quality: Getting Started

2010-04-29T15:00:00.003-05:00

If you happen to be a "mover and shaker" or if you aspire to that role, then you'll be looking for access to one or more key decision makers of influencers closest to the top of your organization. You'll be determined to convince those people that

investing in data quality is a sound business decision
you are the right person to produce the ROI that they'll be looking for

If, however, you simply want to make things better as soon as possible and create new friends and allies while doing so, then you may want to take a different approach.

My recommendation is to use the tactics of the Special Forces. The "Green Berets" were formed into small teams comprising skills critical to the people they were trying to help. They then went out to those people and lived with them. Doing this allowed them to gain credibility and to learn what kinds of changes might (or might not) be acceptable.

The Green Berets helped the people with their work and, while doing so, offered improvements--small changes that produced higher productivity or more consistent results. The goal was to create allies.

"Data Quality" represents exactly the same kind of productivity and/or consistency improvement for our "indigenous" people in whatever part of the company they may serve. A DQ Team may be as small as one member and can produce results that are shocking in their scope and value as well as in their lack of cost. It isn't necessary to spend long periods of time "living" with the people. In fact, one lunch or a couple of coffee breaks will do IF you

ask the right questions
listen carefully to the responses
offer support
follow through

You'll ask about what happens when they get incomplete or incorrect forms (data is usually thought of collectively as a form) from their internal "customer". You'll ask about the extra work they have to do in such situations. Be prepared for an emotional response, this is what causes them to miss deadlines, work overtime, add staff... Also be prepared to hear that they simply pass the problems on because they aren't staffed to deal with them and don't feel accountable for fixing problems they didn't create.

Listening will uncover the sources of the most frequent or egregious DQ errors. Now you can mention that you are about to begin a project with those dirty so-and-so's, that it's likely they don't even know the hardships they're creating, and that you'll be happy to mediate a discussion amongst the parties to try to find a resolution. You may already have some ideas.

Create the meeting, making sure that ALL parties are represented (you were listening carefully, right?) and facillitate the discussion, if necessary gently guiding the discussion but never offering solutions. When the solution is "discovered", the people will own it and will implement it with minimal assistance from you. If your assistance is required, make certain that you deliver and do not hold them up.

Follow up by monitoring, coaching, facillitating and then ask if they'd like some help in publicizing their success. Because you know important people, they'll almostly certainly jump at this opportunity. You give them all the credit and they'll make sure to let everyone know that your help was both timely and critical.

This approach works and can even result in regular meetings to follow the improvement and to look for new opprtunities.

Two approaches--you choose which one has the highest probability of success for the greatest number of people in the shortest time at the lowest cost.

The Theory of Everything

2010-03-29T10:46:00.004-05:00

The US economy, so far as the majority of citizens is concerned, is in the toilet and swirling rapidly in a clockwise direction. Health Care, long in its own toilet, has at least stopped swirling momentarily. All of us have a stake in what happens in those toilets.

I have a stake in another toilet as well. The portion of the economy devoted to technology has been caught in a vortex since the dot com implosion 15 years ago. I realize it will do no good to link all of these since linkage simply makes the resulting mess seem even more impervious to any corrective action.

However, (deep breath) if we don't consider the nature of the connection between them, we have very little chance of making sustainable progress in any of them. So, as my contribution to posterity, I nominate the accension of appearance to the pinnacle of importance in decision making as the criterion most likely to be acknowledged as the root cause of all three problems.

Since 1950 the rate at which appeareances have displaced substance as the motivation for decisions in the US has increased at a dizzying rate. In the past two years I have seen the nations physicians, as represented by a blue ribbon panel from the Mayo Clinic, state that the answer to the nation's health care woes is better access to insurance. The calling that was Medicine has emerged as a new entitlement program for the elite.

In the economy, fiduciary responsibility has been replaced by revenue numbers as the force that justifies all decisions.

In the Technology world, the means have come to justify the ends. Any decision can be justified if it allows me to position myself as a front-runner, new, hip, cool. "There's an app for that" allows us to spend unjustifiable amounts of money just to have that app in our pocket. Similary, corporations spend unconscionable sums on technology projects for which the need is poorly understood. Because the "solution" has to be new to give the proper appearance of tech supremacy, the outcome is always in doubt. Risk isn't so much managed as PR-ed. Spin control is the name of the game.

In almost 30 years of working with technology I have learned one lesson that transcends all others:

Either control your technology or it will control you.

In assessing the meaning of this for your own situation it is well to remember that

Technology demands consistency
Humans and human organizations are incapable of consistency

Just a few recommendations

Be clear about WHY you want to do something
Insist that others are clear about WHY they want to do something
Choose a path that is known to produce the desired result--or at least choose next steps that are known to produce appropriate results.
If you are unable or unwilling to do the above, choose another line of work if possible and stop complaining if not.

We are all either a part of the solution or a part of the problem. In either event, complaining about what someone else is doing or not doing will produce absolutely no change.

If you can't tell the difference between substantive value and the appearance of value you should avoid positions in which you will be called upon to make decisions. If you can tell the difference, then for all our sakes, make the decision and don't give it to someone else.

Christmas Wish List

2009-12-15T12:05:00.002-06:00

The 13+ years I have spent in Healthcare have sensitized me to some things. I might have preferred to remain ignorant of many of them. In the spirit of Christmas, which is handy at this time of year, I'd like to nominate a list of gifts that would bless all residents and citizens of the U.S.A., regardless of theology or philosophy. Each item in the list relates to health care.

I wish that the role of technology could be clearly understood. There are a vast number of well-funded voices who want us to think that technology is health care or that health care is technology. In reality, technology is best thought of as a tool--an inert and often expensive piece of equipment, which, in the right hands, can produce wonderful results.
"The Patient" or "our patients" is not the same as "my patient" or "Josie Jones, patient". It may not be possible to apply technology designed for delivering care to a generic patient to Ms Jones. That doesn't mean that the technology is bad. It only means that the technology must allow for deviation in procedure. I wish that the role of abstraction is system design could be clearly understood.
I wish that all of the factions in the healthcare struggles were clear about their goals--with themselves and with each other. Only by being self-aware, open and open-minded can the parties negotiate a solution advantageous to all. Doctors, nurses, administrators, vendors, government and patients are currently at odds. The friction is not only between factions but within factions. Who will speak for physicians? The A.M.A.? Mayo Clinic? Who? Who speaks for government, for vendors, for patients, for nurses, for administrators? Each of these groups functions like a mob--surging to and fro as a strong voice emerges and then is drowned out. Each group must organize itself before "healthcare" can be organized.
Though I recognize that individuals and groups may be driven by ego to appear more knowledgeable than the next, I most devoutly wish that each of us might recognize that the person across the table might actually have some knowledge that we don't. I wish that we would listen first and assert only when necessary. I wish that we could see ourselves as occupying the same driverless bus.

There are many more things I might wish for this year but I don't want to seem greedy. May you each be showered with blessing upon blessing as one of God's beloved and may we fully appreciate each blessing as it comes.

Principles of Data Governance

2009-12-08T16:12:00.005-06:00

Malcolm Chisolm, in a recent column in Information Management (A Principles-Based Approach to Data Governance) raises an excellent point. In 2006, when I attended my first Data Governance Conference, there was much discussion around a definition of DG. Implicit in this discussion was the need for something that was concise, yet comprehensive, and on top of that, engaging. The idea was that this definition could be used:

As part of a sales pitch (like a slogan)
To create synergy within the emerging discipline
To provide focus for any ongoing methodology efforts

Some present may have had additional motivations, but I think these were the ones on most people’s minds.

The definition that emerged was acknowledged to be a work in progress. By the 2008 Conference, one of the tutorials quoted three definitions:

Data Governance refers to the organizational bodies, rules, decision rights and accountabilities of people and information systems as they perform information-related processes.

Data Governance is the practice of organizing and implementing policies, procedures and standards for the effective use of an organization’s structured/unstructured information assets.

Data Governance: The execution and enforcement of authority over the management of data assets and the performance of data functions.

These were troublesome to me then, probably for the very reason that Malcolm mentions. All seem to acknowledge a context based on an organization’s information assets, but their focus seems to be quite different. The feeling I have is that they are advocating a judicial, legislative and executive approach to governance.

In the U.S., a Constitution lays out these three perspectives and establishes the mechanics (framework, architecture) within which governance will be administered. Within the Constitution and before any of the mechanical parts are discussed, in fact, within the preamble, first principles are asserted. The writers tell us that what follows will be a system of governance for the purpose of

Forming a more perfect union

Establishing justice

Insuring domestic tranquility

Providing for the common defense

Promoting the general welfare

Securing the blessings of liberty to ourselves and our posterity

While this model would probably work on its own in establishing [data] governance, there are just a couple of nuances that will have to be accommodated because our system will not be working in a representative democracy but in a corporation.

Within the context of our system, leaders are appointed and serve at the pleasure of stockholders rather than the public. The principle of one-man-one-vote does not apply. One person may control sufficient votes to dictate to the Board of Directors. Within the day to day operations, the ability to dictate policy, direct activities and appoint deputies is granted at multiple levels, though always subject to the pleasure of the higher levels.

Having now established a context, it’s time to agree on some first principles for data governance. The candidates are:

The entire corporation must agree to be subject to the system.
While those placed higher may still, at their pleasure, appoint and dismiss deputies, they must agree that [data] governance operations will be a factor in those actions.

It must be understood that within the corporation, domestic tranquility, the common defense and the general welfare are all dependent upon the information assets owned and managed by the corporation.
When the system is followed, all processes will flow smoothly, problems are addressed at the process level, personal antipathies are secondary to process execution and process anomalies such as unplanned rework and delays are greatly reduced in number or even eliminated completely.

Consistency is everyone’s goal.
In a work context, surprises are almost always seen as negatives. Our goal must be to improve the consistency of our processes and their outputs such that surprises become exceedingly rare (six sigma has been suggested as a goal) and predictability becomes commonplace.

These principles should be the touchstone(s) of our efforts. Everything we do should be evaluated on the degree to which these principles are addressed.

I will suggest that these may also be the principles of the corporate Quality Assurance effort and remind everyone that they are also the basis of Deming’s 14 points as well as other quality improvement methods. No improvement is possible without first establishing a stable (consistent) process.

I leave you with one final principle: Data governance will not be implemented as a stand-alone initiative. If we cannot see data governance as part of a larger, comprehensive system of governance, we will not be able to address any of the three principles suggested above.

Survival, Error & Technology

2009-12-04T08:46:00.008-06:00

I'm going to pass on some wisdom here. It's not very often that we encounter wisdom today, especially where technology is concerned, and it's often the case that we don't recognize or acknowledge wisdom until we're looking back over the wreckage and trying to figure out what we should have done. I'm probably also setting myself up by labeling this as wisdom but I am so weary of seeing the same ads with different acronyms and talking to the same people with different names.

You will never find your way out of the current mess you're in or about to be in by searching for and hiring someone with recent experience on a specific product. To put it another way, a specific product, no matter how much buzz it enjoys, is never the answer.

I will be among the first to acknowledge that the use of absolute language (never, always...) and even the use of unqualified superlatives (best, worst, fastest...) is a habit to be avoided, nevertheless, decades of experience have proven that the absolute statements in the preceding paragraph represent wisdom and that failure to heed this wisdom will produce cost overruns, timeline disasters, confusion, stress, employee turnover and a host of other undesirable outcomes.

In large part, the success of the human race has been due to our ability to recognize exceptions without necessarily understanding the rule. My own take on this is that, with today's reliance on technology, we may have reached the point where the process of natural selection that has honed this skill over countless generations has now produced a liability. "Something's different," is enough to put us on guard and may be enough to launch a complex defensive reaction to preserve the safety of the individual or group.

First of all, while it is still good to recognize exceptions, it is now absolutely essential (that's an absolute absolute) that we develop the ability to recognize the underlying rule. A study of human error (Human Error, Set Phasers on Stun...) shows that leaping to conclusions about the rule is what produces the error condition. In fact, if we can't describe the rule in terms of the logic of the computer (if... then... else...), we can't rely on technology at all.

You might ask, as I did, how we might acquire this ability. The time tested way is known as [survivable] experience. There are a host of cause-and-effect analysis tools and techniques that have the appearance of rigor and reliability and are an improvement over experience, especially when combined with exhaustive testing, but you will find that even these are more productive when used by people with experience in the world being analyzed.

Tools are great and another critical human enabler, but--and this can't be over-emphasized--no tool is so advanced that it runs itself. Every tool, no matter how advanced the technology requires human hands and a human mind to guide it. If you were to be presented with the greatest woodworking tool in the world or the most advanced sewing machine or fishing gear or computer, would you immediately become a master cabinetmaker or designer or fisherman or software developer? You might note that the only immediate change is one of expectation.

An experienced person with rudimentary tools is more likely to produce a quality result than the inexperienced person using the "best" tools. The fact that I used a tool just yesterday says nothing whatsoever about the level of my experience in producing the required outcome. I have made the mistake of looking for help and focusing too narrowly on what amounts to recent experience with the tools in my shop. I have learned (through experience) that I will enjoy better results if I'm learning while interviewing my prospective employee. If I'm talking with someone whose knowledge stops at the tool's user interface, then I had better be prepared to devote myself to directing the employee's work. If I have a staff composed of such employees, then I need to possess all of the requisite experience myself or else be prepared to conduct a project whose principle product is more experienced workers.

The challenge is to find the right mix of experienced people in supervisory or team lead roles and people who possess dexterity but are in need of experience. If I'm in a director or management role, I have to have experience producing a product with that scope. A technology "system" has a complexity that is beyond human comprehension. The only way to design and build it is through a process of identifying smaller and simpler pieces, building those and then assembling them into the final product. You need to look for people who have an appreciation for the amount of effort this takes and the discipline--both personal and organizational--that it takes.

Stop looking for Oracle or CRM or Rational or even "use case" or "data model" experience except as clues about the approach that the candidate might be expected to take. I understand that these things are ideal as targets for a logic rule processor, but the rule ("find resumes that include these terms") is so simple-minded as to be useless. If your only goal is to turn 1000 resumes into 100, then proceed, but if your goal is to find someone who can get you out of the predicament you're in, then you should spend more time on your rules so that the exceptions are more productive.

Business, Information and Technology

2009-11-23T10:08:00.002-06:00

Are you in management? Do you have annual/quarterly goals? Will you be held accountable for achieving those goals? Is the accountability expressed in bonus dollars? Is there any possibility of a zero bonus?

Are you still with me?

How will the achievement of your goal(s) be measured? Please note here that "how" has two dimensions: one is related to process and the other to a unit of measure. In all of my vast personal experience, all of the attention has been focused on the unit of measure part (when there has been any attention at all), and the process part has never even been part of the conversation.

Please understand that what follows is not intended to sling mud at any individual or organization. My purpose is to clear the air so that we can talk about how we're really going to achieve our goal(s).

I am going to generalize based on extensive, though anecdotal, experience. In other words, I have not conducted a survey, scientific or otherwise, and cannot produce data to back up anything I'm about to say here, so I'm leaving it up to you, the reader to determine whether it feels like truth or not. Should you feel that this does not ring true, or should you wish to fault me for not being more objective, I would ask that you produce a study or at least a body of experience in support of your position.

Awards of bonus dollars tied to achievement are based solely on whether the holder of the dollars wishes to give them away or not. There is rarely, if ever, any protocol defined for defining metrics, units of measure or measurement process. You will go into an "annual assessment" meeting with your boss and he or she may discuss your level of achievement in very general terms before announcing the amount of your bonus or a recommendation for an increased level of compensation.

Why is this important, you ask? Well, it is important because it's the the way things are done. Despite vigorous protests to the contrary, the business world is set up to run on subjective assessment supporting subjective decisions. What, you say that your decisions are "data-driven" (objective)? I would love to hear the story behind the data that was used to arrive at your most recent decision.

All of this is background for understanding why "data" initiatives so frequently become mired in a swamp of politics and personality. Let's walk back from a data-driven decision.

You are able to make the decision because you trust the data.
You trust the data because you are familiar with and trust its source.
You trust the source because you know that it is reliable.
You know that it is reliable because it consistently produces information that can be relied upon.
The source has been consistent because it always uses a tried and true methodology (set of processes) to produce its product.
The consistency is possible because the methodology includes steps designed to validate the source's inputs.
The validation decision returns us to #1.

How do you feel about standards (e.g., standard operating procedure or SOP)? If you don't currently support the creation, implementation auditable use of standards or, at any time in the past have not done so, you have no right to and almost certainly do not have access to reliable information and therefore no claim to data-driven decisions. Just to drive the point home, when your boss decides that you won't be getting that bonus or increase you were counting on, your only acceptable response is to smile and say thank you.

By the way, if you notice that your bank account (or budget) is suddenly much bigger (or smaller) than it was yesterday, what is your responsibility? Who are you accountable to? How much trust can you afford? Now you have some insight into compliance.

The use of technology introduces an additional huge portion of uncertainty into the trust equation. Take another look at the decision walk-back above and note the points where the use of technology means adding additional paths and complexity to the validation process. This is what your data governance people are trying to get their arms around.

To summarize:

data-driven or intelligence-driven decisions demand trust
trust demands reliability
reliability demands consistency
consistency demands compliance
compliance demands governance

OK, you can go back to work now.

What Is "Data" Anyway?

2009-11-17T07:17:00.003-06:00

If you have any experience with phone support (on either end) you will recognize how easy it is to get deep into a process before realizing that the other person is on a completely different path than you are. RTFM (Read The F---ing Manual) often pops up as the answer to our communication difficulties, but it clearly is not the answer or it would have been universally embraced by now.

I happen to be a proponent of the theory that the answers aren't nearly as important as the questions. As a teacher, I know that learning is happening when the pupil is asking questions--particularly a series of related questions. This has become very important as I have attempted to make headway on [data] governance and [data] quality.

My early attempts assumed that everyone knows what data is--and they do, in the same way that a picture is worth 10,000 words. Each and every person you talk to knows what "data" is and each has a different idea in mind. For most, it's a picture of the last set of values they looked at. This might have been a spreadsheet, a graph, a collection of measurements... The key is that data is a set of values. For some, "data" is a commodity. It is files, stripes on a disk, a percent of capacity, a quantity of bytes measured in "mega-", "tera-" or "peta-". For still others, "data" is represented by a schema, model, definition, or some other abstraction.

Given those varying perceptions or perspectives, is it any wonder that at some point in the quest for "data" anything we find ourselves stuck in the quicksand of confusion. Even when all parties have been saying the same things and have been agreeing on goals, there comes a point when someone will say, "We're not going to do that." or "I don't see why that is necessary." or "But that will change my work flow." This is frequently the point at which everything starts to unravel.

So what I have learned, and what I offer to you now, is that the initial phase of any data initiative must be a carefully constructed education process to insure that the quicksand moment never happens. This must be thought of as risk management. Remember, too, that the really important questions (and answers) initially are not the ones you're hearing. All of the different constituencies are going to be much more comfortable exchanging information (or misinformation) within the tribal group than with "outsiders."

The best way to head off this risk is to carefully choose allies from each constituent tribe and use informal conversation about their pain points and the ways that data figures in the relief of that pain. You will be setting these people up to be the "experts" within their respective tribes. Part of this will be coaching them in how to respond to questions and discussion in which they don't feel themselves to be on firm ground. They need ways to postpone a response until they've had a chance to confer with other experts. This is easy to do by setting up a collaborative model.

Rule one of this model is that I never answer for someone else. Everyone can understand that a situation involves yet another perspective and that it is necessary to involve someone from that tribe in order to get a complete answer. The most common danger here is "We don't have time for that." Everyone must understand that this is an absolute red flag event. It signals that we still have not achieved a universal understanding of objective.

When a red flag event happens, it isn't the same as finding ourselves up to our necks in quicksand. It just means that we need to engage in some risk mitigation. It's a sign saying "Quicksand ahead." We will need to bring this person into the fold--usually through informal and non-threatening discussion with at least one peer or trusted expert.

Your role, should you choose to accept it, is to be a non-judgmental, constant, committed, and helpful presence that can be relied upon to be a neutral mediator and facilitator who feels like a friend in any need. Your motives must be above reproach. You cannot count on and should not hope for recognition. All around you will be better off for your presence.

If you are senior in the organization to this person, you should make sure that you are appreciating their contribution but they will appreciate non-public affirmation since putting them in a spotlight may negatively impact their ability to continue to function in the same way.

"What is data anyway?" is a question that requires many answers initially and one answer eventually. Remember, though, that many people are really only interested in what they have to do differently. "Data" may have no meaning whatsoever in their day-to-day responsibilities even though they may be monitoring real-time run charts with instructions to take a specified action when the line goes above or below a certain point. You can't possibly know where to start or where to stop in defining data for them. That's why you need the tribal expert.

Don't seek "important." "Helpful" will take you much farther more quickly.

Disturbances in the Force

2009-10-23T10:58:00.001-05:00

So, if we know how to make things better in terms of data quality and we're motivated to do so, what's stopping us? A word of caution; what you're about to read may be harmful to your health.

Maybe you're old enough to have lived through the Watergate fiasco and can remember the facts coming to light one by one in the press until they eventually began to make a complete picture. Maybe you remember the Hollywood version, All The President's Men, in which the whole picture was produced in two+ hours rather than months, or maybe the whole thing is in the same category as the Crimean War for you and is nothing more than a question on a pop quiz in one of your least favorite subjects.

I'd like to suggest for your consideration that if we want to track down why we are having such a difficult time accomplishing something that we all claim to want, we need look no further than the paragraph above to get all the answers we need.

First, let's imagine that data quality is like truth in government. It's a good thing and we would like to assume that we have it. If, in fact we do not have truth in government (or data quality), who benefits? The answer is that it is in the interests of those who believe they can/will be blamed for the status quo to cover up the problems and subvert efforts to get at the facts that can provide the complete picture. This is especially true if they are responsible for the problems. Even if the only identifiable responsibility is that the person is the supervisor or manager of the function(s) that owns the troubled processes, they still may elect to resist and subvert in order to avoid becoming responsible for the fix.

If we want to avoid this situation, we should absolutely avoid any questions that seem directed at why or who or even how. We should avoid to the extent possible, any investigation into the past. Try to keep all discussions focused on process-based causes that might be producing the effects you are seeing. Do not zoom in on isolated instances but look for trends. Remember, your goal is not prosecution but consistent quality.

In the words of Bob Woodward's source, Deep Throat, "Follow the money." The programmer-champion will struggle against this repeatedly. There is a perception that implementing integrity checking at the point of input represents added cost. Like any other complex process, system development should seek to minimize total cost of ownership rather than any single cost line item. If it takes an extra day of programmer time to ensure that we get 99.99% consistency of integrity in the database and thereby avoid dedicating multiple full-time staff to data clean up, this is a net cost reduction.

Our system design and project management processes may not be mature enough to assign dollar values to this, but it should be easy to determine how much money we are spending on fixing poor quality data every month (or year) and then amending the design and development processes to devote a fraction of that amount to prevention.

The final perspective to be extracted from our example is that a short attention span provides little hope of even recognizing that a problem exists let alone understanding it enough to develop a mitigation strategy. Data quality (and truth in government) requires that everyone be involved. People are capable of recognizing self interest within the corporate interest and enough people will be motivated to act that the ball will be kept moving, but the media of the late 1960's is not the media of 2009. In the 60's there was an interest in the truth that perhaps doesn't exist today. In your corporate environment, you may find it easier to maintain a constant pressure of communication directed at a single theme. The widespread motivation will not be produced by a single appeal surrounded by banners and fanfare and free cake. A communications campaign must be designed for the long haul with continuous refreshing of the message.

You don't even have one percent of your employee workforce today who are ready to grapple with the issue of data quality. You are going to have to break it down in multiple variations and start with the concept of data itself. What is data? You'll be surprised at what you uncover when you go out to talk to people about their data. Stay tuned for some samples.

Programmer as Data Quality Champion

2009-10-22T09:36:00.005-05:00

The programmer is the one who takes all the wish lists and turns them into something that a programmable logic device (computer) can execute to fulfill the wishes. Today, several additional roles have attached themselves to this process. The architects, designers, modelers, testers... all play an important part in the final product but it is important to remember that these roles have motivations other than the ability of the product to satisfy wishes. At best they satisfy a different set of wishes that have more to do with the process than the product.

In the not so long ago days when I started in the information systems industry, none of those other roles even existed. It was all about programming.

In talking with a programmer, you will detect a hint of pride and superiority based on the sure knowledge that none of "them" could produce a program that ran without error and produced a useful result. Other than the "end user" or simply "user", there may be no one lower on the respect totem pole than the "data" people. The programmer only needs to know what you want it to "do"; data is just something that you move from one place to another.

In those bygone days before information technology there were organizations known as data processing. I'm leaving out broad segments of programming known as systems programming because at the operating system level, the data really is a commodity consisting of groups of on/off bits known as bytes. In the very act of ignoring this segment of programming we stumble over the origins of our problem. In early computer systems, there really was no data as we think of data today.

A programmer could grant wishes by making a process that took large amounts of "data" values from one file, combined them with large amounts of "data" values from other files and deposited the resulting "data" values in a new file from which address labels or paychecks were printed. The programmer's responsibility was simply to make sure the program didn't unexpectedly halt.

At first, they just told the users what the data values had to look like in order to ensure that the program kept running. When the users proved incapable of guaranteeing the necessary consistency, programmers took matters into their own hands and created scrubbing programs that would for example guarantee that a file contained only values that looked like $nnnnnn.nn where the value of n is from the set (0-9). Now everyone was happy until one day a big order came in for $1,250,000.00 and was thrown out as erroneous. At the same time, someone figured out how to divert the fractional round-off amounts into a private account.

I'm leaving out some reasoning steps in an effort to keep this to an essay length. If you get lost, just drop me a note and I'll be happy to fill in any missing pieces.

Eventually it was realized that we don't have to store data in a form recognizable to humans--the computer could be taught to present a data value in any format that a human might care to see. This leap forward allowed programmers to distance themselves even more from the data. The idea to take away from this is that programmers may not have the same concept of data that you do.

When non-programmers talk about data, they are typically talking about instances rather than types. To a non-programmer, "Walgreens" is an example of a piece of data as is "sea foam green" and "$900 billion." To a programmer, these are all character strings or text values and may be of three different subrange "types". The subrange (store, color, gross revenue) determines how the value should be handled and the value may be acceptable if it fits the pattern defined for the type.

Today, there are many opportunities to enforce patterns on data values and most of them require no programming at all. The problem is that they all produce errors and error messages that the typical user could not hope to comprehend. In effect, they cause the program to terminate unexpectedly. So, despite all the advancements in technology, we are still having to scrub data files. The alternative is for the programmer to think like a human instead of like a programmable controller and the problem with this alternative is that it introduces orders of magnitude increases (x10, x100...) in complexity and corresponding increases in development costs.

So how can programmers become champions of data quality? One relatively simple way would be to avoid accepting text values as program input. This tactic is a favorite because it defers many decisions until a later time when "we know more" and the big problem here is that we never have to go back and change it. An example here might be useful. Imagine that you are programming a system that accepts input from nurses who are taking vital signs (temperature, BP, pulse, respiration, height and weight) in a patient exam room. You take the usual shortcut and implement all the fields on the screen as text.

Everybody is happy because the nurses don't ever have to go back and correct anything and the program runs without apparent error. One day, though, a health insurance company decides to reward its contractual clients by paying a slightly higher rate to those who document that they are doing a consistent job of collecting vitals at every patient visit. Now we're asked to verify that we do an acceptable job of collecting and recording vital signs. Since the values input to a screen go directly to a database, we should have no problem. It is, in fact, no problem to count the records for which there is or is not a value in those fields, however, when we attempt to aggregate those values to show the range of values or the average value, our query fails. the aggregation query must convert the text values in the pulse field to integers and the text values in the temperature field to floating point (real) numbers in order to compute an average.

We finally discover that pulse contains some values like "100.4", "98.5", "SAME"... that cause an error because they can't be converted to an integer value. When we look at this as a nurse or physician, we can see that the mind ignores the labels on the screen and simply produces a picture of the patient based on the values displayed. Our poor computer, though, is unable to continue. The database architect could have made pulse an integer type and the DBMS would have enforced that typing by not allowing these values to be stored in the database. Using a text type allows the DBMS to accept any value for storage. The programmer could enforce a text value that is guaranteed to convert to an integer or could enforce integer types directly but in order to do so he or she must handle resulting errors in a way that is understood and accepted by the nurses.

More often, though, the nurse managers show the incorrect data to the nurses and exhort them to pay more attention. Do you believe the nurses will respond better to blame and exhortation or to assistance from the program? Check out W. E. Deming's Red Bead Experiment to get your answer.

The programmer champion will be suspicious of a discrete valued field whose data type is text. A value that may be used in a computation or any other operation where a conversion must be done must be investigated carefully. Any value that may be used as a tag for identifying rolled-up aggregations, such as store name, must get additional attention if we don't want to see quarterly sales for "Walgreens" and "Walgreen's" and "Wlagreens". The time to catch and repair these data quality errors is the very first time they are captured by a computer program. That makes the programmer responsible. Other roles have a duty to identify situations where these problems might arise, but only the programmer is positioned to do anything about it.

I realize this is asking a lot. A programmer is only human and can't be expected to know everything (right?). This suggests another way in which the programmer can become a champion. Since it isn't possible for one person to know everything that must be known (hard though that may be to swallow), the programmer must develop enthusiasm for consultation and collaboration. Every role in your environment was created for a reason and each has its own goals and responsibilities. The programmer is accustomed to the data people coming with requests. The requests are nearly always framed in terms of something that the programmer should do to make the [modeler's, architect's, steward's...] life easier and improve overall quality.

It's easy to understand how this can get old in a hurry. The solution is for the programmer(s) to sit down with these other roles and get everyone's needs on the table. All of the other roles mentioned have a different view of data than you do and here's the thing--their view is much closer to that of the customer/user than yours is. You need each other.

Accept that you are a key member of a team and as such the team can't succeed without your commitment. The flip side is that you will not be able to enjoy the success you dream of without the commitment, skills and knowledge of the rest of the team. Be a Data Quality Champion--it's within your grasp.

Next we'll take a look at some forces that act to keep the team from being all they could be. Stay tuned for Disturbances in the Force.

The Problem With Quality

2009-10-22T08:43:00.005-05:00

Meeting the spec[ification]
Documented adherence to established [process] norms
The product's effects are primarily positive (for example, it tastes good and doesn't make me ill)
I'll know it when I see it

Multiple choice: which of the above (choose only one) is the definition you want applied to your new car?

Does the answer change if we apply the definition to your morning cup of coffee?

Last question: which definition do you apply to the next example of "business intelligence" that comes before you?

I have two points in mind. First, defining quality is not an exact science even within a specific context. Second, #4 may be the deciding factor regardless of #s 1-3. In the end, the consumer/customer merely has to say, "That's not what I was looking for" to relegate a product to the trash heap. We all know that it does no good to say, "This is what you asked for" (meeting the spec) or "I did it just like you told me" (followed established procedure) or "One won't hurt you" (tastes good and not sick--yet).

So what is quality and especially, what is data quality? How we obtain data quality is completely dependent on the answer to this question.

I'd like to suggest that we divide the question in order to produce at least one useful answer. If we examine data quality from the perspective of a computer and its logic, we can come up with an answer that will allow us to progress. The second perspective is obviously the consumer/customer or the human perspective.

Recently I received an email with what at first glance seemed like an innocuous statement full of typographic and/or spelling errors, but when I actually looked at it, it was a nearly perfect illustration of a principle I have been talking about for years.

Teh hmuan mnid is cpalbae of miankg snese of amslot atynhnig taht ftis smoe bisac ptratnes.

This is the principle that draws the line between computer logic and human "logic". It is also what makes programmers (an outmoded term, I know, but best suited to the point I'm making) so vitally important. There is only one role in the continuum of roles involved in producing an information system product that must bear the full weight of responsibility for the integrity of the data quality at the boundary between computer and human. That role is best thought of as programmer.

Unless you earn your living as a programmer, some alarms may be going off. In fact, if you are a programmer, some alarms should be going off. I earned a degree in computer science and made a living as a programmer. From there I moved into data modeling, data administration and database administration. Now I'm involved in data quality and governance. In all of that time, I have never come into contact with any training, education, book or even a job description that addressed my accountability for preserving data quality at the man/machine interface.

This may be a poor forum for this, but my intention is to change this situation right here. My next few posts will present some background on how a programmer might live up to this responsibility and some of the forces that will need to be fended off in order to make it a reality.

Next: Programmer as Data Quality Champion

Answers and Questions

2009-10-01T08:47:00.005-05:00

We are living in a time in which seemingly everything is driven by the need for answers. You may be saying, "So, what?" as you read this but the impression I have developed as I have moved through this time is that in many, if not most instances, this amounts to nothing more than "trivial pursuits." A quote found online recently sums it up perfectly:
"We are compelled to drive toward total knowledge, right down to the levels of the neuron and the gene. When we have progressed enough to explain ourselves in these mechanistic terms...the result might be hard to accept." [Edward O. Wilson]

Unfortunately, though having answers readily available can get you around the game board ahead of everyone else, it can't necessarily produce the big win for a cooperative venture. I don't know what research exists on this subject, if any, but it seems to me that living in "the age of information" is vastly overrated. If you think it is good to have a library of information at your fingertips then you may want to think about some desireable properties of information, for example:

Is it true?
Is it accurate?
Is it current?
Is it useful?

A relatively recent recent development has been the creation of a new area of specialization around data quality. This is not a technical post, though, so I leave it up to the reader to go as deep as necessary. The point I would like to make is that the useful property trumps all the others.

The publicity that surrounded recent preseidential elections in the U.S. shows us that "information" need not be true or accurate or current in order to be useful. If nothing else, this should cause us to pause and consider our thirst for information.

The bottom line is that a command of facts is useless unless those facts enable us to accomplish something. This is the function of experience, education, training, practice... In truth, a person who commands an encyclopedia of facts and is fluent in acronymese (see previous post) may or may not be able to accomplish a goal. This is also why we tend to look for experience when we need help.

The catch is that a person who has no experience, training, or applicable knowledge has no way of recognizing experience that will be useful. This leads to insistence on things that may have little or no bearing on eventual achievement. For example, when we need a new roof on our house, we don't ask that the roofer have experience with a certain brand of shingle or a certain kind of fastener. We don't specify how the work will be done. We simply insist on straight lines and no leaks. It's inexplicably strange that we do not take the same approach to technology goals including those relating to information. Read some recent position descriptions or postings on Monster or Dice or CareerBuilder. They are chock full of brand names to the extent that the real goal is obscured.

The proof of my point is the extent to which the information management situation is becoming more complex, with dramatic increases in the amount of work and the specialization of that work. These increases in turn produce increases in costs. It's hard to point to achievements today. Micromanagement, based on a swarm of facts with no understanding produces a lot of activity and few results.

Complexity and [Over]Simplification

2009-09-22T13:10:00.005-05:00

Pardon my absence (if anyone noticed) while I recovered from a bout of depression brought on by repeated exposure to thought streams defined by buzzwords and marketing hype. Ulysses (James Joyce) was memorable for steam of consciousness paragraphs that ran on for page after page. I did not find the process of navigating someone else's thought stream enjoyable in the least. This is the same feeling I have been experiencing of late. The process of mapping the path from stimulus to response (or problem to decision) in humans is poorly understood at best.

My thought was to try to zero in on one kind of problem to see if I could cast some light on the decision-making process and find out how it gets caught in the deep ruts so often.

If you are a technophile who buys in at the bleeding edge and who speaks in acronymese (let's say it's pronounced uh-kron'-uh-meez): a language based on acronyms or sets of initial letters chosen or pronounced in the form of words, for example SQL (for Structured Query Language), pronounced see-kwel, then you may as well find something else to read because I'm about to ask you to take some real responsibility. Actually, I'm going to suggest that those who have to listen to you should demand that you take responsibility.

Many of you will be familiar with the Ishikawa Diagram though you might recognize it as the fishbone or cause-and-effect diagram. If you have ever used this tool to identify the cause(s) of a problem, you may remember just how quickly the diagram can become unmanageably complicated. A few years ago S.M.Casey wrote a book titled Set Phasers on Stun: And Other True Tales of Design, Technology, and Human Error. This book looks at some of the biggest man-made disasters of the past twenty years in an attempt to identify the cause(s).

While human nature demands that we be able to blame someone for anything that goes wrong, this research very pointedly shows that each time we identify what we believe to be the cause, it is always possible to say, "But if x had been alert, the damage would have been minimal or avoided entirely." In short, the cause is always a related set of events that may have been initially set in motion by a "proximate cause" (http://en.wikipedia.org/wiki/Proximate_cause). The problem, if we're interested in insuring that the damage is not repeated, is that legality is irrelevant as are the needs of human nature. The only way to guarantee that a certain problem never arises is to guarantee that anything that could be contributory is not allowed to happen.

So, "What does this have to do with me?" you ask. A brief example being better than a long explanation, here is an actual exchange that happened at a DAMA meeting recently. The chapter President called for suggestions for small-group discussion following the main presentation. One of the suggestions was, "Why do we keep making the same mistakes?" The group noted that

it may not be possible to answer the question and
an answer might well be useless in avoiding the mistakes.

Discussion proceeded to other topics. A specific question was asked involving response-time performance in a database application. Many possible causes for extended response times were trotted out without shedding any light. Someone asked about the possibility that a single query, executed repeatedly might be the point of "failure" and included the suggestion that the DBA (database administrator) should be able to say whether this was the case or not. The response: "We don't have a DBA. We thought we could get along without that additional expense."

Another example: Hammering a nail to hold together two pieces of wood is a simple and straightforward operation that can be accomplished by almost anyone. A group of eight-year-old boys could hammer many nails in a relatively short time. If the goal is a habitable dwelling, though, or even a serviceable garage or potting shed, no sane person would entrust the job to the boys. Note that the issue isn't motivation or lack thereof and it isn't tools nor skill per se, it's really about basic knowledge concerning the desired result.

Some theoretical knowledge concerning material characteristics and structural formats is required to deliver a result that will be in service for more than a few minutes.

In the information systems world, the analogy is much more apt than we might be comfortable in admitting. It isn't possible to be involved in a database design discussion (or even a data modeling one) without the term "denormalization" popping up. I'm going to use denormalization as a placeholder for a host of other bits of conventional wisdom in the discussion that follows. Relational data design is an application of set theory which, in turn is a branch of mathematics. Normalization rules (or forms) were developed to ensure that set operations would be applicable and would produce the expected result when applied to a database. Denormalization means avoiding the application of normalization; in practice, it is rarely an activity. There is no methodology for denormalization.

My advice to the supervisor, manager, project manager who is told that the database will be (or has been) denormalized is to ask to see the normalized design and to ask questions about how far the normalization was taken. My guess is that you will get a lot of verbal tap dancing and arm waving. Ask that normalization be explained to you at least through the third normal form. Remember that you will never be able to utilize the full power of your relational database engine using set operations on a non-normalized database. You will always need programming to get at the data.

I have participated in many conversations in which people stated unequivocally their feelings about denormalization and normalization and could not articulate any of the normal forms. The same holds true for statements about standards, methods, tools... It is apparently necessary today to have a strong opinion about any topic that arises. I haven't heard, "I don't know enough about that to have an opinion yet." in several years.

George Santayana may be the first to say that those who cannot remember history are doomed to repeat it. He certainly won't be the last. Data and information design theory has not evolved much in the past 20 years nor have software engineering or project management. Despite that, every day there are dozens of exciting press releases trumpeting the newest (always trademarked) approach to data management, project management, system development...

In the words of the Wizard, "Don't look behind the curtain!" The same difficult, complex work must be done today as twenty (or 30 or 40) years ago. We do have better ways today of dealing with simple repetitive tasks but all that really means is that what is left is more difficult, more complex, more rigorous. This is no place for amateurs nor lone rangers. Every system will be the result of a team of experts working for a common goal and relying on one another completely for their individual expertise. The manager or project manager had better be able to recognize when tap dancing, arm waving and smoke emission is taking place.

Or maybe all you want is cheap and/or quick.

The Control Myth

2009-08-18T06:47:00.005-05:00

Control is a very much misunderstood concept. The bottom line is this:
Self control is a good thing, an essential thing. Attempts to control others are doomed and will be harmful to all concerned.

This is difficult to write and it is difficult to publish in a public forum. I know it will be resisted and may stimulate reactions from others that will not be beneficial to me. I have made a conscious decision (self-control) and hereby renounce any expectation with respect to responses, reactions, and results.

I retain my hope that the statement may create thought processes that lead others to alter the ways in which they interract with their world.

The conscious reader may note that my statements regarding control are framed relative to people. Control of inanimate objects and such abstraction as process is not only good, it is mandatory. Please note, however, that control of (for example) a process is not the same as control of the people involved in it.

You may have heard of Total Quality Control (TQC) or Statistical Process Control (SPC) and the need to have and use controlled process in order to insure high quality production. A study of these methods has led me to a new understanding of control.

When we think of control we typically associate notions of power. I control something when I can make it bend to my will. A manager is said to control an organization or function. A driver may be fined for failure to have control over their vehicle. This is one of the reasons why TQC and SPC have such a diffcult time gaining traction in the business world in the U.S. (the environment with which I am most familiar). The control that is the core of Quality methods has nothing whatsoever to do with my or anyone else's will. If anything we need to look at it from the opposite direction.

A process is either functioning within understood parameters (in control) or it is not. If it is not, we say it is out of control. What we mean is that we have just discovered that we don't understand the parameters as well as we thought we did. Now we can assert our will to change the process so that the new (improved) understanding becomes part of it. It is potentially life changing to realize that the process is literally in control--that it, is has the control. The evidence is that it produces what it produces. If we desire to change what is produced, we must listen to the process, understand its needs, and give it what it needs.

To take a giant step that may require backfill later, if I have a need to control something, the variability or consistency of output for example, I must give up control to the process. If I have a need to control the people, I must give control to them as keepers of the process. They have a far better ear for what the process is asking for and can give it what it needs. When they can't give it what it needs, they will come to me and tell me what I must do.

To have control, I must give up control.

Life is a process.

Enforcement and Accountability

2009-08-09T11:26:00.003-05:00

Recently I responded to a discussion question on a LinkedIn group forum. The question dealt with how to enforce standards in a data management and stewardship scenario. The other responses mentioned the use of various committees and steering groups as well as management partners for enforcement of standards. One response suggested that a lot of messy people problems could be avoided if automated tools were used to find areas of non-compliance.

I can't help but think that we, as a society, must be nearing the pinnacle (or the pit) of buck passing. When I as an individual choose to ignore an incident in which an action by someone else either ignores the general good or threatens the welfare of all, I am turning my back on accountability and passing the buck to "someone" else.

There have been many instances in which a malefactor, caught in the act, has told me, "What's it to you? What do you care?" If I point out that the action was, for example, in violation of published standards, I might hear, "Nobody follows that. I didn't know it existed until you showed it to me."

The point is that it takes a lot of will in the face of widespread apathy to be accountable for not only following standards, but insisting that others follow them. A study, which I am unable to cite, showed that the rate of deterioration in a neighborhood increases when individual incidents are ignored. For example, a window broken by vandals goes unrepaired or "tagging" of a wall is not erased. Ignoring an incident encourages similar incidents and then worse ones.

I realize I am comparing failure to follow standards with vandalism and I do this with intent. If we assume the existence of a set of standards, they must have a purpose. Normally, the purpose is related to quality. Every manager wants their organization to run like a "well-oiled machine." When an organization does runs this way, we say it is a quality organization. Ignoring a standard is like dropping a grain of sand into the machine. Everything may proceed with one grain, but as one grain encourages two and then three, ..., eventually the machine will break down.

Bottom line: if you're looking for someone else to enforce standards, you're looking in the wrong place. It's up to me and it's up to you. If it's a bad standard, then it's up to me and you to get it fixed. Without personal, individual accountability, you will never get adherence. Enforcement is an empty concept even outside the workplace. People will not endure coercion for long.

Managing Technology and People

2009-07-30T12:17:00.004-05:00

The thing about management is that it works great for money, time or countable things, but management of complex, uncountable things is at best a dream. You say "people are countable" and I'll agree, but only to the extent that the management is about bodies (head count).

Technology and its applications are so complex as to be unmanageable. Edsger Dijkstra spent much of his career trying to get that message across to the business world and the budding computer industry. Today, he is remembered more for optimized search algorithms. If that isn't the industry in a nutshell...

Many years ago I tried to tell people that technology must be controlled, not managed. My slogan was "control technology or it will control you." We are so in love with the notion of management and have so much antipathy for control, that this message, like Dijkstra's, falls on deaf ears. I am not, in any way, comparing myself to Dijkstra. I am comparing his "audience" to mine. Today, the tag line on my web site (www.m2dxtx.com) is "Leadership for change, Management for effectiveness, Governance for stability." The three are not mutually exclusive but the probability of finding all three in one person is quite small.

Control is not a bad thing--it is an essential thing. Automobile travel without control would involve so much risk that no reasonable person would attempt it. The control starts with the design and production of the machinery itself. An automobile is designed to be controlled. It is also designed to function within a larger system of controls. Awareness of this allows the designers to prioritize their efforts and focus on differentiators suggested by the system rather than on mere "performance" factors. For example, it would be a waste of time designing a vehicle for mass production that could negotiate a 90-degree turn at 80 mph. The system of controls insures that this level of performance is unnecessary.

Use of technology without controls is also fraught with risk. We require control over the design and production of technology to insure that the product is useful and usable within the larger control framework that is our business context. Control over the use of technology is needed to insure reliability and safety for all just as our traffic laws and their enforcement produce a sense of safety and predictability for those of us on public roads.

Because no one likes the idea of control, we are calling this "governance" but make no mistake, governance must be about controls or any effort is wasted. People want and need consistency. Consistency produces contentment and the role of government, according to SunTzu (The Art of War) is a contented populace.

So, if by management, you mean counting (or accounting), you won't have success applying it to personalities or to technology. If, by management, you mean coercion, you can, for a brief time, deliver the appearance of consistency and contentment with personalities or technology, but you will only be masking a growing problem. If, by management, you mean a system of controls (governance) that produces consistency, predictability and reduced risk, only then will you be able to say that your technology (and your people) are being managed.

Fortunately, the controls necessary for effective implementation and application of technology are well understood (if largely ignored). The Software Engineering Institute's CMMI and ITIL are but two specifications for a system of controls. These are thorough and consistent and understanding them will enable you to create a system tailored to your organizational needs.
The future starts when "control" is accepted and welcomed.

Healthcare and Health Care

2009-07-21T10:34:00.004-05:00

I have to revisit this subject in light of recent news and developments. It pains me to see the confusion that has caused the pollsters and pundits to be able to take shots at something that everyone wants.

There is concern about cost that is well founded. The problem is that costs come in a variety of disguises. Now discussions of cost have assumed inordinate importance in the questions of access. Once again, healthcare is about cost and health care is about access. These are two distinct issues.

There is virtually no one who advocates the denial of medical care on the basis of inability to pay. The entire health insurance industry emerged in response to a rise in costs driven by improvements in the science and technology of medicine. These improvement demanded better education for medical practitioners (added cost) and the technology has become more complex resulting in increased cost for both development and support.

While all this was happening, we became a nation of sedentary, over-eating, narcissists who believe in the idea that modern medicine can fix whatever we do to ourselves and make us all(well, me anyway) into beautiful people. We have learned to game the system to get the plastic surgeries we want and the pills we want and the therapies we read about.

Insurance coverages have been broadened continually in response to forces too numerous to mention with the result that more premium dollars go out requiring more premium dollars to come in. The insurance companies have developed bureaucratic defenses, requiring second opinions, demanding justification based on diagnostic testing, even making a practice of denial of the initial claim to filter out those who aren't really serious. The additional staffing and data handling is paid for by premium increases and forced out to the medical providers who increase their charges. Rising costs are everywhere and no one can join the debate with clean hands. Everyone wants someone else to absorb the costs.

In the midst of all of this, we sometimes lose sight of those who simply stay away from health care because they don't have enough money to pay for the other, even more basic necessities. We always lose sight of those who try to take care of themselves by buying health insurance, which they can afford only by accepting caps and deductibles or limitations on coverages. These people looks good in the statistics but rarely show up in the doctor's office because paying the premiums has put them into the same category as the uninsured in that they have no financial resources left over to pay for the visit. Further, they now have to live within the insurance bureaucracy that demands diagnostic justification, turning what might have been a $100 office visit into a $300 one.

Access to health care for everyone should be the sole topic in Washington. Access has relatively simple solutions. Let's solve that problem first, and, by the way, we have already agreed that ability to pay will not limit access.

Costs are a completely different issue and one that will require all interested parties to make substantial changes in thinking, planning and delivery.

It is deplorable (to use a word employed by a past President in a slightly different context) that we continue to allow doctors, administrators, insurance CEOs, technology vendors, pharma, and politicians to continue to point fingers at each other while the full cost of inadequate health care is borne by me and you, the patient/consumer.

Haves and Have Nots

2009-07-15T09:24:00.004-05:00

When I speak of have here, it should be clear that I'm referring to resources. Less clear but no less important things to have include:

need (acknowledged)
commitment
known cause of pain

First and foremost is availability of resources to be applied to making improvements. Data and information quality diseases have much in common with human diseases in terms of diagnosis and treatment. There is much discussion today concerning the state of health care in the U.S. The discussion focuses not on diagnosis or treatment--those aspects are well understood (if imperfectly practiced)--but on paying for the diagnosis and treatment.

It seems that financial resources or the lack of financial resources is the single most important determinant of physiological well being. If we examine the whys behind this, we soon see that expectations have much to do with it. The person without financial resources learns to expect that some problems will be chronic and learns to live with them, perhaps at a lower level of function. The financially well-off person learns to expect that every problem has a cause and a cure and that time and money will produce the expected well-being.

Neither is absolutely correct and both sets of expectations produce advantages as well as disadvantages.

We can apply the lessons of health expectations to data quality. Larger or wealthier companies expect that they will be able to attack a quality issue with sufficient resources to conquer it. Smaller or less well-off organizations will not feel able to dedicate one or more people to the issue and will elect to "do the best they can" (see previous post). Small business leaders will see that everyone must be involved in the solution for it to work and that alone will cause them to turn away from a frontal attack and "make do." Large business leaders may believe that the right manager or leader with sufficient resources can bring it off.

Again, neither is absolutely correct.

A person or an organization resigned to living with pain is always going to find it difficult or impossible to improve while a person or organization immersed in full scale battle with the problem may well miss opportunities for improvement.

As it turns out, a "data quality" campaign is like a campaign against bacteria--almost meaningless. Because the scope and scale of the campaign preclude considerations of nuance, we find that we make enemies from within the ranks and everything degenerates until nothing is happening. We can make progress against a specific bacterium or a specific quality issue but we soon realize that we can't hold those gains without creating a framework within which we can establish trust, confidence and consistency. That framework has come to be called data governance. In the case of physiological disease, the framework is Medicine.

Whether you're a have or a have not, the resource issue turns out to be far less important than we might have thought. Consider expectations first.

Can we live with or adapt to the pain?
Have we already adapted? How?
What limitations are imposed by the adaptation?
We can choose to treat symptoms, cure the disease, and prevent the disease. Which is within our reach? What can we do? What should we do?

In most cases, the best choice is to treat symptoms while making lifestyle changes to prevent the disease. Sometimes we have to cure the current disease or we die before we can implement the lifestyle changes. The point is that we always have options. A specific option must consider the past, present and future. A combination of options may produce the best result. Last but not least, have and have not is not really about resources but is about expectations. Commitment is often born of desperation when we realize that we just can't tolerate the future implied by our current expectations. Now we're really ready to do something meaningful.

Can and Should

2009-07-09T12:33:00.004-05:00

Can and Should are in constant tension. They both imply something that has not yet happened--in other words, they both are in the future. So here's the key question:

Do you want your future to be composed of cans or do you want a future of shoulds?

Should is closely related to could.

If you could do what you should do, would you do it? If you should and could but don't, what kind of future do you have before you?

Is your past characterized by "might have", "could have", "would have", "should have", or as my father was fond of saying, "mighta, woulda, coulda, shoulda?"

What's the difference between could and can? It might be knowledge or it might simply be practice. For many people, the biggest difference is the realization that there is something beyond "I can." Parents fill this role as do teachers, mentors and good friends. The process of revealing the new world of could is known as coaching.

What we should do is a function of goals, history and current context. Most of us get paid to know what what should be done. Most of us also take the easy way out and do what we can rather than what we could or should. In fact, "Do what you can," has become a universally accepted surrender. When the boss says it, it means that

they don't know what should be done
they don't know what could be done
they don't want to be bothered with knocking down roadblocks
they don't really care about the outcome

When I say it ("I did what I could.") it means

I know what should have been done
I know that I could have done more
I told them but they wouldn't listen
I was not committed to a quality result

We nearly always allow ourselves to choose the familiar path. When faced with a choice between can and could, we choose to do what we have done in the past--can.

We cannot get the data quality we need unless we have the governance we need and we can have neither if we continue to do as we've always done. This is macro as well as micro advice. Governance is not committees and steering groups, though it may have need of such. Data quality is not one definition, though that may be helpful. Both are about contextual consistency and predictability. This goal could and should be achieved in whatever ways are appropriate to the context within which the consistency is desired.

Consistency is a product of process and the foundation of improvement. Once the process produces consistent output, you have freedom to classify and categorize its output in whatever ways are suitable to its customers. We are currently engaged in trying to classify, warehouse and use inconsistent products created by inconsistent processes.

What could we do? What should we do?

We The People

2009-07-03T07:33:00.003-05:00

We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution...
Article 1 of this constitution describes a representative form of governance, recognizing that the needs for deliberation and timely decision making can best be met in this way. This was particularly true in a time when travel was by foot or by horse (or other animal propulsion) or by water propelled wither by wind, oar or paddle.
Two thoughts come to my mind:

What might this article say if written today?
There has been no need to modify the principles set forth during the ensuing 222 years.

All of this leads to a third thought. If the goals of corporate governance are substantially the same

more perfect Union--Every CEO wants the company to operate as a unit, with a single purpose
establish Justice--A sense of justice is a prerequisite for people to focus on their duties and responsibilities.
insure domestic Tranquility--Inter-personal and inter-organizational dissension is a primary cause of lost productivity.
provide for the common defence--The company must defend its position in the marketplace and each employee is critical to that defense.
promote the general Welfare--This goes hand-in-hand with justice. It's human nature to want things to be better.
secure the Blessings of Liberty--Personal liberty is always subject to the other goals.

then maybe we ought to consider whether the method should be the same.

It's hard for me to consider data governance (which is where I'm coming from) in a vacuum. The goals of data governance are substantially the same goals outlined above. Defense is about defending the integrity of the data resource. Union is about consistency. Justice and welfare is about everyone living by the same rules (which produces consistency).

I don't want to make data governance sound so impossibly complex that we throw up our hands in surrender. The message I'm transmitting is that we have models to use. We do not have to reinvent governance.

One of the difficulties in any governance model is to come up with a definition or picture of "the governed". We go through life happily assuming that everyone else is "just like me" in terms of their wants and needs. Mostly that works, but every now and then, we run into someone who isn't "just like me." When that happens we have two choices. Either we try to make the other person just like me or we adapt our view of "me" so that it includes some new parameters. In corporate life, it is exceeding dangerous to assume that anyone in a role different from ours is "just like me."

Even if we restrict ourselves to data governance, we find that we have to include as "governed" many who are filling different corporate roles and are definitely not "like" us. Again, I go back to the American Colonies in the mid eighteenth century. Imposing or trying to impose a set of rules on people whose life and needs I don't understand is destined for failure. The secondary message is: either include everyone in designing the rules or (poor second choice) understand the needs of the others before designing the rules.

Everything I see and hear about data governance is from the point of view of the person whose role is management of the data resource. There isn't a single person in the marketing department who would ever conceive of the need for data governance. Of course, we can spend time in learning to talk the marketing language and becoming familiar with marketing problems, then we can show them that some kind of governance is needed and they will agree. They might even agree to invest some time on a committee. Eventually, though, they're going to wonder if this is a good use of their most precious resource--time.

Making laws (standards) is a messy process. Much of the data governance effort is about the process--identifying stakeholders, building consensus, the political side of things, while the standards and processes become a very small box on a big diagram. My thought is that we don't even know the stakeholders until we understand the processes. The political side is essential, but there is a lot of good we could be doing if we would focus on the processes and standards.

I keep saying this because, while there may be similarity in the way two corporations handle governance, I have serious doubts whether it will ever be possible to export one company's solution to others. The political implications of forcing an outsider's will on a population would cause "failed" to be stamped on the effort nearly immediately.

Bottom line: You're on a burning platform. Don't wait for someone to save you. What do you have? What can you do? Do it!

Winning the World Series

2009-07-02T09:30:00.002-05:00

How do you win the World Series? Implementing good [data] governance is a lot like winning the championship, whether World Series, Super Bowl, Stanley Cup, US Open... It's a big goal accomplished through thousands of smaller ones. It's also similar to achieving Level 5 on the CMMI.
Let's take a look at how to win the World Series and see if we can learn anything about how to implement governance.
The first thing to recognize is that it takes an entire season--it isn't done in only a couple of weeks in October.
It takes the cooperative efforts of an entire organization.
Management must find and hire the right set of talents and abilities.
Coaches must turn the collection of talents and abilities into a team.
Each person must have the desire to excel as a part of a team.
Each person must come to share the vision of Winning The World Series.
Leadership must emerge to keep the vision in front of everyone.
We must win today's game (over and over again).
I must become a baserunner (if a batter) or keep the batter from becoming a baserunner (if in the field). I recognize that I won't succeed all the time but that doesn't keep me from wanting to succeed every time. Winning today's game means winning more of these smaller contests than I lose.
In order to win the small contests, I am prepared. I practice, I consult coaches, I talk with my teammates. I cultivate the knowledge as well as the abilities required.
I choose equipment that fits my needs.
I learn to win the contests in my own environment and in foreign environments.
I cultivate personal and team consistency.
When all of these things are done consistently and well, we find ourselves with at least the opportunity to win the World Series when October finally arrives.
What jumps out at me in all of this is the need for planning, preparation, patience, desire and commitment. I'm sure that no one out there believes that a governance implementation can be launched and completed in a few weeks. How long do you think it should take? Months? Years? Decades? Since their is no finite season or schedule to constrain us, maybe the best answer is that it will take as long as it takes.
That said, it seems incumbent on us to decide how we'll know when we have completed the task we have set for ourselves. I realize this seems self-evident and trivial but as I visit with people and groups I have developed the impression that the stable state is still undefined. What that means is that we are eternally implementing when we should be improving.
In the absence of another definition of the stable state, I have offered two principles for that state:

No [data] pollution and
No nasty surprises

Since these represent a whole series of contests, each of which we are committed to winning, while understanding that we won't win them all, another important property of the stable state is that it embody learning and self-modification (improvement). When we have created the property and the principles, we will have "won the world series". The next step is to understand the contests that make up "today's game" and equip ourselves physically, mentally and emotionally to win those contests.

Today's problem is that we are losing contests that we don't even know we're involved in. There's an old poker adage that says "If you look around the table and don't recognize the sucker--it's you." In [data] governance terms, if we look around and don't see the loser--it's us.

Feng Shui and Data Governance

2009-06-27T16:49:00.003-05:00

I don't know if feng shui is classified as a science, a craft, an art or if it is part of a spiritual discipline, but whatever it is those in the midst of setting up data governance may benefit form some of its principles.

Wikipedia (http://en.wikipedia.org/wiki/Feng_shui) provides a nice discussion of the history and principles of feng shui. If we think of data and information as the qi (ch'i) or life force of a business, we can see parallels. We want to encourage the life force to flow through our "dwelling" (the business). We want to hold the good qi and and allow the bad qi to pass through without being retained.

Feng shui begins by studying the physical geography of the dwelling. If the structure is not yet built, the site or sites are examined so that it may be built to take advantage of the natural path of qi. If the structure is already built, everything about its natural geography is taken into account before any suggestions concerning arrangement of furnishings are made.

Holding an understanding of feng shui in one's mind while architecting a governance solution might lead to

spending more time understanding what currently is before seeking to change it
including all necessary elements (5 in feng shui) in the design such as process, meta data, master data (conformed dimensions), data quality, metrics, intelligence, ??
acting small but always within a larger vision
recognition of yin and yang (actor and receiver) and the need to consider both within all of the elements

We need our business qi, data and information, to flow freely into the business, through every part of the business, and out of the business. We need to discourage bad qi by showing it for what it is and directing it back where it came from. We need to create harmony within and between business units and functions for the well-being of the business as a whole.

I realize that some readers have already dismissed this with a snort and a sneer. I'm not saying anything about the effectiveness of feng shui. What I am saying is that the goals of feng shui should be the goals of data governance and that it is possible to discover some clues as to good approaches by studying something that has been around for more than 5000 years.

Governance and Control

2009-06-24T21:58:00.004-05:00

There seems to be some confusion in data governance circles concerning the application of governance--how to make it work. I sat through a tutorial at a recent conference in which the expert emphasized the need for authority as the key (or a key) to successful governance.

It was never clear to me what the scope of this authority was to be or how it was to be used. I finally asked the question, "Authority for what?" You may have heard that responsibility without authority is the recipe for stress and burnout. I thought to pursue this line of thinking as a way to discover what was meant by data governance. If I know the nature of the authority, I should be able to deduce the nature of the responsibility. The question never received an answer. What I got was blank looks.

I felt a strong need to get to the bottom of this since the word "enforce" or "enforcement" was also used several times. I was becoming extremely uncomfortable.

Friends, if people do not accept governance and cooperate with it, then the governance model needs to change. We do not need enforcers. We need arbiters, mediators and facilitators. More than anything else we need teachers. I've heard it said that we all do the best we know how and when we know better, we'll do better.

Controls and attempts to control do not work in governance. They only create bottlenecks and delays that encourage people to find other ways. In our local civil government, we call it red tape and bureaucracy. For example, building permits are required for many home improvements. The reasons for this requirement are excellent. The permit and the resulting inspections (audits) protect the current and future homeowner by insuring that the project is safe. In spite of the obvious benefits, many do-it-yourself homeowners avoid the permit process because the process is obscure, the standards must be discovered, it can be inconvenient, it adds to the cost and is known to produce delays. Furthermore, the only way for the scofflaw to be caught is through an inspection and the authority has no reason to inspect other than the permit. Note that contractors licensed by the authority are much more likely to comply.

Contrast this to the governance of traffic on roadways. Standards are clearly displayed, drivers must pass a licensing test demonstrating both physical capacity and knowledge. Law Enforcement (To Serve and Protect) is primarily tasked with monitoring compliance (which their mere presence guarantees). Compliance metrics are gathered via various kinds of technology and governance changes (to speed limits, traffic signals, etc.) are made based on these audits. What if we had a committee at each intersection with the sole authority to direct traffic?

As you can see, governance requires an initial framework (competence, licensure), a coherent set of standards (coherent in the sense of both understandable and integrated), and monitoring/audit capabilities. Anything else is extra and may even get in the way.

The result of good governance is a community that enjoys consistency, predictability and safety and is mostly free from nasty surprises. The authority that is present is passive and present only to deal with issues that don't fit within the governance structure. If authority is needed everywhere, there is no governance anywhere.

Guerrilla Governance

2009-06-22T09:00:00.003-05:00

In the March 14, 2009 post "Guerrillas and Governance" I introduced the notion that, because of long-time inattention to the needs of the people/workers on the frontier (organizational boundaries), systems of governance will have been developed there and may have been in use for a long time.

In most cases, this governance will be relatively crude and inadequate. In the modern context, it might be something as simple as "We don't accept those after 2 PM so that we give ourselves time to get them done before 5:00."

What we, as guerrilla leaders should perceive is in two parts:

This group is dealing with a problem and has a "process" in place for doing so.
There is a problem. It is recurring. It has a cost.

If we feel the need to introduce a new level of governance that eliminates the problem rather than dealing with it on a repeating basis, we must take into account both of these parts.

There is a ready-made community here and they have banded together for mutual protection. We dare not dismiss that fact or we will create opposition that will resist us to the bitter end. Until we take the time to make them feel (not just understand) that we really want to help them with their problem--not ours--they will resist all of our efforts.

The dialogue goes something like this:

you: It looks as though you are experiencing problems with [form, file, request...].

they: You wouldn't believe the kinds of /@#*(^ we get. And it's most of the time.

you: So what do you have to do when you get one like that?

they: When that happens, we have to [lists multiple process steps needed to remediate]. That's why we have to have a cut-off at 2:00.

you: So, if I understand this right, you are getting unusable or unacceptable input from [another boundary function]?

they: That's right. They just don't seem to care how much we have to work.

you: What happens when you complain to them?

they: They just say that it's their job to generate [forms, files, requests] and it's our job to process them.

you: I think there's a good chance that we could guarantee that you wouldn't have to do any of those process steps you told me about or, if you did, it would be rare. Would that make your lives easier?

they: Absolutely. How would you do that?

you: First, we should put together a meeting. I've already talked with them and, believe it or not, they are dealing with similar problems and similar frustrations. I think the solution to your problem is the same as the solution to theirs. To make sure we need to meet because there are still a couple of things I need to get clarified. Will you help?

they: Tell me when and where. I can't meet on Tuesdays at all.

And so it begins. You will use their pain to elicit their cooperation. Their cooperation creates a new community. Community action guarantees compliance. A newly empowered community is a breeding ground for improvement of many kinds.

This is guerrilla governance. The only requirement to get started is a goal. You will need to be able to articulate the goal over and over again in many different dialects. In many cases, you will only want to expose the part of your goal that your audience is able to comprehend. Never try hide the fact that there is more. You'll simply answer all questions openly and honestly and never insist that anyone needs to understand your perspective. "We'll improve our understanding as we go." is a good way to postpone dealing with difficult questions until more education has occurred.

Always remember, you can't do this without them. Their commitment is vital. Talk freely to management about progress and remember that management has pain as well. You're a leader.

Coaching

2009-06-11T11:11:00.004-05:00

It occurs to me that many people probably don't understand what coaching is or how they might benefit. Since I am advertising myself as a data management coach, the first task in marketing myself may be to do some education on what should be expected from a coach and differentiate coaching from consulting.

If your only exposure to coaching is youth activities or watching your favorite team on TV you may have an idea that coaches call the shots, that they direct, and are to be obeyed. Nothing could (or should) be further from the truth. My job as your coach is to understand what your capabilities are (as well as those of your "team") and to use that knowledge to help you find ways of attacking your goals that are likely to lead to success.

You might also have developed the idea that coaches are cheerleaders and that one of their main jobs is motivation through exhortation. Again, not true. While I will be quick to affirm strengths and celebrate success, I will not create unrealistic expectations. A coach's goal is to help you to understand the most effective ways at your disposal for addressing the problems and challenges that will confront you.

A youth soccer example will illustrate. If you are fast and by nature aggressive, you can succeed as a defensive player by attacking the ball and taking it away from your opponent before they have a chance to score. If you are not the fastest player on the field and are a bit passive or hesitant, you can still produce a good result for your team by merely staying between the ball and the goal and delaying your opponent until help arrives or forcing the play out to the edge of the field.

In data management, similar principles can be applied. An aggressive, direct approach may succeed for some while a more calculated and collaborative approach may work better for others. In any case, you will want your coach to be able to help you find the successful path which calls for experience as well as expertise on his part. One of the least appreciated values a coach provides lies not in what you do but in what you DON'T do. Your coach wants you to be successful and will help you avoid situations in which you can't or are unlikely to succeed.

You have knowledge, talent--all the raw materials for success or you wouldn't be where you are. Sometimes what you don't have is time or some specialized expertise and in that case you will want a consultant who can come in and get 'er done. But sometimes this is counterproductive because you won't be able to keep calling the consultant back each time you need a change or repair. If you have some time, a coach may be a better alternative since he will leave you with success strategies and tactics that you can continue to apply.

You want your coach to be at your shoulder, ready to answer your questions but also to be asking you questions continuously to help organize your thought processes. In that sense a coach is more than a teacher and more than a mentor. A teacher will not be responsible for the application of the subject matter. A mentor may be standing by at the end of a phone line. The coach will be there with you.

Note to Sec. Sebelius

2009-06-08T12:05:00.003-05:00

Secretary Sebelius,

I appreciate very much your stated position (according to Healthcare IT News) that technology adoption is in healthcare is not enough, that interoperability of technology is also necessary for healthcare reform. I wonder how much you know about interoperability of healthcare information systems. I wonder only because there is nothing in your published biographical information that leads me to believe that you have any background-in-depth in a technology discipline.

I don't mean for this to sound like criticism--it isn't--I think your position is a correct one and your advisers have done a good job. I wonder if you are aware, though, that there has been talk of interoperability for several years within the healthcare marketplace and there have even been claims of the achievement of interoperability. There has even been a "certification standard" published purporting to validate system interoperability.

All of this isn't worth the effort it took me to type the words. The reason for this "much ado about nothing" is simply that there is no incentive within the marketplace for the level of cooperation it would take. Technology of all kinds is the cash cow of healthcare and no one involved has any reason at all to kill that cow or even to bring it into the barn.

In the early 1980's, the Department of Defense had a very similar problem. Each branch (Navy, Army, Air Force, Marine Corps, Coast Guard) had its own procurement structure and its own pet contractors. There were no standards and all that was necessary for a contractor to be successful was to maintain some level of credibility with the procurement officer(s) involved. The result was that (for example) Army units in the field couldn't talk to units of other services because their communications equipment was incompatible. Logistics was a nightmare because of the variety of spare parts that had to be maintained and computer systems incorporated the "dialect" of the purchasing service and could not exchange information with the systems of the other services. This is the surface of the problem. The technological diversity went much deeper as well to the point where it was a major procurement effort to get two systems to cummunicate.

NASA was developing plans for an international space station and realized that they were going to have to fundamentally change the way that systems were specified, developed, and implemented if there was ever to be any hope of success.

The Defense Dept. took control of the situation through an initiative called Software Technology for Adaptable, Reliable Systems (STARS). DoD mandated that processes and methods (and their documentation byproducts) as well as tools and other technology used in the creation of systems be standardized for the purposes of reducing costs and delivering a level of interoperability.

Healthcare operations and all of their vendors--virtually everyone outside the walls of the DoD and the Software Engineering Institute at Carnegie-Mellon University remain blissfully unaware of any of this history, all the while enjoying its fruits.

I want you to know that I believe interoperablity can be obtained, but not without the institution of new paradigms and some major upheavals in the technology vendor community. I have dedicated 13 years of my life to laying some foundations where I can and I fervently hope that you have the commitment and the political will to see this through. Without that, government efforts are likely only to increase costs.

Changing

2009-06-01T06:45:00.006-05:00

I have been doing a few presentations of late on the subject of "Guerrilla Governance" which is about the application of guerrilla principles to the quest for good [corporate and data] governance.

The central theme is commitment founded on a vision and how to use that to create community, communication and credibility. Through it all, the message is that complaining, wishing and waiting has not produced results, is not producing result and will never produce results.

I learned that I already have what I need and now I'm working to get that message out. The raw material, the resource used to power the change we need is in plentiful supply. It's the pain, frustration, and lack of fulfillment encountered in everyday work life. Even if I do not feel it, others all around me express these feelings every day.

The norm of work life is approval-seeking. The rare business has created a system of standard processes and metrics that frees its employees from the need to seek approval. These are the CMMI Level 5 companies and the Malcolm Baldridge Award winners. The vast majority invest a handful of people with authority by virtue of a title and force everyone else to seek their approval in order to change anything.

If you get this, it's up to you to change it. Alignment is the grail sought by management. It is thought that alignment will produce the "well-oiled machine." The problem is that the "folk wisdom" of the executive suite and the board room seems to be that the basis of alignment--vision--is something best kept close. Rather, alignment springs from a common vision. A shared vision is the shortest path to alignment.

If the leader of the company isn't actively sharing their vision with each and every employee of the company, then it isn't happening. Reliance on staff meetings to promulgate the vision is very much like the old party game of telephone. Who knows what the person at the other end is really hearing? There are other visions out there--I have one myself. Whoever you are, whatever your job, I urge you to hold yourself accountable to the grandest vision within you until it is replaced by one even more grand. Be responsible for the change you need, but remember that the change IS you.

The Status of [Data] Governance

2009-05-28T10:31:00.007-05:00

In the past 10 days I have addressed DAMA chapters in Iowa, Minnesota and Wisconsin on the topic of governance. As part of the presentation, I attempted to learn the status of governance initiatives within the participating organizations. These organizations ranged from the small (< 1000 employees) to the very large.

There was no enthusiasm surrounding the state of any governance activity and only a few who were even willing to say that they had any governance in operation.

As a result of this admittedly informal survey, I am willing to state that recognizable [data] governance is virtually non-existent. But wait, you say, I have read press releases about enterprise data governance being rolled out at some really big-name corporations.

I have done some personal research at one such big-name company by interviewing in person or on the phone several individuals within the corporation who are directly involved at several different levels. This more formal survey revealed a considerable degree of anxiety among those directly responsible for some piece of the effort. At the same time there was a sense of interested detachment from those involved in the "governance" of Enterprise Data Governance. Meanwhile those who should have been heavily involved by virtue of their job responsibilities but weren't formally part of the structure had a pretty fatalistic attitude about the whole thing.

The overall impression I took away was in the nature of the Emperor's New Clothes. The comment I heard most frequently (from every one of those interviewed) was, "We're making progress." These people all have a history with this company that goes back to 1992 and earlier so I imagine that when they stop to consider the difference between then and now, progress of many kinds is apparent.

I'm all for making progress, but I have to wonder if we aren't too easily satisfied. If those who set out on the Oregon trail had been satisfied with progress at this rate, their great-grandchildren would have been overtaken in their Connestoga wagons by the construction of Interstate 80. It would seem that one of the worst things that can happen to an organization (company, corporation, institution) is to create a bubble within which to operate.

Many organizations today seem to have done this and those with the most identifiable corporate culture and the strongest brand have done the most to create their own distinct and separate reality in which "making progress" is not only good enough, it is the pinnacle of achievement.

In the USA, we have a model of governance that was the first of its kind and, because of the model's structure, is viewed as something that can be duplicated elsewhere. We are accustomed to thinking of this model as democracy but that is a mistake. John Adams (successor to George Washington) got it right when he analyzed it this way, "We are a government of laws and not of men."

The corporate equivalent of laws is standards. Governance based on men (and women) runs on approval while governance based on standards) laws runs on compliance. Clearly compliance based governance where compliance can be verified by audit is vastly preferable to approval based governance in which approvals are both slow and subject to reversal for any of a myriad of reasons.

No system of governance is perfect, but a living system in which standards are subject to periodic review and can be modified to accommodate external changes, must be preferred over the alternative. Where will the leaders come from who will do for corporate governance what Jefferson, Madison, Adams, Franklin and others did for national governance?

BI and Re-Branding

2009-05-22T12:38:00.003-05:00

And by the way, there seems to have been a re-branding of "BI" for the mass market.

As recently as a year ago, BI (Business Intelligence) meant something special. Different kinds of information displayed for very specific purposes. Now it seems to mean "reporting" (although "BI" is a lot more edgy than "reporting" so probably worth more money.)

If you are buying "BI" and paying BI prices but getting basic reporting then you are a victim of mass marketing and re-branding (see previous post).

Re-Branding

2009-05-22T11:35:00.003-05:00

Mass marketing seems to be an American (United States) invention and may be the single most impactful innovation of the last century. Please note that I made no value judgment. We each have to make up our own mind whether the impact was positive or negative.

Certainly, it has served to increase wealth so if your standard is ROI then you would have to view mass marketing as a positive development.

The downside effects are much more difficult to measure--plus virtually nobody wants to talk about the downside. Just as clearly, people have been convinced that they "need" something that they didn't even know existed. To that extent, a lot of raw materials were consumed and a lot of byproducts were produced because of the success of marketing.

Perhaps the biggest downside from my own perspective is the continual re-branding of technology practices. The effect of this is that everyone is on their heels all the time. We are bombarded with new acronyms and substantial effort must be expended to learn about them. Unfortunately, the common result of this effort is the realization that this "new" thing is really a 20 year-old concept with a new name.

Those not equipped to realize this invest even more time and energy in trying to make this new thing be their magic carpet without ever discovering what it was that kept the rug from flying the first time around. Technology folks are easy marks since they often are completely unconcerned with history--newer is obviously better after all.

Lots of money is being made but society is paying the price. Healthcare is the perfect example. Technology churn is costing billions at a time when everyone recognizes that costs are out of control. No worries though, we'll just focus all the lights on insurance premiums, thereby diverting attention away from the decision makers.

Governance in Context

2009-05-21T11:56:00.003-05:00

Every time I talk with a group of people I come away more convinced that the lack of governance is the underlying reason for the "failure" of so many technology initiatives.

I have addressed two DAMA chapters this week and those in attendance included a CIO, managers, data and business architects, data modelers, warehouse analysts... I would guess the average age of the groups to be about 40 and they came to their current job roles for the most part via a technology path. They shared stories of corporate in-fighting, closed-mindedness and self-centeredness that have produced some incredibly poor decisions (or non-decisions). The general mood seemed to be resignation if not acceptance. They asked repeatedly what could have been done differently.

Data governance is on many minds today because of some horror stories involving costly mistakes that were avoidable (see Confessions of a Data Governance Sponsor). Anyone can find success if they find the right expert partner and if you are really committed to governance, that's what you'll do. The devil is, as always, in the details.

First of all, how is commitment generated? How does the commitment become focused on [data] governance? Finally, how can we envision and create something that is independent of individual champions (not built on a cult of personality)?

Imagine that we live in a nation without governance--the strongest or the most charismatic become "warlords", accumulating bands of adherents who follow orders and share in the spoils. Now imagine that we are somehow able, through commitment to a vision, sacrifice and patience, to create a system of governance in one city. Present day Afghanistan springs to mind as a real-life example. What will happen to the governance in the city if it can't be extended into the rest of the country?

Now imagine an example of a country with good governance in which a single city has resisted or expelled governance. Hollywood has produced many examples of this story.

Which scenario has the best chance of producing uniformly good governance?

One of the companies represented at one my presentations is a very modern one in which all employees are "team members" and much effort has clearly been spent to create a uniform image. The team members are proud of the identity that they share. At the same time, this companies refers to its business units as "pyramids." What message does this convey? I can think of few metaphors that indicate monolithic autonomy better than pyramid (unless perhaps "silo"). If I work in a terrain of pyramids and want to institute governance, I really have only one choice--to create governance within a single pyramid. This is analogous to creating governance in a single city of a lawless land.

Only the person responsible for all the pyramids could turn a commitment to governance into a common system of governance in all the pyramids. Of course, if we had an organization that had a presence in all of the pyramids, we could delegate the task of creating governance to them. They might even have a chance to succeed if everyone understood that their efforts had the blessing of the supreme leader.

I believe that data governance is somewhat analogous to streets governance or sewers governance. It is absolutely necessary for the community but doomed to fail unless the vision and commitment become widespread. Neighborhood Watch can go a long way toward eliminating unpleasant surprises within a community and one successful neighborhood watch will stimulate surrounding communities to emulate this "best practice." There is a real limit, though, on what a neighborhood watch can accomplish and many, independent and uncoordinated such efforts will provide many gaps through which unpleasantness will find its way.

A company that is unable or unwilling to do process management does not have a sufficient level of governance to support a data governance initiative. If standard processes are anathema, forget about data governance. There must be a level of maturity to set the stage for successful governance or there must be a universal system that indoctrinates new community members with governance principles and assigns and explains their role(s). Military organizations understand this. Our school systems understand this. Every corporation has new employee orientation programs, most of which contain no reference to standard process and the employee's responsibility to adhere to standards.

"We are a government of laws and not of men." except when we step into the corporate bubble. At that point, it is understood that we are to work for the approval of the boss.

The Problem With Telepathy

2009-05-15T08:27:00.002-05:00

The problem with telepathy is that we rely on it but it isn't real, or it's sort of real--well, you know what I mean.

I have noticed that over the past decade or two we (humans in the United States) have virtually abandoned communication as an active effort. The expectation today seems to be that whatever I received must be what what was transmitted. We commonly leave an interaction in one of two states:

we have more questions than we had prior to the interaction and we take the questions to others (who typically were not present and never even had the benefit of the transmission)
because we didn't understand what we received, we label the transmitter as a poor communicator (or an idiot) and assume that we don't really need to know what they were trying to say

It's the rare individual who actually takes an active role during the interaction to ask for clarification or context.

When someone says "like, you know" and we smile and nod, we are either relying on telepathy or intuition or body language or prior knowledge or we simply don't care and we just want to get away.

Obviously this isn't really a recent development--not even if you consider two decades "recent"--but it does seem to me that the problem is worsening. I sit in meetings and watch people. They get wrinkled brows briefly and then they disengage. I know that they should be engaged as stakeholders, but they aren't. What's the problem?

One thing that we could all work on is finding ways to ask questions that--but wait, that would be active communication.

One of Stephen R. Covey's 7 Habits is "Seek first to understand, then to be understood." [italics are mine] The other six are personal and could be honed by a hermit. This one actually assumes relationship. Relationship is a real thing, unlike telepathy. I'm going to be a lightening rod here but anyone whose idea of "working on a relationship" is based on the central idea, "You don't understand me." is going to be disappointed repeatedly. This applies to every kind of human to human relationship and is the essence of Covey's effectiveness habit.

Before I lose myself in this, I'd better stop. My advice to anyone is

learn what active communication is (if you aren't participating, you aren't communicating)
listening (actively) is probably 90% of communication
ask questions when they arise and ask them of the right person
be present in the communication

We're in this together and we are going to sink or swim together. Let's start acting like we know this.

Pardon My Absence

2009-05-11T09:46:00.002-05:00

I have just returned from a 3,300+ mile odyssey whose central purpose was to be present at the birth of our first grandchild. Despite feeling pretty worn down from the drive to Vermont, I am excited to report that the world has been brightened with the addition of Stella Michael (both grandfathers are Michaels).

I'll return to my more normal topics very soon.

Everything From The Center

2009-04-24T09:55:00.003-05:00

Does your life have a center? What is it? You don't have to visit with someone for very long before you can begin to see the answers to these two questions. The challenge then is to avoid associating the person with your perception.

A key principle of relationship and communication is that a person can --and will--change. The change may be profound as in moving the center of their life. In relationship, it is best to assume the best possible motivation for the other person and, at the same time, give them room to make mistakes. It is just possible that, even if we share a common center, are motivated by the same high purpose, and are both "good people", that we might not take exactly the same path to the goal.

In the best of all worlds, we would be attentive to each other and aware when we seem to be diverging. When we see this happening, we would talk about the divergence and the reasons for it. At this point, there have been no mistakes, no recovery is needed. All we need to do is realign ourselves with our shared goal. The path chosen might be mine or yours or some new one created out of the improved understanding derived from our conversation.

These principles are as applicable to groups as they are to individuals. Many a "leader" has brought disaster to himself and his followers by becoming focused on the problem in front of his nose and losing contact with the overall objective. This happens when the leader's center either moves inside of himself (glory, revenge, hate...) or was there to begin with (advancement, wealth, recognition...).

Don't lead your followers into a trap. Keep (or move) your center outside of yourself.

Keeping Your Own Counsel

2009-04-23T07:22:00.003-05:00

"The man who says what he thinks is finished, and the man who thinks what he says is an idiot."
Rolf Hochhuth

What do you think of this sentiment? Would it help to know more about Herr Hochhuth?

I believe that a significant portion of humanity operates as though these two statements were true. I'm sure there are many reasons for this, but the root of all the reasons is fear. Fear is such a powerful emotion in us that many can't even acknowledge it--even to themselves. The result is that it transforms and appears (I like the word manifests) as another emotion altogether.

Earlier I posted regarding controlling personalities and said that insecurity and low self-worth, two manifestations of fear, were at the root of the need for control. When I saw this quote on MSN today, I immediately went back to the control issue. These two statements nullify everything we teach about collaboration and teamwork. If they ring true to you, may I suggest that you ask yourself why.

This isn't a casual investigation. You're not going to be truthful with yourself at first. You'll have to do the kaizen "seven whys" and do the seven whys repeatedly because, if you hope to help yourself and are attempting to be honest with yourself, you will get different answers each time you get to the seventh why. One day you will name a specific thing that is the cause of your fear and you will feel good because knowing is better than not knowing. But this is just the first plateau.

Now you have to ask why this particular thing is producing fear in you.

Each time you reach a new plateau, you will look around and see your world differently.

Do you recognize in yourself the need for control? Do you share your vision, plans, objectives freely and listen to the responses? Do you say what you believe and believe what you say?

How can your company, organization, team function productively if you answered no to any of these questions?

People First--Technology Somewhere in There

2009-04-21T15:10:00.003-05:00

Very briefly, if you don't have the human side in order, meaning attitudes, capabilities, interest, no technology can be successful. Of course, it is possible to define "successful" so that your implementation passes the standard, but if that's how you work, you're probably not reading this anyway.

It certainly does help to have an idea of what kinds of technology benefits you are looking for and the types of technology that might help to deliver those benefits BEFORE you begin to design the change management program. Countless technology implementations costing untold millions (or billions) of hard-won capital dollars have been scrapped or have taken so long that the purchased technology was obsolete before the implementation was completed. The reason (and the solution) is in these two paragraphs.

Best Practice and Best

2009-04-21T10:14:00.003-05:00

It occurs to me that many people may be confusing best practice with best. I thought I might devote a few lines here to separating the two concepts in the hope that better (more productive) use of effort might result.

Best Practice is the survey of how others are doing whatever it is that we would like to do, picking out those that seem most successful, and documenting how they are doing it.

Best is an objective determination, based upon comparison, and using some standard of comparison, of several alternatives within a context.

Humans being what they are, we always have to carefully define the context we are working in. Humans being who they are, the definition is never quite good enough to resolve all doubts. I am cursed with the gift (cursed with a gift--who's with me?) of immediately seeing similarities and differences. Show me a rule and I'll find the exceptions. Show me several random situations and I'll show you the similarities. It's a gift in that I find it very useful--a curse in that it drives other people crazy.

I'll get right to the bottom line. A best practice effort is only going to produce results for you if your investigation examines contexts as well as process, tools, governance... To put it another way, do you have the same history, culture, experiences, skills and attitudes as the organization you're comparing yourself with? If not, that doesn't mean there's no value for you in their practice. It does mean that you're going to have to know more about them and about yourself before you can put your own version of that practice into effect.

As far as best goes, it's time that we gave up on the idea of supremacy. Best in a particular competition among equally matched competitors is still only good for the moment. That's why we have continuous improvement. It's why there are frequent surprises at playoff time.

The best you can find is still just the best that you can appreciate. When we know better, we'll do better.

The Aging Workforce and Your BI

2009-04-20T13:55:00.002-05:00

First, in the interest of full disclosure, let me say that I have passed my 60 birthday, though am not quite old enough to apply for social security benefits.

I saw a piece on the Today Show this morning in which one of those being interviewed made a statement to the effect that agism or age-based discrimination, is to be found at both ends of spectrum. That rang true for me. I guess the only question to be answered would be the placing of some scale on the spectrum so the appropriate remediation planning could be done.

We seen many articles and interviews regarding the characteristics of various generations from Gen Y to Baby Boomers (pretty much the endpoints of the employment spectrum). I always have to remind myself that these "portraits" are generalizations only. It's easy to latch on to a singe trait within a generational portrait and, having seen that trait exhibited by at least one member of that generation, apply it to all within the age group. Then it's only another small step to say, well, if that one is true, they're all probably true.

It's also good to remember that tendencies are statistics. An average (or a tendency) requires that some be above and some below. It isn't necessary for any individual to be average for there to be an average for a population.

This is all on my mind because I have been following some discussion forums recently and have begun to form some opinions about abilities of generational groups based on the content of the posts. I'm struggling against this because, when I stop to reflect, I know I'm making a statistical mistake.

Here's the thing: If it's happening to me, it's happening to anyone (everyone). It's very dangerous to attribute knowledge or maturity-related issues to age. Every young person is not immature or ignorant and every older person is not mature or wise. So this is a reminder that each person must be appreciated for who they are and the unique contributions they are able to make.

There's a corollary: You can't really know a person until you've met that person face to face and have shared some of his/her life. Do you know me because you're reading this or because we've exchanged emails or tweets? There are hard limits to what can be accomplished remotely, without personal contact. Some younger people know this and many older people do.

If you happen to run a company, you have a bigger labor pool than you need right now. This won't be the case for long, especially if you're in a scientific, engineering or high-tech market. People pushed to the side now and left to rust away may not be able to step back in after several years (or even several months) have passed. When you push people out of one end of the pipe without making sure there are people entering at the other end, eventually you will find that your pipe is empty. How will you adapt your business then? The knowledge, creativity and energy that created your business will be needed to transform it. It won't be about age. It will be about maturity, wisdom, creativity and energy.

The Health Care (or Healthcare) Vision

2009-04-17T10:54:00.003-05:00

I have a vision that can only be fulfilled from the office of the CIO. This vision is characterized by

The patient of one is the patient of all
The history will be available wherever and whenever the patient presents
The patient may know immediately about the services delivered and the charges for those services.
The revenue cycle can be reduced to same-day for covered (under contract) services
Costs can be reduced by as much as 50% through attention to the quality of the information captured and used within the system
People will accept a system of standards when they see that their own interests are satisfied
The people in health care are highly motivated and need only a shared vision to be fully productive
Alignment is possible when the vision is clear
Good, reliable, current information is the key to all of this and getting that will demand the careful cultivation of cultural and attitudinal changes.

I firmly believe that this vision can come to pass with blinding speed in an environment of openness and honesty in which people are encouraged to care about and for each other.

Arm Waving and Obfuscation

2009-04-17T08:42:00.004-05:00

"Pay no attention to the curtain!" "Do not look behind the curtain!"

There are two areas in which it seems necessary to ignore the Wizard. One is in the nature of "healthcare" itself. Most spell checkers refuse to recognize healthcare as a valid word, suggesting "health care" instead.

Clearly, spell checkers have not been sensitized to the politics involved. Since the dawn of the HMO nearly 30 years ago, health care has gradually given way to healthcare. What does that mean for you and me? I don't know if you've noticed, but even the physicians have been shifting attention away from their own role toward the insurance end.

More than 15 years ago, I attended a software engineering conference in Chicago at which [former] Surgeon General C. Everett Koop was a keynote speaker. I became disturbed during his speech when he made comments seeming to grant insurance companies a gate-keeping role in health care. When I got back home, I wrote him a letter mentioning my impressions and asking whether that was his intent. Some weeks later I received an envelope with Dr. Koop's return address and which contained my letter. My "gatekeeper" question was circled (red pencil) and "NO!" was written (large) in the margin.

Today, physician leaders don't even blush when they tell us that the solution to our healthcare problem is better access to [insurance] coverage.

Health care can be linked to medicine. Healthcare is a business, pure and simple. When people are talking about money and where it will come from, they aren't talking about health care.

I do not question the motives of individual physicians in health care, but I question everyone's motives in healthcare unless they are talking about profits. Even the insurance companies are being edged out for control. Today, healthcare is controlled by third-party (neither doctor nor patient) vendors of everything from pharmaceuticals, to technology--especially technology. This isn't necessarily a bad situation--it all depends on what the goals are. Pick your favorite goal from this list (or suggest another):

Profit
return on investment
market dominance
a healthy populace
access to medical care
control
benevolence/compassion
giving back
???

If you are a person who has a need for (or interest in) health care, you really do need to peek behind the curtain of healthcare to avoid wasting your time, money and energies.

Standards Clarification

2009-04-16T20:49:00.002-05:00

A bit of a postscript to the last post:

I can almost hear the snorts of disgust. Many in healthcare will be quick to dismiss the last post by telling themselves that "we have standards." I can't allow them to let themselves off so easily.

Of course healthcare employs standards. I was, for a brief time, part of a newly formed HIMSS task force on standards. Healthcare has a wealth of standards, none of which are truly standards in that all use words such as should and unless and if possible.

Healthcare has not seen fit to develop a framework for standards and no ontology by which to bring sense and meaning (and thereby value) to the hundreds of standards vying for attention. In truth, anyone in healthcare can say without fear of contradiction that "we have standards" and none of those assertions mean anything.

If n people or organizations are doing the same work using n (or even n-1) standards, it cannot be said that the work is being done in accordance with standards. This is said routinely by each of the workers but to those who view the work from an objectifying distance, it is quite clear that n standards is no better than no standard.

Measurement of process in healthcare has a long way to go before SPC principles can be applied.

How, by what process, can healthcare practitioners be brought to believe in the power of process standards through which measurement standards can be developed? Whose interests are satisfied by the status quo?

Standards Dread

2009-04-15T10:31:00.004-05:00

Healthcare in general and from top to bottom seems to have an absolute dread of standards. Physicians (many if not most) will flop about like a fish on a hook whenever the word comes up. They fight no matter how the subject is introduced. Whether it be "guidelines", "pathways" or even "best practice", it doesn't matter. As an interested consumer, I find this disturbing on several accounts.

It demonstrates such a me-first, tunnel vision mentality that, if I were given a clear choice, I would run as fast as I could away from this and toward anyone who could demonstrate that they do, in fact, adhere to standards.
It has a huge--and negative--effect on everyone else in the healthcare organization. The physician role is so central in healthcare that, if there is no reliable standard (process) for the physician, then nursing, registration, coding, billing, systems developers..., no one can predict from one day to the next or from one patient to the next what they're going to be asked to do.
It holds the practice of medicine back to a level not much different than was seen in the 18th century. Oh, sure, we have better drugs now and diagnostic magic is performed many times a day using the lastest technology, but the outcome for me, the patient, is so dependent on the physician I see that "bedside manner" often seems to be the most critical factor in outcomes.

In my 60+ years of life on this planet, I have seen that humanity can be put into two groups--those who appreciate standards and those who do not. Further, it has always seemed that the second group is the cause of problems not ony for themselves, but for everyone. Let's take a kind of standard that we're all familiar with--traffic laws. Those who flout the traffic laws are a hazard to everyone else on or near the road. And note that consistency and predictability are key. Traffic laws work because they produce consistency and the ability to predict with some assurance what the other guy is going to do. Those two principles keep everything flowing smoothly and with minimal (and manageable) disturbance.

We do not realize fully, the value of the standards we employ in this country. John Adams is the one who noted that we are governed by laws not men. Bribery is a recognized way to get things done in many parts of the world. Imagine having to find the right person (how do you do that?), paying to get their attention, then finding out that they weren't the right pewrson after all and having to start again. We sometimes go through that with building and remodeling contractors and it makes the evening news. Healthcare isn't much different EXCEPT that we don't get as heated about it because someone else is paying.

I constantly wonder at the inability of the insurance companies to get provider organizations to create and use standards.

Today, we are being told that technology is the key to the healthcare crisis (which is a crisis of out-of-control costs). I am going on record here that technology will only drive costs up unless the healthcare "system", beginning with physicians, learns to cherish standards, utilize standards, trade on standards.

We, the patients, must demand a system in which we can rely on standards to produce outcome and efficacy data allowing comparison of physicians and organizations. It's a sad system in which the only statistics available for comparison are mortality numbers.

Healthcare: More on I.T. Capabilities

2009-04-12T10:49:00.005-05:00

I realize the need to be a bit more specific with respect to capabilities. I have seen a virtually unbroken string of capabilities discussed that related to the practice of medicine or the business of healthcare delivery, but in almost fifteen years of experience in healthcare I.T., I have yet to see a capability goal that is about I.T.

An example may help. Suppose you are the city planner. You could take one of two approaches.

You could develop a comprehensive plan for the city that takes into account various expansion directions and encourages some while discouraging others. The plan would include both residential and business expansion.
You could attempt to satisfy each request that comes before you. You could allow various developers and politicians to set the agenda for the next six months and continuously play one against the other for priority.

In the second approach, a deft manipulator can get by for quite a while. The city, on the other hand, will always be in turmoil and unable to staff its various departments effectively. Budget dollars will go to the current top project(s) at the time of the budget approval process. But, because project priority will be ever changing with the political situation, those projects may start and then go into hibernation. Workers with specific skill sets will find themselves with little to do while the new high priority project has tough going because there is no money to bring in the right skills. Equipment winds up sitting in a warehouse or worked far beyond recommended maintenance cycles.

In the first approach, remarkably modest effort goes into an expansion model. Productive and non-productive alternatives emerge. The productive ones are fleshed out and the unproductive are set aside for contingencies. potential streets, sewers (storm and sanitary), gas, electric, phone, Internet, and other necessary utilities are blocked in. Alternatives and contingencies are identified and capacities are established. With all of this in place, it is now relatively simple to budget for the next year's projects and all will be well as long as we don't cave into to political expediency and begin to allow projects outside the scope of the plan.

There is no problem as long as all negotiation goes on in broad daylight. If a particular out-of-scope project has popular appeal, it will be a simple matter to say, "OK, here's what it will mean for our other projects, for our long term plan and for the budget. If it's still popular, we do it.

Back to I.T. capabilities; server capacity, mass storage capacity, network bandwidth and access points, space, cooling--these are all obvious capacity planning considerations and for that reason they are at least on the radar for everyone. More subtle be no less important considerations include the architectures (technology, network, data, and communications) that we want to have, skills required as we work within the architectures and, not least, the standards architecture that will be the governance glue that holds everything else together.

Depending on your own experience as a technology worker (rather than as a manager), you may not have a "feel" for the people, skills, and experience that will be required, nor even for the implications of various architectures. Again, this is no place to guess. Get your direct reports together. If they don't seem sure or can't tell a story that is meaningful to you, then get some outside advice. The skills and attitudes required for a successful SOA (service-oriented architecture) are much different from those needed in a more "traditional" approach.

If you intend to rely on vendors for everything, you will still need to be able to tell which vendors fit into your architecture and which don't. They will tell you that they are "architecture-agnostic" or that they can fit into any archtecture--don't believe it. Know about the characteristics of things that will fit and things that won't. Attempting to force-fit something that doesn't belong is the quickest way to throw everything you've built into turmoil.

If you're uncomfortable, I apologize for being so open. The answer to getting comfortable isn't avoidance, though, it lies in making a concerted effort to bootstrap yourself. No budget, no resource plan, no allocation discussion--management of one thing is not the same as management of anything. Technology is to big, too complex, too fast-moving to be brought into line by the classic "cost management and allocation of resources" approach. It needs a team approach because there's just too much for one person to know.

Do what you're able to do and know what your boundaries are. Be honest with yourself first of all and with your peers, reports and management. You may wonder if you are the only one at times, but it is not possible to manage technology without honesty. Unlike people, technology can't be coerced or manipulated.

Advice Re; Healthcare IT

2009-04-11T13:27:00.004-05:00

If you are a healthcare CIO or the top IT director/manager and happen to be reading this, or if someone has forwarded this post, and if you feel under some pressure more accomplished (or even to get something accomplished), I have some gentle advice. You're going to get phone calls from angry physicians no matter what you do, so you may as well do the right thing.

It is a fact that, in the history of computing (I.T.), no application or system has ever been delivered to universal acclaim (despite what the marketing people tell you). Medicine has more than its share of curmudgeons and you're going to have to let them become part of the background noise. If you're afraid of losing your job because of the indignation of a few people then you're probably on the wrong career path.

I hope you knew that already, but it can be reassuring to hear it from an "outsider".

Now, if you're still reading, you may be wondering about what the right thing is that you should do. The first thing is to get your priorities squared away. Then you you will have to work very hard to get your priorities into the cognitive space of the rest of your organization's upper management. What are your priorities? What are your 1, 3, 5 year or long term objectives? Are they written down and visible to others? If your answer is no, then your job just became much more difficult. By the way, if you are a CEO, and the only I.T. objectives you know about are the ones you handed out, you may want to re-think your approach to I.T.

What does an objective/priority look like for I.T.?

The big rocks (the ones you put into the container first) are capabilities and alignment. In my career, the most pervasive problem I have seen is that healthcare CIOs invariably focus their attentions outward from their I.T. organization. In nearly 15 years, I have yet to see a single initiative directed at developing or improving capabilities come from the top of the I.T. organization. When such things have happened, they seem always to arise from the introduction of a new application or a new technology. Just once, maybe we could spend some time to actually imagine the kind of I.T. organization we think will be needed five or more years from now. Capabilities, in the healthcare I.T. world come about by accident. The current best case is when there are a few people within the organization doing this kind of thinking on their own.

Of course, when there are uncoordinated pockets of effort, you don't get the other big rock, alignment. By the way, if one of your objectives is something like "reduce or contain costs", you may want to rethink. Cost management is like breathing if you're a manager. To call it a priority or a goal is to divert attention away from the big rocks. Similarly, any objective that includes the word "continue" is a waste of paper. Holding the gains must also be at the level of breathing for managers. You will want to keep measuring to make sure there is no backsliding and take action if there is, but holding onto gains has to be part of the original planning process and should be automatic.

Alignment begins with a picture of your organization now and five years from now. Every manager and supervisor/lead within I.T. must share that picture and understand their own specific role(s) in it. This is something best accomplished as a team. You can't mandate alignment. You will want to decide how you will recognize and/or measure the alignment within I.T. How will you know when you have it? Everyone should understand and be on board. This is the first step.

It is up to the CIO to bridge between the business strategy and the I.T. strategy to make certain that as alignment develops within I.T. it is congruent with (aligned with) the business' strategic plan.

You are fortunate in that your direct reports want all of this as much or more than you do. All they need in the way of motivation is the belief that you are committed to seeing it through. If you don't feel confident that you can produce the I.T. vision, don't attempt it on your own. This is the place for leadership that is confident and sure-footed. Start with a clear and concise vision--it has to be your vision or you won't be able to commit to it, but you can get help to create it.

Healthcare I.T. cannot get where the nation wants it to go as long as are dissociative in our thinking. Things aren't the way we want them to be simply because we say so in a presentation to the CEO or the Board. You know that buying and installing a new system is only the first step of implementation. What we're talking about here is very similar. Forming a committee or naming a Director is only the first small step. If you want to know if you have actually achieved your goals of capability and alignment, I would recommend strongly that you bring in an outside person or group to conduct an audit.

Good luck! You are the leading edge.

Programmers Need Leadership

2009-04-08T14:59:00.005-05:00

Many programmers are also musicians. Many have fluency in multiple spoken languages. Many have a gift for mathematics. It seems that these abilities are somehow related in the human brain. This has been apparent for decades to those who guide students into appropriate careers.

A bit of reflection reveals that these are all somewhat solitary pursuits involving individual dedication and a large degree of creativity. Of all of these career paths, software development may be the most accessible and the most remunerative. Pair this with the tendency of many socially gifted personalities to throw up their hands when confronted with technology or mathematics beyond what can be done on an adding machine and the corresponding inability or unwillingness to grasp qualitative differences in abstractions such as software, and you get a recipe for significant problems.

People who are intimidated by the technology of a personal computer or laptop are glad to grant credibility to the first person who can make the technology perform the desired tricks. The non-IT parts of the business have come to terms with this and simply compartmentalize I.T. so that as few as possible must have anything to do with those people.

Now add an additional dimension: the belief (common these days) that a good manager can manage anything. This may be true, but having worked with and come to know many developers (programmers), the credible manager is an absolute rarity in I.T. Just ask any developer (or network admin, server admin, DBA...). You'll find that a manager who enjoys the credibility and respect of the "troops" is the very definition of raris avis.

The bottom line is that software developers and the associated technology disciplines comprise people who have had to figure things out for themselves and who know that their bosses don't understand what they do. The combination of these two (an ability and an awareness) produces people who have an approach to life that is similar to that of a cowboy or possibly a farmer. They are independent and like it that way. They won't turn from a challenge, even if that challenge is doing something that they believe is appropriate despite known management objections and even obstacles. They simply know that when it works, everything will be forgiven.

And it is.

I am disheartened when I follow discussion boards on the Internet. A provocative question is posed concerning methods and the discussion immediately goes to tools. Terms that have been around for decades are redefined without any acknowledgement of the accepted definition. Adding insult to injury, this is done even by data architects who, more than anyone else, should know better.

What I am seeing in these discussion boards is a playground full of gifted five-year-olds with absolutely no supervision. They are capable of amazing feats, but at what cost? If you are the CEO or the CIO of a company, do you have ANY idea what it costs--not in salary, but in uncontrolled complexity and corresponding maintenance costs--to allow this?

You can't blame the five-year-olds. They aren't the problem. They are doing exactly what they were put there to do. If you can't find managers who are capable of establishing some level of respect and control, then you must at least find leaders among the children and give them a mentor.

Even mathematicians respond to this. Remember the Manhattan Project? This project produced the atomic bomb that ended WWII. Dr. Robert Oppenheimer was the mathematician/physicist leader who guided the work and Gen. Leslie Groves of the Army Corps of Engineers was the administrative mentor. One mathematician, no matter how capable and creative, could not have developed the weapon in the time available. But 5000, without leadership could not have done it either.

Data is available through the Software Engineering Institute at Carnegie-Mellon University on the cost benefit of a managed development team using defined processes. Once the processes have been defined (which takes leadership), the manager has only to believe in and rely on the processes in order to achieve predictable, low-cost results of known quality.

Whatever path you choose, it is you, the executive leader, who is responsible.

Why Management Isn't Enough

2009-04-06T12:09:00.004-05:00

If you've been reading this series from the beginning, you may have noticed that, as of now, there has been no mention of that traditional management responsibility, resource allocation. To review, we started by listing four critical characteristics of business intelligence

meaning (semantic) understood
applicability (use/utility) understood
currency (timeliness) understood
source/lineage/pedigree understood

Clearly, none of these is about allocation of resources. Just as clearly, resources will have to be allocated to create and maintain those characteristics. The point is that management skills do not come into play until the organization has decided that business intelligence is a capability it must have.

Prior to this, we have also discussed three distinct functions that are typically thrown together in the basket labelled "management." They are

Leadership: moving an organization in the best direction by motivating change
Management: making the organization effective by managing costs and productivity
Governance: keeping the organization's productivity high by ensuring consistency and stability

Creating, developing and maintaining the four essential characteristics of business intelligence is going to demand all three of these functions. Leadership will be required to define and effect necessary changes. People accustomed to doing their job as they see fit, may need help in adjusting from that paradigm to one of "best for the organization." In many cases, managers have interpreted "organization" to mean the group for which I am responsible. They will need some leadership to make the adjustment to thinking of the the good of the organization as a whole. The scope of organization will vary depending on the process under discussion.

By now, you may be entertaining doubts. The doubts may center on the organization's ability to carry this off. They may concern whether the right people, experience, skill sets are available. They may also be about whether this business intelligence thing is worth the effort. I can't answer any of those doubts for you, but I can say, without equivocation, that getting to business intelligence is going to demand change and that the change will demand strong and consistent leadership.

The people within an organization are fully capable of governing themselves. Indeed, the United States of America is founded on that premise. We are "a government of laws and not of men" said John Adams. In the corporate world, laws are known as standards. Employees representing various business functions and sub-functions will be the best ones to define the standards that they will follow as long as leaders keep before them the corporate interest. There is a long history of failed attempts to impose standards on people, including those by King George V that resulted in the American Revolution. There is no need to repeat those failures.

Once the standards have been defined and agreed to by those affected, and once they have been reviewed for costs and efficiency by the managers, then what remains is to establish the equivalent of legislative subcommittees to monitor their consistent use and consult with the parties when the standards must be improved or abandoned. The cost lies in the change that produces the standards, not in the governance of the standards. In fact, many organizations already have a function dedicated to the monitoring of standards application. Quality Assurance has frequently been given a bad name because it is charged with creation of the standards as well as their monitoring.

Do you want business intelligence? Do you need current, unambiguous, actionable information with provenance, or can you be satisfied with really cool charts and graphs in three (or more) dimensions and lots of colors? You will have to decide.

The Beginning (5)

2009-03-27T08:04:00.002-05:00

In order to get where we want to be (business intelligence) and have the characteristics we are looking for, we will have to make an alliance with our systems vendors.

Our vendors are grown accustomed to thinking of themselves as "partners" and now it's time to call their bluff. We have learned that the long term success of our business depends on the quality of the information that it possesses. Without reliable information, the most efficient and productive process will fall immediately into non-productive and costly tachycardia. Tachycardia is the disorder in which the heart's productive rhythm is disrupted and it's consistent, productive beat turns into an erratic, non-productive, and potentially life-threatening arrhythmia.

How can our partners help us deal with this? They must be responsive to questions about data models, semantics and relationships. They must show evidence that their internal consistency is at least as good as ours. You may want to audit their process consistency. Companies that have achieved the Malcolm Baldridge Quality Award have suppliers who allowed themselves to become this kind of partner.

You will have a robust Quality Assurance capability and you will want them to be linked with a similar capability in the suppliers' organizations. Quality Assurance has been alluded to in previous posts and the QA involvement is data quality should not be minimized.

Quality Processes Produce Quality Data

Processes that are uncontrolled or out of control are incapable of producing reliable data--whether they are in your organization or in a supplier's organization.

Next time we'll address Management specifically.

The Beginning (4)

2009-03-25T14:03:00.002-05:00

In my experience, those who can do the least about data quality feel most responsible, while those who have quality literally at their fingertips feel no responsibility whatsoever. This creates many difficulties for one who wants business intelligence.

I'm going to collect receptionists, customer support, sales, anybody who enters data into any of your computer information systems under the heading of "the business." We could also include those who collect or transcribe data to/from paper if it eventually winds up in a computer-accessible data file.

Maybe we have done too good a job in convincing the people seated at computer keyboards that they have nothing to fear. In any event, many simply do not pay adequate attention to what they are doing. There are many reasons for this including:

heavy work volumes causing a pace that is too fast for error recognition or for going back to correct errors
inadequate training
inadequate guidance built into the user interface
inattention/distraction

On the other hand, I have encountered way too many instances in which people were actually aware that they were producing garbage but didn't care enough to do anything about it. Sometimes it's sabotaging the people in the next department. Sometimes it's a statement to the supervisor and sometimes it's "so what."

It is true that some data can be repaired after the fact, but the sad fact is that "cleansing" can never be 100% effective and in some applications cleansing isn't even possible--if it isn't captured correctly the first time, there's no going back. Repairing bad data is very expensive for your business. Experts estimate that, for anyone who receives data as part of their work process, from 30-60% of their work day is spent in rechecking or validating or repairing the data so that they can do their job.

The data architect will have worked with the folks to understand their data needs so if we can prove that they have participated in that process and that the architects and developers have complied with their defined processes, then what we are left with is attitude or training as problem sources. These are issues for management to resolve.

One final area for the business to think about: there is a need to include problem recognition in training and provide safe reporting paths for those who do take note and take action. Those at the front line typically have little awareness of the business that they front-end for. They don't know that someone cares or that someone can actually do something to fix a problem that they struggle with on a daily basis.

You will want to include a module on the data resource in the New Employee Orientation. You'll want to take remediation to department meetings to catch all who missed the new employee offering. It must be simple to report a problem and there can be no "grilling" of callers by the support line triage staff. Take the call and send someone to see the problem in person while the reporting employee works. Use a remote desktop capability to watch the problem happen. There are many options, but if the caller is made somehow to feel guilty or foolish or ignorant, they will never call again.

Next time we'll take on the vendors.

The Beginning (3)

2009-03-24T08:45:00.004-05:00

Before we explore the data architect role in terms of our overall goal, I should say that I do not intend to go into depth for any of the roles. I mentioned the need for standard processes but didn't specify any of the necessary processes. For one thing, that level of detail is well beyond the scope of this (or any) blog. For another, the processes and standards must be compatible with the organizational culture. I wouldn't go so far as to say that any path will do, but I will say that, if you live in Minneapolis, there is more than one way to get to Miami. The thing to remember is that there are more constraints as you depart and again as you arrive. We ask, "What's the best way out of town if I'm headed south?" And then we ask, "What's the best route to city hall if I'm coming from the north?" But in between we can take the scenic route, the shortest route, the fastest route, the Civil War route or any other route that seems good at the time as long as we keep our eventual goal in mind.

From the standpoint of business intelligence and our four characteristics, we would want to pay special attention to what the programmers are doing or not doing with respect to definitions (semantics). The data architect will have spent considerable effort in researching and compiling information about the data. They will have learned about how various kinds of data relate to each other for different business functions and users and they will have defined quality rules for each kind of data.

The process standards, to be monitored by Quality Assurance and warrantied by Quality Control, will ensure that the programmers have those definitions and rules in a format that they can use and that they do, in fact, use them.

If your programmers work for someone else, the processes and standards will be about acquisition. They will ensure that the data definitions and relationship rules embodied in the application or system are compatible with those of your business. You are going to have to lean hard on your vendors and they will squirm and plead "proprietary." The best advice that I can offer is to walk away from this vendor. Another vendor will be happy you asked because it will allow them to really get close to you and they will be proud of their quality processes. The ones who drag their feet do so because they aren't able to produce the assurance you need. Proprietary is a euphemism for we don't know.

Become interested in these things. Ask questions. Expect answers that you can understand. Don't accept arm-waving and diversionary tactics. You will be well on the way to business intelligence from a high quality, reliable data resource.

Next time: the "business" role

The Beginning (2)

2009-03-23T14:31:00.003-05:00

Because we want our intelligence at our fingertips, easily accessible, we collect and store data in computer-based filing systems. In return for the convenience of this and not having to store mountains of virtually useless paper (the bigger the pile, the less useful), we have taken on the responsibility of hiring and working with various kinds of technology-savvy people.

At the foundation is the programmer, also called a developer or a software engineer. These are the people who actually create the scripts for the computer to execute. 20 years ago, programmers were visible to the rest of the organization. Today they are typically segregated and largely invisible except to the CIO.

These are the people who create the screen forms and buttons and functionality that your front line employees actually use to capture and look up information. In larger organizations today, the programmers do not create the filing system for data. Instead, that is designed and built by someone else (the data architect) and the programmer merely connects to it. The power that the programmer holds in terms of our four characteristics of good BI is the power to knowingly or unknowingly subvert the quality of our data.

This is a good time to introduce the concept of the data resource. It is essential that today's business view the data that is captured, modified, stored, retrieved and archived as a resource in the same way that capital is a resource or buildings and property is a resource or employees are a resource. The business must devote the same kind of attention to the data resource as it does to the financial resource of the company. Neglect or failure to do so will render the data resource valueless at best and a liability at worst. In between those two extremes, the business will experience increased costs as your workers struggle to get the quality they need from the data that feeds the processes they work within.

So the programmers who build screens and functionality that allow corruption into the data resource are slowly destroying the business just like termites in the framing of your house. It is vital, in order to end up with the BI we need, that programmers employ processes that are controlled for quality purposes. Employing process standards for programming will not guarantee that our four characteristics will be delivered, but NOT doing so will guarantee that we will never be able to produce the BI that we need.

Before we wrap up this installment, it's good to recognize that there are two kinds of programmers; those who work for you and those who work for someone else from whom you purchase or license the programs (or software or applications or systems or...). No business today is unaffected by programming.

When you are getting control of your processes and instituting quality assurance, you will want to ask the same questions of your software or application vendors. Today, they are often unwilling to talk about this and will use deflectors like "proprietary" to avoid the questions. You need to ask yourself whether you can bet your data resource on "proprietary."

We'll get into this a little deeper next time when we discuss the role of the data architect.

Begin at the Beginning (but with the end in mind)

2009-03-22T12:17:00.003-05:00

Business intelligence, data warehouses, data stores, stewardship, governance... Where do we begin?

Let's begin by assuming that we want to have something we can recognize as "business intelligence" when we're done. The most important characteristics of business intelligence will be.

meaning (semantic) understood
applicability (use/utility) understood
currency (timeliness) understood
source/lineage/pedigree understood

There may be other characteristics that are of specific interest to a business customer, but these will put us squarely in the middle of the ballpark.

Before we go any farther, this is going to take more than one post so let's think in terms of a series of posts addressed to specific roles that, together, create the governance road map.

If business intelligence is the goal, then we must begin with the business and we'll start right at the top with the CEO. Of all those involved in the business, the CEO has the greatest power to influence, for better or for worse, the attitudes and motivations that will eventually produce business intelligence. Others in leadership roles will have similar effects to lesser degrees.

Mr., Mrs. or Ms CEO, begin by absorbing the four characteristics above. Ask yourself what it would take for you to feel good about the intelligence presented to you. Once you feel comfortable about your ownership interest, you will need to become single-minded in your pursuit of these characteristics.

Practice asking for proof, for information about the intelligence (which will be known as meta data by some of your staff) and insisting on answers that give you comfort. You may view this as self protection, which is a very good viewpoint to have. In fact, if you are consistent in this, you will find that Sarbanes-Oxley is a piece of cake for your business.

This is where it will start. The next post will address sources of the information that will eventually emerge as business intelligence.

Fear, Accountability and Approval

2009-03-19T09:23:00.002-05:00

Fear is an interesting thing. Fear is an emotion that is at the root of many other emotions. If negative emotions comprise a spectrum, then fear is like the sunlight, which, passing through our situational prism, produces the stress, anxiety, mistrust that we actually feel. It takes a lot of self-examination and hard work to be able to let the lesser emotions go and find the fear and its source.

Of all the ways that fear manifests, perhaps the most destructive for a business is that of controlling behavior. The need for control is based on feelings of inadequacy. Many people feel inadequate and still manage to function well in a cooperative environment. Sometimes, though, a person finds himself in a position that he never dreamed of being in and inadequacy, fueled by the fear of losing it all (by proving that he really is inadequate) creates a desperate need for control.

This person will find a way to insert himself into as many important committees as possible and will create new committees if there seems to be a gap in the information flow. This person can't tolerate subordinates who are successful because they become a threat. They have a dislike for group contexts and prefer to use one-on-one meetings to better control the message.

The single worst thing effect of controlling behavior is that the controller manipulates everything so that he has the key decision. This produces several negative impacts including

an entire organization is slowed to the pace of one individual
decisions are arrived at through discussion with peers rather than with knowledgeable subordinates
information needed by subordinates may be concealed in order to preserve the decision authority
frantic scrambles to meet deadlines arrived at without benefit of process
no closure on "projects" because of information hiding and diminished credibility
frustration among subordinates--although a really talented controller will be able to keep this frustration focused among and between subordinates
after-meeting meetings among subordinates for the purpose of validating perceptions
much talk about accountability without any accountability
lack of standards because control diminishes in an objective environment

Do you have someone like this in your organization? How do you deal with it?

One approach might be to create standards around process and measure compliance. With good, useful and actively used standards accountability can be made real. Without them, the best we can do is approval. Accountability is objective. Approval is subjective. Accountability creates no fear. Approval is all about fear.

Your best people have a set of internal standards to which they hold themselves accountable and they won't stay long in an approval environment once it becomes clear that they will have to compromise their standards. The ones who do stay...

A Hammer in Search of a Nail

2009-03-18T13:52:00.003-05:00

Are we all familiar with this metaphor? Abraham Maslow ("Hierarchy of Human Needs") is said to have originated this, but it hardly matters. The concept rings so true that it has probably been known since the first tool.

I have lately participated in some discussions on LinkedIn with the result that I now believe that the vast majority of workers in information technology are in possession of a tool that they are seeking to apply to every problem that they encounter. The really dangerous ones are creating problems to use their tool on. Oh, wait, there's a name for that--it's called marketing.

Notice, too that I said "in possession of" a tool. It is apparently no longer necessary (if it ever was necessary) to actually be skilled in the application of your favorite tool.

I once worked with a guy--a programmer in this case, but I'm not looking to single out programmers--who said something rather like, "We don't support the business. We are the business." The business was rail transportation and it is true that if all of the applications used by this particular company were to suddenly disappear, those "left behind" would no doubt have had to cease operations until they could be reorganized.

That's not the bad part though. the bad part is that this is a really good example of just how far the notion of "where's the next nail?" can take us. When I believe that the world as I know it is held together by this tool that I hold in my hand, I am in the midst of a dissociative process. I don't go home and act this out. When I'm not at work for example, I'm just the guy next door. When I do get to work, though, I'm still the guy next door--it's just a different door. We--most, if not all--live in two separate realities. About the only thing that keeps us from being diagnosed with a dissociative identity disorder (DID) is that we (usually) remember what happened in the other world.

I could go on at length but the bottom line is that, not only is our work life a separate reality from our real life, but there is a completely different reality in the executive suite than there is on the floor, and (for me) most importantly, a separate one for I.T. Leave aside for a moment, the variety of realities we might encounter as we go from networks to servers to DBA, to data architecture, to development to QA--it is absolutely amazing that we get anything at all accomplished.

The poor data architect finds himself stepping into and out of a dozen distinct realities every day. It is certainly a defense mechanism to take refuge in a favorite tool--the "data model." This is the talisman used to shield against the swarm of alternative realities. Unfortunately, the tool was designed for a different purpose, to capture and integrate all the different realities. Nails come in many forms, too.

Relationships

2009-03-17T15:25:00.001-05:00

Check out this SlideShare Presentation:

Relationships

View more presentations from mpmeier.

Assistance vs Solution

2009-03-17T09:46:00.004-05:00

How can we believe that someone can give us a "solution" when we haven't even been able to frame the question? Are we that simple minded? Have we been brought to this--that we are puppets, manipulated by marketeers?

We all have need of assistance at one time or another. This is a good thing to recognize. I've run into something I don't understand. I'd better find someone who does understand so that I can move on.

If we feel ill, we might look for some relief from the cold remedies or pain relief aisle at the grocery store. We might even go to a drug store and ask for help from the pharmacist on duty. We may do some research on line or at the library in search of relief. Do we expect to be cured like this? Most would probably admit that they aren't seeking a solution--just some relief from a particularly unpleasant symptom while they wait for "natural" healing to happen.

Some, on the other hand, will look for a "solution" by visiting a medical doctor (or maybe a chiropractor). They will get relief from the most unpleasant symptoms with the advice to "do this for a week." The result is the same except for the cost. Of course there are times when consulting an expert is indicated without question. When I have severe, unexplained pain or profuse bleeding or when a bone suddenly develops a new joint, then it is time to involve outside expertise.

When we ask for assistance and then insist on telling the expert what to do, we can sometimes get what we want (say antibiotics), but doing so will not provide a solution and will, in fact, be detrimental to ourselves and to others.

How does this relate to the business world? Pretty directly as it turns out. When we experience symptoms, we could look for the source and begin lifestyle changes that will render the source harmless in the future. We can even apply topical analgesics in the form of temporary hires, a revenue bond issue or something of that nature to relieve the symptoms while we wait for the cure to develop. Or we can buy some technological antibiotic at considerable expense and increase the general frustration/pain level as we try to graft the "solution" into our corporate body.

When dealing with issues of personal health, does it make sense to ask the pharmaceutical rep what to do? He or she will undoubtedly have the "solution" in their inventory (or pocket). No matter how you ask the question, they will have a product that will provide relief. This is exactly the approach we take when it comes to technology.

What we should do, as a business or an individual, is engage a personal trainer to show us the lifestyle changes and the personal discipline that will be needed to break out of the cycle of pain. The trainer will even recommend some aids when appropriate.

Please, please stop going to technology vendors asking for solutions. Please, please stop and think when "your account manager" calls to tell you about the newest solution available through him.

No two people and no two businesses are the same. A real solution must fit the individual physiology and lifestyle. It must come from within with the aid of expert assistance.

Practical, Pragmatic, Productive

2009-03-16T12:32:00.003-05:00

We kind of like to think of these three as synonymous. Don't they all require a frame of reference, a context, a perspective? Certainly they do if we want to think of them as synonyms. Further, in order to be synonyms, wouldn't they all need the same frame of reference, context, perspective?

I've noticed that people have a hard time with those (like me) who split semantic hairs. And yet, the long-term success of anything relating to data depends on grasping the nuances of differing frames of reference, context and perspective and integrating them into something that will be useful to all.

Beware the architect who listens to your 30 second description of what the strategic information system must do, then asks "when do you need it?" and disappears.

My son at 6 years asked me for help in building a submarine. I asked him a few questions and learned that he intended to actually travel under water--not just a play submarine. He had corresponded with his teacher's son who was an officer aboard an actual nuclear submarine. He had diagrams and photos. I asked him what he proposed to build it out of. He took me to the garage and showed me some 2x2 and 2x4 lumber and some chicken wire left over from building a rabbit cage. I asked how he proposed to keep the water out and learned that this was to be my contribution.

In my career, I have seen the equivalent of this scenario reenacted many times. It's absolutely amazing how much effort is expended on these projects. Yes, it is hard to tell your 6 year old son (or your boss) that the project can't be done and it is possible that your credibility will be diminished. I wonder whether the cost of information technology couldn't be cut by 90% if we simply learned some better ways of saying "no."

Oh, and, by the way, was it practical, pragmatic, and productive to gather diagrams, photos and written material, identify materials and find an "expert" to assist with the hard parts? Was I practical, pragmatic and productive in my approach? What was the result?

Sometimes the impossible must be done but far more often it's better to simply move on to something more in line with our actual capabilities. And, in the cases where we do have do accomplish something "impossible" the very first thing we have to do is discard practical, pragmatic and even productive because those are what brought us to impossible in the first place.

Guerrillas and Governance

2009-03-14T09:57:00.004-05:00

Back to the American colonies in 1775.

Over the course of nearly 200 years, the colonies had grown from the equivalent of a handful of lemonade stands into a dozen or so franchises with new locations opening up in the undeveloped markets to the west. Since the appointed governors liked comfort and their delegates had few qualifications beyond their socio-economic status or their relationship to someone important, and since communication was so poor (or slow, which amounts to the same thing), governance was like the light of a lantern.

Close to the governor things looked good but you didn't have to get very far away before the light began to fade and things became less and less distinct. By the time a traveler left the last organized settlement, he had to be prepared to take care of himself.

The problem for the governors--and the leader back at empire headquarters--was that those people out there got to be pretty good at taking care of themselves and actually began to enjoy being out of sight and out of mind. When the leader needed to have better productivity from the empire, there was some push back.

The governors were under pressure to produce more and the workers out there on the boundaries were pretty satisfied with things the way they were. The governors asked the leader for more resources to improve their ability to govern. The leader sent in mercenaries to quiet things down and implement better governance, by which was meant to suppress dissent and put everyone back to work for the king.

Of course, the outsiders didn't know anything about the locally designed governance and stepped all over it because they only had one tool, force.

Enter the guerrilla. There are leaders everywhere and sometimes they have a vision for governance and the future. When the locals see a better life with the guerrilla governance than they do with their formally appointed governance, the guerrillas will eventually have their way.

There are many points to be made from this story but I leave them up to you. This is history, but it's also today.

The one point I do want to make is that governance has to work for the governed as well as for the empire and that replacing something that is poorly understood but seems to be working may, if it is not handled well, produce more harm than good.

As always, your comments are welcomed and appreciated.

Pragmatism and Sacred Cows

2009-03-13T12:53:00.003-05:00

Something happened to me today that has happened before. I thought I had filed the barb off the hook and that I wouldn't be snagged by this any more, but I was wrong.

I made the statement that my work would be done when people no longer talked about data governance having realized that all varieties of governance are "governance." The response was to remark that statements like that are blue sky and that people will turn me off and look for someone more pragmatic.

Knowing, as I do, that the statement represents the ultimate in pragmatism--the application of what is known to make sense of what is unknown--I proceed (very quickly) through several emotions. I guess I have blunted the hook to the extent that I recognize the emotions as they appear--which is good. Where I wound up, after passing through the "I don't need this" phase, was appreciating Galileo, Darwin and Einstein.

I'm not comparing my contribution to theirs. I am comparing my emotions to what they almost certainly experienced as they struggled to break through the thickest of all possible walls--common knowledge.

Sometimes we have other names for common knowledge. We say "everyone knows that..." or "best practice states..." or any of a host of other codewords for what amounts to a sacred cow.

I have learned that the words "practical" or "pragmatic" or even "heresy" are silver bullets most often applied to protect the sacred cows from attacks by, well, me. I just want you to know that if you want practical solutions, you should get as far from best practice proponents as possible. You might even want to venture out to the boundaries of your organization to see what pragmatic really looks like.

Measuring Governance

2009-03-12T09:48:00.009-05:00

I apologize. I said I would address this yesterday. We do have to get back to the guerrilla movement on the frontiers of the empire, but let's take a little time to look at measurement of governance.

First, let's agree that data governance is like any other governance except that it focuses on data. A governance program directed at process or at competency or whatever, would have the same characteristics? OK, I'll attempt a justification for that statement.

What do we ask of a data governance process? What are the objectives? By the way, I use the term process here in the sense of a set of activities that are ongoing and have a consistent purpose. The purpose of the data governance process is to:

Optimize the value of the data resource by insuring that the capture, storage, retrieval, destruction and use of the resource is done in accordance with established policy, procedure and standards.

Do you buy it? If not, I'd be pleased to discuss alternative purposes, but the remainder of this discussion is based on this purpose.

Based on the purpose of data governance then, several perspectives on measurement suggest themselves. The most obvious one is the QA (quality assurance) perspective. How are we doing at following established standards? It is tempting to count the number of standards, policies and procedures because counting is easy to do and there is a tendency among the governors to equate many laws with good government. Strangely enough, among the governed the emphasis is on the quality of the laws rather than their quantity. A small number of effective and easily understood standards may deliver more benefit than a larger number of over-specialized or esoteric ones.

The most effective measurement will be part of the standard or process itself, but some organizations may find it useful in getting governance going, to do retrospective analysis to see how well/consistently processes are being applied. Health care makes extensive use of the "chart review" to gather this kind of data retrospectively. Measurement intrinsic to the process or standard has the potential to be much more nuanced and useful than that done retrospectively simply because all of the context is available.

Clearly, though, the nature of the metric(s) is very much determined by the process or standard itself. For this reason, it makes no sense to discuss metrics or KPIs (key process indicators), a special kind of metric, without first establishing the process context.

Other perspectives might differntiate among standard, process, and policy or might measure in conjunction with the data life cycle, specific subject areas or specific usages.

One last point, should you be tempted to think in terms of measuring accountability.

Accountability in the absence of a standard is really approval.

No governance mechanism can exist for long based on approval. Each change in "leadership" will create massive turmoil as everyone seeks to reorient to a new approval model.

Oxymorons in Abundance (part 3)

2009-03-11T09:49:00.007-05:00

Yesterday I stated that governance already exists and we ignore it to the detriment of all. A bit more discussion on this will be helpful.

Back to the frontier first. A settler had the problem of governing the household. There isn't much that need be said about that since we still deal with that issue today and probably in much the same ways. When a settler (at the boundaries of the empire) encountered others like him/her, they had to come up with ways to govern those relationships. I'm not going to go into those mechanisms here--cultural anthropologists have published a lot of theory and case studies about this process. I will say that whatever was negotiated fell into one of two broad categories:

One party was clearly dominant and dictated the terms

The parties created a contract that was seen as mutually beneficial.

Even in the cases of mutually beneficial agreements, there is often a competitive aspect. One of the parties will think, "Well, I had to agree to this but I'm going to stick to the letter of the agreement and if they think I'm going to go out of my way to make their lives easier, they'd better think again."

Fast forward to our modern equivalent. It's easy to see that, at the boundaries of our corporate empire, where the other party is the source of revenue (client or customer) this is counter-productive and a sure path to failure. It isn't quite as easy to see that the same holds true at organizational boundaries and, most importantly, process boundaries. Because of the way businesses operate culturally, each employee is competing with every other employee in the same way that settlers at the frontier were often forced to compete with each other.

The result (Dr. W. Edwards Deming exposed this very effectively) is sub-optimal performance for the process, organization and empire. We can hear this any time we choose to listen. "I know that data is incomplete, but I don't have time. They're the ones who need it--let them clean it up." "Yes, it's wrong but it's what they asked for. They're the ones who will have to do it over."

So here is the role of a governance program. Governance exists, but its goals are not those of the process, organization, empire. In order to replace naturally occurring, organic governance with governance that is aligned with the corporate vision and carefully designed to further the strategies and goals of that vision, the organic governance structures, including the attitudes that created them, must be identified and understood. Then they must be replaced with equally effective structures.

Are you familiar with the idea of a cow path? This picture shows cow paths--no, not the broad "road" that the cows are on--the cow paths are those faint lines that meander across the fall line of the hill. Why are there so many? The cows have their reasons but they aren't talking. Those paths are like the organic governance in your organization. You've heard the expression paving the cow paths, which is not considered a good thing to do.

I'll leave you with a final thought. Is redesigning the cow paths a productive effort? The answer is that it may be and it depends on the objectives, BUT if there is no way to train the cows to use the new 21st century solution, then even the best vision, strategy and goals will be for nought.

Oxymorons in Abundance (continued)

2009-03-10T10:37:00.019-05:00

Leadership is about Change.
Management is about Effectiveness and Efficiency.
Governance is about Consistency and Stability.

The discipline of Change Management is well worth our time. It consists of theory and practice involved in getting a change implemented. http://www.m2dxtx.com/images/Change_Components.gif makes an excellent reference whether you are a Manager, Governor or Leader. In everyday life, we can find ourselves in situations where there is an atmosphere of confusion around some initiative. As the chart directs, this is evidence that the vision driving the change has not been communicated well enough. In other words, the effort still needs leadership.

If the general mood is one of frustration, then this is where the manager must step in to make certain that sufficient resources are available.

To understand the role of Governance better, let's examine the life of a guerrilla movement such as the early days of our own American Revolution. The various colonies existed under charter from the King of England--not unlike departments in any corporation. As far as the Leadership was concerned, the colonies' purpose (the vision) was to produce wealth to allow expansion of the empire. The colonies were OK with that since they were able to keep a portion of the wealth for themselves.

The Leadership had placed Governors in each of the colonies to make sure that everything functioned well and that the colonist/workers could and did focus on their production. Leaders like to be able to rely on their empire to generate wealth consistently and predictably. The problem was that the Leaders and the Governors were out of touch with life at the boundaries. Governance attenuated pretty rapidly as you moved westward, away from the seaport communication links and seats of governance.

It was hard to focus on making the leaders wealthier when you had to worry about coming back from a hard day in the fields to find your home burned down and your family gone. Even a relatively small thing like an illness meant that you might have to cease all "normal" activities in favor of ministering to the ill or traveling days to either bring the ill to a physician or the physician to the ill.

In all of this people learned to manage their own situation. In so doing, they developed their own leadership and governance skills and processes. They learned to manage the processes and assessed their effectiveness by the stability they delivered. Ineffective processes were modified or abandoned.

The lesson here is that there is little need for governance in the boardroom. Governance is most valuable at the boundaries, where control is weakest. If there is no stability, no consistency and no predictability at the boundaries, then there is no governance. If Leaders attempt to hold Managers and "official" Governors accountable for something that doesn't exist, they will find themselves more and more out of touch as the accountable individuals scramble to create evidence that they are being effective.

What is data governance accountable for? What do we measure? What are the goals? What are the trends? Is business intelligence defined for data governance? Do we have data governance?

Tomorrow: Measuring Data Governance

Oxymorons in Abundance

2009-03-09T08:20:00.009-05:00

No Governance in Data Governance.
No Intelligence in Business Intelligence.
No Leadership in Corporate Leadership.

Let me hasten to say that these are not intended to be value judgements. To be fair and truthful (which are tough sells) I should modify these statement a bit.

There is little governance in data governance, little intelligence in business intelligence and little leadership in corporate leadership. The question to be asked is not "why" but "how". Whenever we are faced with something unexpected, we are used to responding, "Why?" I've learned that why? almost always puts people on the defensive and that communication effectively shuts down when people become defensive.

If we change our question to how? we can focus on processes and look for cause rather than fault. So, how does it come about that data governance so often lacks any vestiges of governance?

The first task is to differentiate between leadership, management and governance. This isn't about people--an individual may be capable of doing all three--but it is about tendencies. Here's a breakdown that might help this to make sense.

Leadership is about change.
Management is about effectiveness and efficiency.
Governance is about consistency and stability.

My take, after watching these processes work for many years, is that we find it nearly impossible to keep these three functions separated. When we are doing management while talking about leadership or doing leadership when we are talking governance, we not only confuse ourselves but also the community we are working within.

For example, we make a leadership decision to create data governance. Then we turn the task over to managers. In reality, leadership is required all the way out to the line organizations. Managers cannot make effective or efficient something that they do not understand and have not bought into. A too-early transition from leadership to management will give birth to confusion, frustration, burn out and the failure of the initiative before it even gets to the governance phase.

Continued tomorrow.

Governance and Data Governance

2009-03-05T12:53:00.003-06:00

It occurred to me today that one of the reasons for so much confusion in the data governance ranks today is that businesses have a hard time with governance in general.

The literature I've seen (and I'll admit I'm not ready for a thesis defense) is focused on the what I would call the mechanical aspects.

What kind(s) of committee(s) and at what level
Who should be members
What are the roles of the committee(s)
Who holds the "decision rights"
What are the decision domains

When discussing committee membership, the choices are framed in terms of role and level within the business.

Nowhere is the question of competency introduced. To me, this is cause to wonder about the purpose of governance. If competency is not a requisite quality of governance, then why do it? It seems clear that, even in areas that most would gladly cede to executive management such as strategy formulation and prosecution, there is an aspect of competence that, if missing, will cause decisions to be ineffective and/or impossible to implement.

We are accustomed to seeing leadership, management and governance used interchangeably when, in fact they are three different activities with three separate purposes.

Leadership has the mission of (productive or positive) change.

Management has the mission of effectiveness and efficiency.

Governance has the mission of stability, consistency and predictability.

I submit that questions about whether a certain initiative should be funded or not is NOT a question for governance but for management. The governance question is whether doing this will upset the applecart. Can we continue to produce expected results if we do this? If we need to do this because of a leadership imperative, how can it be accomplished such that predictability is preserved?

Without intending any disrespect to executives, it is doubtful whether they could productively be involved in answering those questions.

Information or Systems?

2009-03-03T11:18:00.004-06:00

For those of you who work in I.S. or I.T. or any of the variants, a question. Is your organization about information or is it about systems or maybe it's about technology?

My sense is that many more people are in it for the systems (programming) or the technology (networking, servers, wires, boxes) than for the information.

Just to get all the cards on the table, I'm asking this from the perspective (there's that word again) of someone who has been having his nose bloodied for years because of a stubborn streak that keeps on insisting that it is about the information and that everything else is supporting cast, walk-ons and extras (the Academy Awards are a recent memory).

The term, ontology, has become trendy in the relatively recent past. It simply means a specification for a concept. A concept is an idea and many times it never progresses beyond that. Rarely, an idea like freedom or liberty needs little or no specification to make it useful, Many ideas, like stewardship, on the other hand, need quite a lot of specification before they become useful.

Information Systems/Technology appears to be in need of some onological work. I.S./I.T. are ideas that require a context. They are found in the context of a business. The business, in turn, has a context but we don' need to go that far for the purposes of this discussion.

Businesses need to produce, dispense, store, manage many kinds of things and all of them are physical save one--information. Because information is a concept in its own right, it quite often gets pushed out of the way in favor of the physical things that compete for our attention by virtue of form, color or sound. These things require physical space and unless we do something, they will soon pile up and make it impossible to get anything done.

Quiet information or data, on the other hand makes no demands and is consequently ignored. Remember, though, that the business has an I.S. or I.T. organization because every now and then someone needs a specific piece of data or a chunk of information and needs it now. Sometimes the data has just come into existence and other times it has been languishing in a "data file" for days, months or years.

How do we find that set of ones and zeros and turn it back into the concrete abstraction that the business needs? Friends, that takes data as well. Device names, drives, folders, files, instances, records, fields, indices, values--all of that is data. In I.S., we understand the need to keep that kind of data reliable. We create systems and they are data as well. We understand the need to maintain our system data: product, version, build, component, QA status... and the implications of not doing so.

Frequently the Data people (data administration, data architecture, data stewardship, data governance, database administration...) are part of I.S. or I.T. and we're content with that as long as they are directing their attention outward, toward the business. As soon as they begin to exhibit interest in us and our handling of our own data, we start to feel resentment, frustration and even anger. "Who are they to tell us how to do our job?"

Friends, and I am sincere in my use of the term, programming is programming and data is data. The Data people can help you and they want to help you and, most of all, they need to help you in order to close the loop. They are being held responsible for the quality of the data resource and the processes that create and manage the resource. You represent a huge exposure as far as they are concerned. When you re-learn to associate your system with the information that flows through it, I hope you will also learn to value what the Data people are offering.

Information Systems, Information Services, Information Technology: let's refocus on the reason and purpose of those efforts. You can benefit from the consistency that results from standard processes. You can benefit from better data management capabilities. We can all benefit from understanding our shared purpose--the best information for the business we're part of.

Questions and Answers

2009-02-28T12:19:00.002-06:00

No learning is taking place if there are no questions.

How good does it have to be? Good enough.

Good enough begs for more questions. Who will decide? What will they base the decision on? What will be used for comparison? What is the standard? Each of these questions will lead to more questions and--here is the key--someone will tire and yield the field.

The answerer, let's call him A, and the questioner, Q, will continue the dialogue until either A or Q tires or loses interest or runs out of time. When the process doesn't run to completion the result--let's say BI, but it could be almost anything--will be based on whatever perspective Q has developed from the answers already received.

But here we are back at "how good does it have to be?" How many questions and answers does it take? Of course it varies with the quality of both the questions and the answers. Some people are good at 20 questions and some never get it. In order to zero in, we should look at the track record.

If you're A and you're never satisfied--never get what you need--no matter who is in the Q role, then I suggest that we focus in on A as the probable source of whatever problems there are. If you're Q and what you're delivering is never good enough, then we should focus on Q.

What's your track record? Can you offer any insights? Do you need to stick with it longer? What can you do to keep the other guy engaged longer?

Tools and Products

2009-02-27T18:03:00.000-06:00

Very briefly, I have been seeing many instances of confusion about tools and products. If you're going to make chisels or scissors, you really have to keep the eventual product firmly in mind.

A tool maker who loses track of the application for his tool runs the risk of producing something that has no utility. Something that has no utility or whose utility can't be easily recognized is not a tool. It may be an art object or it may just be a poorly designed piece of junk. It may also simply become so costly that the product can no longer afford it.

How good does it have to be? Good enough is the right answer. As with so many things in life, the right answer isn't nearly as satisfying as we might have hoped.

More on Perspective

2009-02-27T17:27:00.000-06:00

I worked for a company that had the hardest time getting new things started on schedule. Some new product or line of business would be announced with a grand opening date. Often the date was only a few weeks away. The problem was--over and over again--that the people who had to actually make the new thing happen found out about it at the time of the announcement.

So guess what happened? Either the schedule slipped and slipped again or the doors opened on something that was incomplete and holding all the pieces together was extremely hard on everyone involved. Is that what you guessed?

The root cause of this, as it turned out, was lack of the needed perspectives. Highly placed persons believed that they could make all the commitments for many of the enabling functions. In principle, this was true and needs were recognized and responded to AT A HIGH LEVEL. As any general or coach will tell you, the best strategy is only as good as the troops or players who have to execute it.

When we finally got a group together to analyze the situation, it became clear that no one person could know all that must be known to plan and implement the project. What was required was a meeting--as soon as possible--of representatives of all the business functions involved as well as those who are involved in everything (facilities, telecom, network...). All the perspectives only emerged in a group setting where people with specialized knowledge could bounce ideas around. Personal perspective expands in a group setting.

Again I ask, what perspectives are required for a successful BI implementation?

Perspectives

2009-02-26T09:38:00.000-06:00

I have come to view perspective as a (if not the) key to success. Before I can discuss perspective, though, I guess I had better define what I mean by success.

Success means

the achievement of some agreed-upon goal or
demonstrable progress toward a defined objective.

In life, success means doing the above while remaining within your vision. If you don't have a vision--some ideal that guides your decisions--then you are allowing others to define success for you.

In order to achieve success, then, it's important to understand who else is involved. If no one else need be involved in your achievement, then we're talking about a PR (personal record) and one of those is irrelevant to anyone other than you. Now, if you establish a trend of new PRs or if your PR is also a WR (world record), then others will take notice and may want to become involved. Let's leave aside these kinds of individual achievements for now because they are individual and the definition of success that we are using calls for "agreed-upon" goals or "defined" objectives. That implies more than one person is involved.

So who else is involved? There is a commonly used term, stakeholder, that labels a relationship between a person and a goal or objective. People who are stakeholders have an interest in this particular success (positive or negative). There are many experts on identification and management of stakeholder relationships, so I'm going to leave further discussion of stakeholders up to them.

My conviction, developed from years in the trenches and awareness (a combination of observation + curiosity + caring + reflection), is that perspective may be the single most important dimension affecting success.

For one thing, each of your stakeholders will have a perspective and it will be what you need to understand in order to bring that person into your team. But beyond stakeholders, there are still perspectives that must be discovered and managed. If we miss a particular perspective, we may miss or misunderstand entire classes of potential stakeholders.

A perspective may be based on a functional role. For example, there is a perspective associated with "father" that is different from that of "husband". We are all capable of juggling several perspectives--that's what we refer to when we say things like "on the other hand..." They are also based in part on prior experience. The father of four has a different perspective than the father of one or the fater-to-be. The best outcomes (successes) come when we are able to include as many perspectives as possible AND avoid excluding any that shed critical light on our effort.

What are the perspectives that must be part of a business intelligence success?

Introduction

2009-02-25T17:04:00.000-06:00

I never had any intention of becoming a blogger--not that there's anything wrong with blogging. It may be too little, too late and I certainly feel like the boy with his finger in the dike, but there needs to be a voice of reason in cyberspace when it comes to the latest HOT topic, Business Intelligence.

I am leveraging more than 20 years of data management experience here, so I'm going to try to avoid the bling and get to the meat. Since so much of the BI (short for business intelligence) stream is bling, frills, bells and whistles, I should be able to keep these relatively short.

I'll inaugurate this spot with the assertion that if there is no meat to what you're seeing--if the information isn't "actionable", then it doesn't matter how it's presented. That means that if you've blown your budget on presentation (ex., dashboard) tools but you've never spent a cent on a data quality assessment and you don't have process consistency or even documented processes, then you won't get intelligence.

That's enough for an intro. More tomorrow.