Jon Stewart Don't Know Squat About Data

This past Wednesday (Mar 27, 2013), the Daily Show included a segment on the backlogs within the VA and found the cause to be

• The reliance on paper to transfer medical records between the DoD and the VA

So far, so good.  Good investigative reporting.  If they had only left it at that and allowed viewers to form their own conclusions...

Instead, John Stewart made a joke about validating his preconceived ideas concerning Republican responsibility.  He played a clip of a Republican member of the Defense Appropriations committe comparing AHLTA (the DoD healthcare information system) and VistA (the VA's Healthcare Information umbrella) to Play Station and X-Box which, though both can use the same TV, cannot talk to one another.  AND THEN Mr.Stewart suggested that the parent of the household, if he/she wanted to minimize contention and confusion, could impose a single solution on the household.  A photo of President Obama was displayed during this, clearly indicating that all the blame could be laid at the President's feet.

Nothing could be further from the truth.  The VA has been assembling the components of VistA since 1987  and it was fully formed by the late 1990s.  AHLTA was introduced amidst much fanfare at the beginning of 2004.  The birthing process for AHLTA overlapped the adolescence and maturation of VistA, indicating a conscious decision (pre-2004) by the DoD to ignore the VA's efforts.

This entire mess is too complex in its origins and effects to discuss in a single blog entry so I'll content myself with one last piece of exculpatory evidence on behalf of anyone who has assumed elected office since 2004.  The only way to move information from AHLTA to VistA is

1. Generate paper from AHLTA
2. Send the paper to the VA
3. Enter the information from the paper into VistA

This is also the reason why the DoD can't simply replace AHLTA with VistA. 

There is a lot more to be said on this subject but, among other things, it seems clear that this is but one (albeit very painful) example of the inability of our health care "system" to adopt anything approaching a standardized view of the enterprise.  Interoperability is something to be wished for but which cannot exist in an environment in which healthcare systems vendors are the X-Box, the Play Station, and Wii.

Data is like Scat

We (data people) like to talk about data as if it were the most important thing in the world when, in reality, it is exactly like turds.  Yes, that's what I said, merde, gavno, scheiss, shit. 

Of course, it could still be the most important thing is the world but whether it is or is not depends on many things, not the least of which is what we're trying to do.  Data is the byproduct of process much as turds are the byproduct of digestion.

For naturalists and scientists castings or anaimal feces provide important information on a wide range of subjects including the animals diet, where it has been, and its general health.  For a tracker, a different set of information is deducible from the spoor.  The tracker tells which species, how recent, whether the animal is moving or is likely to be nearby and other things that may help him earn a bonus.

No one says, "This is quality shit." or "We need a governance program to improve the quality of our shit."  They simply learn what they can from it and move on.  The kind of value available changes over time but even petrified feces have a story to tell.

If we presume to be data experts, maybe we should be focused on extracting value from our data and understanding the kinds of value that can be extracted as well as how to recognize the data that will tell us what we need to know.

Even the most inconsistent set of data has a story to tell and we should listen to that story instead of wailing about the story we wanted to hear.  Because data is the byproduct of process, inconsistent data should tell us that the process that produced it is inconsistent.  If this is the case, how can we expect consistent data unless we can create a consistent process.

The bottom line is that when we focus our efforts on data quality we are misunderstanding the world we live and work in.  We are creating additional and entirely unncessary complexity.  How does this happen?  The cause lies in the storage of data for input to and output from computer systems.  We have allowed ourselves to institutionalize the cart before the horse.  What computer systems require is consistent input.  A system can be designed to deal with quality issues as long as they are predictable.  A system (process) will always produce output of a consistency equivalent to its input.

If we were pursuing consistency instead of quality, our disagreements would be fewer, our stress would be lower and our impact would be greater.  Consistency a readily understood concept while quality will always be the most elusive of quarries.

Reach the Unreachable Goal

Information quality, like many information and technology subspecialties, seems to drift in and out of focus.  Sometimes I find it difficult to understand why the subspecialty exists.

A recent epiphany concerning words and language has sent me on an outward-bound trajectory.  I used to chafe at the "definition" of quality in a data context--the one that defines quality as "fit for purpose."  I am now ready to break with it completely. 

My epiphany had to do with the fact that meaning is not packaged up in words and phrases.  Rather meaning is hiding behind and beneath them.  Well-chosen words can be used as markers to stake out the boundaries of meaning or even, if we're careful, to constrain meaning much like a fence.  When we focus on the stakes or the enclosure we risk losing the meaning that's inside.

We have all sat in high school (or college) literature classes and debated what the author or poet meant--what was the meaning that he or she had captured within the fence of language they had constructed.  What we have failed to see in our information technology context is that business leaders, managers, consultants and others are exactly like those authors (only usually not nearly as careful in their use of language).  We always have to ask what they meant and when we investigate, we invariably find that they didn't really know.

For more than 30 years we have labored to constrain the use of terms and encourage (or even enforce) the standardization of terminology.  The issue seems to have taken on even more importance with the widespread adoption of newer reporting (business intelligence) technology that brings the data full circle.  The leaders who were unable to stay focused long enough to generate good requirements are now launching their BI desktop and seeing the result.

I think that if we are honest with ourselves we will realize that though we now have better titles for what we do and we are getting paid better to do it, we would have to admit that our goal is not attainable.  We are being held accountable for The Quality of Data because that's how we have labeled ourselves.  We haven't even learned from the tribulations suffered by the other quality disciplines.  At least we should begin calling it data quality control or data quality assurance.  We have to focus on improvement in processes rather than improvement in data.

Look inside the fence I have built here and see if you can find some meaning that will help you.

Language, Information, Data and Quality

Hat in a Puddle

Language is a slippery thing.  For many years I conducted my life in the firm belief that if I were only precise enough in my selection of vocabulary, clear enough in my choice of syntax, I could convey an idea without ambiguity to any audience.

I have frequently been disappointed with the result. There is a force at work that allows people in the audience to navigate their own way through the meaning of my language.
Those in the data and information industry are accustomed to thinking of meaning as the semantic of the data.  Nothing could be further from the truth.

The dictionaries are full of semantics.  We can choose from a rich set of words (semantic tokens) to describe the situation represented in the picture above.  Note that we can change the meaning of the set of semantic tokens by several non-verbal (without words) methods including tone and inflection.  For example "hat in a puddle" has a different meaning than "hat in a puddle" or "hat in a puddle."

When we talk about semantics, we mean the meaning denoted by the words.  Alas, as humans we must also deal with the connoted meaning that each of us associates with the words.  Words and collections of words invoke in us memories, hopes , desires that are not part of the semantic but are part of the meaning.

The situation is very much like the puddle above.  Most people will accept the puddle at "face value" and simply avoid it so as not to get wet or muddy.  Others will make assumptions and develop expectations based on their personal experience with puddles.  Some of these will not change course, especially if their experience and their current situation allows them to expect that they won't get muddy.  Note that a new dimension was just introduced--a temporal dimension that allows us to react differently now than we might have an hour ago.

All Meaning Not Apparent

A man walking along a road saw a hat in a puddle and recognized the hat as one usually worn by his neighbor.  He thought to pick it up and return it.  When he picked up the hat, however, he saw the face of his neighbor.  He asked whether the neighbor needed help.  "I'm all right.  I've got my horse under me," was the reply.

The face value of words (and appearances) is accepted by most people and used to support decisions of all kinds.

Poets understand that meaning is not conveyed by words.  "Wait a second!" you say, "Poetry is composed of words."  We're both correct.  The meaning of a poem (or a story) is created by all the images, memories, hopes, dreams and desires that those words evoke in us.  This is why everyone who makes rhymes is NOT a poet and why everyone who has a command of vocabulary and syntax is NOT an acclaimed author.

This is the world in which we attempt to improve data quality.  While we may aspire to improve the quality of the semantics, it seems clear that we will never influence the quality of meaning.  This is, perhaps, what "fit for intended purpose" tries to convey.  What if the semantic tokens were musical notes instead?  What if they were colors or smells?  Would we be as confident?

What if we ceased our attempts to control the perceptions of an audience and instead created ways for our audience to explore the boundary between semantics and meaning?

Whole Cloth or Patchwork

Data Quality is too big to conquer. 

Customer mailing addresses or patient demographics are big enough challenges for most.  What is the common thread that, once followed will allow for a holistic approach to data and information quality? 

The picture at right (captured from Wikipedia and source unknown though the file name was in German) gives an idea of what I mean.  The weft (or woof) is the continuous thread while the warp is comprised of individual threads such as tools, methods, metadata, process, culture, management, governance and so on.

The whole cloth can smother any initiative while the individual threads of the warp provide gainful employment and career advancement for (at least) thousands.  Anyone with thread or yarn experience knows that, wound on spool or into an organized and well-designed ball, it is useful and can be applied effectively to many purposes.  Off the spool and out of control it is trash that must be thrown away.

If the whole cloth is data quality, then what is the weft that makes it more than a collection of individual fibers?  When we wish to weave data quality, we don't get to choose our warps.  We have to take everything as we find it and somehow turn it into the blanket that everyone wants.  Sometimes we find that we have a small patch of that blanket but no way to merge it with other patches because the weft is too specific and insufficient.

Sometimes to find a way to take several such patches and turn them into a quilt.  Rarely, the quilt provides the security needed.

Some questions that occur to me:

• Can data quality be a patchwork?
• What are the candidates for the weft that will bring all the variations of warp together into a consistent fabric?
• Can we content ourselves with becoming masters of a specific warp thread?  If enough such masters emerge, can they collaboratively create data quality?
• Is computer science part of the warp?  How about MIS?  Psychology?  Anthropology?  Or are these kinds of things that can be twisted into the yarn of the weft?
• How do management, governance, leadership, software development, system design and architecture, data design and architecture, process design and architecture... fit?
• How do (data) modeling, metadata management and various other data-specific technologies fit?
• How do commercial products and tools fit?

I'm on the lookout for new questions but if I ever come across an answer, I'll certainly scoop it up and add it to my basket.

Now

We don't value now.  We talk about it all the time.  We use it for emphasis.  We mostly use it to separate the past from the future.

Often when we talk about now we have only the fuzziest of notions, the flimsiest of definitions in mind.  "Now is the time for men and women of conscience..."  The concept of now in this instance could mean anything from a generation to a session of Congress to a particular crisis.  Sometimes we use now to mean very soon or as soon as humanly possible (the future) as in, "I need it now!"  Occasionally we use it to bound some time in the past as in, "Now in those days..."

Now has power that, for most people, is unrealized because it is unrecognized.  For our purposes now is the moment of decision.  If we are able to grasp that moment, that now, and use it, we can change our own life and the lives of those around us.  We can use now to create a new future.

In the gap between stimulus and response there is a piece of eternity--now--in which we can decide what the future will look like and decide on the response that will launch that future.  Please be aware of now and use it as it is intended to be used. 

When we are present in our life we are conscious of each now and we use them to create a future that matches our vision.  What is your vision, your ideal?  What will you use to direct the decisions you make in each of your nows?

All Governance (like Politics) is Local

Tip O'Neill, the Speaker of the House of Representatives during the Kennedy/Johnson years, is famously said to have offered the advice that "All politics is local."  If there is anyone out there who doesn't understand that Data Governance is politics then wait.

If we're to gain any advantage from the former Speaker's wisdom, we going to have to pull it apart and take a look at all the pieces.  Clearly he wasn't denying the existence of national and even international politics.  He had participated in politics at every possible level so what did he mean and how can we benefit?

First of all the context (which is always eliminated from "sound bites") is that of successful politics.  Which of us doesn't dream of successful data governance?  If we can accept that DG is political rather than technological or administrative or managerial, then we're ready to make use of political wisdom in our quest for successful data governance.

Successful politics is about getting enough people to come with you so that you can accomplish a vision.  Because we're human, we look for shortcuts.  We start by assuming that if we can convince the right person then that person will bring everyone else along.  So we start with our elevator speech in case we find ourselves confined with an influential person for any period of time. 

We also adopt the position that money will equate to support.  We pursue funding which requires approval at the executive level.  In short, we focus much if not most of our efforts on the critical few in the blind hope that all others are followers.

For sheep this may work.  Substantial research has been done on flock or herd behavior in an attempt to understand how humans are influenced to move one way or the other.  We have all seen a flock of starlings or sparrows or a school of fish suddenly change direction--apparently with a single will.  What magic would get people to act that way?  Leaving aside the question of goal or vision, which may or may not involve the common good, if we could master this magical force, think of all the effort that could be put to better use.

I have read some of this research and at the risk of oversimplification the answer lies not in identifying the leader but in identifying the first followers.  When one bird or fish or wildebeest, in motion, changes direction it may be for any reason or no reason at all.  If no one comes with them, they will very quickly rejoin the mass.  If another individual comes along then two going in the same direction exert some "gravitational" attraction that acts to influence others in the vicinity.

In the human world we divide people into leaders and followers.  More generally, we try to create leaders by assigning titles or creating org charts.  As mentioned above, we intend to leverage leadership by devoting our efforts to affecting their path, trusting that they will bring with them enough followers to makes our effort successful.  The problem with all of this is that titles do not confer leadership. 

What lesson can we learn then from Tip's advice?  My own take is that, rather than search for a leader, we might better be a leader, campaigning locally and helping our neighbors and those in need.  When we have one or more others with us because they are benefiting form the relationship we become much more effective in changing the direction of the heard.  Tip understood that grand political movements arise from individual voters recognizing common goals. No legislation is effective when the governed choose not to obey.  Devote your efforts locally and pay attention to what your neighbors in the next block are saying.

Politics is local.

How Would Jesus Do Data Governance (Part 3)

Having tried the preaching route, attracting big crowds, name recognition and a following, (are you with me?) Jesus recognized that the results he came to provide were not being realized. People listened, cheered and went home to the same life they were living before. Anyone who has been involved in the data industry for any length of time will immediately recognize this situation.

Data-driven development, Data Administration, Information Engineering, Data Architecture, Data Quality, Data Management, Data Governance... Cheering followed by more of the same.

What now? Even if you are God and are omnipotent, you still have to get humans to change in order to bring them to the new world--one in which they can rely on one another and in which each acts in light of a defining principle and for the greater good. How long have you been a human? I have more than 60 years of experience in being and being with humans. Jesus was obviously smarter than I. He realized after only 30 years that preaching and teaching just wasn't going to work--even though the Ideal was attractive and clearly in the best interests of all. He was getting, "Well, even if..." and "But everyone would have to buy in..." and "That's not how we understand the policies..."

Maybe this is what you're hearing as well. Maybe you are beset on all sides by scribes and pharisees, nit-pickers and policy wonks. Maybe they are constantly trying to trap you in a bit of heresy or a policy violation. Maybe you've thought about giving up because for some reason the people would rather listen to them than to you.

What did Jesus do? He changed his emphasis from preacher/teacher to minister. He modeled the changes he was talking about and he did it consistently with each and every person he met, meeting each one where they were. He showed them that the past was irrelevant and the future was not a given. He gave them the present and they experienced for themselves that their lives were better.

He did not give them rules, instead he gave them hope and someone to come to. He gave them someone who understood them and who showed them how, by subtly changing their perception, they could obtain victory over the troubles that plagued their lives. He never showed them judgment or condemnation. He always cared about their welfare and he planted seeds of change.

He gave himself for the people long before he gave his life for them.

If you believe that better policies, better standards, better rules, better laws will force people to care about data, I'd like to help you and I hope that I may already have helped you by planting this seed. Sitting in an ivory tower and sending out criers to inform the people of the duke's latest whim will never (as in NEVER) be productive. Maybe you'll think about giving up some day and you'll remember this seed.

How Would Jesus Do Data Governance (part 2)

First of all, as I mentioned last time, I am NOT comparing data governance to the Salvation of humanity--I am simply examining methods in light of the fact that both ideas resist description, can't be marketed and are not for sale.

So, I've got an idea that will make the lives of many easier to get through, make the company more profitable, reduce the level of risk involved in making use of information technology, and improve the ability to communicate. My problem is that the idea is based on something I can't describe. I can provide plenty of examples but that only seems to increase the confusion. So far I'm in good company. We've all been exposed to the parables of Jesus. He used them as everyday examples to shed light on some seriously abstract concepts. His disciples scratched their heads and asked each other, "What did he mean by that?" Two thousand years later, they're still being explained to us on Sunday mornings around the world.

Fortunately my idea has no expiration date. There is no competition to put the idea across--no race to the finish. The pain will continue until a critical mass of enlightenment is achieved. Enlightenment is essential because without it all we'll ever have is a collection of examples and stories that, to the average person, seem completely disconnected. Because people like for things to make sense and to be predictable, this disconnectedness leads to alienation.

When Jesus stuck to his idea, looking for better examples, better stories, he found himself on the outside. He was a different sort, something of a kook, but certainly not threatening. He had his small following who were largely content to be associated with him for the limited notoriety that the association provided. They knew him, liked him, respected him. The powers-that-be were not threatened and left him free to try to get his idea across.

Up to this point we have already made us of his methods and find ourselves in exactly the same position. We are tolerated and even receive minimal support from those who hear that others are doing it and seem to derive some non-quantifiable benefit. Unfortunately, as a band of disciples we lack much. We either lack a common cultural foundation or we ignore it. perhaps Jesus' disciples were fortunate in that they weren't constantly bombarded with "fresh, new takes" on their central idea. They were able (forced to) discuss the examples and stories among themselves. This process no doubt kept them cohesive as a group and, in the end was the springboard that helped them to launch the Idea out into the world.

How Would Jesus do Data Governance?

Recently I've been led to a reassessment of my approach to this whole blogging thing. Originally I had the idea that people would be attracted to a common sense, git'r done view of data governance. This quaint notion was driven by the fact that, although a lot of money was changing hands under the label "data governance" not much was really getting done to realize the vision.

There have been several incidents of late that have honed this approach by grinding away some remaining misconceptions on my part. What is left is the sharp edge--no assumptions, no vain hopes, no vanity, no illusions or delusions--just a cutting edge that can be applied to any knotty issue in the data governance/data quality landscape. Actually this edge can be applied to any knot whatsoever.

First I had to let go of the idea that anyone would listen to and heed an idea just because it's good. We've come to expect marketing glitz. A wise person once said, "All that glitters is not gold." The best ideas are seeds that must be tended over time but which will, in their own time, produce fruit.

Then I had to abandon the notion that people would rally to an idea. As it turns out, people rally to people--to leaders. Here I would ask the reader to consider leaders in general and some specific leaders. What was the basis of their "leadership?" In most cases it was charisma. Too often their followers realized too late the direction in which they were being led. I'm not charismatic.

Finally, I have understood that people will follow reward (most often money) rather than ideas. The great idea of data governance has been crushed, chopped, sliced and diced in search of marketing leverage and greater monetary reward.

All of this honing has left me with a better appreciation of Jesus of Nazareth, known as the Christ. I'm not comparing his mission with data governance--that would just be ridiculous. I do want to look at his methods, however, since many of the problems are strikingly similar.

Stay tuned here for an exploration of "What Would Jesus Do?" applied to data governance.

Chapter One

This will be chapter one of the eventual book. Some things have to laid out very explicitly. No one should ever be able to complain that they didn't get their money's worth or that they thought they were getting something different.

Some basic principles that will govern everything else that is said:

• Data is part of language
• All communication about data is, itself, data
• Language consists of denoted meaning (denotation) and connoted meaning (connotation) and on top of these is layered implication and inference which involve human perspective
• Nothing about language guarantees communication
• Communication requires a minimum of two entities from the following set [human, machine, logical construct (e.g., software)]

If we are to get a grasp on data quality we have the best chance of success if we restrict our discussion to ONLY that data that is part of communication between machines or between machine and logical construct. Of course this is neither exciting nor even very useful. We have many specifications formats (ethernet, etc.) that guarantee that communication on some level will take place between machines and logicals. ASCII or EBCDIC are the most basic of communication specification. It doesn't take very long for the alert observer to notice that unless a human is involved somewhere in the process, it doesn't really matter what the communication is.

"Matter" implies human involvement or at least we can infer human involvement from a statement that something does or does not matter. Matter is a value judgment couched in an emotional context. It's only when we start to peel back the layers, asking why or in what sense something matters that we begin to get to the idea of quality. Our exploration, then, will follow the trail of what matters.

As we follow this trail we're going to encounter the idea that what matters is, in many ways, distinct to the judge. What matters to the reader of a graphical novel (formerly comic book) may not be the same things that matter to a reader of War and Peace or a viewer of The Mona Lisa. What matters to someone watching Wile E. Coyote fail in yet another attempt at catching the Road Runner is not the same as what matters to someone watching Being John Malkovitch or Inception.

How then do we determine whose perspective to assume? Whose view matters?

The answer of course is that, where communication is concerned, everyone's perspective matters. It would be a great feat of communication if we could present a context-free (perspective-free) discussion of data quality. In fact, it would be such a feat that we're not likely to ever see it and it certainly won't happen here. Our intent is to zero in (or home in but NOT hone in) on a very small number of perspectives to see what matters to them and then step back to see if there are any common themes that can be exploited. If we are successfull in that, we may have created a springboard for the one who comes after.

In the next chapter we will nominate some key perspectives and begin to investigate what matters to them.

DQ: More Than Meets The Eye

As we move forward toward a view of data quality that allows us to create and use a language specific to DQ issues, descriptions and solutions, let’s take a minute here to examine the behavior of data.

Certainly, one of the attributes of quality data is that it is well-behaved. In other words it consistently delivers value according to principles that are applicable because of its type, domain, range, relationships, maturity, purpose(s)…

It is useful at this point to differentiate between static and dynamic properties of data. Any DQL (data quality language) that we might define should work well where static properties are concerned. When we begin to consider dynamic properties, the task becomes much more complex. The greater the number of dynamic properties, the greater will be the complexity.

Our chances of designing a DQL will be significantly greater if we can restrict ourselves to static properties only. Before we can do that, we have to understand the dynamic properties and assess their relative importance. Can we carve them out of the discussion? Will excluding them compromise our DQL’s capabilities?

Looking back at the list in paragraph 2, the first three properties might be thought of as static. These are the focus of our modeling efforts or, if we only pretend to do modeling, of our programming efforts. There is a tangent here that we’ll resist for now, but at some point we have to come back to it. The question of how data is initially defined is huge and the effect of initial definition on the lifetime of a datum and in particular on its quality is not to be underestimated.

For now, though, we’ll put that on the back burner. We expect the individual pieces of data to possess a definition (usually called a description), and our DBMS requires that we say what kind of data it is. Is it variable length text strings, a specified number of characters, integer, floating point, money, date/time, etc. It is surprising how many data are defined to the DBMS as varchar. It shouldn’t be surprising since all of our modeling tools allow us to set a default type and the default for the default is always varchar(n). This is popular because it guarantees that any value supplied will be accepted. Oops, another tangent almost sucked us in.

The final three items in the list are dynamic in the sense that their values can and will change, sometimes rapidly and usually unexpectedly. Let’s take the last first. Purpose, as “fit for…,” will change whenever we’re not paying attention. We hope that our stewards will be on top of this but pragmatically (everyone likes pragmatism), they may be too close to the business itself so that changing business needs or drivers loom so large that defined purpose fades to insignificance.

Maturity is also dynamic. We expect maturity to change over time. When we think of data maturity (if we do) we include stability (of all the other properties), quality metrics that have flattened out, recognition within the enterprise and probably several other aspects.

Finally, we have to face relationships. We’re not very good at relationship management. Some of us wouldn’t recognize a relationship if it sent us a valentine. Others pile all sorts of unwarranted expectations on top of our relationships and then wonder where has the quality gone.

It all starts in the modeling phase. Chen, when he invented a graphical notation for describing data, gave equal weight to entities and relationships. Both had a two dimensional symbol and the opportunity to possess attributes. For many reasons, not least perhaps that tool developers didn’t grasp the importance of relationship, “data modeling” tools eventually turned a multi-dimension, real thing into a single one-dimensional line that is only present all as a clue to the schema generator to copy the identifier from one of the linked entities into the attribute list of the other and label it as a foreign key so that the database engine can build an index.

Although I find examples are often counter-productive in the discussion of data quality, one example may illustrate the role of relationship in completing the semantic of a data set. PATIENT is such a common entity in the health care marketplace that no one even bothers to define it. It is a set of “demographics” by which we mean the attributes and it has relationship with PHYSICIAN or PROVIDER. It probably also has relationship with Visit or Admission, Order, Procedure, Prescription, Specimen and other entities of specific interest to the enterprise such as EDUCATION_SESSION, CLAIM…

It deosn’t take long to figure out that the relationship between patient and physician is more complex than can be accommodated by a single foreign key. A physician can “see” a patient, refer a patient, treat a patient, consult (with) a patient, admit a patient…the list goes on and on. Each of these relationships has real meaning or semantic value and may even be regulated by an outside body. Typically, these are implemented by a single foreign key attribute for each.

Now, imagine a situation in which an in-utero procedure is scheduled on a fetus. You may be aware that transfusions, heart valve repair and a host of other medical procedures are actually performed on the fetus while it is still within the mother’s womb. So, who is a patient? If the facility also terminates pregnancies for any reason you can see the conundrum. Medicine doesn’t allow for terminating the life of a patient (Dr. Kevorkian excepted). At the same time, we would like to sometime treat the fetus as a patient, perhaps for reasons of safety. We also experience the lack of values for attributes that we may have viewed as mandatory, e.g., DOB, SSN.

It is only when we explicitly talk about relationships that these issues emerge. Relationships cast light on the entity from all angles.

Relationships also represent the business processes that inform the purpose of the data. Often, undocumented meaning gets attached to data. Two analysts will get together and agree that for the purpose of this analytic, this combination of attribute values will be included (or excluded). For a given ETL job, we decide that an attribute value that isn’t on the approved list will be replaced with “&”. The adjustments to business processes are constant and usually undocumented and unnoticed. Until we can point to a documented process/relationship, we have no way of capturing and dealing with changes.

What’s the difference between an association and a relationship? Somewhere in there we’ll find clues about dynamic quality properties. One thing leaps out as a property of quality and a property of relationship—expectation. When we claim that something has quality, we establish an environment in which it is permitted to have certain kinds of expectations. The same is true of relationship. When two parties or entities enter into relationship they agree as to the expectations they will have of each other.

In our quest to define quality for data, we will be forced to document expectations and to monitor accountability with respect to those expectations.

Data, Governance and Data Governance

There have been some great discussion threads on the IAIDQ LinkedIn group recently. One thread that attracted a lot of attention started with a question from a PhD candidate in Data Quality. It simply asked whether there is an accepted definition of data quality. 200 replies later, most people would say, "No." More recently a thread began by bemoaning the fact that there is no accepted definition of Data Governance. A lively discussion followed that continues even now. Yet another refers to an article on five reasons to cleanse downstream instead of preventing upstream.

I am about to shed some light, however feeble, on the subject. Allow me to start by admitting that I am a person who likes to do the analysis necessary to solve a problem. Though my patience is improving with practice, those who know me will back me up when I say that I want to solve a problem ONCE.

Previous posts here have explained the abstract nature of data and all that implies in terms of getting people on board when it comes to doing something about quality. People will listen to or read a horror story about some preventable data issue that cost Company ABC $umpteen million. They will nod sagely and say something like, "They should have seen that coming." They are simply unable to see that their own company is engaged in the exact same practices and completely at risk for the $umpteen million.

Friends, it isn't just our favorite whipping boys, Management. There is no more recognition within IT than there is in the boardroom. Our boxes and wires friends think of data in terms of DASD and Raid configurations or bandwidth and throughput. Our developer pals don't really think of data at all except as the fuel that activates their code. Architects appear to be concerned with the storage and throughput views overlaid with an access management filter. They seem more concerned with making developers and DBAs happy than with the quality of the asset.

Enter Data Governance, which in most instances wants to be about definitions, rules and "enforcement." Often Data Governance tries to heap another thick layer, called meta data, on top of all the data that is already being mismanaged in the organization. It's often the case that Data Governance fails to practice what they preach.

Here's the revelation: Data Governance isn't about data. Data Governance is about process. It is the means to the Data Quality end. I have already said that Data Governance is that part of corporate governance that is dedicated to stewarding the corporation's data asset. It is exactly analogous to the role of Finance/Accounting with respect to the capital asset. Unfortunately, Finance has two things going for it that Data Governance doesn't have: GAAP and audits.

Generally Accepted Accounting Practice is a set of guidelines for money management processes that are accepted as the name implies and USED nationally and internationally. The use of these practices insures that processes will be auditable. The audit process verifies that GAAP was used and if there were exceptions, that they were clearly noted with enough information to allow the results to be brought back into alignment with GAAP. The underlying theme is that if the processes were sound then the result is believable.

Imagine if every company of any size whatsoever were able to devise and use its own bookkeeping structure and process. There could never be a stock market. Equity trading would be too risky for anyone and all business would essentially be sole proprietorships. Moreover, there would be no chance of oversight by outside bodies (Government).

This is a picture of the situation with respect to data today. When will it get better? Data Governance has no power to make the situation better. Without an externally defined data management framework and periodic audits by independent auditors, there will be no improvement. In the meantime, if data quality metrics improve, it's only because some particularly strong and charismatic personality is present.

No one questions the need for accounting nor the rigor of accounting procedures. Actually the same can be said for data governance and data management procedures. The difference being that in the case of money, the lack of question results in compliance while in the case of data it results in apathy or confusion.

Does the data world have something like GAAP that could become the necessary process infrastructure to support data management audits? I don't see it. Data is still too personal, too subjective, too misunderstood to attract the attention of researchers. Data management is a black box to virtually everyone and they like it that way.

People prefer to cleanse downstream data because their customers fell their pain being relieved. Happy customers is the goal after all. The bonus is that cleansing provides an unending source of employment for those doing the cleansing. It's win-win! People aren't going to be highly motivated to change a win-win scenario any time soon.

Whose Job Is It, Anyway?

There are several reasons why you are reading this blog post. Leaving out all the self-aggrandizing ones, let's focus on those who actually have the title question in their minds. You may have been drawn here by an interest in BI (business intelligence), the latest name for "reporting." It is likely that you have had some bad experiences involving reports or dashboards, or mash-ups or some other information display/access effort.

You spent what seemed like way too much time getting to an understanding of

• how this this was to be used
• the kind(s) of content that would be useful/acceptable
• how the information should be arranged/displayed

If you are part of a really accomplished organization, you may also have had seemingly endless discussions concerning

• How "bad" data would be recognized
• How "bad" data would be handled
• Remediation or cleansing processes to reduce the incidence of "bad" data

And, finally, if your organization is in the six-sigma population

• What is "bad" or poor quality data?
• Where does it come from?
• What does it cost?
• Where should we devote our efforts?
• What kinds of efforts offer the greatest ROI?

If you follow the various discussion forums concerning data quality, you will find one question popping up with regularity: "Who is responsible for Data Quality?" I asked myself why the question is asked. What prompts the question? It seems not to matter whether the organization has a reputation for quality, nor whether it has a history with data quality, nor whether the questioner is experienced or inexperienced, executive, manager, or front-line production. Why?

Having talked with some of these folks and researched the situations of several others and then simply meditated on this over an extended period of time, I have come up with a few likely scenarios.

1. The questioner knows the answer but either wants validation or a sufficient number of the "right" answer from people who are likely to be respected.
2. The questioner has encountered roadblocks from unexpected directions and is dealing with surprise and disappointment.
3. The questioner is curious about what others are doing.

What I have NOT seen is any evidence that the questioner is sincerely trying to determine how best to attack the problem of poor data quality. It would be easy to assume from these same discussion forums and conversations, that nearly everyone knows what they're doing and has either solved the problem or is well on their way to a solution. When we learn enough about human nature we understand that these people are all feeding their egos (or rather that their egos are feeding themselves since much of this is unconscious) and that they are taking an incidence of limited or small-scale success and projecting it into an eventual enterprise level solution. I have done this myself.

I know this--that data quality, or any other variety of quality for that matter, will not bend to advanced degrees, nor to mastery of data and information design concepts, nor to any product or set of products, nor to any effort by marketers, nor even to the best-designed procedures, methods, governance structures, architectures...

The ONLY way to data quality lies through the hearts and minds of each and every person in your organization. Each of them has the power to subvert any plan, procedure or method; to render ineffective any tool or product; and to humble the greatest of egos.

The bottom line is that it must be everyone's job but of course that isn't a satisfying answer because the very short list of things that everyone believes are important includes things like breathing, eating, elimination of wastes, procreation, maybe community, relationship, acknowledgement... This list of universally agreed-upon important things will never spontaneously include data quality. In fact, if a poll were conducted in the boardroom, it is unlikely that data quality would appear on the list of things important to the company.

Don't misunderstand--the quality, security and accessibility of your data assets is at least as important to the continued health and well-being of your company as that of your capital assets. The problem lies in the fact that data isn't real and tangible. If bad data smelled bad or rusted or developed crumbling holes, or if it resigned and went elsewhere where it was more appreciated or was subjected to audits by outside entities, or showed up on a P&L or balance sheet where it was reviewed by prospective data contributors--THEN it might get some attention.

It is true that anyone can produce an example of data but virtually no one--even those responsible for collecting and storing its instances--will understand that data is something other than what they are holding or pointing to or storing. But I digress.

If we can't accept the answer that data quality is everyone's job then we need to move on to identify the person or corporate function who will be accountable for the quality of the company's data. It's not possible here to put a name to this accountable party. What we can do, though is itemize some of the skills and abilities required to help point the way to your unique name.

First and most important, let's agree that what we are talking about here is cultural change and cultural change, more than any other kind of change, requires leadership. Already we see that a corporate function can never be accountable although you can tape a job title on the person's door when you identify him/her. This leader will be able to move freely across the company and will be able to give everyone the feeling that they have been heard. This DQL (data quality leader) will be conversant with principles and practices of quality improvement. The DQL must be completely comfortable with the nature of data and will not be confused by the display of samples. Attributes of the data asset as a whole will be the focus of all of the DQL's efforts. S/he may well choose to shine the spotlight on a segment of the population and may delegate someone more familiar with that segment to assume the leadership of this effort. That surrogate DQL will also deal only with population attributes (metrics).

The DQL will never fall into the trap of confusing examples with anything else. An example may be representative or it may be an anomaly. Only the population metrics allow us to tell the difference.

Further attempts at guiding your choice may be counter-productive if you begin to feel manipulated or otherwise used.

A final caution concerns those characteristics that will render a person unsuitable. Just as you might wonder about a carpenter who feels compelled to talk interminably about his hammer or his saw, the person who leads with the name of a tool, tool vendor, methodology, author, book, etc., is unlikely to be the one you're looking for. The last thing that the leader/agent of cultural change needs is to divert any attention away from the primary focus. Products and tools may be useful for producing the population metrics discussed above, but beyond that should be well in the background and completely invisible to the majority of those you are attempting to influence.

Those who confuse files or [mega/tera]bytes with data are likewise unsuitable as are those who confuse a spreadsheet, chart or report with data.

I hope I haven't made it seem like an impossibility. Talk with people about this and over time you'll begin to get a feel for what to look for and what to avoid.

Leadership, Management, Governance

Leadership for change

Management for effectiveness

Governance for stability

It is way too easy to become confused where these three functions are concerned. We like to think of ourselves as leaders when what actually consumes our time and attention is hitting deadlines and deliverables. Leaders emerge when change is in the air.

When the marketplace is shifting; when the economy is deteriorating and taking our profitability down with it; when we're no longer able to keep up--that's when leadership is required and leaders step forward. Anyone can steer a straight course through calm seas with good charts. It takes a master to sense the environmental factors, inspire confidence among the crew and make the continuous changes required to keep the ship from breaking up or running aground until things settle down and we get back into familiar waters. We risk disaster when leadership isn't acknowledged and permitted to assume control. When the crisis is over, we often find that the leader has little or no interest in the day-to-day operations of work rosters, schedules, performance reviews... The leader may make a very poor manager.

The manager is the master of routine. She is the one who keeps the machinery humming and the product going out the door. He makes sure that time boxes are hit, that deliverables are delivered and that budgets are created and followed. When the winds of change begin to blow, the manager who fails to recognize the need for leadership or believes in error that he can handle the leadership role can cause massive and sometimes irrecoverable damage before she agrees to [temporarily] relinquish the helm. Exceptional management defines the team, builds the team and keeps the team vision alive. This management is essential.

Governance is ubiquitous and invisible. Governance is observing, analyzing, formalizing, monitoring, measuring, improving. Governance establishes the standards to which managers hold themselves, each other and their teams accountable. It should be readily apparent that governance is every bit as necessary within a high-performance organization as is management. An exceptional governance function is a combination of historian, engineer and seer. A liberal dose of management is required to insure that governance doesn't degenerate into approval by the de facto expert.

The six sigma governance function will have incorporated leadership into the system of standards. A set of standards and standard processes may be of little use to the leader in the midst of the storm but may have helped to prepare that leader to be able to step forward.

Leadership for change, management for effectiveness, governance for stability is the tag line of WhiteLake Data Management. Data Management will be a microcosm of the enterprise as a whole. Every point of view (perspective) within the enterprise will be represented in Data Management. Virtually every process within the enterprise will be examined by Data Management in an attempt to "get it right." The risk associated with unreliable information (data) can only be assessed in light of the process(es)--and the personalities-- involved. Data Management is not governance but must include governance as an essential component. As its name implies, management is its bread and butter. Because it is frequently considered the homely step child, however, the availability of leadership may well be the key to its success.

Data Governance Is...

Recently someone on a LinkedIn discussion forum asked for a definition of data governance because he had yet to find one that was universally accepted. Now he has about 30 definitions and all of them are "personal" in the sense that they apparently work for the person who responded and/or that person's company.

In fact the lessons of the past 10 years or so produce the conviction that data governance is anything, everything and nothing.

It is anything we need it to be that serves our immediate purpose.

It is everything in that it spans all corporate functions in order to produce the needed results.

It is nothing because we always have to define the term whenever it is used and no two definitions are the same.

My own input to the discussion was that

Data governance is that part of corporate governance that is concerned with insuring the integrity of the corporate information resource.

The only useful application for this definition is in establishing a context for any initiatives and establishing responsibility or accountability. This definition cannot be used as a strategy or a vision to drive results. It doesn't suggest any metrics. It doesn't help us to isolate key processes nor does it suggest any standards.

Other definitions you may have seen involve "decision rights". The problem with "decision rights" is that there are always individuals who will pop up once one of these rightful decisions is made and insist that the decision in this particular case was rightfully theirs. Quite often this individual will make a good case and the "rightful" decsion will be overturned--often at considerable cost. When this happens, it calls into question the makeup of the existing decision-making bodies and can cast a long shadow over the entire concept.

My definition contains a problem in that it invokes another poorly defined concept--that of corporate governance. As I have discussed in previous posts, corporate governance in any form that would support the needs of those desperate for data governance is as rare as a polar bear in west Texas. It is the most challenging, most demanding and most thankless job imaginable to create a system of governance in the midst of a feudal culture.

This is not to say that results--even valuable and far-reaching results--can't be obtained. Such results are possible for those who are dedicated, courageous, knowledgeable and visionary. If one can keep the vision of data governance as part of a corporate culture and pursue integrity for all information but do it one relationship, one entity--even one attribute--at a time, then real progress can be made.

I have often wondered why we think we need a definition for data governance when it is so obviously subjective. Of course data integrity and data quality and even data itself are equally subjective. None can be discussed without first offering a definition ("What do you mean by that?") and we don't have definitions that we can quote that are meaningful to "this" audience. In fact, we are given to definitions that are nearly meaningless even to our colleagues and serve only to get us all in the same ballpark.

It is possible to take any result and call it data governance (who could argue?). It is possible to take any corporate initiative and use it to promote data governance. Why don't we simply get busy and spend our time discussing results instead of definitions? Show me the data that demonstrates you have brought processes into control for some subset of your company. Let's talk about which processes are most critical and which represent the greatest opportunity. Let's get moving.

Principle Before Practice

Before I begin, a caution. Do NOT think that I am taking a negative position re: data governance. On the contrary, I firmly believe that the concept is essential. But what is the concept?

As I watch data governance related position descriptions parade by on DICE.com, MONSTER.com, etc., I am struck by the focus on the practice of data governance. They are all about tools and skills and methods and all either assume that everyone knows what the goals are or (worse yet) that goals for this company are the same as those for every other company.

I have learned at least one thing in my 27 year adventure in all things data-related and that is that doing something, no matter how efficiently or effectively, is very often a wasted effort without some forethought about the principles that we are attempting to implement.

Principles provide the glue that links all of our efforts together and the medium that allows us to be productive even when our last effort was a failure.

As with so many initiatives around data, this wave (governance) has crested and achieved the status of "best practice" without ever achieving measurable ROI. We have seen smaller-scale successes--enough to keep hopes high--but not, to date, the enterprise wide success that was used to sell the initiative in the first place.

But maybe I'm getting ahead of myself. What is the principle that data governance represents? Some candidates are:

• Make life easier for DBA's by reducing the rate of database schema changes
• Make life easier for developers (programmers) by making requirements less ambiguous
• Reduce cost due to rework by making the rules for data quality and completeness more accessible at the point of capture
• Make the data warehouse more useful by applying the same rules to data everywhere it is captured.
• Reduce the number of sales lost because we can't keep track of our customers
• Establish process consistency so that we can begin improvement efforts
• Stop those sobs from delivering junk and expecting me to fix it
• Define and establish (standard) processes to reduce variability in output quality
• your favorite here

Which one(s) do you like? I believe the answer is all of the above and then some. What is the principle that unifies all of these goals? I discussed this in a broader context in a prior post. My point though is that if we don't have a firm and commonly held idea of the principle(s) we are attempting to implement, then no matter what we do, which tools we use, or how skilled the workers are, we aren't going to accomplish anything meaningful. To put it another way, "If you don't know where you're going, any route will do."

What Do You Do When Things Aren't Working The Way You'd Hoped?

Let's pick a context first because a) this problem is pervasive in the world I live in (how about you?) and b) the context will determine our course of action. I'll use my own life as an example.

I have spent my life seeking to understand my environment so that I could have a chance of staying out of hot water by being able to predict outcomes. I actually got pretty good at the predicting part but was never able to translate that into the staying out of hot water part. It turns out that when you see a result coming that is unwelcome to everyone, hot water is the least of your worries.

Of course I could have kept quiet and just let things happen but the problem with that is that almost invariably a minor course change would have prevented the outcome. It always seemed reasonable to attempt that minor change. Just as invariably there were political implications involved in any changes to the published plan. Bottom line: my career is littered with "you were right's" that came three years after I moved on.

So. if you would learn anything from my example, maybe it would be that "being right" carries no value. Maybe it's that you should just keep your head down and wait for the seniority promotion of for retirement. Maybe the lesson is that you do what you can and the rest belongs to someone else.

I will say that over time I have achieved objectives that others considered "impossible" because I was willing to take risks. The problem there of course if that if the objective was considered unachievable then no one is prepared when the find themselves standing inside the walls.

I think that this is also the story of data management (to include what has come to be known as data governance). Organizations have been talking about data management for nearly thirty years now and there are hundreds if not thousand of experts who will tell you exactly what you should to enjoy the benefits of good data management practice. What none of them will tell you because a) you don't want to hear it and b) you wouldn't hire them is that there is no proven methodology--no set of practices and tools, skills and technology--that will guarantee results.

Why should this be? You would think that in 30 years someone would have stumbled across something that will deliver predictable results. The answer lies in the subject matter. "Data" is a concept understood by everyone. Everyone in the boardroom has their favorite data. The issue at the root of all problems is that "everyone" is seeing data "as through a glass, darkly."

The inability to communicate about data and reach a consensus is what is keeping us from our objective. To this add the cult of personality that defines the management--let's call it governance--of the corporation. The decision makers understand nothing of the underwater portion of the data iceberg, seeing only the table, graph or dashboard that's in front of them. What must be managed is the abstraction that is data and not the values that are only the visible portion. When we try to do anything with the abstract, we find that there are side effects on the visible portion that cause VIP personalities to convulsively respond in exactly the least useful way.

You can get useful results if your objective is modest. For example, it is possible to get two business functions who are exchanging data or three or more that have a symbiotic relationship based on data to take consensus action to stop what is often a great deal of daily pain. The intractability is encountered when we attempt to broaden the scope to cross departmental or divisional boundaries. The goals and methods of data management are counterintuitive to those raised in the power politics of corporate "success."

We usually find ourselves managing data as a commodity, "how much", "how many", "what is the cost", "who produces", "who consumes", "spoilage rate", "how fast"... While these all have an attraction in that the answers can be easily captured in one of those tables, graphs, dashboards, none deal with the underlying problem of managing the abstraction. Data is the most complex thing that a corporation attempts to manage. It is more complext even than money.

The pity is that we treat data as if putting it into a "piggy bank" solves all our problems. You heard it here first:

• Technology is no answer--technology can help us sort different kinds of values into different piggy banks, no more.
• Technical skills (modeling, DBA, quality...) are no answer. The cashier makes use of such skills to keep his/her drawer in order and reconciled.
• People skills by themselves can't achieve any result except perhaps building meaningless consensus.

This is enough clues. If you call, don't bother to tell me what DBMS or CRP system or BI tools you're using. None of those things are of interest until the final stages of a solution. I don't expect any calls because too much credibility is wrapped up in the current initiative--whatever it is. When it fails to produce results, a new personality will step in and you'll start the cycle anew. Someone, someday may actually be willing to take a risk to stop the pain. I'll be retired or deceased by that time but maybe you'll have learned from this what you should be searching for.

Data Quality: Getting Started

If you happen to be a "mover and shaker" or if you aspire to that role, then you'll be looking for access to one or more key decision makers of influencers closest to the top of your organization. You'll be determined to convince those people that

1. investing in data quality is a sound business decision
2. you are the right person to produce the ROI that they'll be looking for

If, however, you simply want to make things better as soon as possible and create new friends and allies while doing so, then you may want to take a different approach.

My recommendation is to use the tactics of the Special Forces. The "Green Berets" were formed into small teams comprising skills critical to the people they were trying to help. They then went out to those people and lived with them. Doing this allowed them to gain credibility and to learn what kinds of changes might (or might not) be acceptable.

The Green Berets helped the people with their work and, while doing so, offered improvements--small changes that produced higher productivity or more consistent results. The goal was to create allies.

"Data Quality" represents exactly the same kind of productivity and/or consistency improvement for our "indigenous" people in whatever part of the company they may serve. A DQ Team may be as small as one member and can produce results that are shocking in their scope and value as well as in their lack of cost. It isn't necessary to spend long periods of time "living" with the people. In fact, one lunch or a couple of coffee breaks will do IF you

1. ask the right questions
2. listen carefully to the responses
3. offer support
4. follow through

You'll ask about what happens when they get incomplete or incorrect forms (data is usually thought of collectively as a form) from their internal "customer". You'll ask about the extra work they have to do in such situations. Be prepared for an emotional response, this is what causes them to miss deadlines, work overtime, add staff... Also be prepared to hear that they simply pass the problems on because they aren't staffed to deal with them and don't feel accountable for fixing problems they didn't create.

Listening will uncover the sources of the most frequent or egregious DQ errors. Now you can mention that you are about to begin a project with those dirty so-and-so's, that it's likely they don't even know the hardships they're creating, and that you'll be happy to mediate a discussion amongst the parties to try to find a resolution. You may already have some ideas.

Create the meeting, making sure that ALL parties are represented (you were listening carefully, right?) and facillitate the discussion, if necessary gently guiding the discussion but never offering solutions. When the solution is "discovered", the people will own it and will implement it with minimal assistance from you. If your assistance is required, make certain that you deliver and do not hold them up.

Follow up by monitoring, coaching, facillitating and then ask if they'd like some help in publicizing their success. Because you know important people, they'll almostly certainly jump at this opportunity. You give them all the credit and they'll make sure to let everyone know that your help was both timely and critical.

This approach works and can even result in regular meetings to follow the improvement and to look for new opprtunities.

Two approaches--you choose which one has the highest probability of success for the greatest number of people in the shortest time at the lowest cost.

The Theory of Everything

The US economy, so far as the majority of citizens is concerned, is in the toilet and swirling rapidly in a clockwise direction. Health Care, long in its own toilet, has at least stopped swirling momentarily. All of us have a stake in what happens in those toilets.

I have a stake in another toilet as well. The portion of the economy devoted to technology has been caught in a vortex since the dot com implosion 15 years ago. I realize it will do no good to link all of these since linkage simply makes the resulting mess seem even more impervious to any corrective action.

However, (deep breath) if we don't consider the nature of the connection between them, we have very little chance of making sustainable progress in any of them. So, as my contribution to posterity, I nominate the accension of appearance to the pinnacle of importance in decision making as the criterion most likely to be acknowledged as the root cause of all three problems.

Since 1950 the rate at which appeareances have displaced substance as the motivation for decisions in the US has increased at a dizzying rate. In the past two years I have seen the nations physicians, as represented by a blue ribbon panel from the Mayo Clinic, state that the answer to the nation's health care woes is better access to insurance. The calling that was Medicine has emerged as a new entitlement program for the elite.

In the economy, fiduciary responsibility has been replaced by revenue numbers as the force that justifies all decisions.

In the Technology world, the means have come to justify the ends. Any decision can be justified if it allows me to position myself as a front-runner, new, hip, cool. "There's an app for that" allows us to spend unjustifiable amounts of money just to have that app in our pocket. Similary, corporations spend unconscionable sums on technology projects for which the need is poorly understood. Because the "solution" has to be new to give the proper appearance of tech supremacy, the outcome is always in doubt. Risk isn't so much managed as PR-ed. Spin control is the name of the game.

In almost 30 years of working with technology I have learned one lesson that transcends all others:

Either control your technology or it will control you.

In assessing the meaning of this for your own situation it is well to remember that

• Technology demands consistency
• Humans and human organizations are incapable of consistency

Just a few recommendations

• Be clear about WHY you want to do something
• Insist that others are clear about WHY they want to do something
• Choose a path that is known to produce the desired result--or at least choose next steps that are known to produce appropriate results.
• If you are unable or unwilling to do the above, choose another line of work if possible and stop complaining if not.

We are all either a part of the solution or a part of the problem. In either event, complaining about what someone else is doing or not doing will produce absolutely no change.

If you can't tell the difference between substantive value and the appearance of value you should avoid positions in which you will be called upon to make decisions. If you can tell the difference, then for all our sakes, make the decision and don't give it to someone else.