Search This Blog

Monday, May 14, 2012

Whole Cloth or Patchwork

Data Quality is too big to conquer. 

Customer mailing addresses or patient demographics are big enough challenges for most.  What is the common thread that, once followed will allow for a holistic approach to data and information quality? 

The picture at right (captured from Wikipedia and source unknown though the file name was in German) gives an idea of what I mean.  The weft (or woof) is the continuous thread while the warp is comprised of individual threads such as tools, methods, metadata, process, culture, management, governance and so on.

The whole cloth can smother any initiative while the individual threads of the warp provide gainful employment and career advancement for (at least) thousands.  Anyone with thread or yarn experience knows that, wound on spool or into an organized and well-designed ball, it is useful and can be applied effectively to many purposes.  Off the spool and out of control it is trash that must be thrown away.

If the whole cloth is data quality, then what is the weft that makes it more than a collection of individual fibers?  When we wish to weave data quality, we don't get to choose our warps.  We have to take everything as we find it and somehow turn it into the blanket that everyone wants.  Sometimes we find that we have a small patch of that blanket but no way to merge it with other patches because the weft is too specific and insufficient.

Sometimes to find a way to take several such patches and turn them into a quilt.  Rarely, the quilt provides the security needed.

Some questions that occur to me:
  • Can data quality be a patchwork?
  • What are the candidates for the weft that will bring all the variations of warp together into a consistent fabric?
  • Can we content ourselves with becoming masters of a specific warp thread?  If enough such masters emerge, can they collaboratively create data quality?
  • Is computer science part of the warp?  How about MIS?  Psychology?  Anthropology?  Or are these kinds of things that can be twisted into the yarn of the weft?
  • How do management, governance, leadership, software development, system design and architecture, data design and architecture, process design and architecture... fit?
  • How do (data) modeling, metadata management and various other data-specific technologies fit?
  • How do commercial products and tools fit?
I'm on the lookout for new questions but if I ever come across an answer, I'll certainly scoop it up and add it to my basket.