Tag Archives: Data Quality

Selected data modeling best practices

Recently I was in a conversation about data modeling standards. I confess that I’m not really the standards type.  I understand the value of standards and especially how important it is to follow them so others can interpret and use work products. It is just that I prefer to focus on understanding of the principles behind the standards. In general, it seems to me that following standards is trivial for someone who understand the principles, but impossible for someone who doesn’t. But there doesn’t seem to be general understanding of data modeling principles. Continue reading

The data quality challenge, in pictures

Data quality in most large organizations is commonly known to be rather lacking.  Most would argue that things haven’t gotten much better since this 2007 Accenture study found that “Managers Say the Majority of Information Obtained for Their Work Is Useless”. To some, quotes like that are shocking, but if you think about how information is processed in most Fortune 1000 sized organizations it is surprising that data available to managers is as good as it is. These slides have been useful in my efforts to explain the persistence of data quality problems in large organizations. Continue reading

The gnarly, subtle-seeming data quality question

I’ve posted a couple of articles at my company’s blog site that reflect my view on data quality efforts:

  • Yes, there is a business case for improving data quality, and I’ve got real business value examples. If you look for real money where you anecdotally know there are data quality problems, you’ll likely find it in high costs of data correction and rework, and savings related to business process improvements that reliable data enables.
  • There are distinct things an organization can do to reap benefits of improved data management and data quality.  (1) Get started in the first place, (2) find the tangible benefits, (3) cross the departmental silos that exist in every large organization, and (4) promote sound data management practices.

Continue reading

Data quality and data governance lessons from national health care

Who would want to be a national health care administrator?  Who would want the responsibility for managing health care and formulating health policy for tens or hundreds of millions of people?  It seems obvious that such decisions would rely on quality data.  A recent interview impressed upon me how much data managers can learn from a field where data recording millions of separate life and death decisions aggregates to support decisions on the future allocation of health care resources.

Continue reading

Consider the source in health care data integration

The Atlantic, not typically a technical rag, recently presented an article by business and economics editor Megan McArdle on health care data integration entitled “Paging Dr. Luddite”. The article brings to a mass audience an understanding of both the importance and difficulty of data integration, but the title and general anti-healthcare-professional tone seem counterproductive.

Continue reading

Special considerations in health care data

I’ve worked with health care data for the past few years, and in a recent conversation I realized it might be valuable to detail some of the complexities of health care data for those who might enter this growing field.  Of course these considerations aren’t unique to health care, but they are typical of the challenges that the new health care application developer or analyst might face. Continue reading

Metadata goals, ROI, and point solutions

Recently there has been a long, and very interesting, discussion of do-it-yourself versus third-party metadata tools on LinkedIn’s TDWI BI and DW discussion forum (membership required to follow the link). I have followed but haven’t commented, but I suppose I contributed when Information Management kindly published my article on DIY metadata.

The discussion is extremely informative, presenting the views of a variety of knowledgeable professionals in different situations, and describing successful and sometimes not-so-successful efforts to solve the essential metadata challenge: how to document what information is locked up in databases. Continue reading

On DW federation, whac-a-mole, and integrating business data

Information Management recently sent around their pick of best IM blog articles of 2009.  Among them was Forrester’s James Kobelius’s reaction to Bill Inmon’s “incineration of a straw man concept that he refers to as ‘virtual data warehousing (DW).’”

According to Mr. Inmon, virtual data warehousing reminds him of the carnival game called whac-a-mole.  He says “just when you think this incredibly inane idea has died and just when someone has delivered what should have been a deathly blow, out it pops again from another hole.” Continue reading

Coming soon: data like money

It is a commonplace to say we should manage data like a resource. But when you think about it, data is an asset but not a resource.  Data isn’t a thing like real estate, employees, or customers, but rather it represents all of those things.  In data-geek-speak, data is a meta-resource that holds information about resources.  That makes data a lot like money. Continue reading

DQ, he isn’t so dumb he just needs glasses

In a recent very thoughtful post on data quality, Paul Erb plays out an analogy comparing data users with Don Quixote and data quality professionals with Sancho Panza, then reverses the analogy to cleverly coin the “Sancho Panza” test of data quality professionals.  He encourages data quality professionals promoting the critical role of data quality to apply a what would Sancho say test to ensure that they are aligned with the needs and interests of data consumers. Continue reading