Tag: Data Quality
-
Guiding Principles for Data Enrichment
The data integration process is traditionally thought of in three steps: extract, transform, and load (ETL). Putting aside the often-discussed order of their execution, “extract” is pulling data out of a source system, “transform” means validating the source data and converting it to the desired standard (e.g. yards to meters), and load means storing the…
-
Thoughts on Healthcare Data Quality
The well-publicized problems with healthcare.gov are disturbing, especially when we remember they might result in many continuing without health insurance. But it seemed a step in the right direction when recent a news report differentiated between “front end” and “back end” problems. The back end problems were data issues, like a married applicant with two…
-
Data Governance Begins At the Spreadsheet
Data management professionals have long and sometimes rather Quixotically driven organizations to “get past the spreadsheet culture.” Maybe that’s misguided. The recent furor over a widely read social science paper may show how we can look to scientific peer review for a way to govern data, spreadsheets and all. Recently, it was found that a…
-
Data Design Matters
As important as it is, data modeling has always had a geeky, faintly impractical tinge to some. I’ve seen application development projects proceed with a suboptimal, “good enough”, model. The resulting systems might otherwise be well-architected, but sometimes strange vulnerabilities emerge that track directly to data design flaws. Recently I saw an example where a…
-
Lessons from the puppy poster
In some presentations, I assert that top-down data modeling should result in not only a business-consistent model but also a pretty well normalized model. One of the basic concepts behind normalization is functional dependency. In layperson’s terms, functional dependency means separating entities from each other and putting attributes into the obviously correct entity. For example, a…
-
Selected data modeling best practices
Recently I was in a conversation about data modeling standards. I confess that I’m not really the standards type. I understand the value of standards and especially how important it is to follow them so others can interpret and use work products. It is just that I prefer to focus on understanding of the principles…
-
The data quality challenge, in pictures
Data quality in most large organizations is commonly known to be rather lacking. Most would argue that things haven’t gotten much better since this 2007 Accenture study found that “Managers Say the Majority of Information Obtained for Their Work Is Useless”. To some, quotes like that are shocking, but if you think about how information…
-
The gnarly, subtle-seeming data quality question
I’ve posted a couple of articles at my company’s blog site that reflect my view on data quality efforts: Yes, there is a business case for improving data quality, and I’ve got real business value examples. If you look for real money where you anecdotally know there are data quality problems, you’ll likely find it…
-
Data quality and data governance lessons from national health care
Who would want to be a national health care administrator? Who would want the responsibility for managing health care and formulating health policy for tens or hundreds of millions of people? It seems obvious that such decisions would rely on quality data. A recent interview impressed upon me how much data managers can learn from a…
-
Consider the source in health care data integration
The Atlantic, not typically a technical rag, recently presented an article by business and economics editor Megan McArdle on health care data integration entitled “Paging Dr. Luddite”. The article brings to a mass audience an understanding of both the importance and difficulty of data integration, but the title and general anti-healthcare-professional tone seem counterproductive.