I hold a strong prejudice that IT paradigms are useful for about 30 years. The PC was dominant from 1980 to 2010, “online” mainframe systems from 1970 to 2000, and so on. If that’s the case then time’s up for Bill Inmon’s data warehousing framework. So far no widely held pattern has emerged to help us envision data management in today’s big data, mobile BI, end-user visualization, predictive analytics world, but at their recent Business Technology conference, Forrester Research took a swing at it by presenting their 2009 “hub and spoke” organizational strategy as a data management vision. Continue reading
Data management professionals have long and sometimes rather Quixotically driven organizations to “get past the spreadsheet culture.” Maybe that’s misguided. The recent furor over a widely read social science paper may show how we can look to scientific peer review for a way to govern data, spreadsheets and all.
Recently, it was found that a key study underpinning debt-reduction as a driver of economic growth based its conclusions on a flawed spreadsheet. As this ArsTechnica article describes, Carmen Reinhart and Kenneth Rogoff’s Growth in a Time of Debt seemingly proved a connection between “high levels of debt and negative average economic growth”. But, per a recent study by Thomas Herndon, Michael Ash, and Robert Pollin, it turns out that the study’s conclusions drew from a Microsoft Excel formula mistake, questionable data exclusions, and non-standard weightings of base data. The ArsTechnica piece finds those conclusions fade to a more ambiguous outcome with errors and apparent biases corrected. Continue reading
Out of curiosity I recently reviewed articles critical of Agile Methodologies. I had expected agile-versus-waterfall arguments and attacks from vendors selling new alternatives, but even given the reputation that advocates have for flaming well-intentioned critics, I wasn’t prepared for the level of emotion I found.
My opening position was that Agile techniques are great, but like any other tool there are limits and prerequisites. The critical articles I read strengthened that view. Let’s review three examples that stood out, in reverse order: Continue reading
Recently the BBC posted this video. On first view it is just funny, but watching those dogs learn to drive really reminded me of personal experiences with IT teams making big learning transitions. To represent those real situations let’s consider a fictional team of SQL developers facing the daunting task of deploying a functional Hadoop-based analytics prototype in two months. The video parallels their critical learning success factors: (1) set audacious goals, (2) learn bit by bit, and (3) know your limits.
Recently I was in a conversation about data modeling standards. I confess that I’m not really the standards type. I understand the value of standards and especially how important it is to follow them so others can interpret and use work products. It is just that I prefer to focus on understanding of the principles behind the standards. In general, it seems to me that following standards is trivial for someone who understand the principles, but impossible for someone who doesn’t. But there doesn’t seem to be general understanding of data modeling principles. Continue reading
When Tom Petty sang, “Hey baby, there ain’t no easy way out” he wasn’t referring to business intelligence (BI) reporting but he might have been. Current generation reporting engines, AKA data visualization or data discovery tools, market their products with statements like these, emphasizing quick development and ease of use:
- “The democratization of data is here. In minutes, create an interactive viz (sic) and embed it in your website. Anyone can do it— and it’s free.” (Tableau Products Page)
- “Easy yet sophisticated report design empowers your employees to design professional and telling reports in minutes not days” (Windward)
I like these tools, and I do believe that they can provide a leaner, more productive, and more informative approach to BI reporting than some more mature products. However, none is a silver bullet for all data integration and reporting woes. Continue reading
I’ve written a number of posts about agile techniques of project management on this site, all in a spirit of advocacy. A comment on the most recent reminded me that agile/scrum isn’t necessarily the right solution in all situations, and in some it may work but needs to be applied carefully. After that comment I thought it would be interesting to write about situations in which agile methods should be applied with care, if at all: Continue reading
As a relational database professional I couldn’t help but feel like something would be lost with the emergence of the new Big Data/NoSQL database management systems (DBMS). After about two years of buzz around the topic, I’m really excited about the emerging possibilities. However, I’m pretty sure we’ll miss the relational model’s strengths in requirements definition and conceptual design. Continue reading
Data quality in most large organizations is commonly known to be rather lacking. Most would argue that things haven’t gotten much better since this 2007 Accenture study found that “Managers Say the Majority of Information Obtained for Their Work Is Useless”. To some, quotes like that are shocking, but if you think about how information is processed in most Fortune 1000 sized organizations it is surprising that data available to managers is as good as it is. These slides have been useful in my efforts to explain the persistence of data quality problems in large organizations. Continue reading
I’ve never understood the obsession with “green” status among IT application development project managers, and the intense pressure put on them to “stay green” by the program management offices (PMOs) they report to. We would benefit from a cultural shift away from avoiding yellow status.
For those not in the field, it is in vogue to express IT project status using a stoplight analogy, where green means things are going well, yellow indicates some quality, schedule, or budget risk, and red means there’s imminent risk of failure.