I believe that early, effective big picture diagrams are key to application development project success. According to the old saw, no project succeeds without a catchy acronym. Maybe so, but I’d say no project succeeds without a good big picture diagram. The question: what constitutes a good one? To me good high-level diagrams have four key characteristics: they are simple, precise, expressive, and correct.
I recently stumbled upon one of The Martin Agency’s hilarious Geico caveman ads and wondered, rather geekily, why they didn’t do one about data analysis. I think if a caveman suddenly arrived in the 2010s he or she would see parallels between his life and the activities of today’s knowledge worker. When I thought it through, it seemed obvious that knowledge workers need to be more like farmers and less like hunter/gatherers if they want to achieve the full potential of business intelligence.
I hold a strong prejudice that IT paradigms are useful for about 30 years. The PC was dominant from 1980 to 2010, “online” mainframe systems from 1970 to 2000, and so on. If that’s the case then time’s up for Bill Inmon’s data warehousing framework. So far no widely held pattern has emerged to help us envision data management in today’s big data, mobile BI, end-user visualization, predictive analytics world, but at their recent Business Technology conference, Forrester Research took a swing at it by presenting their 2009 “hub and spoke” organizational strategy as a data management vision. Continue reading
Data management professionals have long and sometimes rather Quixotically driven organizations to “get past the spreadsheet culture.” Maybe that’s misguided. The recent furor over a widely read social science paper may show how we can look to scientific peer review for a way to govern data, spreadsheets and all.
Recently, it was found that a key study underpinning debt-reduction as a driver of economic growth based its conclusions on a flawed spreadsheet. As this ArsTechnica article describes, Carmen Reinhart and Kenneth Rogoff’s Growth in a Time of Debt seemingly proved a connection between “high levels of debt and negative average economic growth”. But, per a recent study by Thomas Herndon, Michael Ash, and Robert Pollin, it turns out that the study’s conclusions drew from a Microsoft Excel formula mistake, questionable data exclusions, and non-standard weightings of base data. The ArsTechnica piece finds those conclusions fade to a more ambiguous outcome with errors and apparent biases corrected. Continue reading
Out of curiosity I recently reviewed articles critical of Agile Methodologies. I had expected agile-versus-waterfall arguments and attacks from vendors selling new alternatives, but even given the reputation that advocates have for flaming well-intentioned critics, I wasn’t prepared for the level of emotion I found.
My opening position was that Agile techniques are great, but like any other tool there are limits and prerequisites. The critical articles I read strengthened that view. Let’s review three examples that stood out, in reverse order: Continue reading
Recently the BBC posted this video. On first view it is just funny, but watching those dogs learn to drive really reminded me of personal experiences with IT teams making big learning transitions. To represent those real situations let’s consider a fictional team of SQL developers facing the daunting task of deploying a functional Hadoop-based analytics prototype in two months. The video parallels their critical learning success factors: (1) set audacious goals, (2) learn bit by bit, and (3) know your limits.
Continue reading
Recently I was in a conversation about data modeling standards. I confess that I’m not really the standards type. I understand the value of standards and especially how important it is to follow them so others can interpret and use work products. It is just that I prefer to focus on understanding of the principles behind the standards. In general, it seems to me that following standards is trivial for someone who understand the principles, but impossible for someone who doesn’t. But there doesn’t seem to be general understanding of data modeling principles. Continue reading
When Tom Petty sang, “Hey baby, there ain’t no easy way out” he wasn’t referring to business intelligence (BI) reporting but he might have been. Current generation reporting engines, AKA data visualization or data discovery tools, market their products with statements like these, emphasizing quick development and ease of use:
I like these tools, and I do believe that they can provide a leaner, more productive, and more informative approach to BI reporting than some more mature products. However, none is a silver bullet for all data integration and reporting woes. Continue reading
I’ve written a number of posts about agile techniques of project management on this site, all in a spirit of advocacy. A comment on the most recent reminded me that agile/scrum isn’t necessarily the right solution in all situations, and in some it may work but needs to be applied carefully. After that comment I thought it would be interesting to write about situations in which agile methods should be applied with care, if at all: Continue reading
As a relational database professional I couldn’t help but feel like something would be lost with the emergence of the new Big Data/NoSQL database management systems (DBMS). After about two years of buzz around the topic, I’m really excited about the emerging possibilities. However, I’m pretty sure we’ll miss the relational model’s strengths in requirements definition and conceptual design. Continue reading