Bob Lambert

A recurring theme in the literature on IT over the years has been frequent failure of IT projects.  Most studies lay the bulk of the blame on requirements (examples here and here).  One way to improve accuracy and fit-to-purpose of requirements, and thereby promote project success, is to include data analysis as well as process analysis in the requirements plan.

I’ve cited here the need to start data interface analysis early to avoid budget and schedule blow-ups when, as a result of not thinking early about interface complexity, data integration work turns out to be bigger and nastier than anticipated.

Early data study also helps business analysts elicit more detailed and accurate business requirements.  Say a mid-level football (soccer) team in the UK is looking to recruit a couple of strikers who can reliably punch home goals for the club.  The obvious data they seek is (1) the number of goals scored per game by each prospect, and (2) over their careers how much time have they spent on the bench due to injury.  At the same time, this club is building a strategic recruiting system to support growth into the higher echelons of English football.  A process-oriented requirements strategy (like the one described here) asks the team’s recruiters what they need to in order to get good people into the club, and often emerges with a list of statements about what the system will do (”The system shall provide an interface enabling entry of the following player statistics” or “The system shall provide a report ranking players by the following criteria:…”).

It isn’t necessarily wrong to start with process analysis, especially when backed up with formal techniques like use cases, data flow diagramming, or others, but addition of data analysis early provides ability to be far more perceptive into the real business needs.  Without interviewing anyone a data analyst can know that there are many goals in a game of soccer (OK, to some not nearly enough, but that’s another story), that the attributes of a game include location, weather conditions, date and time, whether it’s regular season or playoff, and more.  Attributes of a goal: time during the game; left foot, right foot, or head; did it come from a set play or in the run of play; from the left or right side of the field, and much more.

The analyst who knows the data and understands its structure can probe with questions like whether a player tends to score at the end of games, or would it be useful to find one striker who tends to score from the left side of the field and another who scores from the right?  By understanding the data an analyst can understand the business problem more deeply, build better rapport with business people  by asking more informed questions, and cross the business/IT communications gap to define the right requirements so that the right system gets built.

It may be just the organizations I’ve been exposed to, but in my experience data analysis isn’t typically part of the requirements effort.  Supporting this point, the author of the wikipedia page on business analysis entirely omits data analysis, apparently favoring a process-only approach.  On the other hand, object-based techniques offer a balanced approach, studying both data and process by representing things like goals, games, and players as objects with their own attributes and behaviors.  In addition, the International Institute of Business Analysts (IIBA) includes data-oriented along with process-oriented techniques in its Business Analysis Body of Knowledge (BABOK).

As process/data balance early on in the application lifecycle becomes more widespread analysts should generate more insightful requirements and, other things being equal, the success rate of IT application projects should improve.

It seems I’m frequently in conversations about using null to represent a business value.  To paraphrase, say there are credit and cash customers, and there’s a suggestion to set “Customer_Type” to “C” for credit and null for cash.  To data and database professionals this is obviously a bad idea, but it’s not obvious from a business point of view.

In a database null means that there is literally no value, or the value is indeterminate.  Null is not the same as zero or blank.  When a database operation involves nulls the result can be difficult to predict for someone not practiced in SQL.  In many cases the answer is null.  For example, 1+0=0 but 1+null=null.  In plain English, what you’re asking the DBMS to do in the latter case is to add 1 to [I don't know what], and of course 1+[I don't know what] equals [I don't know what].

So, if you use null to represent a business value then you might not get the results you’re looking for when you try to get business answers out of your database.  For example, say “C” represents credit customers and null represents cash customers, and you have 2 cash and 1 credit customers.   In SQL Server if you use a Count function to tally all of your cash customers the answer isn’t 2, it is null.

That’s one example of why it’s not a good idea to try to represent a business fact with a null value.  It doesn’t make business sense and in this case the DBMS, correctly, won’t make sense of it for you.

To be clear, whether or not a given database column permits null values is an entirely different question, best left to database designers.  For example, a database table might record which patient occupies which hospital bed.  It may be reasonable and correct to assign a null patient ID if the bed is currently available.  However, there are alternative methods of representing this situation, and the database designer should be free to choose the right alternative taking into account the specifics of the application under development.

Recently I was asked for a high level requirements plan for a large IT conversion.  I googled around a little for something standard.  I found some good references (see links at the bottom of this post), but not exactly what I was looking for: a simple, method-agnostic layout of the high level steps and checkpoints in requirements for a big project, emphasizing interactions with business people.   I then rifled my files to find the example below.

This summary plan frames up interactions with business subject matter experts and their review of results.  The table lays out the steps, “granularity” (meaning how often each step is carried out), what comes out of each step, who does the work, and offers a few notes.

Before the table, here are two important definitions:

• Sponsor: Very early on in your project you should identify the one person who you need to make happy in order to succeed.   The bigger the project, the higher up the sponsor.  Keep in touch to make sure they know what’s going on, keep them happy, and if something happens that will make them not happy don’t keep it secret.  Maintain a vibrant risk/issue process so that you can give them early warning of bad possibilities and they can help early.
• Stakeholder: A stakeholder is anyone who will benefit from or be harmed by your project.  Requirements come from stakeholders.  Be sure to build support even with the latter group if at all possible, and at least make the adjustments that will keep them from working to prevent you from succeeding.

Careful, this is just an empty vessel.  Within this high level framework a team can apply whatever requirements techniques they want.  (In fact, I highly recommend structured analysis techniques like use cases or process models, but that’s for the requirements team and this framework is for the PM).  Of course a smaller effort suitable for agile techniques wouldn’t need something like this.  This is for big transitions like conversion to a new COTS package, for example, where it is easy to get lost in the detail.

Hopefully if you’re a PM on a big project you’ll find this framework as useful as I did.

 Here are some of the other references I found along the way, caveat emptor:

• http://www.outsource2india.com/software/RequirementAnalysis.asp
• http://en.wikipedia.org/wiki/Requirements_analysis
• http://www.jiludwig.com/HCI_Journal_article.html

Free form diagramming doesn’t only mean “no rules”, it also means “just right”.

This post, last in a three part series on free form diagramming, gives some simple guidelines for getting the technique right.  Part one talked about the tension between rigor and expression in diagramming for analysis and design, and how more precise diagrams can hinder rather than help communications with business people.  Part two reviewed free form diagramming in practice.

While it is impossible to specify format and structure of a free form diagram in advance, here are some useful guidelines to follow when building your free form diagram:

•    Rule number one: draw it as you see it. Typically, an analyst uses a free form diagram because he/she already pictures a business process.  Trust your mental picture and get it down on the page.  Then, go through the following checklist to make sure it says what you want it to say.

•    Model real world processes, things, and events. Free form diagrams have one great advantage over Use Case Diagrams, Data Flow Diagrams, and the rest: they are concrete rather than abstract.  For example, in a free form diagram you can symbolize a shopper with a clipart picture of someone choosing a soup can from a grocery store shelf.  The free form diagram should clearly represent things from the real world: organizations, locations, business processes, interfaces, etc.

•    Use symbols consistently. Look at each repeated rectangle, line, circle, icon, etc, and verify that everything with the same shape represents the same type of thing or event.

•    Speak the language of the audience. A free form diagram should depict things business people care about in recognizable terms.  For example, accountants might readily understand boxes labeled GL, AP and AR for general ledger, accounts payable, and accounts receivable.  A shipping clerk might quickly interpret a process illustration showing labeled icons shaped like trucks and warehouses.

•    Arrangement on the page conveys meaning. Frequently, free form diagrams group objects that belong together on the page.  In other cases, the diagram shows a definite process flow by the arrangement of objects.  For example, Business Intelligence Architecture diagrams typically show information flow from source systems on the left to business reporting on the right.  Could this kind of flow or grouping improve your diagram?

•    Limit the number and complexity of objects on the diagram. Most often, a meaningful diagram shows relatively few objects, organizes them in a sensible way, and does not cross lines.  If you need many objects to tell the story, reduce complexity by arranging them logically.

•    Work at one level of detail, or clearly indicate differences between levels of detail. If your diagram includes a box labeled “AP System” then it would not likely make sense for it also to contain another labeled “Journal Voucher Key”.  Diagrams that communicate well are all at the same level of detail or clearly indicate differences in level of detail.

The free form diagram can be an essential part of a successful IT application project by enabling all to understand the target system in the same way and helping business people understand critical functionality.

This is part two of a three part series on free form diagramming for IT projects.  This entry reviews free form diagramming in practice. Part one talked about the tension between rigor and expression in diagramming for analysis and design, and how more precise diagrams can hinder rather than help communications with business people.  Part three will provide a few simple guidelines for getting it right.

Some current development tools and methods do include the free form diagram in their toolbox.  For example, Scott Ambler’s Agile Modeling site includes a page on free form diagrams (http://www.agilemodeling.com/artifacts/freeForm.htm), and the IBM tool XDE has provided an integrated free form diagram (http://www-128.ibm.com/developerworks/rational/library/915.html).

A Free Form Diagram Showing the Broad Outlines of a System

Free form diagrams can be especially useful in illustrating the overall scope of an IT development effort.  For example, XDE product literature cites the free form diagram’s “capabilities to communicate broad ideas about the direction of [a] solution”.  IT software product marketers frequently bear out this advantage over structured modeling with attractive diagrams describing their products and services, as in the diagram above.  Such diagrams enable companies to “level set” the terms of a conversation, providing a reference point for providing more detailed information about the product.

A similar diagram can provide a reference for IT project participants.  One, produced jointly early on in custom development of a financial system by the primary business contact and project architect, helped build consensus with designers, database administrators, programmers, and business people.  Some on the project considered this shared vision of the planned system one key to the project’s success.

Both of these examples illustrate the most common use of the free form diagram: to depict “the fundamental organization of a system [or business function], embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution” (http://en.wikipedia.org/wiki/System_architecture).

Because the free form diagram can seem more concrete than other, more abstract, models, a well-constructed free form diagram is a great way to communicate complexity to an audience of both technical and non-technical participants.

The free form diagram is also useful in other situations.  Sometimes a requirements team works with business users who have a diagram describing their current process.  In that case the team might use the same format to describe the new process.  In other cases it might be useful to improve readability by drawing a process flow on a map of the shop floor, or using icons representing real objects and events rather than class symbols on a class model.  The point is to create a medium that precisely matches the audience with the message.

Part 3 of the Free Form Diagrams series provides a few simple guidelines for success with this technique.

One effective way of communicating complexity, especially in the overall architecture of a system, is the free form diagram.  A free form diagram can directly address unique characteristics of a system in a way that business people can understand.

Out on a walk some years ago I met an acquaintance who happened to be a professor of Computer Science.  We talked shop (I worked in the IT department of the local electric utility) and he asked me what methods we used for mathematically proving program correctness.  I confess I laughed.  My team was struggling to figure out just what the business really wanted – forget mathematical proofs!

Our conversation highlights the tension between rigor and expression in diagramming to support IT projects.  Developers need precise diagrams rigorous enough to accurately reflect complex processes.  However, that precision and detail can make the diagrams at best boring and at worst intimidating to business people.  Often they don’t have the time or patience required to sit through the explanations needed to understand such diagrams. Requirements meetings frequently end early with business participants walking off grumbling about IT non-alignment.

UML is today’s de facto analysis and design diagramming standard. To the OO professional UML provides a rich, expressive, and consistent set of conceptual tools that continue to evolve.  Work is underway to improve the accuracy and precision of UML with respect to the target code. (http://www.omg.org/docs/ad/00-01-07.pdf).
The problem is that more precision will make the UML diagrams more complex and less understandable to non-coders, making the diagrams even less useful in communication with business people.

For project success business people need to be able to communicate their needs freely and completely.  Requirements analysts need to capture, record, and replay business needs for confirmation, and let the business expert return to other work as quickly as possible.  These communications often work best with pictures rather than words, and of course when everyone understands the picture the same way.  Asking a busy floor supervisor to review a formal class model is literally having him or her review specifications in a foreign language.

One solution is for developers to communicate with business people using free form diagrams, expressive yet rigorous diagrams with drawing conventions tailored to the audience.

More to come in parts two and three of this series.  Part 2 talks about using free form diagrams in practices, and Part 3 provides some simple guidelines for successfully using free form diagrams.

Everyone involved in managing or improving a business process should understand data modeling.  For real. And almost everyone is in a position to improve a business process by understanding the current one and making suggestions to improve it.  Understanding a business process means understanding business objects, events, the relations among them, and the business rules that govern the process, and that’s what data modeling is all about.

Sure, data modeling is the first step to designing a database, but that’s just a coincidence.  A well designed database is well designed both because it’s efficient and because it matches business needs.

The first step in data modeling is understanding entities.  An entity is like a business object: examples may include customer, order, product, patient, blogger, post, or whatever.  The next step is to understand relationships among the entities, like  a customer may place many orders or  a post is written by a blogger.   Then the data modeler thinks about the attributes of the entities, like the name of the blogger, the price of the product, and so on.  The attributes and relationships are where the business rules come from.  Examples of rules may be “a post is written by exactly one blogger” or “every order must have a shipping address.” It’s not really a sequential step by step thing, more like a series of really interesting brainstorm sessions, but you get the idea.

Data modeling can get really complex, especially when it includes enough detail to generate an actual database, but that’s beside the point.  We’re talking about clear thinking about business things, events, relationships, and rules.  The point is that this kind of thinking can enable a business person to understand better the things, events, and rules of a business, and then to define more rational processes based on that understanding.

Today a lot of the problems that data modeling helps reveal relate to overcomplicated org charts and artificial complexities of legacy information systems.  Often the business evolves around the system, and it takes clear thinking to untangle the process spaghetti that results.

I worked with one company that organized itself by four different product line channels, served by matrixed support functions like accounts receivable and claims.  Worked pretty well, except when you wanted to know all of your contacts with a given customer, for example.  The same customer could have had a different record for each channel, then more information was socked away in the matrixed support functions.   Furthermore, the customer records were all laid out differently, and had different addresses, contact information, billing instructions, and so on.

Maybe I’m oversimplifying, but shouldn’t one customer have one file with the same information that everyone uses?  It seems to me that if applied in the real world this data-oriented perspective could really make things simpler and more cost effective.