By now Agile has taken over waterfall as the dominant app dev project pattern*. In many large organizations, the traditional waterfall PMO also owns Agile projects. One aspect of PMO oversight that can work against Agile culture is the project audit. If the goal of an audit is to ensure the project reflects Agile values, it can help ensure working software and a satisfied customer. If not, an Agile project audit can reinforce process, documentation, and other values that don’t directly promote project success.

In this post I’ll briefly review the Agile Manifesto, recount some prominent advice for auditors of Agile projects, and offer suggestions for auditors who want to reinforce rather than suppress Agile values.

Reviewing the Agile Manifesto

The Agile Manifesto is a 68-word, three-part statement of values concerning software development, accompanied by 12 supplementary principles. Here’s the Manifesto proper in its entirety (their emphases):

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• • • Individuals and interactions over processes and tools
    • Working software over comprehensive documentation
    • Customer collaboration over contract negotiation
    • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Take a moment to consider what’s not included in the Manifesto. There’s no mention of scrum, SAFe, iteration, sprint, retrospective, train, standup, scrummaster, product owner, and so on. Mirroring Frederick Brooks’s “No Silver Bullet”, I believe that the Manifesto emphasizes the essence of successful software development rather than its accidents, without discarding care of the accidents. Remember, the Agile founders said that there’s value in the formal structures around software development, but they valued individuals and interactions, working software, customer collaboration, and responding to change more.

Prevailing Advice for Auditors

Two representative sites I found in June of 2019 with this search seem to focus more on the values on the right rather than those on the left:

• This one, focusing on government projects, provides a nice introduction to Agile and makes the valid point that “Agile development does not negate the need to capture information needed to ensure continuity, consistency, and repeatability when team members turn over.” However, it undercuts its repeated praise of Agile values with the direct statement that “user stories should be written to the most granular level possible.” Part of the point of the Agile method is to avoid trying to define things you can’t know too early, and instead to define just enough to get started and let requirements evolve naturally with development of the application.
• This one came up first on my search. It gives these “reasons…one would carry out an Agile audit:”
  • “Ensure the project is being delivered using an agreed Agile approach.
  • “Validate that the Product Owner and Scrum Master are competent.
  • “Assess the Backlogs of work to be completed.
  • “Ensure engagement for the Sprint Teams.
  • “Ensure that the customer’s requirements are being managed.”

While these checks are useful, none directly concerns individuals and interactions, working software, customer collaboration, or responding to change.

Values-Based Audit of Agile Projects

Rather than focusing on structure and process, auditors can promote healthy Agile projects by emphasizing Agile values in their evaluations.  Here’s how I would approach it:

1. Assess individuals and interactions: Interview team members, product owners, and scrummaster to evaluate team camaraderie and morale. In the interviews, assess skills and performance versus project needs, and individual satisfaction/workload. Look for recorded notes of retrospective meetings, or other forums in which the team reflects and improves capabilities and performance. In an effective Agile team, the auditor should find that normal project self-reflection and open communication reveals and resolves challenges as they arise.
2. Assess working software: After familiarizing her/himself with the application and associated documentation, the auditor both reviews records from the test process and interviews representative customers to determine whether or not the system works to their satisfaction. Subjective responses from the customer are valuable regardless of how well grounded. Part of the team’s responsibility is to communicate effectively with the customer, and if the product works but the customer doesn’t know it, then the team should take steps to improve customer communications so that requirements are more accurate to their needs.
3. Assess customer collaboration: Step 2 will reveal much about customer collaboration, but the auditor should explicitly ask the team how they collaborate with customers, then similarly ask customers the methods and effectiveness of collaboration. Again, lack of consensus between the two parties will indicate need for improvement.
4. Assess response to change: Change management should come naturally to a healthy Agile project. Short (two- to four-week) iterations offer opportunity to change direction quickly as business needs change. In the conversations noted above, the auditor should be on the lookout for indications that this normal process isn’t operating as it should, like long iterations or the team being resistant to changing existing logic.

Only after evaluating the primary values on the left should auditors turn to the values on the right. Likewise, projects that show significant problems with respect to primary values should work to correct those issues before focusing on procedure or documentation. Of course, a poorly performing project will likely show problems on both the left and the right, but correcting process and documentation issues without correcting software quality and collaboration is just a temporary fix.

* There are plenty of signs that Agile project methods have taken over as the predominant app dev pattern. For just two examples, there’s this undated study declaring that Agile is “the new norm”, and an article published in Forbes over a year ago with the headline “Why is Agile Eating the World”. As a project manager, SQL developer, and data analyst for various Fortune 1000 companies, almost all of the work I’ve done in the past 15 years has been in an Agile context; two of the four exceptions were notable failures in part because they used waterfall to pursue poorly defined objectives.