4 ways to make agile and waterfall work together

In theory, the two methodologies don’t mix. In practice, they almost always have to. Here are four tactics to make these opposites work together.

Comments

Whether it’s a consulting engagement or an internal software project, IT departments like the flexibility and productivity of an agile project. But in selling the project to upper management and the finance organization, waterfall concepts will nearly always be introduced as requirements. If you think hard about it, agile and waterfall are nearly contradictory philosophies. So how do you get them to work together? In too many software projects, particularly those involving consultants, it’s a pretty uncomfortable marriage. Unlike the song, opposites do not attract.

Coping Tactic #1: Vigorous account management

Since waterfall involves a fixed specification as well as a fixed budget and schedule, the flexible (or only vaguely-defined) outputs for sprints will mean a fair amount of tap-dancing. However, this can work when waterfall’s “specification” is ill-defined (because the customer does not know what they really need) or malleable (because an executive decides to preempt things). But beware: this tactic only works when vigilantly and aggressively applied, forcing conversations about de-prioritizing deliverables and avoiding scope-creep throughout the project.

Best Practice: Explicitly manage expectations in writing at the start and end of every sprint. Further, have a customer signoff for each feature delivered during the sprint.

Worst Practice: Head-in-the-sand behavior, hoping that the issue will just take care of itself.

Coping Tactic #2: Change-order extravaganza

In waterfall projects, clients are used to the idea of change orders (and of course your contract stipulates them, right?). So, use that mechanism early and often. In one $700K project we were involved with, over 50 change orders were issued (even though there was only a tiny overrun). The change orders need to be included as formal addenda to the statements of work (SOWs) and include a customer signature so they have contractual impact.

Best practice: Promptly issue a change order for every vaguely-specified item when the vagueness gets ironed out. Change orders should include both schedule and cost impacts, as well as feature details.

Worst practice: Delay or even skip the issuance of change orders, leaving it to “a verbal understanding” that will quickly be forgotten.

Coping Tactic #3: Explicit closure of discovery

In typical software projects, an initial fixed-cost phase (comprising 10 percent to 25 percent of the total project effort) is devoted to the requirements discovery process. The output of that process is the statement of project functional requirements. Even though it is common to discover many details after that first phase, these new items are not the project implementer’s problem: they are the consuming or client organization’s problem. They should not be accepted as binding requirements without a change order. Unfortunately, I’ve seen projects that were still in discovery after deployment.

Best practice: The project requirements document produced by the discovery process should include verbiage indicating that the contents are the totality of binding requirements for the contract, and include a VP signature from each major department in the consuming or client department.

Worst practice: Let it slide, accepting new requirements at any stage of the project without a change order.

Coping Tactic #4: Stipulating a not-to-exceed for each major functional area

The tactics so far have focused on coping mechanisms during the project, when it’s already too late. Instead, this tactic focuses on the verbiage of the SOW, in an attempt to meld just one agile principle into a waterfall framework. In the SOW’s deliverables section, each major section (typically between 5 and 10) needs to have the following words at the end:

Notwithstanding the foregoing, the total effort involved in implementing and deploying this functional area shall not exceed X hours. When the effort for this area reaches 25 percent, 50 percent, and 75 percent of that limit, consultant may notify client that a task review meeting is required. During that meeting, the consultant will brief the client about project variances or discoveries that endanger deliverables, and a recommended path forward. After that meeting, client will issue a change order request indicating the desired modification to requirements, schedule, or budget.

Best practice: Take the small stuff in stride, but request a task review meeting for any task that goes into red status or is blocked for more than a few weeks. Failure to put tasks in red status is a firable offense for the project lead.

Worst practice: Omit this verbiage from the SOW, and during the project avoid the whole issue until UAT.

But then…

While all of the above tactics work, to a degree, they are trying to bridge a contradiction. Of course I advocate real agile projects, but many a client organization or its management just can’t deal with flexibility and insists upon checklists of fixed deliverables. If so, you need to know that before you sign the contract. You may be better off just sticking with pure waterfall and managing things that way. Of course, you’ll need to jack up the bid and lengthen the schedule, but at least you will be managing risk in a coherent way.

The root cause

In most custom software projects, the requirements are not specified in advance with enough detail to truly support waterfall bids and tight project management. For CRM projects of any complexity, I’m going to go as far as to say it’s an impossibility. Why? What are the odds that:

The users will state their requirements with such thoroughness and precision that you won't misunderstand them?
The users won't change their minds about details and priorities during the course of the project?
The client’s channels, competitive environment, and internal business rules won't change during the life of the CRM project?
The client understands their data, business processes, and cross-system integrations so well that there won't be any surprises during the project?
The client is able to foresee all the government regulations, customer contractual stipulations, pending litigation, and internal standards you'll need to comply with?

Consequently, some level of discovery will be happening throughout the project…which means your bid will have to have some padding to deal with the “dark matter” that is scattered throughout the deliverables list.