The status quo doesn’t work

Traditionally, software has been built using a waterfall methodology because that was the go-to option taught in most project management classes. Waterfall stems from early assembly-line principles—sequential, noniterative subject assembly. Each discipline that participates in the project operates independently. For example, analysts work independently from developers, and developers work independently from designers. They have their turn building the deliverable and then passing it to the next team. The practice received a lot of attention and widespread adoption once the Department of Defense formalized its approach to creating software through the military standard DOD-STD-2167.

The waterfall practice is attractive because, at least in theory, it allows the business to lock in scope, investment (or resources), and schedule. First, a comprehensive business case is built by an analyst. Then detailed requirements are collected by the subject matter experts, while data models get locked in by architects. The designs are crafted by the design team. Implementation is performed by engineering, testing executed before market release, and all while the project manager coordinates the delivery based on a fixed schedule, budget, and scope. The stakeholders and sponsors do not get to see the product until the very end after it has been tested and approved for market release.

Using these traditional methods, companies have been less than successful. Some stats reveal waterfall’s shortcomings:

• Microsoft research has shown that 50 percent of all software built using waterfall methodology is either never used or does not meet business objectives.
• Poorly defined applications (miscommunication between business and IT) are implicated in 66 percent of project failures, costing US businesses at least $30 billion every year (Forrester Research).
• 60–80 percent of project failures can be directly attributed to poor requirements gathering, analysis, and management (Meta Group).
• 40 percent of end users reported problems (Gartner).
• Up to 80 percent of IT budgets are consumed in fixing self-inflicted problems (Dynamic Markets Limited 2007 Study).

The evidence is clear: waterfall doesn’t work. Let’s deconstruct the process to learn from the mistakes of those that came before us. You’ll find that we need to solve more than just the delivery process.