Firstly, let’s get this out of the way: I like Waterfall. It’s definitely got a place in the toolbox. As a methodology, it’s been around for a very long time… I’m pretty sure they didn’t use Agile to build the Pyramids. It’s a great methodology if you have to make complex but atomic changes, such as cutting over a road traffic management system, or doing a PR launch: there are many situations where iteration isn’t possible, or the iteration length would be months (at which point, you don’t have an Agile process, you have a series of waterfall projects running in consecutive phases, regardless of what you may think is actually happening…). But here’s the thing: Waterfall predates the creation of the Corporation by thousands of years, so has become part of the invisible fabric permeating the enterprise, like Payroll services or Legal departments. It’s the default position and like any tool it shapes the wielder’s perspective: if you have a hammer, all problems tend to look like nails. You can tend to deliver a project or create a product using Waterfall with a known and understood success rate, which makes it a favoured option for the enterprise since there’s a lot of emphasis on reliability of outcome.

There’s a higher level question though. If you didn’t have Waterfall, would you find new ways to do things? Would they actually be more appropriate? This leads to the question of whether doing things differently increases the success rate enough to offset the risk and cost of retooling the enterprise to a new methodology.

Hardcore Agilistas tend to insist that if everything’s done to the letter of the manifesto, impediments will be overcome, management structures will melt away, everyone will be empowered and the people will get what they want. To me (and leaving aside the echoes of Karl Marx) this misses the point of Agile, in that it is a process that introspects itself. This is its superpower: where Waterfall has shaped the entire enterprise over time to look like it, Agile is capable of adapting to its environment as well as shaping it. You don’t have to be a Prussian Field Marshall to understand that no battle plan survives contact with the enemy; a methodology that is able to adapt will survive.

Westfield is a large organisation, and specialises in digging very big holes in the ground, pouring in concrete and earning rental income for the next two decades.  Fair to say that it does this very well in that they’re able to repeat the process successfully and have most of the processes down to a fine art. It’s Waterfall’s home ground.  But how does a property management company used to dropping a billion dollars to rebuild a sizeable chunk of the East End of London address a clear and present danger to their core business with the seismic changes due to the rise of retail ecommerce?

Let’s look at the ingredients of the problem: lack of deep knowledge of the domain (online retail), needing to create an ecommerce offering in a rapidly-evolving landscape, while building a new business from the ground up.  By the time that a traditional Waterfall project had gathered the business requirements, they would already be out of date.  This was why we went for an Agile approach, focussing intently on building and releasing in tight iterations, always delivering something new into the production environment, looking at our direction versus the landscape and modifying accordingly. It’s fair to say that the feature list we started with was not the one we ended up delivering; by deploying something to the market we were able to get an early insight into what was and what wasn’t working and act accordingly.  Another important factor is accommodating a rapid growth in team size and team functions; roles were defined and redefined several times as operational pain-points became clearer.  Often it seemed like whack-a-mole with new problems arising in one area as previous problems were resolved in another.  Here, choosing a particular Agile methodology such as Scrum is not as important as its underlying principle of continuous introspection and improvement through use of the end-of-sprint retrospective to air pain-points and a commitment to re-engineer processes to remove them.

So for us, was the cost of retooling for Agile outweighed by the benefits?  It’s certainly reshaped (and is still reshaping) the way we do things. In retrospect, there really wasn’t another option.