Project mismanagement

In 1995, the Standish Group published its first Chaos Report. The introduction refers to a humorous comparison between building bridges and developing software-based business solutions. In 1986, Alfred Spector of Transarc Corporation suggested that bridges are normally built on time, on budget and do not fall down. In contrast, software never comes in on time or on budget - and it always breaks down.

Based on assessments of 8 380 software application development projects, the Chaos Report found that only 16.2% of these projects could be regarded as successful: delivered on time, on budget and with their features and functions working as initially specified.

The remainder of the projects were either challenged (52.7%) or cancelled (31.1%). A challenged project was defined as completed and operational, but with overruns on budget and time. It also delivered less features and functions than originally specified.

What's really significant about the budget overruns is that they averaged a whopping 189% of the original forecasts.

Back in 1995, the failure-cost to American corporates and government agencies was estimated at $140 billion, with cancelled projects costing $81 billion and challenged projects accounting for $59 billion.

The report also argued the regimented and inflexible 'Waterfall' approach to development was a major contributor to the dismal success rates. That opinion was reinforced by a study of 6 700 projects featured in the 1997 book: Applied Software Measurement: Assuring Productivity and Quality,by Capers Jones.

Commenting on this study, the renowned software development methodologist, Craig Larman, says: "Four out of the five key factors contributing to project failure were associated with and aggravated by the Waterfall model."

In response to growing acknowledgement of the inadequacies of traditional project management approaches, greater attention began to be focused on so-called lightweight methods. In 2001, a group of 17 eminent software engineering practitioners, including Martin Fowler, Steve Mellor and Alistair Cockburn, published the Manifesto for Agile Software Development.

Although not unique in its advocacy of the principles, practices and methods that underpin Agile - much of which had been highlighted by people such as Dan Gielan and Tom Gilb - the manifesto did start a movement which fundamentally changed the way software-based solutions would be built.

Notwithstanding Agile's early detractors, Gartner stated in 2006: "It's a fact that Agile efforts differ substantially from Waterfall projects. It's also a fact that Agile methods provide substantial benefits for appropriate business processes."

These benefits can be summarised as:

* Greater responsiveness to changing business needs;
* Improved return on investment resulting from quicker delivery times;
* More effective risk management;
* Greater predictability of costs and schedule;
* Improved project governance due to greater visibility of project status;
* Higher productivity of the development team; and
* Increased quality of the delivered solution.

The most recent Chaos Report indicates the success rate of projects during 2012 has reached an all-time high of 39%. Although the success rate has more than doubled since '95, many billions of dollars were still allocated to projects that were either cancelled or did not deliver on expectations, and overran on budgets and time.

What's really significant about the budget overruns is that they averaged a whopping 189% of the original forecasts.

This is despite a substantial uptake of the Agile approach. About 49% of businesses surveyed in 2012 for The 7^th State of Agile Development Survey from VersionOne* stated they apply Agile methodologies.

Agile is not a panacea for project success. Nor is it an antidote to project failure. Many companies have lost sight of the fact that Agile represents a 'philosophy' and is not in and of itself a methodology. In fact, there are many methodologies that implement the principles of Agile to varying degrees. Examples include the Unified Process, Extreme Programming, Scrum and Lean Software Development.

All too often, companies will 'standardise' on a single, specific Agile methodology (Scrum being by far the most popular), and will build the capacity and capability of their development organisation around that choice.

Just as the nature of the bridge to be built will determine how best to build it - think of a wooden footbridge over a small stream versus the 4km Akashi Kaiky? suspension bridge in Japan - it should be clear that a one-size-fits-all development approach is not realistic.

It should also be clear that Agile was initially positioned as setting the right mindset when embarking on development projects. The attributes that comprise the right mindset were defined in the 2001 manifesto's Twelve Principles of Agile Software. The key point here is that each of the succinct principles relate to what must be done, not how to do it.

All Agile-driven methods are equal. Depending on the task, some are more equal than others.

When selecting a project methodology, companies must consider a number of factors that will influence the success of a specific project. These include: the degree of novelty of the solution; the maturity of the company (both the development team as well as the business it supports); the business and technical risks associated with the initiative; the extent to which requirements are known and the ability to make appropriate trade-off decisions as the project unfolds; the scale of the solution; and its effect on the business.

Understanding such factors informs the selection of methods and techniques that are most suited to delivering a successful project. To stress that point, it's perhaps worth considering that while a GPS will show and tell a user the way, it has no influence on the transport best suited to either the route or to carrying whatever must be conveyed along it.

This means companies have to adopt a toolbox approach to their development. They must also place far greater emphasis on truly understanding the motivation for developing software solutions, rather than simply going through a recipe-based mechanistic application of a specific methodology.

In other words, businesses shouldn't ask if their project teams are Agile. They must instead ask if teams are meeting business needs.

* Of the 4 048 individuals polled in VersionOne's survey, over 80% were in software development or IT departments, and most commonly (34%) worked as project managers, Scrum masters and team leads - followed by 27% who were members of a software development team.