When bad software goes good

Many software developers still get it wrong. The three original tenets of good software development remain entrenched - meeting customer expectations, on time and within budget - but only 60% of process improvement projects show anywhere near the results they promised.

The figures vary by researcher and the individual definitions of success that they apply, but the end result is the same: a significant proportion of software projects still fail to deliver the intended value.

An August 2007 Ambysoft report based on 586 responses states: "Respondents reported that agile software development projects have a 71.5% success rate, traditional projects a 62.8% success rate, and offshored software development projects a 42.7% success rate." The full report can be viewed at http://www.ambysoft.com/surveys/success2007.html.

Ambysoft has a looser definition of project success than does Standish Group`s Chaos Report, an industry standard that demonstrates more alarming figures.

Chaos 2004 shows that 53% of software development projects were challenged, 18% failed and 29% succeeded.

In the 1990s, the predominant strategy was to employ all-encompassing processes deployed by a flood of consultants. But the processes were too cumbersome and certainly missed the low-cost, rapid development boat.

Another issue was getting people to follow processes. Processes are not designed for people because people do not work sequentially. They switch between tasks. They start with something and midway continue with something else.

In software development, people are knowledge workers. Consider how much time developers spend thinking versus actually typing code. In my experience, developers spend 80% of their time thinking. The problem in a process world is how to map the thinking process. It`s almost impossible because people don`t think sequentially.

Ironically, another problem with processes is that they try to be complete. What starts out as a good idea for one person rapidly becomes a gargantuan and unwieldy book - that incorporates every eventuality - which not many are going to read. People just want to focus on what they need to do their jobs and they quickly skip most, if not all, of the steps.

With the failure of the bulky, unwieldy processes, vendors took the next natural evolutionary step and suggested developers and their organisations select parts of the processes they needed; a great theory that never cracked it in practical application. The problem was that a shopping list mentality was pervasive as people decided they needed all of the processes because, logically, they were all deemed necessary.

When that failed, vendors came up with agile programming. It`s an approach that recommends that good developers will naturally figure out how to deliver good software. But that too has evolved since agile advocates realised that at least some process is absolutely essential.

The next most recent result, currently employed by many organisations, was a conglomeration of the different schools of thought as to just what is necessary: another process book from which they`re supposed to select the most suitable but which nobody has the time to read, and which led to yet another misalignment of described processes and application.

The corporate world has rejected both the teen whizz-kid and double-breasted suit approaches to coding.

The new kid on the coding block is a fusion - to borrow modern culinary parlance - of the two. Organisations are beating the challenge to deliver good quality software at low cost by promoting practices, not processes, to first-class citizen status.

The world first heartily shook hands with practices under a different guise - best practices - in the 1990s. The problem was that best practice for one business was not necessarily best practice for another. Integrating different strains of heavily dogmatic best practices proved nigh impossible and, like their forebears, they were rejected. Simply put, practices are a subset of processes, each complete in themselves. They can be selected individually, implemented and measured. For instance, think about requirements capturing as a practice. Two example variants would be use case modelling and user stories. On the one project I could employ use case modelling as a practice and on the other user stories - both in the same organisation. This allows me to select the best fit for the project context.

Today practices - not best practices - describe what we are doing and process follows how we do it.

Practices must focus on the essentials; they need to be quickly read and absorbed; and they need to let people do what they do best: figure out the rest for themselves.

Organisations that adopt this approach should distil practices to the bare essentials and flesh them out in the context of their own operations.

Over time, they will employ development practices and extend them to become best practice for their own business. Essentially, they start small and finish the picture themselves.

Building practices enables a rich process ecosystem contextualised to specific businesses.

Practical application develops the agile kernel, the essential practices, and groups additional and specifically necessary practices around that for governance and compliance, configuration management, quality management, architecture management, requirements definition and management.

Practices, not processes, will give organisations good software, quickly and at low cost. But remember, while organisations may have the necessary practice and process tools, they still require motivated and competent people.