Executing test estimation

After establishing what needs to be estimated, when test estimation should take place and why it should take place, there are various approaches to executing the estimation.

Intuition or guesswork can be used as a starting point. This would typically not give good results, but may form the basis for improvement. Making some estimate for later comparison is better than not making any estimate at all.

If an estimate is based on previous experience (on the same or similar systems) it can lead to fairly good testing effort and duration estimates. Some companies have estimation standards in which they have formally documented an approach incorporating their successful estimation methods.

Creating a detailed work breakdown structure of all anticipated or planned test activities is a good starting point for test estimation. If this is prepared in a spreadsheet then cost in hours, prerequisite tasks, schedule and monetary value can be estimated and planned for.

A further benefit of using a detailed work breakdown structure for test estimation is that other stakeholders in the project can understand how the estimates were derived, and they may be able to point out problems with the work tasks chosen or the detailed estimates for one or more work tasks. (An example might be for a colleague to remark that no time estimate was included for test set-up and breakdown.)

Detailed work breakdown structures allow early, wide participation in - and buy-in to - the test estimation process, and will be more likely to motivate management spending when testing tasks and schedule are clearly described.

A shortfall of this approach is that it is time-consuming, and a more rapid approach might have demonstrated that the order-of-magnitude of costs for testing might have disqualified the project at the outset.

In this approach, use function points to indicate system size and suggest test estimates related to this size. Test points use function points, and then go on to modify the test effort/duration by applying modifiers specific to testing.

Test estimation can be top-down, bottom-up, or a combination of the two approaches.

Wayne Mallinson, chairman of Test and Data Services.

A simpler, formula-based test estimation method might simply be to take a percentage of the software development effort for testing. Examples would be that 50% of the project budget should be devoted to testing. These formulae are often experienced-based and relate to benchmarks in various industries. Collected company metrics can be used as a kind of recorded experience; so if, for example, 100 tests on a system took two days to complete, we might expect 150 tests to take three days to test on the new, similar system.

Test estimation can be top-down, bottom-up, or a combination of the two approaches. Top-down approaches have the disadvantage of not involving all the testers and thus not getting full buy-in from them (or easily overlooking important experience and knowledge held by them).

A bottom-up test estimation approach can be slower and more costly, as the participating team has to be informed and trained, and then allowed time to come to a result. Combining and balancing the two approaches tends to balance the expectations of the test manager, and at the same time get good participation from the testing team.

It is a good idea to have test estimates reviewed by all project stakeholders, as they may find problems with the estimations or their underlying assumptions. If other stakeholders are invited to review the estimates and find them to be reasonable, then testing buy-in, budgets and co-operation will be easier to attain.

Before holding such a review, note that it is always good practice to use at least two estimation methods and be sure the results correspond fairly closely. I personally like to use three methods to check the reasonableness of testing estimates I make.

This should hopefully have provided a way to begin testing estimation and helped you develop a method that is right for you.

In my next Industry Insight, I will discuss the influences that affect test estimation, the benefits of doing a good estimation, and finally, what to do afterwards.