Benefits of good test estimation
While there are many benefits of good test estimation, it is important to note the influences that are involved in an overall test estimation project.
Test managers, in particular, have specific dependencies on other stakeholders, but do not always carry as much authority as project managers, or the development director. Each of these role players influences the test approach and therefore the test estimation process.
Test managers` plans have dependencies on:
* Senior management for company culture and priorities;
* The business for test budgets;
* Business analysts for the quality of the requirements specifications;
* Systems analysts for the quality of the functional specifications and designs;
* Developers for the quality of the software components, the date, mode of shipping (phased or incremental), and the sequence of the software shipped;
* Project managers for actual project drivers (cost, quality and/or schedule);
* Other projects for access to key testing resources (such as the testing laboratory) which might need to be shared; and
* Infrastructure personnel for the availability, suitability and reliability of the test environments or to supply required testing hardware or permissions.
Better testing estimates lead to better planning, execution and monitoring of tasks under a test manager`s attention. Better testing estimates allow for more accurate scheduling, realise results more confidently, and form the foundation of good management credibility.
Good testing estimates and negotiation skills will improve the working conditions of the test manager as other stakeholders learn by experience to respect the manager`s estimates.
Monitoring of testing progress is more meaningful and motivating if the test progress was based on accurate estimates under normal or predicted circumstances.Wayne Mallinson, chairman of Test and Data Services
The extent of the testing depends on the amount and type of testing motivated by the testing policies, the strategic approach to testing, and the specific test plan details. Often the extent of testing is determined by some prioritisation scheme. Risk-based testing prioritisation is an example: higher risk applications would normally be more extensively tested than lower risk applications, and therefore a greater amount of testing would be planned and need to be estimated. This increased testing work will often be reflected in increased testing skills required, effort, duration, equipment, schedule and related testing costs.
Projects may be time-, quality- or cost-driven, or driven by various combinations of these factors. The test plan and estimation must take into account the most appropriate testing activities to meet these project drivers.
Monitoring of testing progress is more meaningful and motivating if the test progress was based on accurate estimates under normal or predicted circumstances. Reasonably accurate estimation of testing effort is important as management promises and expectations could rely on this. Payment of contracts, and performance evaluations of staff, could in some way rely on good testing estimates.
It is easy to forget certain elements from test estimates. Downtime or slack time may occur when systems fail and require time to restore. Difficult to repeat incidences may require more time than planned. Time for planning, preparation, results analysis and communication should not be forgotten.
Testing estimates based on experience can be improved by taking a past as well as a current and future view of the testing. For example: we were able to do 50 tests a day last year, now we can manage 60 an hour, and we believe that in three months this will improve to an average of 70 tests an hour as we advance on our learning curve.
Test estimation methods should be trainable and should achieve similar results from different, but similarly skilled or experienced testers. Inviting testing colleagues from other projects to review your estimates - or better still, to independently calculate and compare their estimates with yours - is a good idea if high accuracy and confidence in the estimates are important.
Peer reviews for estimates that carry a high risk is advised. For example, a company may be quoting for a fixed cost testing assignment on a project against other bidders; correctly estimating the task is important for the sustainability and credibility of the company.
Test estimation is an essential part of any project, and should be done as early as possible in the project. It sets realistic targets (budget, staffing and time) for the team. The testing approach strongly influences test planning and therefore the associated estimation of testing.
Test estimation should be top-down as well as bottom-up (participative), involve detailed work-breakdowns where time allows, always use three methods in order to triangulate and ensure reasonableness, attempt to use past, current and future indicators as inputs to the estimation, and obtain similar results from independent, but similarly skilled persons following the approach. (These results should give consistent results to the actual experienced results.)
The following groups should be asked to review an estimate:
* Experienced peers - for high-risk projects or assignments.
* Project stakeholders - to gain their support, and the benefit of their insight, experience and objectivity.
The test estimation approach should be standardised and include measurements where possible. It should be systematically reviewed and improved from experience. In-project improvement (as opposed to next project improvement) should preferably be sought for greater benefits.