Agile development needs a new testing strategy

Recent studies have confirmed the increased popularity of agile software development. For test managers accustomed to the traditional waterfall approach, this means a change in testing strategy.

Automated software testing will play an important part in the new test strategy, specifically in agile software development projects.

Adopting agile poses some challenges to testing and specifically to automation. In traditional automation, an automation specialist will automate a test as part of the regression suite, after the test has been executed manually by a test analyst. This approach still has its place in agile, but not as the only use of automation in the testing department.

An automation tool in the right tester`s hands can add enormous value to the manual testing effort. Computer-assisted testing focuses on tools to empower a tester to test better and faster.

Not all testers have development skills, but most junior testers entering the market today have these skills, and computer-assisted testing will benefit them the most.

I have seen a tester pair with a developer for one sprint. Both understood what was required and set about performing their particular tasks. While the developer did the coding, the tester developed the test cases, and then wrote a data-driven script to execute these test cases on the GUI. On completion of the development task, the automation script was executed against the development environment.

Almost 30 defects were found on the first test iteration. The good news: the developer could fix these immediately, recompile and retest using the same automated script. This also freed up the tester to focus on integration and exploratory testing.

This approach might not work for every story, or even every sprint, but it proves that automation can add value earlier and not just after the story is marked as done. The challenge will be to identify tasks suitable for automation.

As we adopt agile, we need to change our automation strategy.

For automation to succeed, there must be an effective strategy covering tools and resources using these tools, and how both of these add value to the overall testing strategy. Automating everything might be a strategy, but not an effective one. It will not only create a maintenance nightmare, but it will be very expensive and take a long time.

As with any project, automation benefits must be measured in relation to their cost and also to what extent the original objectives have been achieved. Before embarking on any automation exercise, be clear on why automation is considered in the first place.

For example: if the objective is for automation to cover regression testing, ensure there is no duplication of effort in the manual testing strategy. Don`t spend money on something that is not going to be used or add value.

An effective automation strategy will focus on immediate, short- and long-term benefits:

Immediate benefits (small investment, long-term benefits): Scripts that take one or two days to develop and can add value immediately, such as scripts that will assist testers to set up test data, which normally would have been a time-consuming task, and scripts developed when pairing with a developer in a sprint.

Short-term benefits (medium investment, long-term benefits): Scripts that can perform environment build verification or smoke testing. These scripts will test basic functional areas of an application after every new build, and should execute in under 15 minutes. A variant of the smoke test script can be a mission-critical verification script, which executes against high-risk areas of an application.

Long-term benefits (long-term investment, long-term benefits): Regression test suites covering horizontal and vertical tests within the application. Horizontal scripts will cover high-level functionality across an application, and vertical scripts will test each high-level functional area in depth.

The testers of the future will use automation tools on a daily basis, not just for regression testing, but also functional system testing. This will create more challenges, especially in companies using traditional, heavyweight, costly record-and-playback tools because expensive user licences will not be available to every member of the testing team. An alternative is to use open source test tools for sprint testing, and commercial tools to handle the regression test suite.