Software engineering quality is about more than finding bugs

Nelisiwe Mbense, QA Lead at Retro Rabbit / SmarTek21.

---

Nelisiwe Mbense, QA Lead at Retro Rabbit / SmarTek21, explains why organisations need to test beyond the obvious, and why getting software quality right takes more than automation.

Software can pass its tests but still frustrate users and expose problems that nobody anticipated. These gaps can become punitively expensive for companies relying on digital platforms to serve customers and run critical business processes.

Mbense says one of the biggest misconceptions about quality assurance (QA) is that it's optional. “Some see it as a simple job to just find bugs before having your software go live. It's not just about checking whether a button works. It's about making sure the client's expectations are met, plus understanding how the software behaves when people use it in ways not expected. More importantly, what will the end-user see when something does go wrong.”

Mbense expands: “A banking application, for example, might work perfectly when a customer follows every step correctly and has a stable internet connection. But what happens when the connection drops halfway through a transaction? Does the application recover properly? Could the customer submit the same instruction twice? And what happens if they leave the application open and return after their session has expired?” She confirms these are the questions QA teams need to ask before customers encounter the problems themselves.

“For large organisations, particularly those running complex digital platforms, the consequences of getting this wrong can extend beyond a frustrating user experience. Defects can disrupt business processes, create additional work for support teams, delay releases and undermine customer confidence as well as cost the business more resources to fix it. Sometimes a feature works exactly as the developer intended, but that doesn't necessarily translate into it working well for the end-user. Both scenarios need to be examined.”

Looking beyond what should happen, Mbense notes one of the challenges in software testing is that it's easy to focus on what’s supposed to happen rather than what can happen. “A developer might test whether a registration form accepts a name using a straightforward entry such as John Smith. But will the same form accept a much longer South African name, multiple surnames or a name with special characters? Will the information display correctly across the application, or will something break along the way?” she asks.

Mbense says these may seem like small details, but they can determine whether someone can use a digital service successfully. She says this is where experience and exploratory testing make a difference. “Rather than following only a predetermined set of test cases, a QA specialist actively investigates the software, tries different approaches and looks for weaknesses that may not have been anticipated during development. I spend a lot of time trying things that developers might not immediately think of, such as clicking on something in an unexpected order, leaving an application open, interrupting a process or trying something that isn't part of the normal user journey. My goal is to determine how the software responds in these scenarios.”

She says this approach also means thinking about how people actually use technology. “Users switch between applications, change device settings, lose connectivity and make mistakes. Testing needs to account for these realities, not just the ideal scenario described in a requirements document. At Retro Rabbit, we endeavour to predict the user’s preferred browser, screen size and responsiveness to make sure that the software is seamless no matter what platform is being used.

“For organisations with several systems working together, the same thinking needs to extend beyond individual features. A function may work in isolation but fail when it interacts with another system or when information moves between different parts of a business process. The aim is to identify these gaps while the team can still address them, rather than discovering them after a release.”

With AI increasingly finding its way into software development, organisations are also looking at how it can speed up testing. Mbense believes it has a role to play but cautions against assuming AI tools or automated tests can replace experienced QA professionals.

“AI can help us work faster and can be useful for generating test ideas and handling repetitive tasks. But someone still needs to understand what we're testing, why it matters and whether the results make sense. AI is a tool that predicts patterns; humans need to think first and consider behaviours and then address how they would do it.”

Automation is particularly useful for regression testing, which checks whether changes to an application have affected functionality that previously worked. These tests can be repeated whenever the software changes, giving teams quicker feedback without having to perform every check manually. “But automation can only check what it has been designed to check. If an important scenario has not been considered, a test can pass without revealing the problem,” cautions Mbense.

She notes exploratory testing addresses a different need as it allows an experienced tester to investigate unexpected behaviour, question assumptions and follow up on issues that emerge during testing.

For Mbense, the two approaches work best together.

“Automation is great for the things that need to be checked consistently. But when you're exploring an application, you might discover something you weren't even looking for. That's where human judgment comes in.”

She says the biggest question for businesses today is not whether to use AI or automation, but how to use these tools alongside the right skills and testing processes to improve the quality of what they deliver.

If it's not documented, can you prove it was tested? “Finding defects is only part of the job. QA teams also need to keep a record of what they tested, what the results were and which issues still need attention. This becomes particularly important in cases where several teams are working on the same product; or a release decision needs to be made; or when someone other than the original tester needs to pick up the work,” she says.

Mbense confirms that at Retro Rabbit / SmarTek21, tools such as Xray are used to manage test cases, record execution results and track defects. “This gives the wider team visibility into the testing process, rather than leaving that information in an individual's notes or memory. If the test cases, bugs and results are documented properly, other people can see what has been tested plus the detailed steps around how it was tested, what has failed and what still needs to be done,” says Mbense.

She says that shared information can help developers, business analysts and product owners understand the status of testing and work together to resolve outstanding issues. “It also makes it easier for another team member to continue the work when necessary. For organisations with complex applications and multiple delivery teams, this kind of visibility matters. A test report should do more than confirm that testing took place. It should help people understand the results and make informed decisions about what happens next.

“The level of documentation and reporting will vary depending on the system, the risks involved and the organisation's requirements. The underlying principle, however, is the same: testing needs to be traceable, and the team needs to be able to explain what it has done and what it has found.”

Mbense highlights that quality assurance can sometimes be seen as the part of software delivery that slows everything down, but this is the wrong way to look at it. “Developers finish building a feature, the QA team finds problems, and the release is delayed while those problems are fixed. People sometimes joke that QA is a developer's worst nightmare because we're the ones who find the things that don't work. But we're not there to catch developers out. We're there to help the team deliver a better product.”

The difference often comes down to when QA gets involved and how closely the team works together. “When testers are brought in only at the end, problems may be harder and more expensive to fix. When they work alongside developers and other stakeholders throughout delivery, they can raise questions earlier and help identify gaps before they become bigger issues. This is particularly relevant for organisations modernising existing systems or introducing new digital services. While a new functionality needs to work, it also needs to work with what is already there. Testing must be implemented that examines those connections and the impact that changes may have on the wider system.”

Mbense says for technology leaders, the value of QA should not be measured by the number of defects found. “It also lies in helping teams understand the risks, improve their processes and make better-informed decisions about software releases. Ultimately, QA engineers want the same thing as the developers and the business. We want software that works, that people can use without unnecessary frustration, and that does what the organisation needs it to do.

“That starts with asking better questions, testing beyond the obvious and making quality a shared responsibility right from the beginning of the development process,” Mbense concludes.