The landscape is constantly changing for modern application architectures today, as multiple parties inside and outside of companies add or modify the individual components that the application is dependent on.
Since composite applications involve many changing parts that are interdependent, testing capabilities must evolve from: "What went wrong?" after the fact, to a proactive approach of: "If I do this, will something go wrong?" It must continuously and proactively be validated that changes in one component within the application will not adversely affect other components and systems downstream. Unlike the client-server days, the test phase does not end after deployment, but instead is needed continuously, including production.
If a company increases an application's complexity and interdependency while holding the delivery schedule and costs constant, its quality will suffer. However, it is possible to improve quality and efficiency with the same resources if the company improves collaborative testing across the life cycle.
Barriers to entry
To achieve this, tools and methodologies must overcome certain obstacles:
* The defect resolution process between development and QA is typically manual, iterative and costly, and scales poorly as application complexity grows. It requires expertise and detective work to trace the problem, and may not reliably resolve the issue, as manually recreated test conditions may not reflect the live failure scenario or may be inconsistent when repeated.
* Test and validation responsibilities must be better shared and co-ordinated across development, QA and other stakeholders. For example, developers may perform unit testing and code coverage to find structural bugs within a component, but when done in isolation, it does not verify that a component will meet real world business requirements.
* Agile programming methodologies, which consist of shorter development iterations of a smaller feature set, can enable organisations to be more responsive to changing needs. However, without equally agile test and validation efforts accompanying each iteration, a company won't be able to release new features with quality, effectively making quality assurance a governor on the velocity of the company's release cycles (this was discussed in the previous Industry Insight).
Other key enablers to collaborative quality include: portable, executable test cases that are easy to extend, easy to chain into workflows with other tests, and simple to integrate with existing test repositories.
These test cases must be loosely coupled from upstream/downstream dependencies so they can be easily shared across different teams and environments, including the ability to execute them with different underlying data, along with the ability to easily attach prior results and other artefacts to extend them further.
With better collaboration and shared testing assets, each team can contribute earlier to the quality of the application while leveraging the efforts of other teams. For instance, developers may initially deliver a few simplified "happy path" tests to QA. QA can then take that test and iterate on it to ensure it holds up to the expected scenarios in use. These same tests are handed to production teams for systems integration tests and continuous monitoring.
Business analysts can also leverage these to test at a higher level, while still communicating issues efficiently back to the larger team, and in doing so, the software development life cycle becomes more streamlined.
Continuous monitoring is an integral part of any DevOps adoption strategy. Back in the mid-1990s when client-server architectures gave way to multi-tier applications, no one realised that application performance management (APM) would contribute to the delivery of new services to market - faster, with higher reliability, and at a lower cost. This is exactly what APM brings to the software delivery life cycle today.
On your marks
Most companies already have a head start in their DevOps adoption, since APM is integral to any large company's success in managing an ever growing number of applications and services in production. Leveraging their investments in APM, IT operations can collaborate better with development to ensure that crucial feedback loop is closed.
The test phase does not end after deployment, but instead is needed continuously.
IT operations are sitting on a huge amount of production behavioural/performance data. This data can then be used to create virtual services, which perform with production characteristics in order to simulate 'lifelike' production conditions during pre-production development and testing.
Take a look at the following example:
A company is developing a new order management service (OMS) using agile methodologies and continuous delivery mechanisms, as discussed elsewhere in this tech series. IT operations will use APM to monitor the performance characteristics of both the OMS system but also the called back-ends (mainframe, Web services, ERP, etc). IT operations also notices that certain back-ends are much slower to respond under very high loads, causing the end-user experience to suffer.
In order to accelerate product development, the CIO decides to leverage service virtualisation in the SDLC to reduce the time to market for OMS 2.0. Using data mining, the development team is able to create virtual services for all the out-of-scope systems, and decorating it with the performance characteristics of the production systems. This allows the performance engineering team to create a 'lifelike' test environment to sign-off the behaviour of OMS 2.0 under high loads.
The CIO is then happy, since it's proven that OMS 2.0 will be able to cope under excessive load in production with an improved end-user experience.
The bottom line is:
It is important for companies to understand that DevOps is more than just a set of 'tools'; it's a process, and companies should embrace all parts of this process to fully realise the benefits it offers.
Share