Software testing in the future - Interview with Deutschen Bahn

Bettina Buchholz is Strategic Lead for Quality Assurance & Test at DB Netz AG and is Product Owner of the test-focused CI/CD pipeline MoQ-AP (Modular Quality Assurance Pipeline). Andreas Neumeister from CGI is an expert in test data management and test management. Tom Kober from CGI is a specialist in test automation architectures and test automation strategies. Together they are realigning the topic of quality assurance at DB Netz AG, with a focus on a sustainable test strategy and continuous testing.

What challenges will software testing have to face in the future?

Bettina Buchholz: In the past, there were dedicated testers who were responsible for “quality”. Nowadays, we no longer talk about testers, but quality coaches whose job it is to get the quality mindset into the agile teams. In doing so, they help the development teams and product owners to integrate an understanding of testing into every stage of the SDLC and uncover errors at an early stage using shift-left. At some point in the future, there will no longer be any quality coaches and quality engineers, as every single person involved in software development will intuitively pay attention to quality assurance and continuous testing. A continuous quality culture will be established.
In future, testing activities will no longer be perceived as a burden/additional work, which is still often the case today. They will become an integral part of the toolchain. The quality of the software product will be checked intuitively and automatically with every single software development activity.

Andreas Neumeister: As the level of automation will continue to rise sharply in the future, the manual work previously required will be freed up for other important topics. These include: Security testing, portability testing, testing accessibility, high availability and many other topics. With regard to test data generation, a mix of requirements-based automated microservices and AI-controlled data generation will become established. This will make many manual efforts in the collection of data requirements, data generation and data manipulation obsolete. At the same time, however, more effort must be invested in planning and setting up the test data strategy.

Tom Kober: Due to the current global situation, security vulnerabilities are of the utmost importance. Testing must therefore focus more closely on security aspects in order to avoid vulnerabilities at an early stage and in an integrated manner. In addition, new technologies and processes also require new testing approaches, which is why testing of serverless services, AI and machine learning, for example, will increasingly come to the fore. This includes not only the development of test methods for these technologies, but also the resulting opportunities in software testing in general.
A major factor when designing a test automation architecture is the effort required to maintain and develop automated tests. Self-healing tests and supporting test design tools are already being used to address this issue. However, these are usually not yet mature enough to create sustainable and reusable tests.

What ideas or solutions could address these challenges?

Tom Kober: Integrated and standardized scan tools enable proactive error prevention and create better awareness for software quality and security through continuous integration even before a test environment is provided. This allows valuable information on software quality to be shared among stakeholders and improvements to be implemented at an early stage. At the same time, compliance checks and standards can be defined on an organization-specific basis using individual rule sets.
Thanks to intelligent analysis options and the associated views, better assumptions and more precise tests can contribute to an improvement in quality assurance. As a result, testing efforts can be deployed in a more targeted manner, which can lead to faster test feedback or a reduction in costs, among other things.

Bettina Buchholz: Machine learning will become an increasingly relevant part of software development. Today, testing activities are still perceived as additional work. Machine learning will increase the acceptance of software quality and software testing activities and ideally make them a matter of course.
Test processes must be integrated with all activities within them, for example in CI/CD pipelines, and cover all test levels and test types, resulting in almost complete automation.
However, I see the biggest challenge in switching the mindset and establishing a continuous quality culture. Nowadays, we think too much in terms of “shift-left and shift-right” approaches. We need to move towards “quality overall”.

Andreas Neumeister: In line with the quality culture described above, development, security, operations and quality assurance will become more closely interlinked. This will lead to a development of the skillset of employees in the projects, but also to a further development of the technical procedures for these areas. Particularly in view of the increasing degree of automation and the use of AI technologies, the clean composition of test strategies, including automation and data, will become increasingly important. This is particularly helpful when carrying out comprehensive integration tests across heterogeneous system landscapes. As companies and technologies are constantly evolving over time, the ability to test continuously and across technologies and systems is becoming increasingly important.

What does future testing look like? How will we test?

Tom Kober: Open source tools are becoming increasingly widespread in connection with container-based test applications and form a cornerstone for the test infrastructure of the future. This enables the sustainable use of test automation tools and reduces the initial effort involved in developing the test infrastructure. Local test tools are only used for test development and static test environments are only used for manual tests and the development of automated tests. Due to the associated high degree of automation, particular attention must be paid to reusability so that these costs can be amortized.

Bettina Buchholz: I think we are well on the way to the Future of Testing. We have become much faster in test execution, but the feedback cycles are still far too long.
If we look in the direction of machine learning and AI, there are completely new possibilities for testing.
Test cases and test suites must become more maintainable or, ideally, maintain themselves. AI will make it possible for test cases to learn for themselves whether and in what form they are needed and what requirements the user or the system under test has.
“Experience-based testing” is also relevant for a risk-based testing approach, and here too AI will provide us with significantly more support in the future. For example, in future test data will be reproduced based on experience from previous test executions or test cases will be designed automatically based on evaluations of previous error accumulations. Test automation will also identify and rectify error states in its own system itself in order to eliminate sources of error from its own system. Quality assurance becomes a constant companion that is perceived as support in software development.

Andreas Neumeister: In addition, future AI quality assurance systems will analyze user behavior in existing systems. In this way, errors and weak points will be identified, repaired by the system itself and then automatically tested according to basic quality assurance guidelines and normal user behavior. Thanks to good monitoring capabilities, quality assurance is shifted more to the operations area. To make this possible and reduce the time-to-market for new products and features accordingly, sustainable analysis and monitoring functionalities must be designed and established. Knowledge of the function and status of a system in production is of fundamental importance for quality assurance in operations.

How can testers and test managers prepare for this today?

Andreas Neumeister: Testers and test managers should regularly reflect on their test procedures, preferably with the entire development team, and implement any optimization potential found. They should also exchange ideas with colleagues from other projects and read success stories from other quality assurance projects. Furthermore, they should constantly educate themselves in a rapidly changing world. The quality assurance approach of 5 years ago is no longer relevant today. As automation will become increasingly important in the future, test managers and testers should already be thinking about the topic in broad terms today. It is not a question of being able to implement automation technically. Automation potential must be recognized at an early stage in order to be able to set up an automation strategy for your own project.

Tom Kober: It is not only important to share an understanding of testing within the team, but also to involve testers and test managers in the various processes and technologies used in software development and projects. This promotes the exchange of information between different stakeholders and thus also passes on and internalizes awareness of software quality. This is essential in order to take the first step towards a collective understanding of software quality. Only in this way can the state-of-the-art technologies of our time develop into new software testing innovations in conjunction with proven testing practices.
There is already a demand that test automation engineers are no longer just responsible for their tests, but also for the automated test execution and integration of their tests into the SDLC. It is therefore important that they deal with topics such as DevOps, CICD and cloud platforms in order to create a holistic understanding of Dev, Ops and test.

Bettina Buchholz: From my point of view, the first step is that we shouldn’t be afraid of trying new things and being able to fail. Only by making mistakes can we improve.
Our main task at the moment should be to help the team members, product owners and project managers of this world to develop an understanding of quality assurance. As a coach, to support the mindset shift towards a quality culture.
As Tom Kober has already indicated: from a technical point of view, we testers need to get more involved with CI/CD and continuous testing. We need to find a way to make testing an integral part of the SDLC. Only if we achieve this will we make it easier for non-testing team members to get to grips with the topic and develop a tester mindset. This will create the basis for a future quality culture