Test Intelligence

We take a trip into the world of test intelligence, coverage profiling and test gap analysis. How do you analyze development and test artifacts to efficiently identify errors? How do you evaluate the discrepancy between expected and actual results? And how do you incorporate all of this into the pipeline? Analysis tools promise continuous improvements in test quality and efficiency. But what is behind them?

“We did a study ourselves to see whether we as developers really manage to cover the functionality in the developer tests” - Elmar Jürgens

Dr. Elmar Jürgens completed his doctorate on static code analysis and was awarded the Ernst Denert Foundation’s Software Engineering Prize for his doctoral thesis. He is co-founder of CQSE GmbH and has been helping teams improve their quality assurance and testing processes for ten years. Elmar regularly speaks at research and industry conferences and has received several awards for his presentations. In 2015, he was appointed Junior Fellow of the German Informatics Society.

Highlights of this episode:

• Elmar Jürgens and I talk about Test Intelligence and how it helps to find bugs more efficiently
• We discuss the importance of test gap analysis and coverage profiling
• Elmar explains how manual tests can be included in the analysis
• We will discuss how Test Intelligence can improve development processes and what the future trends are
• We talk about the challenges and benefits of introducing Test Intelligence in teams
• Elmar gives insights into the importance of change management when implementing analytics tools
• Discusses how Test Intelligence can help save resources and work more efficiently
• Elmar shares how Test Intelligence is used in practice and what it holds for the future

From test gap analysis to test suite minimization: Test Intelligence

In this episode, I share my insights from an insightful conversation with Elmar Jürgens about Test Intelligence. We uncover the importance and application of analysis techniques in the development and testing process to find bugs more efficiently and improve the quality of software development.

Test Intelligence

Today I had the pleasure of welcoming Elmar Jürgens to the podcast. We talked about the topic of test intelligence. This term may seem abstract at first, but it encompasses a range of analysis techniques that aim to find bugs in less time and thereby significantly improve the software development process. From static analysis to coverage profiling, our conversation covered a wide range of topics that are all brought together under the umbrella of Test Intelligence.

The test gap analysis

One of the most prominent topics of our conversation was undoubtedly test gap analysis. This technology enables teams to uncover untested changes before the release. By merging data from the version control system and coverage profilers, a complete documentation of all test steps is created - a significant step towards improving quality assurance. It is particularly impressive that not only automated tests are taken into account, but also manual ones - a decisive extension of the conventional understanding of test coverage.

Coverage beyond automated tests

Elmar emphasized that coverage profilers have been used primarily for automated tests for years. The fact that we can now successfully use these technologies for manual testing marks a significant step forward. This development clearly shows that the boundaries between manual and automated testing are blurring and we are moving towards a more comprehensive view of testing.

First-hand experience: surprises from the field

Elmar’s experience with various projects revealed some surprising insights: In one case, particularly low test coverage even led to the termination of a project. Such moments highlight the importance of thoroughly analyzing and monitoring test coverage. It also shows that even in technically skilled teams, not all changes are always adequately tested - another argument for the use of test intelligence.

A look into the future

Our conversation also provided an exciting outlook on what we can expect in the future in the field of test intelligence. From using the same technologies in production to introducing new analytics such as test smells or requirement smells, the possibilities seem endless. This continuous progress promises not only more efficient error detection and correction, but also an overall increase in quality in the software development process.

Static and dynamic analyses combined

One particularly interesting aspect is the combination of static and dynamic analyses. By merging both approaches, we can get a more precise picture of where our code stands and what needs to be improved. These hybrid analyses represent an innovative approach to better support both developers and testers and ultimately develop better software.