We take a trip into the world of test intelligence, coverage profiling and test gap analysis. How do you analyze development and test artifacts to efficiently identify errors? How do you evaluate the discrepancy between expected and actual results? And how do you incorporate all of this into the pipeline? Analysis tools promise continuous improvements in test quality and efficiency. But what is behind them?
“We did a study ourselves to see whether we as developers really manage to cover the functionality in the developer tests” - Elmar Jürgens
Dr. Elmar Jürgens completed his doctorate on static code analysis and was awarded the Ernst Denert Foundation’s Software Engineering Prize for his doctoral thesis. He is co-founder of CQSE GmbH and has been helping teams improve their quality assurance and testing processes for ten years. Elmar regularly speaks at research and industry conferences and has received several awards for his presentations. In 2015, he was appointed Junior Fellow of the German Informatics Society.
Highlights of this episode:
In this episode, I share my insights from an insightful conversation with Elmar Jürgens about Test Intelligence. We uncover the importance and application of analysis techniques in the development and testing process to find bugs more efficiently and improve the quality of software development.
Today I had the pleasure of welcoming Elmar Jürgens to the podcast. We talked about the topic of test intelligence. This term may seem abstract at first, but it encompasses a range of analysis techniques that aim to find bugs in less time and thereby significantly improve the software development process. From static analysis to coverage profiling, our conversation covered a wide range of topics that are all brought together under the umbrella of Test Intelligence.
One of the most prominent topics of our conversation was undoubtedly test gap analysis. This technology enables teams to uncover untested changes before the release. By merging data from the version control system and coverage profilers, a complete documentation of all test steps is created - a significant step towards improving quality assurance. It is particularly impressive that not only automated tests are taken into account, but also manual ones - a decisive extension of the conventional understanding of test coverage.
Elmar emphasized that coverage profilers have been used primarily for automated tests for years. The fact that we can now successfully use these technologies for manual testing marks a significant step forward. This development clearly shows that the boundaries between manual and automated testing are blurring and we are moving towards a more comprehensive view of testing.
Elmar’s experience with various projects revealed some surprising insights: In one case, particularly low test coverage even led to the termination of a project. Such moments highlight the importance of thoroughly analyzing and monitoring test coverage. It also shows that even in technically skilled teams, not all changes are always adequately tested - another argument for the use of test intelligence.
Our conversation also provided an exciting outlook on what we can expect in the future in the field of test intelligence. From using the same technologies in production to introducing new analytics such as test smells or requirement smells, the possibilities seem endless. This continuous progress promises not only more efficient error detection and correction, but also an overall increase in quality in the software development process.
One particularly interesting aspect is the combination of static and dynamic analyses. By merging both approaches, we can get a more precise picture of where our code stands and what needs to be improved. These hybrid analyses represent an innovative approach to better support both developers and testers and ultimately develop better software.