How do you automate tests when the applications are used by a wide range of devices - from new to outdated? The desire to automate everything cannot be fulfilled here, as the differences between the various operating systems are too complex. A combination of automation and manual tests manages the balancing act here. Anika and Benedikt take care of quality assurance for ZDF’s mobile applications and share their strategies for finding errors and how they decide which tests are automated and which are carried out manually.
“You have to find a good balance: What do I automate and on how many devices and what do I actually want to map? And how can I close this gap?” - Benedikt Broich, Anika Strake
Benedikt has been part of Appmatics since 2016 and supports the development of QA processes. As team lead, he works at the interface between test management and partners such as Klett, Kicker & Eurowings.
Anika has been working at Appmatics since 2019. Initially, she worked as a QA automation engineer. Today she is TeamLead Automation and supervises test procedures for partners such as ZDF, Migros & BVG.
Highlights of this Episode:
In this episode, I talk to Anika Strake and Benedikt Broich about the challenges of a test automation project for ZDF. They emphasize the importance of striking a balance between manual and automated testing to ensure both efficiency and flexibility.
Benedikt begins by explaining how Appmatics was awarded the contract by ZDF. The original tender called for complete automation of the tests. However, it quickly became clear that this was not the most efficient solution. Many apps did not have tags and IDs set correctly, which made automation more difficult. They also had to ensure that older devices were also covered. The solution: a mixture of automated and manual tests.
Anika explains in detail the importance of manual tests in the project. Especially with old devices or specific OS versions, they encounter unexpected problems. Manual testing allows them to isolate and fix bugs faster. While automated tests can provide a broad overview, manual tests allow a deeper investigation of specific problems. This two-pronged approach significantly increases test coverage.
Automated tests offer many advantages, especially for repeatable tasks such as checking links or performing standardized actions in apps. However, Benedikt also points out the limitations of this method. Changes to the front end or updates to operating systems can easily interfere with automated tests. This once again demonstrates the value of manual tests in order to be able to react to such changes at short notice.
An important aspect of the project is the regular evaluation of which test cases should be automated and which should be carried out manually. Anika mentions that they pay attention to several factors: Does automation increase test coverage? Are the functions critical? And how complex is the manual testing effort? These considerations help to optimize the use of resources.
After a year of intensive work, the team was able to make significant progress. The combination of manual and automated tests proved to be extremely effective. They found errors that they would never have discovered through pure automation, especially on edge devices or in specific usage scenarios. This experience clearly shows that a balanced approach to test automation is crucial for success.