As a deep learning enigneer, David explores the possibilities of using AI. It’s about his approach to AI-generated test cases, the limitations and the possibilities. We also talked about prompt patterns, what the smallest changes in the prompt do and how they can be designed to increase the quality of answers. Finally, David assesses the development of AI and has interesting tips for testers who want to start integrating AI into their workflow.
“However, I didn’t improve this one prompt in the traditional way, but then moved on to a conversation. It doesn’t have to stay with a single prompt” - David Faragó
David is a deep learning engineer at mediform, a company that develops communication solutions such as telephone bots for doctors’ surgeries. His work there includes fine-tuning large language models, prompt engineering and data science, as well as software engineering, e.g. microservices, Kubernetes and quality assurance. He also has his own company, QPR Technologies, which offers consulting and development services for innovative quality assurance, e.g. using static code analysis or AI. He founded this company with colleagues after completing his doctorate in symbolic AI, formal methods and model-based testing. He is a member of the steering committee of the GI specialist group Test, Analysis and Verification.
Highlights of this episode:
Left:
Today we enter the world of prompt engineering and how it is revolutionizing the quality of software testing. David shares his experiences and insights on how artificial intelligence and prompt engineering can be used to create and optimize test cases.
Artificial intelligence has the potential to eliminate the infamous writer’s block when creating test cases. However, despite the enormous possibilities, he makes it clear that complete automation is not realistic. His main message was clear - AI can do a lot, but it is not a panacea.
One critical issue is the question of the correctness of generated test cases. David emphasized that AI-generated tests must be viewed particularly critically. Despite the impressive ability of AI to generate useful content, errors and inaccuracies cannot be ruled out. This is particularly relevant in the context of test generation, where accuracy is essential.
David shared a story about his experiment with commit messages using AI for quality assessment. This experience revealed both the potential and the limitations of AI - a single small change could completely reverse the AI’s judgment. This experience taught him to always critically question the results.
We then talked about David’s journey to becoming an expert in the field of prompt engineering. By experimenting with different patterns, he found ways to improve test quality. But despite significant progress in test coverage and structuring, challenges in correctness remained.
David is optimistic that further developments in the field of AI-supported testing are imminent. Nevertheless, a certain skepticism remains regarding the reliability of these methods. As always, it is important to strike a balance between innovation and critical observation.