Test Pyramid - A Critical Look

Everyone knows the test pyramid, everyone immediately has its image in mind. Sometimes it has three floors, sometimes four. But it is worth considering whether it is slowly becoming obsolete. For example, it makes sense to change the focus when testing microservice architectures. Do you always need a lot of unit tests? Perhaps not! And: unit tests are addictive! Ronald tells us his view on this “package deal” of the test pyramid.

“I think a lot of the tests we do when developing services are less unit tests than actual integration tests” - Ronald Brill

Ronald Brill has been designing, implementing and testing software for over 30 years. Parallel to his work as a principal at GEBIT Solutions GmbH, he has been involved in the testing of web applications for more than 10 years and has developed testing tools with Wetator and BingoBongoTesting. He is the maintainer of HtmlUnit and a committer for various other Open Source projects.

Highlights of this episode:

• Ronald Brill shares his critical view of the test pyramid
• The test pyramid is a widely used model in software development that provides lower layers with many unit tests and upper layers with few GUI tests
• Ronald argues that the test pyramid may no longer be up to date, especially in relation to modern development practices such as microservices, agile methods and DevOps
• It is discussed that tests should not necessarily be performed separately, but must be considered as an integral part of development
• An important point is that tests should build on each other instead of repeating themselves
• Ronald emphasizes the importance of integration testing and argues that in some cases too many unit tests can be inefficient
• The discussion also highlights the challenges of testing UIs and the efficiency of integration testing in service-based architectures
• The importance of communication and collaboration between developers and testers is emphasized
• Ronald encourages critically questioning existing testing approaches and adapting them where necessary
• The episode concludes with a call for open discussion and exchange of feedback

The test pyramid: an outdated model or still relevant?

Today I sat down with Ronald Brill to take a critical look at the test pyramid. We discussed its relevance in modern development approaches such as microservices and DevOps and explored alternative testing strategies that might better suit current technologies.

The test pyramid concept

Today I had the pleasure of talking to Ronald Brill about a topic we are all familiar with but not always clearly defined: the test pyramid. This concept, although it has been anchored in various literature and everyday practices for years, deserves critical consideration. The pyramid suggests an ideal test structure - lots of unit tests at the base, some GUI tests at the top - but the reality is often different. Ronald’s skeptical perspective sheds light on an important question: Is this model still up to date or do we need a new paradigm?

The historical development of the test pyramid

Ronald looks at the origins of the test pyramid. His 30+ years of experience in the field give him a deep insight into the evolution of testing practices. The pyramid was created in response to the problem of inefficient testing in large applications. Its main aim was not to establish a strict hierarchy of testing, but rather to embed the concept that testing is an integral part of the development process - not an afterthought or an isolated task for a separate team. Of particular interest is his realization that the real power of the pyramid lies in making testing more efficient by drawing on artifacts that developers are already using.

The challenge of existing models

While the classic test pyramid provides a solid foundation, Ronald points out that it may no longer keep pace with today’s technological developments. Agile methods and microservice architectures require flexible and adaptive testing strategies. The strict structure of the pyramid - many unit tests at the bottom, few UI tests at the top - may not always be the best approach. Especially in the context of microservices, integration tests might be more promising than an overabundance of unit tests. This insight leads to a key question: how can we adapt our testing practices to meet the requirements of modern software development?

Alternative approaches to the traditional pyramid

The discussion turns to alternative concepts. Ronald emphasizes the importance of project-specific adaptation of test strategies. Rather than sticking rigidly to the pyramid model, teams should be prepared to shape their approach based on the specific requirements of their project. He also mentions interesting variations such as the ‘test cup’ or ‘hourglass’ model as examples of creative adaptations. These flexible models recognize that an effective testing strategy must cover the full spectrum from unit to integration testing - always aiming to avoid redundancy and maximize efficiency.

The role of communication and analysis

A key takeaway from our discussion was the importance of communication and continuous analysis within the team. To test effectively and ensure the quality of the software, developers and testers need to work together and continually rethink their strategies. This also means recognizing when certain tests are redundant or when an error occurs - to learn from this and adapt the methodology accordingly. This continuous reflection and adaptation is crucial for success in a rapidly evolving technological landscape.

A call for flexibility and discussion

To conclude our discussion, the world of software testing is complex and dynamic. While the test pyramid once provided a useful model, today’s technologies and methodologies challenge us to think more flexibly and find more customized solutions. I invite you to continue this discussion - whether through feedback or your own experiences with alternative testing models. Let’s explore together new ways to test our software more efficiently and effectively.