E2E Test Automation Framework Selection

Test automation is important. This saves us an incredible amount of work and we make faster progress. But the framework and the tools need to be carefully considered. You know my favorite quote: “If you automate crap, you have crap - only faster” - that doesn’t help anyone. And that’s why the requirements have to be very clear. These differ not only from system to system, but also regionally, as Mesut explains to us using a real-life example. He lives in Japan and gave us an insight into the modernization of his framework after coming across some inconsistencies.

“I live in japan and in that area especially safari as browser usage is very high. because people love using iOS and MacOS systems. So the most of the traffic is coming from safari. And then I figured out, the tool that we were using is not support exhibit test cases on safari” – Mesut Durukal

Mesut has over 15 years of experience in areas such as industrial automation, IoT, cloud services and defense industry, complemented by his expertise in test automation and CI/CD integration. He has held multiple roles in multinational projects including Quality Owner and Hiring Manager and is well versed in CMMI, Scrum & PMP. As a recognized speaker on international stages and winner of the award for the best presentation, he is also involved in various program committees.

Highlights of this episode:

• Mesut Durukal talks about his experience of choosing an end-to-end test automation framework
• Mesut realized that the existing tool did not support all the browsers he needed, especially Safari, which is popular in Japan where he is based
• He conducted a benchmarking study, evaluating pros and cons to select the best framework for his needs
• Mesut emphasizes the importance of choosing the right tools to cover all testing needs
• Ultimately, he chose Playwright for his project, but emphasizes that the choice depends on the specific project requirements
• Mesut also discusses the importance of architecture and reusability in test automation environments
• Machine learning can help with converting specification files during a framework transition
• Continuous evaluation and improvement of tools and practices are essential in test automation

Evaluating E2E test automation frameworks: Mesut’s insights into choosing the best tool

Choosing the right end-to-end test automation framework is critical for effective quality assurance. The episode provides a detailed discussion with Mesut about evaluating different frameworks, considering strengths and weaknesses, and making informed decisions based on specific project needs.

The need for effective test automation

Automation plays a crucial role in quality assurance by enabling comprehensive test coverage beyond manual capabilities. Mesut explains that choosing the right toolset is essential to ensure comprehensive testing on all required platforms.

The Safari browser dilemma

Mesut recalls his realization that the current tool could not test on Safari, a critical requirement due to high usage rates in Japan. This limitation prompted him to look for alternative frameworks that could fulfill this and other specific needs.

Benchmarking of framework alternatives

A methodical approach led Mesut to evaluate different frameworks by listing their pros and cons and prioritizing features based on their relevance to his project. Such careful benchmarking ultimately led him to select a framework that improved test automation capabilities.

A mixture of open source and paid functions

Mesut discusses how he navigated the mix of open source elements and paid features within various tools, highlighting how cost considerations influenced his decision-making process as he built an effective automation environment.

Consulting leads to comprehensive requirements

Combining personal experience with insights from product owners and developers, Mesut created a list of necessary features for his automation environment. He emphasizes that understanding stakeholder needs is key to determining the suitability of the framework.

Strengths and weaknesses

Each framework presented unique advantages - the reporting capabilities of Cypress, the speed of Playwright, the simplicity of Nightwatch - and disadvantages such as limited browser support or less community support. Mesut’s narrative illustrates how these factors played into his final choice.

Architecture is important

Mesut emphasizes the importance of a well-structured architecture that facilitates an easy transition between frameworks and avoids unnecessary duplication of work. He shows how foresight in design can save time when making major changes.