Implementing Test Automation

At best, the introduction of test automation in a company takes place gradually. This serves to adapt the techniques and tools to the needs of the company. But also to adapt the processes and, in particular, to align the company’s expectations with the possibilities of test automation. In practice, an introduction in 3 basic phases, which represent this step-by-step introduction, has proved successful for us: Pilot project, application and roll-out.

Pilot project test automation - proof of concept

The pilot project phase comprises several activities: the preparation, implementation and evaluation of the pilot.

Preparation of the pilot project

During the preparation phase, the current status is analyzed. To this end, the process descriptions, tools used and existing dependencies are recorded by analyzing the associated documentation. Furthermore, interviews are typically also conducted with all those involved in order to record the processes in place and the comparison with the documentation. The difference is sometimes significant. During this analysis, priority is given to identifying potential for improvement resulting from the reduction of previously manual activities. Corresponding activities to leverage this potential (e.g. use of tools, need for modeling, test coverage or traceability) are also analyzed. This results in a catalog of criteria that a test automation solution should fulfill in the best possible way.

The next step is the market analysis of the available concepts, the associated tools and the corresponding dependencies based on the previously identified list of criteria. All available solutions are analyzed and evaluated and then compared. Depending on the time and financial possibilities, the pilot phase is then carried out with the best x solutions. Since there are most likely some offers that obviously do not fit the company’s needs, it makes no sense to implement all solutions. I also advise against carrying out the pilot with only one (the best-ranked) tool, as this narrows down the view of the advantages and disadvantages considerably. I recommend carrying out the pilot with the 3 top-ranked tools.

Implementation of the test automation pilot project

The pilot is now carried out with the aim of solving a smaller task that only takes a few person-days. The implementation itself can be carried out with the involvement of the technical experts. In any case, the experts for test automation should be in charge. The path to the solution, the advantages and disadvantages experienced in the process and the solution itself are recorded for each tool used.

Evaluation of the test automation solution

Following implementation, the experience gained is evaluated and presented to the entire group of participants in a workshop. Here, the differences between the tools used are reassessed. In particular, the practical experience of using the tools can revise the assessments made previously. Two fundamental questions will be clarified during the workshop:

• Should the (locally limited) use of test automation in real projects be recommended after completion of this pilot?
• Which tool should be used for this deployment?

If the first question has been answered positively and an answer has also been found for the second question, then the last milestone of this phase has been mastered and the application phase begins.

Application

The application phase essentially comprises two steps: The selection of the department / group in which the commercial application of test automation is to take place, and the application itself.

When selecting the department, one of the key points is to reuse the experience from the pilot project and make the necessary adjustments to ensure the smoothest possible transition from the pilot to the application. I recommend that every decision-maker pay attention to the selection of suitable people, as the introduction is still in a sensitive phase. Furthermore, the appropriate tool must be selected and the processes adapted as part of the adaptation. To evaluate success, it is also advisable to carefully select the relevant key figures.

The preparation phase is followed by the application phase. Especially during the initial application, it is recommended that users are in constant contact with experts in order to discuss and reflect on their experiences, obstacles and progress. Initially, this is done with training and coaching sessions accompanying the application with the aim of independent work in the department.

With the help of the key figures defined at the beginning, the selected test automation technology can be finally evaluated after the application phase with regard to commercial suitability. This experience should provide enough information to decide whether the technology should be rolled out company-wide.

Roll-out of test automation

In the roll-out phase, the use of test automation is formalized. The development process is adapted to the test automation and a corresponding training program is set up. Responsible persons are appointed in the departments to monitor and control the process improvement. Application coaching should be provided, especially during the initial application. The long-term goal of the roll-out is for employees to internalize the process without external support and adapt it to the changing business environment as required.

Test automation in agile projects

In agile projects, these phases are shortened. In most cases, a test automation tool is tested on a section of the application during the ongoing project. The result is evaluated in a retrospective, for example, and rolled out successively to the project if successful. Here, the use is much more dynamic, tools are changed occasionally and there is often a tool landscape instead of a single test tool.