The acceptance test is the highest of the test stages. The theory says that the customer tests here and realizes with a smile on his face how wonderfully all his requirements have been implemented. Then he gives his ok, the contract is fulfilled and everyone is satisfied.
Unfortunately, things usually look a little different in practice. Crashes, unclear operation, missing or incorrect functions. Instead of a satisfied smile, there are outbursts of anger, frustration and arguments about what was meant and how.
One way to prevent this is to make sufficient use of the previous test stages such as unit test, integration test and system test. Or to involve the customer as early as possible. This is also a key driver for agile methods and agile testing. Early, short feedback loops and reviews by the customer and user. All of this helps to avoid a rude awakening at the end.
The ISTQB defines acceptance testing as: “A test stage focusing on determining whether a system can be accepted.”
Everyone can decide for themselves whether this definition is helpful. Essentially, the difference to previous test stages is that the customer or user now acts as the tester. This naturally brings a new perspective on the test object and therefore new questions, opinions and ideas.
For projects that do not run on a customer environment, e.g. a mobile app for the consumer market, alpha or beta tests are often used here.
The technical specifications of the customer, client or user serve as the test basis for the acceptance test. Examples include business processes, user stories, epics, use cases, etc.
In the case of migrations, the operating instructions or documentation of the old system are sometimes used as a test basis.
As contractual acceptance is often also involved here, the decision as to whether a test is successful or not has far-reaching consequences.
It is equally important to check non-functional requirements and confirm their implementation, e.g. by means of an operational acceptance test.
Technical requirements are often available in text form. How can test cases be created from these requirements? Similar to system testing, it is best to combine structured test case methods with experience-based test case creation.
One problem here is that the customer or user is not familiar with structured test methods such as equivalence class formation, limit value analysis or decision tables. They also often lack knowledge of how test cases are documented, structured and processed. It has proven to be a good idea to provide a short training session at the beginning of the test activities to impart this basic knowledge. A few tips and ideas are enough to massively increase the quality of a department’s tests.
The test object for the acceptance test is the integrated overall system. The application is tested from the outside, usually in the same way as users will do every day in the future.
Interfaces are already integrated and connected to the other systems. The aim is to test very close to the future production status.
The aim of the acceptance test is to check whether the functional and non-functional requirements for the application have been fulfilled and sufficiently implemented. From the customer’s point of view. The customer ultimately decides whether their ideas and concepts have been implemented as they imagined.
The result of the acceptance test is the final acceptance. This means, for example, that the project is completed and moves on to the next phase of maintenance. Payment flows, training and going live also often depend on the result of the acceptance test.
A valid statement on the test result can only be achieved during the acceptance test if the tests are carried out in the customer’s final infrastructure. This is the only way to rule out incompatibilities and side effects.
When a system is put into production for the first time, the future production environment is sometimes also used for the acceptance tests. In times of virtual systems, the provision and management of the infrastructure is much more time and resource efficient.
The test data should also be provided or at least controlled by the customer or client. Anonymized production data can be used. An alternative is to fall back on the test data management of the system test. Regardless of which option is used, it is important to adopt the subsequent user perspective as well as possible.
As the acceptance test is usually only carried out once in traditional project procedures, test automation tends to play a subordinate role here.
Things are very different in agile acceptance testing. Test automation is mandatory here today, as they have to be executed over and over again.
Robot-controlled process automation (RPA), which can be used from the acceptance test right through to the production environment, is currently experiencing a trend in this context.
The acceptance test is essential in an agile context. Here, the focus is very much on the customer’s benefit, their feedback and their ideas. Due to the usually short iterations and the intensive exchange with the customer or subsequent user, the acceptance test becomes an ongoing process. The user’s constant feedback is taken on board and incorporated into the next steps. Of course, this also massively reduces the potential frustration factor at the end of the project. In some agile projects, each accepted sub-step is considered to be completely finished and is only checked via automated regression tests. No further manual testing then takes place.
I have hardly ever encountered any problems with acceptance tests in agile projects. Quite the opposite. Everyone involved benefits from the intensive and ongoing exchange.
Unfortunately, the world looks different in traditional projects with an acceptance test at the end. There are always challenges here:
I have been amazed to see projects where the acceptance test consisted of three clicks. Then acceptance was explained and anything that didn’t work was corrected via maintenance contracts. At the other end of the scale, I have seen customers who have set up professional test centers to carry out acceptance tests. The range is wide, but so is the consistency.
Things can get heated and the air can get thick in many an acceptance meeting. This wouldn’t have to be the case if we had worked together for months beforehand towards the moment of acceptance. It is therefore always important to me personally to plan and carry out this phase together. Client and contractor.