Shift Left but Right
The integration of Shift Left and Shift Right into the development process offers holistic quality assurance. Shift Left promotes early testing to...
Test data plays a central role in modern software development. It enables data errors to be identified at an early stage and quality assurance to be optimized. The use of AI and low-code solutions makes test automation more efficient. Close cooperation between IT and the specialist department is crucial in order to harmonize data processes and increase quality.
In this episode, I talked to Joshua and Hermann about quality, test automation and agility. Hermann explained how important it is to detect data errors early on and how to systematically generate target results in order to compare them with actual results. He emphasized that there is often a lack of suitable tools to carry out these tests efficiently. Joshua added that their methods help companies to harmonize and test data from different systems. We also talked about the role of artificial intelligence in the testing process and how it can help make suggestions for testing and improve collaboration between IT and business departments. Finally, there were insights into the challenges and benefits of visualizing data processes to optimize quality assurance.
“The problem is that you have data errors. They often appear far too late. In other words, you actually want to test data during development.” Hermann Friebel
Hermann Friebel, founder and Managing Director of FINARIS Financial Software Development GmbH since 2001, has almost four decades of expertise in software development and testing in the areas of securities trading and risk controlling.
Since joining FINARIS in 2015, Joshua Claßen has established himself as a senior consultant for backend test automation of complex banking applications. Through his work with RapidRep, the predecessor of SQACE, he gained extensive experience in automated data testing and data quality assurance with various clients.
In many companies, data errors are often only recognized at a late stage. Although the IT department usually has technical access to the data, it often lacks the necessary business expertise to identify data errors at an early stage. In most cases, the specialist department develops specific test cases and checks them, but this is often cumbersome and inefficient.
A promising method for solving this problem is to systematically generate target results and compare them with the actual results. However, this technique often only covers about 70% of the relevant cases. To ensure complete coverage, the remaining 30% should be supplemented by generated test data. In this way, almost comprehensive coverage of all possible scenarios can be achieved.
Close cooperation between IT and the specialist department is essential in order to improve the detection of data errors. By using special tools, employees from the specialist department can actively work on the data processes and support their further development. This collaboration not only promotes the quality of the test results, but also mutual understanding between IT and specialist departments, which leads to better data processes in the long term.
The use of low-code components helps to simplify the testing process. With these tools, many tasks can be implemented more quickly and without in-depth technical knowledge. Especially in large companies with many departments in which similar problems often occur, low-code components help to avoid redundant solutions and make processes more efficient.
Artificial intelligence (AI) has the potential to revolutionize test automation. AI can help to automatically generate suggestions for tests or create code, making the process more efficient and saving valuable time. However, it is important to remain realistic: AI-supported systems make work easier and optimize processes, but they do not take on all tasks completely autonomously.
Test data is specially created or selected data that is used in software testing to check the functionality, performance and security of a system. They simulate real data and use cases to ensure that the software functions without errors and meets the requirements.
There are different types of test data, including synthetic data (artificially generated data), production data (real user data) and masked data (real data that has been anonymized). Each type has its specific use cases, advantages and disadvantages in terms of accuracy, security and privacy.
Test data can be created manually, generated automatically or derived from production data. Automated tools, such as test data management solutions, make it possible to create large amounts of data quickly and efficiently and cover specific test scenarios. The choice of method depends on the test requirements and the available resources.
Synthetic test data is artificially generated data that simulates real data without containing sensitive information. They are particularly suitable for test scenarios where data protection and data security are paramount or where real production data may not be used.
Anonymization is important in order to comply with data protection guidelines and legal regulations such as the GDPR. It protects users’ personal data and minimizes the risk of a data breach. Anonymized data is often used in test scenarios that require real user data but need to protect privacy.
The biggest challenges include complying with data protection regulations, providing the right amount of data, managing data up-to-dateness and minimizing data redundancy. Effective test data management ensures that tests use realistic and up-to-date data without taking up unnecessary storage space.
The quality of the test data has a direct influence on the validity of the test results. High-quality test data leads to reliable results, while faulty or inadequate test data can lead to incorrect conclusions and undetected errors. Data quality is therefore a central aspect of the test process.
Common tools include TDM solutions such as Informatica Test Data Management, Delphix and CA Test Data Manager. These tools support the generation, masking and management of test data and help to make the test process efficient and secure.
Production data should not be used if it contains sensitive information and there is a risk of a data breach. This applies in particular to test scenarios that do not require access to real user data or where alternative, anonymized or synthetic data can be used.
Up-to-date and relevant test data can be ensured by regularly updating and validating the data sets. Test data management tools and processes help to keep the data up to date and thus ensure the validity and accuracy of the test results.
The integration of Shift Left and Shift Right into the development process offers holistic quality assurance. Shift Left promotes early testing to...
Test data - a tiresome topic for many companies, especially when it comes to cross-system provision. It must be complete, otherwise the tests will...
Integrating quality into the software development process should be a priority from the outset. The experience gained in testing can be a valuable...