Artificial intelligence (AI) is becoming increasingly important in our technology landscape. For AI to deliver good and reliable results, regular appraisals of the systems are required. The AI assessment matrix is a systematic concept for evaluating AI systems. Both safety and innovation criteria play a central role here. The testing of AI systems poses a number of challenges and is still dependent on human expertise. In addition, social responsibility and ethical aspects such as fairness and data privacy are of great importance. These topics form the basis for a well-founded examination of the requirements and standards in AI development.
In this episode, I talk to Christoph Poetsch from TÜV AI.Lab about the testing and certification of artificial intelligence (AI). Christoph introduces the AI Assessment Matrix, a systematic concept for evaluating AI systems. We discuss the importance of safety and innovation criteria, the challenges of AI testing and the need for human expertise in the testing process. Together, we will also address the social responsibility and ethical aspects of AI.
"Our credo is to keep the innovation, but on the other hand to protect against the negative consequences of the technologies." - Christoph Poetsch
Dr. Christoph Poetsch is Senior Advisor AI Ethics and Quality at TÜV AI.Lab, an independent joint venture of key players in the global testing and certification industry. TÜV AI.Lab paves the way for trustworthy AI by developing compliance criteria and testing procedures for AI systems. Dr. Christoph Poetsch's core tasks at the AI.Lab are in the areas of AI quality, AI ethics and systematic groundwork. He holds a doctorate in philosophy from the University of Heidelberg and has completed research and visiting fellowships at Yale-NUS College Singapore, the University of Notre Dame and the École Pratique des Hautes Études in Paris.
n
Artificial intelligence (AI) is increasingly being used in various areas of our lives. It is therefore becoming increasingly important to ensure that AI systems are reliable, safe and ethical. The testing and certification of AI systems is therefore becoming increasingly important.
Testing AI systems is particularly important in light of the complex challenges associated with the use of AI. At a time when decisions are increasingly influenced by algorithms, it is important that these very systems are transparent and comprehensible. The importance of AI testing goes beyond technical appraisal. Testing AI also involves social and ethical dimensions that are important for the well-being of society.
It has already become clear in the first section that testing AI has several dimensions. In principle, the appraisal of AI can be subdivided into testing of and with AI. In the appraisal of AI, we also have to deal with the question: Is there a fair AI that acts according to our values? After all, fairness and justice are crucial for the responsible use of AI systems.
Christoph Poetsch heads the TÜV AI.Lab as an institution that deals with the testing and certification of artificial intelligence. Under his leadership, the Lab has developed an AI assessment matrix that enables a systematic approach to AI testing and certification. This matrix is crucial to ensure that AI systems are not only innovative but also comply with applicable safety standards.
The developed AI assessment matrix comprises two dimensions:
The TÜV AI.Lab acts as a mediator between companies and regulatory authorities and has set itself the goal of creating a framework that enables both technical progress and takes social values into account. The topic of safety and robustness of AI systems is particularly important when it comes to the responsible use of artificial intelligence.
If you want to appraise AI systems, you need a structured approach to ensure a comprehensive analysis and avoid potential negative consequences that can result from inadequate test techniques. To meet this need, TÜV AI.Lab has included the two dimensions test dimension and test areas in its matrix.
The test dimension: This axis includes various dimensions of testing. Various test forms are mapped here, such as
The test areas: This axis focuses on relevant lifecycle areas of AI. It enables a targeted consideration of the phases in which tests should be carried out in order to verify the robustness and effectiveness of the system. These include
By combining these two dimensions, a structured framework is created that enables a systematic approach to testing AI systems. The test dimensions on the X-axis interact with the test areas on the Y-axis, resulting in a multidimensional view. This approach helps to systematically record and evaluate all relevant criteria.
AI systems are tested in three dimensions:
This dimension comprises concrete tests that are carried out directly on the AI systems. The functionality of the system is appraised under realistic conditions. The aim is to ensure that the system works as expected and that no unexpected errors occur.
This dimension focuses on the appraisal of the existing documentation. It analyzes whether the required information about the AI system is properly documented. Comprehensive documentation is crucial for the traceability and transparency of the tested systems.
This dimension looks at the processes used in the development and operation of the AI system. This includes the qualifications of the personnel responsible for the management and maintenance of the systems of systems. Adherence to standards and best practices in these processes contributes to safety and robustness.
These three dimensions provide a structured framework for evaluating AI applications and ensure that both technical and organizational aspects are taken into account.
The AI Risk Navigator is a free risk classification tool that helps companies to identify and assess potential risks associated with AI systems. The AI Risk Navigator thus supports a systematic approach to risk management.
By using the AI Risk Navigator, companies can:
The AI Risk Navigator is not only a risk assessment tool, but also a strategic asset for companies operating in a complex landscape of AI technologies.
There are key core requirements of AI regulation that are important for the development and implementation of safe and efficient AI systems. Several aspects need to be considered in this context:
These core requirements affect not only the AI system in question, but also other systems in its environment. The interactions between different AI applications and their impact on people and the environment must be continuously assessed. This creates a holistic approach to AI testing and certification that considers both technological and ethical dimensions.
Global responsibility plays a central role in the increasing use of AI systems. Several key areas are considered in the certification of AI systems:
These factors are crucial forKI certification processes. After all, only a responsible approach can ensure that new technologies not only bring economic benefits, but also make a positive contribution to society and the environment. With an interdisciplinary exchange between different stakeholders, standards can be developed that integrate both technical and ethical aspects.
As AI systems are increasingly being integrated into various areas of life and have an impact there, it is important to take a holistic view of the development.
The testing of AI systems brings with it a variety of technical challenges. Central to this is robustness testing, which aims to ensure that systems function stably and reliably under different conditions.
Important points in this context are
Comprehensive testing requires not only technical skills, but also an understanding of ethical issues and the impact of AI systems on users and society. A structured approach to these tests is important to ensure the long-term safety and effectiveness of AI applications.
The appraisal and certification of AI systems will become even more relevant in the future. The following developments will have an impact in the future:
It is clear that the testing and certification of AI systems is not just a technical process, but also implies a social responsibility. The coming years will show how successful the integration of innovation, safety and ethical standards will be.
AI needs to be tested to ensure the safety and performance of AI systems. The testing and certification of AI helps to validate the quality of the technologies and create trust among users.
A systematic testing approach is necessary to capture the complexity of AI systems. By introducing a two-dimensional AI assessment matrix, test dimensions and test areas can be clearly defined.
The AI assessment matrix consists of two axes: the X-axis represents different test dimensions, while the Y-axis represents specific test areas that are important for a comprehensive assessment of AI systems.
The AI Risk Navigator is a free risk classification tool that helps companies identify potential risks in AI and act as a mediator between companies and regulators.
One of the biggest technical challenges is robustness testing. This requires a thorough analysis of the system architecture as well as extensive testing to ensure that the AI system functions reliably under different conditions.