Fair, good AI?

Artificial intelligence should support people. Whether that’s counting screws in a factory or assisting the head physician during a complicated operation. However, these different areas of application place enormously different demands on AI. The ethical principles are also different around the world. So what does fairness actually mean? And where do discrimination and justice begin? AI should act in accordance with our values, for which it must be trained - but these values must first be defined.

“If you take a look at the fairness debate surrounding artificial intelligence, a very exciting paper by Kleinberg found that what we so objectively call fairness is partly contradictory” - Marc Hauer, Tobias Krafft

Marc Hauer is a research associate at the Algorithm Accountability Lab at RPTU Kaiserslautern, specializing in the design of responsible AI systems. He heads the DIN SPEC working group “Fairness of AI in Financial Services” and works as a freelance expert and consultant in the field of algorithms and AI.

Tobias D. Krafft is a PhD student in the field of “Algorithm Accountability” at the TU Kaiserslautern and Managing Director of Trusted AI GmbH, focusing on black box analysis and AI regulation. He heads the DIN working group “Ethics/Responsible AI” and received the Weizenbaum Prize in 2017 for his research in the social context of AI. He is also involved in the Gesellschaft für Informatik for the Socioinformatics course.

Highlights of this episode:

• We talk about the topic of fair AI and how to evaluate fairness
• Marc and Tobias explain that fairness in AI is not only a technical issue, but also an ethical and social one
• They present a methodology adapted from safety engineering to assess fairness in AI systems
• The importance of transparency and stakeholder involvement is emphasized
• The importance of testing and reviewing AI systems for fairness is discussed
• The challenges and need to involve the right stakeholders are addressed
• The vision of an open source approach for the development and evaluation of fair AI systems is shared

Further links:

• Use Assurance Cases for Fairness
• The practical testing in the fAIrbydesign project
• The Guidebook for the framework

Fair AI: A guide to ethical technology

Today I talk to Marc Hauer and Tobias Krafft about the concept of fair AI and how to operationalize fairness in this context. We explore how an assurance case framework can help ensure fairness in AI systems and why stakeholder engagement is crucial.

The basis for fair AI

In this episode, I take a critical look at the topic of fair artificial intelligence (AI) together with my guests Marc Hauer and Tobias Krafft. What does it actually mean when we talk about ‘fair AI’? This question is particularly relevant as AI permeates more and more areas of our lives. The two experts discuss how fairness in AI is not only a quality feature, but also has profound social and ethical dimensions.

Operationalization of fairness

Tobias explains that the challenge in assessing fairness is to find objective measures to quantify subjective concepts. Common practice is based on comparing qualitative measures between different sensitive groups. However, even these approaches have their limitations and often lead to complex debates about their correct application and interpretation.

Die Rolle des Assurance Case Frameworks

An exciting development in the quest for fair AI is the use of the Assurance Case Framework. Originating from safety engineering, this framework provides a structured method for arguing and validating safety or, in this case, fairness assertions. Through step-by-step refinement, main and partial assertions are made, which ultimately need to be supported by evidence. This process not only enables deeper reflection on the requirements made, but also more transparent communication to stakeholders.

Fairness as a collective endeavor

A key element in the design of fair AI systems is the involvement of different stakeholders. The definition of what is considered ‘fair’ can vary widely and often depends on the perspectives and needs of the parties involved. Through workshops and discussions with these groups, relevant fairness measures can be identified and translated into testable requirements.

The future of fair AI

While Marc and Tobias admit that the concept of a completely fair AI may remain an ideal, they emphasize the importance of ongoing discussions and improvements in this area. In particular, the role of open source initiatives and community-driven projects could help to establish widely accepted standards for fair AI in the future.