Marvin van Bekkum | Searcharxiv

arXiv2025

Digital welfare fraud detection and the Dutch SyRI judgment

In 2020, a Dutch court passed judgment in a case about a digital welfare fraud detection system called Systeem Risico Indicatie (SyRI). The court ruled that the SyRI legislation is unlawful because it does not comply with the right to privacy under the European Convention of Human Rights. In this article we analyse the judgment and its implications. This ruling is one of first in which a court has invalidated a welfare fraud detection system for breaching the right to privacy. We show that the immediate effects of the judgment are limited. The judgment does not say much about automated fraud detection systems in general, because it is limited to the circumstances of the case. Still, the judgment is important. The judgment reminds policymakers that fraud detection must happen in a way that respects data protection principles and the right to privacy. The judgment also confirms the importance of transparency if personal data are used.

cs.CY↗

arXiv2025

De spanning tussen het non-discriminatierecht en het gegevensbeschermingsrecht: heeft de AVG een nieuwe uitzondering nodig om discriminatie door kunstmatige intelligentie tegen te gaan?

Organisations can use artificial intelligence to make decisions about people for a variety of reasons, for instance, to select the best candidates from many job applications. However, AI systems can have discriminatory effects when used for decision-making. To illustrate, an AI system could reject applications of people with a certain ethnicity, while the organisation did not plan such ethnicity discrimination. But in Europe, an organisation runs into a problem when it wants to assess whether its AI system accidentally discriminates based on ethnicity: the organisation may not know the applicants' ethnicity. In principle, the GDPR bans the use of certain 'special categories of data' (sometimes called 'sensitive data'), which include data on ethnicity, religion, and sexual preference. The proposal for an AI Act of the European Commission includes a provision that would enable organisations to use special categories of data for auditing their AI systems. This paper asks whether the GDPR's rules on special categories of personal data hinder the prevention of AI-driven discrimination. We argue that the GDPR does prohibit such use of special category data in many circumstances. We also map out the arguments for and against creating an exception to the GDPR's ban on using special categories of personal data, to enable preventing discrimination by AI systems. The paper discusses European law, but the paper can be relevant outside Europe too, as many policymakers in the world grapple with the tension between privacy and non-discrimination policy.

cs.CY↗

arXiv2025

Discrimination and AI in insurance: what do people find fair? Results from a survey

Two modern trends in insurance are data-intensive underwriting and behavior-based insurance. Data-intensive underwriting means that insurers analyze more data for estimating the claim cost of a consumer and for determining the premium based on that estimation. Insurers also offer behavior-based insurance. For example, some car insurers use artificial intelligence (AI) to follow the driving behavior of an individual consumer in real-time and decide whether to offer that consumer a discount. In this paper, we report on a survey of the Dutch population (N=999) in which we asked people's opinions about examples of data-intensive underwriting and behavior-based insurance. The main results include: (i) If survey respondents find an insurance practice unfair, they also find the practice unacceptable. (ii) Respondents find almost all modern insurance practices that we described unfair. (iii) Respondents find practices for which they can influence the premium fairer. (iv) If respondents find a certain consumer characteristic illogical for basing the premium on, then respondents find using the characteristic unfair. (v) Respondents find it unfair if an insurer offers an insurance product only to a specific group. (vi) Respondents find it unfair if an insurance practice leads to the poor paying more. We also reflect on the policy implications of the findings.

cs.CY↗

arXiv2025

Using sensitive data to de-bias AI systems: Article 10(5) of the EU AI Act

In June 2024, the EU AI Act came into force. The AI Act includes obligations for the provider of an AI system. Article 10 of the AI Act includes a new obligation for providers to evaluate whether their training, validation and testing datasets meet certain quality criteria, including an appropriate examination of biases in the datasets and correction measures. With the obligation comes a new provision in Article 10(5) AI Act, allowing providers to collect sensitive data to fulfil the obligation. The exception aims to prevent discrimination. In this paper, I research the scope and implications of Article 10(5) AI Act. The paper primarily concerns European Union law, but may be relevant in other parts of the world, as policymakers aim to regulate biases in AI systems.

cs.CY↗

arXiv2022

Using sensitive data to prevent discrimination by artificial intelligence: Does the GDPR need a new exception?

Organisations can use artificial intelligence to make decisions about people for a variety of reasons, for instance, to select the best candidates from many job applications. However, AI systems can have discriminatory effects when used for decision-making. To illustrate, an AI system could reject applications of people with a certain ethnicity, while the organisation did not plan such ethnicity discrimination. But in Europe, an organisation runs into a problem when it wants to assess whether its AI system accidentally discriminates based on ethnicity: the organisation may not know the applicants' ethnicity. In principle, the GDPR bans the use of certain 'special categories of data' (sometimes called 'sensitive data'), which include data on ethnicity, religion, and sexual preference. The proposal for an AI Act of the European Commission includes a provision that would enable organisations to use special categories of data for auditing their AI systems. This paper asks whether the GDPR's rules on special categories of personal data hinder the prevention of AI-driven discrimination. We argue that the GDPR does prohibit such use of special category data in many circumstances. We also map out the arguments for and against creating an exception to the GDPR's ban on using special categories of personal data, to enable preventing discrimination by AI systems. The paper discusses European law, but the paper can be relevant outside Europe too, as many policymakers in the world grapple with the tension between privacy and non-discrimination policy.

cs.CY↗