Automating Society Report 2020

Research

Netherlands

by Ronan Ó Fathaigh and Naomi Appelman

Contextualization

The use of automated decision-making (ADM) in the Netherlands came to international attention in late 2019 and early 2020. This followed the intervention by one of the UN’s top human rights experts in a high-profile court case in The Hague concerning the controversial SyRI (System Risk Indication) system. Professor Philip Alston, the UN Special Rapporteur on extreme poverty and human rights, described the case as the first legal challenge he was aware of that “fundamentally and comprehensively” contested the use of an ADM system in the welfare state on human rights grounds. Particularly noteworthy in the Court’s ruling that declared SyRI to be in violation of Article 8 of the European Convention of Human Rigths (Rechtbank Den Haag, 2020) is that governments have a “special responsibility” for safeguarding human rights when implementing new technologies such as these automated profiling systems (Idem, par. 6.84). The case led to a major public and political debate on ADM systems, and the use of algorithms in the Netherlands, and by the government more generally.

In contrast, the previous edition of Automating Society described how ADM and artificial intelligence (AI) was “predominantly discussed as part of the larger Dutch strategy on digitization,” and that no work, specifically focused on ADM and AI, had been done in the Netherlands on a national agenda. Yet, in the space of a year, a lot has changed. As we will discuss in this chapter, the Dutch government has now published a national Strategic Action Plan for Artificial Intelligence, with actions including major government investment in research on the legal aspects of decision-making algorithms, and the transparency, explainability, and supervision of algorithms. Indeed, in summer 2019, the main coalition government party, Volkspartij voor Vrijheid en Democratie (VVD), published a report proposing a new supervisory authority for the use of algorithms (Middendorp, 2019a) [here]. In a similar vein, in autumn 2019, another coalition government party, Democraten 66 (D66), also published proposals on the establishment of an oversight body for ADM systems. The proposals suggested prohibiting the use of certain ADM systems (like SyRI) and automated face recognition systems until proper legislation and supervision is first put in place (D66, 2019) [here].

In this edition of Automating Society, we aim to highlight new applications of ADM systems in the Netherlands. This chapter provides updates on both the political and public debates, and the regulatory and self-regulatory measures concerning ADM and AI. We also look at how civil society and the media in the Netherlands continue to play a crucial role in the current public debate on ADM systems, and the government’s use of algorithms that affect individuals. For example, in summer 2019, the Dutch public broadcaster, Nederlandse Omroep Stichting (NOS), revealed the widespread use of predictive algorithms by government agencies, based on confidential inventories on the use of such algorithms obtained by the broadcaster (Schellevis; de Jong, 2019) [here]. This report led the Chairman of the Dutch Data Protection Authority (Autoriteit Persoonsgegevens) (AP) to state publicly that the Dutch government must be more transparent about its use of predictive algorithms (NOS, 2019) [here]. Indeed, in November 2019, the AP announced that two of its high-risk focus areas for supervisory work during the 2020-2023 period would be digital government, and the use of algorithms and AI (Autoriteit Persoonsgegevens, 2019) [here] The AP is concerned about the increased use of algorithms and AI by more and more government authorities and private companies, which carries risks of harmful effects - while the irresponsible use of algorithms can lead to incorrect decisions, exclusion, and discrimination. The AP also highlighted an ongoing trend within local government and law enforcement authorities giving them access to large amounts of data on individuals. The AP added that it is important that government authorities handle such data responsibility.

The chapter also shows how it is not just the data protection authority that is increasing its oversight of the use of ADM systems, but that other regulatory authorities are too. A prominent example of this is the Dutch Central Bank (De Nederlandsche Bank) (DNB), which supervises financial institutions in the Netherlands. In 2019, the DNB recognized the risks associated with the use of AI by financial institutions in decision-making processes (De Nederlandsche Bank, 2019) [here]. As a result, the DNB published general principles for the use of AI in the financial sector, in order to ensure that financial institutions use AI in a responsible manner. The DNB will be examining the issue of responsible use of AI in its supervision of financial institutions in 2020.

A catalog of ADM cases

Alternative dispute resolution

Over the past few years in the Netherlands, the use of online and automated dispute resolution systems as an alternative to costly and time-consuming court cases has increased. Organizations, such as the E-Court and Stichting Digitrage (Digitrage Foundation), are the most well-known examples. Legally, these dispute resolution mechanisms are a kind of automated arbitration, where both parties contractually agree to use one specific automated method to resolve a dispute. At the E-Court, the entire process is conducted online. The parties upload the required documents and an ADM system creates a verdict. A (human) arbiter then reviews that verdict and signs it. Especially in sectors with a high turnover of small court cases, these fast and cheap alternatives are attractive. For instance, in 2018 most Dutch health insurers, and many debt collection agencies, included these automated dispute resolution companies in their policy agreements (Kuijpers, Muntz & Staal, 2018) [here].

However, as the use of these automated online arbitration systems increased, a large societal backlash ensued. The E-Court, for example, was criticized for the opacity of its underlying system that produces the verdicts. This was due to the fact that the verdicts were not published, and because people were inadequately informed that they had the choice of going to a state court (Kuijpers, Muntz & Staal, 2018) [here]. There was extensive negative reporting on these practices in the media, and a report by the Landelijke Organisatie Sociaal Raadslieden (National Association of Social Lawyers) heavily criticized the E-Court (Sociaal Werk Nederland, 2018) [here]. These criticisms and concerns grew to such an extent that the lower courts decided to temporarily stop confirming the arbitration decisions of the E-Court, effectively shutting down their operation. This resulted in an extended legal battle (e-Court, 2018) [here]. In February 2020, Dutch state and e-Court were still engrossed in this dispute, now arguing over whether a settlement has been reached (Driessen, 2020) [here]. Despite these concerns, the Minister of Legal Protection recently stated that he views digital innovation and online arbitration, such as the E-Court, as positive developments, as long as they operate within the bounds of the law (Minister voor Rechtsbescherming, 2019a) [here].

Health care

A striking example of ADM is the use of the growth tracking software, called Growth Watch, used in the Dutch health care system. The software tracks the development of children and automatically flags discrepancies that indicate either disease or child abuse (RTV Utrecht, 2019) [here]. The software is said to contain several anomalies that make it unreliable for children under a certain age, or with a specific health history. In one instance the system justly separated from its parents (RTV Utrecht, 2018) [here] and (van Gemert et al., 2018) [here]. This case prompted doctors to advise against using the software without a proper disclaimer or major adaptations (van Gemert, 2019) [here].

Journalistic reporting

Automated news reporting is also on the rise in the Netherlands. The previous Automating Society report discussed several news agencies that had implemented automated recommender-systems that semi-automatically decide which articles are shown to each visitor or subscriber. These trials have continued and have been deemed successful. In the past year, ADM in journalistic reporting has been taken a step further, and now there are several examples of automatically generated news articles. For example, during the provincial elections of March 2019 the national broadcaster NOS published automatically generated articles with updates on the election results. The system works with standard texts and automatically inserts the latest results (Waarlo, 2019) [here] and (Duin, 2019) [here]. A large private broadcaster, RTL, also recently implemented an automatic news generator. The automatic news generating software is called ADAM, short for “Automatische Data Artikel Machine” (automatic data article machine), and is mainly used to create local news, such as traffic updates. ADAM will also be used to generate articles on trends where much of the information is publicly available, such as national trends regarding schools or hospitals. The initiative was partially funded by the Google Digital News Initiative (Wokke, 2019) [here] and (Bunskoek, 2019) [here].

Finance and trading

One of the largest Dutch banks, the ING, has created several financial services that are based on ADM systems. For instance, the ING Smart Working Capital Assistant uses ADM to predict events “that have a positive or negative impact on the working capital of the company in question”. Such assessments are sent directly to the client’s smartphone, who can then use the information accordingly (Visser, 2018) [here]. Another example is the ING Katana Lens that the bank created in cooperation with a pension fund. This web-based application is aimed at helping investors by giving them predictions of price trends for specific bonds. The ADM system works by collecting and analyzing past and current price data, and predicting the future price trends based on past patterns (Banken.nl, 2018) [here] and (Mpozika, 2019) [here]. The ING also has a similar product, Katana, but this is aimed at the sales-side of bond trading and not investors (ING, 2019) [here].

Law enforcement Initiatives

Dutch law enforcement is experimenting extensively with ADM systems. The police are conducting several local pilots, such as the project in the municipality of Roermond, discussed earlier, and several other national projects. ADM is also used to fine people for using a mobile device whilst driving. The Dutch national police issued a statement that, from October 2019 onwards, the police will use advanced cameras to automatically detect drivers using phones while operating a vehicle, as this is illegal in the Netherlands. These so-called “smart” cameras will automatically flag when a driver is using a mobile device. The camera will then automatically take a picture of that person and register the license plate. This information is subsequently sent to a police officer for review. When the police officer determines that the information is correct, a fine of 240 euros is automatically sent to that driver’s home address. (RTL nieuws, 2019) [here] and (Politie, 2019b) [here].

The police are also using ADM to try and solve “cold cases” – or unsolved crimes. The national police conducted an experiment using artificial intelligence to analyze several cold cases, and to help prioritize which cases should be reopened. This experiment has been deemed successful and the police will continue using ADM to analyze cold cases. (Politie, 2018) [here] and (Tweakers Partners, 2019) [here]. Furthermore, the national police are actively collaborating with academia to develop ways in which artificial intelligence can help law enforcement. In 2019, the University of Utrecht established the National Police Artificial Intelligence Lab, where seven PhD candidates will, in close collaboration with the police, research how artificial intelligence can be used in a law enforcement context. This lab builds on the work already done in the Police Data Science Lab that the University of Amsterdam created in 2018 (Politie, 2019a) [here]. Another example of the Dutch police using ADM is in their online reporting system for Internet fraud. ADM is used to adjust the reporting questionnaire, give people advice, and automatically alert the police when there is a high likelihood that a crime is committed (Blik op nieuws, 2019) [here].

Of specific importance is the use of the predictive policing tool ‘Crime Anticipation System’ (CAS, Criminaliteits Anticipatie Systeem). Based on past police reports and conviction records, CAS predicts which places have the highest risk of specific types of violent crimes, with the aim to effectively spread police presence throughout a city to prevent these crimes from happening (Minister van Veiligheid en Justitie, 2015) [here]. On a bi-weekly basis, the system produces a 125 by 125-meter color-coded grid indicating the highest risk areas where police presence is subsequently increased. Different versions of CAS have been in use since 2014 (Politie, 2017).

Finally, the Dutch police’s Catch face recognition system is of particular importance. An investigation published by VICE in July 2019 uncovered the existence of over 2.2 million images in the Catch system of a total of 1.3 million individuals who may be suspected of committing a serious criminal offense (van Gaal, 2019) [here]. In November 2019, the police confirmed that its face recognition system now has access to a complete database of people suspected of serious crimes. Current figures are unknown, but in 2017, 93 suspects were identified using the system (NU.nl, 2019) [here]. Furthermore, the national police introduced a smartphone app last year that allows police officers to send images to Catch (NU.nl, 2018) [here].

Policy, oversight and debate

Having described the ADM systems currently in use in the Netherlands, this section examines how Dutch society is debating these developments. We will look at the debate from the perspective of the government, civil society and academia.

Government and parliament

It was noted in the last Automating Society report that little work had been done on a national agenda concerning ADM or artificial intelligence by the Dutch government. However, over the last year, there have been major developments in terms of new national agendas concerning the use of ADM systems and artificial intelligence.

Strategic Action Plan for Artificial Intelligence

One of the most important developments at a policy level has been the Dutch government’s new Strategisch Actieplan voor Artificiële Intelligentie (Strategic Action Plan for Artificial Intelligence) (SAPAI), which was launched in October 2019, and contains a number of initiatives in relation to ADM and algorithms (De Rijksoverheid, 2019) [here] (see also the letter to parliament from the Minster for Legal Protection, 2019b) [here]. The SAPAI is built upon three tracks:

Track 1 involves “capitalising on societal and economic opportunities” presented by AI, and how the Dutch government will make optimal use of AI in the performance of public tasks. The government also intends to engage in intensive public-private partnerships to realize the benefits of AI.

Track 2 is “creating the right conditions”. This includes ensuring that the Netherlands has access to more usable data for AI applications to realize better AI developments.

Importantly, track 3 of the SAPAI is “strengthening the foundations”. On this track, the government will ensure that public values and human rights are protected. In this regard, the SAPAI’s actions will include research into the legal aspects of decision-making algorithms, research into the risks of face recognition technology, and European certification of AI applications in the administration of justice. In addition, there will be action on the effective supervision of algorithms, including research into the transparency/ explainability of algorithms, and crucially, the supervision of algorithms. The Dutch government will also establish a transparency lab for government organizations, and stimulate the participation of Dutch companies and public organizations in the pilot phase of the ethical guidelines for AI from the High-Level Expert Group of the European Commission.

The SAPAI also tasked the Nederlandse Organisatie voor Wetenschappelijk Onderzoek (Dutch Research Council) (NWO) with funding research on explainable, socially conscious, and responsible AI. In November 2019, the NWO launched the first national AI research agenda, (Artificial Intelligence Research Agenda for the Netherlands) (NWO, 2019) [here].

Dutch Digitalization Strategy 2.0

The previous edition of Automating Society noted that the first Nederlandse Digitaliseringsstrategie: Nederland Digitaal (Dutch Digitalization Strategy) was launched in 2018. In 2019, the Dutch government built upon this by implementing an updated Nederlandse Digitaliseringsstrategie 2.0 (Dutch Digitization Strategy 2.0) (De Rijksoverheid, 2019b; 2019c) [here and here]. This strategy sets out the government’s priorities in relation to the impact of digitization on society, the economy, and government. These include applying AI solutions to resolve social issues and stimulate economic growth and developing digital inclusion, digital government, digital connectivity, and digital resilience. Notably, the strategy recognizes that AI can put pressure on key basic rights and public values (such as human dignity, autonomy, and the prohibition of discrimination). As a result, the importance of inclusiveness and transparency and the explicability of algorithms will be key issues. Furthermore, the potentials and risks of the algorithm-based decision- making will also be assessed in the context of safeguarding public values; and the government will engage in case studies in the area of social security, law, self-driving cars, and content moderation. In March 2019, the Ministry of the Interior and Kingdom Relations also presented the NL DIGITAAL: Data Agenda Government to the House of Representatives (Ministerie van Binnenlandse Zaken, 2019a) [here].

In addition, in February 2019, the first Conferentie Nederland Digitaal (Digital Netherlands Conference) was held, bringing together government ministers, industry, researchers, and civil society, in order to better cooperate on digitization (Nederland Digitaal, 2019) [here]. Notably, Nederland ICT, an organization representing industry, presented its AI ethical code of conduct to the State Secretary for Economic Affairs.

Finally, in the previous Automating Society report, the government was supposed to react to a report on algorithms and fundamental rights during the summer of 2018. In March 2019, the government sent its response to parliament. In its reaction, the government acknowledges that new technologies can have a large impact on fundamental rights. The government mainly focused on the need for procedural safeguards and fairness. Central to its policy is neutrality, non-discrimination, transparency, and data protection. The government is looking into additional legal safeguards for reducing bias in its big data initiatives and it is creating guidelines to be used for the ex-ante review. Additionally, the government stressed the importance of robust data protection laws and effective enforcement (Minister van Binnenlandse zaken, 2019b) [here].

Supervision of ADM System and Algorithms

Members of the current Dutch government have also been quite active in terms of the debate concerning ADM systems and algorithms. As mentioned in the introduction above, in May 2019, the main coalition government party, VVD, published a report proposing a new supervisory authority for the use of algorithms (Middendorp, 2019a) [here]. The following month, Jan Middendorp, a government MP, sent an initiative to the House of Representatives, proposing supervision of the use of algorithms by the government (Midendorp, 2019b) [here].

Furthermore, in November 2019, another coalition government party, D66, also published a report with proposals to limit the use of ADM systems by government and technology companies (D66, 2019) [here]. The proposals include:

  • creating a proper legal framework for the use of ADMs systems, where linking and analyzing individuals’ data would only be permissible where it is required for a compelling purpose, and with sufficient guarantees for protecting individuals’ human rights.
  • establish a central overview of the number of databases, the degree of linking, and the use of ADM systems.
  • establish an algorithm authority, and a reporting obligation for algorithm use.
  • prohibit face recognition systems and certain uses of algorithms until there is proper legislation and oversight.

Algorithms and the Administration of Justice

In the previous Automating Society report, it was noted that the Minister for Legal Protection promised to send a letter to parliament in fall 2018 on the possible meaning of ADM and AI for the judicial branch. At the time of publication, this letter had not been received. However, in December 2018, the Minister sent a letter to parliament about the application and use of algorithms and artificial intelligence in the administration of justice. The overall aim of the letter was to give an overview of current experiments in the field of ADM and the administration of justice and to identify benchmarks for judicial use of ADM, based on fundamental rights. These benchmarks are accessibility, internal and external transparency, speed, independence, and impartiality of the judge and the fairness of the trial. The government goes on to list several elements of the judicial process where ADM systems could be used and also identifies possible pitfalls. The overall conclusion is to tread carefully, to conduct experiments apart from actual cases, and to properly assess each experiment to see what the added value of the use of technology is (Ministerie van Justitie en Veiligheid, 2018) [here]. Finally, in March 2020, the Minister for Legal Protection sent a letter to parliament in response to questions on algorithmic analysis of court judgments (Minister voor Rechtsbescherming, 2020) [here]. The Minister set out the benefits of algorithmic analysis of judgments, including allowing more insight into judicial reasoning and pointed to a scheme at the District Court of East Brabant on AI analysis of court documents and cases, The Minister confirmed that only 2-3% of court rulings are published online in the Netherlands, but it is planned to increase this to 5% over the next three years; and that there is currently no specific legal framework for applying AI to court rulings in the Netherlands.

Algorithms and Law Enforcement

The extensive use of ADM systems by the Dutch police, as described in the previous section, became a topic of debate in the Dutch Parliament after the Minister of Justice and Safety outlined the Government’s policy on the use of AI by the police (Kamerstukken 2019). In response, the Parliamentary Committee for Justice and Safety drafted more than 80 questions. The Minister answered in February 2020 emphasizing, firstly, that the Dutch police currently is not using AI systems on a large scale with only the online Internet fraud reporting system as a concrete example. However, the Minister did confirm that algorithmic systems, as outlined above, are used throughout the Dutch police, and outlined several fields where algorithmic or AI systems are being developed. Examples of these, are speech to text software, image/face recognition, and natural text processing, all to shorten the time needed for desk research. Further, the letter elaborated on the ethical, legal, and privacy safeguards in place for the use of AI. Crucially, the Minister clearly stated that it is the government’s conviction that the future development and use of AI-systems for the police is deemed necessary (Kamerstuken 2020).

Regulatory and self-regulatory measures

Automatic Number Plate Recognition

On 1 January 2019, a law that had been enacted in 2017 on Automatic Number Plate Recognition (ANPR) came into effect [here]. The civil society organization, Privacy First, is currently preparing legal proceedings, arguing that the law violates international and European privacy and data protection laws (Privacy First, 2019) [here].

Data Processing Partnership Act

In the previous edition of Automating Society, it was reported that a public consultation was completed on a proposal for the Wet gegevensverwerking door samenwerkingsverbanden (Data Processing Partnership Act). The act aims to provide a basis for public-private cooperation, and make collaboration between them easier in relation to the processing of data, specifically used for surveillance or investigation purposes (e.g., to prevent crimes or detect welfare fraud). In June 2019, the Minister for Justice and Security sent a letter to parliament on the progress of the proposed law [here]. The Ministerraad (Council of Ministers) have agreed on the bill, and it has been sent to the Raad van State (Council of State) for its advice.

Data Protection Authority focus on AI and algorithms

As mentioned in the Introduction, the Autoriteit Persoonsgegevens (Dutch Data Protection Authority) (AP) announced in November 2019 that two of its high-risk focus areas for supervisory work during the 2020-2023 period would be digital government, and the use of algorithms and AI (Autoriteit Persoonsgegevens, 2019a) [here]. This is because the AP is concerned about the increased use of algorithms and AI by more government authorities and private companies, which carries the risk of harmful effects, while the irresponsible use of algorithms can lead to incorrect decisions, exclusion, and discrimination. The AP also highlighted the continued trend of local government and law enforcement authorities now have access to large amounts of data on individuals, and that government authorities must handle such data responsibly.

In addition, the AP also announced in 2019, that it would be investigating the developments of Smart Cities by municipalities in the Netherlands, to ensure the privacy of residents and visitors is protected. The AP noted that municipalities are increasingly using data-driven and automated decisions and that the digital tracking of people in (semi) public places is an invasion of privacy that is only permitted in exceptional cases (Autoriteit Persoonsgegevens, 2019b) [here].

Finally, in April 2020, in the context of the coronavirus crisis, the AP released a statement on possible government use of location data from telecommunication companies to track the spread of the virus. The AP stated that, under Dutch privacy law and the Telecommunication Act (Telecommunicatiewet), telecom companies “may not simply share customer data with the government”. The AP concluded that statutory regulation would be needed, which would be required to be proportionate and contain sufficient safeguards (Autoriteit Persoonsgegevens, 2020) [here].

Dutch Central Bank recommendations on AI and ADM

In July 2019, the Dutch Central Bank (de Nederlandsche Bank, DNB) published a discussion document titled “General principles for the use of Artificial Intelligence in the financial sector”. This document was created as a response to the growing use of automated systems within the financial sector and is meant to give guidance to financial service providers planning to use ADM systems. The DNB explicitly recognizes the risks involved in using ADM in a financial context. The bank summarized their recommendations in the form of an acronym: “SAFEST”. This stands for soundness, accountability, fairness, ethics, skills, and transparency. The guidelines are meant to start a sector-wide conversation on the use of ADM. The DNB called on relevant stakeholders to send input and will report on the results at the end of 2020 (De Nederlandsche Bank, 2019) [here].

Civil Society and Academia

Civil Society

The previous Automating Society report listed several civil society organizations doing important work on ADM, including Bits of Freedom [here], De Kafkabrigade (The Kafka Brigade) [here], Platform Bescherming Burgerrechten (Platform for the Protection of Civil Rights) [here], and Privacy First [here]. In the past year, these organizations have continued their efforts. For example, Bits of Freedom is actively campaigning against face recognition software and it recently published a report arguing for a ban on face recognition in public spaces. For that report, they conducted an experiment where they showed how even one publicly accessible camera in the inner city of Amsterdam made it possible to identify the people walking past (Hooyman, 2019) [here]. Furthermore, the Platform for the Protection of Civil Rights has been one of the driving forces behind, and the main claimant in, the SyRI system court case. This organization has been campaigning against the system for years (Platform Bescherming Burgerrechten, 2019) [here].

Public-Private Partnerships

Consistent with the Dutch government’s policy of promoting public-private partnership, there are now several prominent examples of such partnerships. Two such examples are the launch of the Strategic Action Plan for AI by the State Secretary for Economic Affairs in 2019, and the Dutch AI Coalition, which, was also launched the same year (Nederlandse AI Coalitie, 2019) [here]. It is a public-private partnership comprised of over 65 parties from industry, government, education, and research institutions (Martens, 2019; Wageningen University & Research, 2019) [here and here]. In addition, the Kickstart AI Initiative is also relevant. This brings together a range of large Dutch companies Ahold Delaize, ING, KLM, Philips, and the NS, to fund AI research and education (Kickstart AI, 2019) [here].

Scientific Research

There is also a great deal of scientific research on ADM systems and AI taking place in Dutch universities. As mentioned previously, the Dutch Research Council (NWO) launched the first national AI research agenda in 2019. In addition, one of the most significant research initiatives has been the establishment of the national Innovation Center for Artificial Intelligence (ICAI). This is a national initiative focused on joint technology development between academia, industry and government in the area of artificial intelligence. The ICAI involves a number of research institutions, including Delft University of Technology, Radboud University, University of Amsterdam, Utrecht University, and Vrije Universiteit Amsterdam (Innovation Center for Artificial Intelligence, 2019) [here]. There are a number of labs established under this collaboration, such as the National Police Lab AI. As mentioned above, at Utrecht University, the National Police Lab AI works in collaboration with the police, to research how artificial intelligence can be used in a law enforcement context.

Furthermore, at the University of Amsterdam, there is the Digital Transformation of Decision-Making research initiative. The purpose of this initiative is to examine how automated decision-making systems are replacing human decision-makers in a range of areas, from justice, to media, commerce, health, and labor. In particular, researchers are looking at how the digitization of decision-making affects democratic values and the exercise of fundamental rights and freedoms (UvA, 2019a) [here]. In relation to this, there is the AI & the Administration of Justice research initiative which is also at the University of Amsterdam. This initiative is engaged in normative research into the fundamental rights, societal values, and ethics of automated decision-making, specifically oriented towards the role and responsibilities of judges, public prosecutors, and lawyers (UvA, 2019b) [here]. And finally, there is the Human(e) AI research priority area of the University of Amsterdam which stimulates new research on the societal consequences of the rapid development of artificial intelligence and ADM in a wide variety of societal areas (UvA, 2019c) [here].

Key takeaways

This chapter reveals several important new findings about ADM and AI in the Netherlands during the past year. Perhaps the most important point is that there is currently an incredible amount of activity in the field of ADM and AI, whether at the government policy level, in political and public debate, or the ever-increasing new uses of ADM systems by both government and private companies. Secondly, it is striking that the use of ADM systems and AI by law enforcement has seen major development in the Netherlands. Thirdly, it also seems that a lot of the use of ADM systems by the government is in the realm of local government and government agencies that have discretion in how to allocate recourse, such as welfare, unemployment, tax exemptions, etc. However, in terms of the challenges that ADM systems present, the case of the Netherlands perhaps highlights major issues concerning the lack of transparency with regards to ADM systems used by government. This has been one of the key criticisms of ADM systems, such as the SyRI system for welfare-fraud detection. Indeed, in the SyRI judgment, one of the biggest faults was the lack of transparency, with the Court even noting the “deliberate choice” by the government, of not providing “verifiable information” on the nature of SyRI. Finally, sector-specific regulators are also stepping-up, such as the data protection authorities and financial institution regulators, which are bringing a new level of guidance and scrutiny to both public and private uses of ADM systems.

References

Andringa, Remco (2018) Politie wil zakkenrollers en plofkraken vangen met data, NOS Nieuws, https://nos.nl/artikel/2250767-politie-wil-zakkenrollers-en-plofkrakers-vangen-met-data.html

AT5 (2019) ‘Dijksma wil scanauto gebruiken in strijd tegen afvalbergen’ https://www.at5.nl/artikelen/196944/dijksma-wil-scanauto-gebruiken-in-strijd-tegen-afvalbergenAutoriteit Persoonsgegevens (2019a) AP legt focus in toezicht op datahandel, digitale overheid en AI, https://autoriteitpersoonsgegevens.nl/nl/nieuws/ap-legt-focus-toezicht-op-datahandel-digitale-overheid-en-ai

Autoriteit Persoonsgegevens (2019b) Waarborg privacy in de ontwikkeling van Smart Cities, https://autoriteitpersoonsgegevens.nl/nl/nieuws/waarborg-privacy-de-ontwikkeling-van-smart-cities

Autoriteit Persoonsgegevens (2020) Gebruik telecomdata tegen corona kan alléén met wet, https://autoriteitpersoonsgegevens.nl/nl/nieuws/gebruik-telecomdata-tegen-corona-alleen-met-wet

Banken.nl (2018) ING verbetert trading met kunstmatige intelligentie, https://www.banken.nl/nieuws/21220/ing-verbetert-trading-met-kunstmatige-intelligentie

Bunskoek, Jasper (2019) RTS Nieuws introduceert redactierobot ADAM, rtl nieuws, https://www.rtlnieuws.nl/nieuws/artikel/4914671/adam-robot-nieuws-journalistiek-innovatie-rtl-nieuws-google-dni

Blink op nieuws (2019) Sneller advies bij aangifte, https://www.blikopnieuws.nl/digitaal/276509/sneller-advies-bij-aangifte-internetoplichting-door-kunstmatige-intelligentie-ai

Driessen, Camil (2020), ‘Robotrechter en staat ruziën over schikking’ het NRC, https://www.nrc.nl/nieuws/2020/ 02/26/robotrechter-en-staat-ruzien-over-schikking-a3991842

Duin, Roelf Jan (2019), Een robot kan dit (nog) niet schrijven, Het Parool, https://www.parool.nl/kunst-media/een-robot-kan-dit-nog-niet-schrijven~bc838dbf/

D66 (2019) D66 pleit voor digitale revolutie, https://d66.nl/digitale-revolutie/.

e-Court persberichten (2018) e-Court sleept rechterlijke macht voor de rechter, http://www.e-court.nl/persberichten/

van Gaal (2019) Gezichtsherkenning op de Nederlandse straten: moeten we dat willen?, Motherboard, https://www.vice.com/nl/article/8xzydz/gezichtsherkenning-op-de-nederlandse-straten-moeten-we-dat-willen

van Gemert e.a. (2018) Pediatric Condition Falsification Misdiagnosed by Misjudged Weight Growth from the Curve of Measured Weights. Am J Case Rep. 2018, https://www.ncbi.nlm.nih.gov/pubmed/29946058

van Gemert e.a. (2019) Limitations of Weight Velocity Analysis by Commercial Computer Program Growth Analyser Viewer Edition,Annals of Biomedial Engineering 2019/47, https://link.springer.com/article/10.1007/s10439-018-02118-8

Hooyman, Paula (2019) Het ware gezicht van gezichtsherkenningstechnologie, https://www.bitsoffreedom.nl/wp-content/uploads/2019/11/het-ware-gezicht-van-gezichtsherkenningstechnologie.pdf

ING (2019) Katana named innovator 2019, https://www.ing.com/Newsroom/News/Katana-named-Innovator-2019.htm

Kamerstukken (2019), 2019-2020, 26 643 en 32 761, nr. 652, https://zoek.officielebekendmakingen.nl/kst-26643-652.html

Kamerstukken (2020), 2019-2020, 26 643 en 32 761, nr. 669, https://zoek.officielebekendmakingen.nl/kst-26643-669.html

Kuijpers, Karlijn, Thomas Muntz and Tim Staal (2018), Vonnis te koop, De Groene Amsterdammer, https://www.groene.nl/artikel/vonnis-te-koop

Landelijke Organisatie Sociaal Raadslieden LOSR/Sociaal Werk Nederland, (2018) Rechtspraak op bestelling?! Stop commerciële rechtspraak, https://schuldinfo.nl/fileadmin/Publicaties/Rechtspraak_op_bestelling_Stop_commerciele_rechtspraak.pdf

Middendorp, Jan (2019a) Initiatiefnota Menselijke grip op algoritmen, https://www.vvd.nl/content/uploads/2018/11/35212-2-Initiatiefnota-van-het-VVD-lid-Middendorp-Menselijke-grip-op-algoritmen-PDF.pdf

Middendorp, Jan (2019b) Initiatiefnota: Menselijke grip op algoritmen, https://www.tweedekamer.nl/kamerstukken/detail?id=2019D22106

Ministerie van Binnenlandse Zaken en Koninkrijksrelaties (2019a) Aanbiedingsbrief bij NL Digitaal: Data Agenda Overheid, https://www.rijksoverheid.nl/documenten/kamerstukken/2019/03/15/aanbiedingsbrief-bij-nl-digitaal-data-agenda-overheid

Minister van Binnenlandse Zaken en Koninkrijksrelaties (2019b) Kamerbrief met reactie op UU-rapport Algoritmes en grondrechten, https://www.rijksoverheid.nl/documenten/kamerstukken/2019/03/28/kamerbrief-met-reactie-op-uu-rapport-algoritmes-en-grondrechten

Minister van veiligheid en Justitie (2015) Antwoord op vragen van het Lid Marcouch, Tweede Kamer, 2014-2015, 1321, https://zoek.officielebekendmakingen.nl/ah-tk-20142015-1321.html

Ministerie van Justitie en Veiligheid (2018) Kamerbrief over toepassing artificiele intelligentie en algoritmen in de rechtspleging, https://www.rijksoverheid.nl/documenten/kamerstukken/2018/12/19/ek-artificiele-intelligentie-en-algoritmen-in-de-rechtspleging

Minister voor Rechtsbescherming (2019a), Antwoord op vragen van het lid Jasper van Dijk over het bericht ‘deurwaarders die onterecht kosten in rekening brengen, https://www.tweedekamer.nl/kamerstukken/kamervragen/detail?id=2019D47735&did=2019D47735

Minister voor Rechtsbescherming (2019b) Brief van de Minister voor Rechtsbescherming aan de Voorzitter van de Tweede Kamer der Staten-Generaal, https://zoek.officielebekendmakingen.nl/kst-26643-641.html

Minister voor Rechtsbescherming (2020), Antwoord op vragen van het lid Groothuizen over algoritmische analyse van vonnissen, https://www.tweedekamer.nl/kamerstukken/kamervragen/detail?id=2020Z03232&did=2020D11984

Mpozika, Nicole (2019) Disentangling the Bond Universe with Katana Lens, Medium, https://medium.com/wbaa/disentangling-the-bond-universe-with-katana-lens-4bd271fe97c2

Nederlandse AI Coalitie (2019) https://nlaic.com/

Nederland Digitaal (2019) Opbrengsten Conferentie Nederland Digitaal 2019, https://www.nederlanddigitaal.nl/actueel/nieuws/2019/03/21/opbrengsten-conferentie-nederland-digitaal-2019De

Nederlandsche Bank (2019) DNBulletin: DNB komt met richtlijnen voor gebruik kunstmatige intelligentie, https://www.dnb.nl/nieuws/nieuwsoverzicht-en-archief/DNBulletin2019/dnb385020.jsp

Niewold, Michaël (2018), ‘Inzet scanauto’s: meer opbrengst, minder agressie’, RTL Nieuws, https://www.rtlnieuws.nl/economie/artikel/3788081/inzet-scanautos-meer-opbrengst-minder-agressie

NOS (2019) Privacywaakhond: overheid moet transparanter zijn over algoritmes, NOS Nieuws, https://nos.nl/artikel/2286880-privacywaakhond-overheid-moet-transparanter-zijn-over-algoritmes.html

NPPL (2019a) Beregenen op maat met Raindancer, https://www.proeftuinprecisielandbouw.nl/beregenen-op-maat-met-raindancer/

NPPL (2019b) Sensor-gestuurde gewasbescherming in fruit, https://www.proeftuinprecisielandbouw.nl/techniek/sensor-gestuurde-gewasbescherming-in-fruit/

NU.nl (2018) Gezichtsherkenning politie Identificeerde 93 verdachten In 2017, https://www.nu.nl/internet/5141948/gezichtsherkenning-politie-identificeerde-93-verdachten-in-2017.html

NU.nl (2019) Gezichtendatabase van politie bevat foto‘s van 1,3 miljoen mensen, https://www.nu.nl/tech/5967571/gezichtendatabase-van-politie-bevat-fotos-van-13-miljoen-mensen.html

NWO (2019) Artificial Intelligence Research Agenda for the Netherlands, https://www.nwo.nl/binaries/content/documents/nwo/algemeen/documentation/application/nwo/beleid/open-science/ai-research-agenda-for-the-netherlands-airea-nl/AI+Research+Agenda+for+the+Netherlands+%28AIREA-NL%29

Platform Bescherming Burgerrechten (2019) Uitspraak in SyRI-rechtszaak 5 februari: lees de pleitnota van onze coalitie, https://platformburgerrechten.nl/2019/10/29/uitspraak-in-syri-rechtszaak-5-februari-lees-de-pleitnota-van-onze-coalitie/

Politie (2017) Criminaliteits Anticipaticie Systeem verder uitgerold bij Nationale Politie https://www.politie.nl/nieuws/2017/mei/15/05-cas.html

Politie (2018) Nieuwe technologie in oude politiezaken, https://www.politie.nl/nieuws/2018/mei/23/00-nieuwe-technologie-in-oude-politiezaken.html

Politie (2019a) Kunstmatige intelligentie vergroot onze slagkracht, 16 januari 2019, https://www.politie.nl/nieuws/2019/januari/16/%E2%80%98kunstmatige-intelligentie-vergroot-onze-slagkracht.html

Politie (2019b) Inzet slimme camera‘s tegen afleiding In het verkeer, 30 September 2019, https://www.politie.nl/nieuws/2019/september/30/00-inzet-slimme-cameras-tegen-afleiding-in-het-verkeer.html

Privacy First (2019) Submission of Privacy First to the fifth periodic review of the Netherlands by the UN Human Rights Committee, https://www.privacyfirst.nl/images/stories/PDFs/Privacy_First_submission_addendum_HRC_2019.pdf

Rechtbank Den Haag (2020), 5 February 2020, ECLI:NL:RBDHA:2020:865. Official Dutch language version: http://deeplink.rechtspraak.nl/uitspraak?id=ECLI:NL:RBDHA:2020:865. Unofficial English translation: http://deeplink.rechtspraak.nl/uitspraak?id=ECLI:NL:RBDHA:2020:1878.

De Rijksoverheid (2019a) Strategisch Actieplan voor Artificiële Intelligentie, https://www.rijksoverheid.nl/documenten/beleidsnotas/2019/10/08/strategisch-actieplan-voor-artificiele-intelligentie

De Rijksoverheid (2019b) Nederlandse Digitaliseringsstrategie 2.0, https://www.rijksoverheid.nl/documenten/rapporten/2019/07/05/nederlandse-digitaliseringsstrategie-2.0

De Rijksoverheid (2019c) Dutch Digitisation Strategy 2.0, https://www.nederlanddigitaal.nl/binaries/nederlanddigitaal-nl/documenten/publicaties/2019/11/13/english-version-of-the-dutch-digitalisation-strategy-2.0/Dutch+Digitisation+Strategy+2.0.pdf

RTL nieuws (2019) Politie zet vanaf vandaag camera‘s in tegen appende automobilisten, https://www.rtlnieuws.nl/nieuws/nederland/artikel/4867361/camera-controle-appen-achter-stuur-auto-politie-boete-240-euro

RVT Utrecht (2018) Wetenschappers kritisch over kinderartsen en Veilig Thuis, https://www.rtvutrecht.nl/nieuws/1793482/wetenschappers-kritisch-over-kinderartsen-en-veilig-thuis.html

RTV Utrecht (2019) Wetenschappers: „Software bij kinderartsen geeft mogelijk foute uitkomst“, https://www.rtvutrecht.nl/nieuws/1970056/wetenschappers-software-bij-kinderartsen-geeft-mogelijk-foute-uitkomst.html

Schellevis, Joost, and Winny de Jong (2019), Overheid gebruikt op grote schaal voorspellende algoritmes, ‚risico op discriminatie‘, NOS Nieuws,29 May 2019 https://nos.nl/artikel/2286848-overheid-gebruikt-op-grote-schaal-voorspellende-algoritmes-risico-op-discriminatie.html

Tweakers Partners (2019) Kunstmatige intelligentie moet politie helpen bij cold cases, 10 April 2019, https://tweakers.net/plan/2072/kunstmatige-intelligentie-moet-politie-helpen-bij-cold-cases.html

University of Amsterdam (2019a) The Digital Transformation of Decision-Making, https://www.uva.nl/en/faculty/amsterdam-law-school/research/research-themes/digital-transformation/digital-transformation-home.html

University of Amsterdam (2019b) Kick-off meeting AI & the Administration of Justice, https://www.uva.nl/shared-content/faculteiten/nl/faculteit-der-rechtsgeleerdheid/evenementen/evenementen/2019/09/kick-off-meeting-ai--the-administration-of-justice.html

University of Amsterdam (2019c) RPA Human(e) AI, https://humane-ai.nl/

Visser, Cees (2018) ‚ING neemt ai-applicatie op in bankomgeving‘, Computable, https://www.computable.nl/artikel/nieuws/development/6417555/250449/ing-neemt-ai-applicatie-op-in-bankomgeving.html

Waarlo, Niels (2019) NOS zet robotjournalist in voor verkiezingsuitslagen, de Volkskrant,https://www.volkskrant.nl/cultuur-media/nos-zet-robotjournalist-in-voor-verkiezingsuitslagen~b0e1159e/

Wageningen University & Research (2019), Precisielandbouw, https://www.wur.nl/nl/Dossiers/dossier/dossier-precisielandbouw.htm

Wesselink, Henk et al. (2018), Datagedreven sturring bij gemeenten. Van data tot (gedeelde) informatie voor beter (samen) sturen, Vereniging Nederlandse Gemeenten, Berenschot, https://vng.nl/files/vng/nieuws_attachments/2018/datagedreven_sturing_bij_gemeenten_lr.pdf

Wokke, Arnoud (2019) RTL Nieuws gebruikt software om lokale artikelen te maken, Tweakers, https://tweakers.net/nieuws/159930/rtl-nieuws-gebruikt-software-om-lokale-artikelen-te-maken.html

Team

Ronan Ó Fathaigh

Ronan Ó FathaighRonan Ó Fathaigh is a Senior Researcher at the Institute for Information Law (IViR), University of Amsterdam. He is a member of the Digital Transformation of Decision-Making research initiative, examining the normative implications of the shift toward automated decision-making and the effect on democratic values and fundamental rights. Ro- nan has published his work in a number of international academic journals and co-authored reports for international institutions such as the Council of Europe and the European Parliament. He has also co-authored a report for the Dutch Ministry of Internal Affairs on online disinformation and political advertisements, and he has conducted research on automated decision-making in content moderation systems, as part of a joint research initiative of IViR and the Annenberg Public Policy Center, University of Pennsylvania. Ronan has a Ph.D. in law from Ghent University, and he is a former visiting research fellow at Columbia University. Ronan is a regulator contributor to the Strasbourg Observers blog, commenting on current human rights issues.

Naomi Appelman

Naomi AppelmanNaomi Appelman is a PhD researcher at the Institute for Information Law (IViR), University of Amsterdam. Her PhD looks at platform regulation and automated decision-making through the lens of democratic values and fundamental rights within the broader context of the digital transition of decision-making research initiatives. She combines legal and philosophical perspectives in her research, which focuses on strengthening the position of people in online speech regulation. Within the IViR, she has worked on research reports on disinformation and political advertisements for the Dutch Ministry of Internal Affairs, and she has worked on an online content moderation report. Naomi also studied philosophy at the University of Amsterdam.