Automating Society Report 2020

Story

Slovenian police acquire automated tools first, legalizes them later

by Lenart J. Kučić

The Slovenian police legalized its use of face recognition five years after it started to use it. Despite formal safeguards, no institution can restrain the Interior ministry.

When Slovenian journalists or activists ask officials whether the police, the secret service, and the army are using any technological tools for mass surveillance, they are often reluctant to admit even the existence of such devices. However, when they meet their colleagues at international security conferences or receive questions from foreign journalists, they want to make a positive impression. “Of course, we are testing (or using) the latest new gadgets to fight terrorism, organized crime or illegal migration,” they say. “We are a modern and technologically advanced security force, after all!”

This may sound anecdotal but it happens to be true.

When Slovenian civil society activists and privacy experts read the 2019 AlgorithmWatch report on face recognition in the EU, they were surprised to find Slovenia listed among the countries where the police use face recognition.

“Slovenian police personnel have been using face recognition since 2014. The software has been developed internally,” the report claimed, quoting the Slovenian Police as the source.

It turned out Slovenian journalists and privacy activists might have been following the wrong sources.

How do you detect evil aliens before they can invade human society? You need to follow the “hot sheets”, said special agent “K”, played by American actor Tommy Lee Jones in the hit sci-fi movie “Men in Black”.

What he meant by “hot sheets” were the supermarket tabloid magazines reporting on alien abductions and other strange and bizarre stories. You may also get lucky by reading the New York Times, “K” explained to the rookie agent  “J” (Will Smith). But tabloids are a much better source of information if you want to control alien activity on Earth and protect humanity.

But the “hot sheets” do not only reveal potential aliens. They can also alert privacy advocates to unusual police activity.

“Aggressive pensioners recognized on Facebook,” claimed the headline in Slovenske novice, the biggest Slovenian print tabloid. A woman from Ljubljana, the Slovenian capital, left her car on a bicycle track. She left it with the blinkers on, and she ran to a nearby restaurant. It only took her a minute to come back, the tabloid wrote, but her parked car had already annoyed an elderly couple.

“I saw my friend in the police database”

The articles in Slovenske novice and Delo about face recognition are not isolated examples.

In January 2017, a woman contacted a journalist writing for the national television website MMC. She was stopped by traffic police. She was not carrying her ID, so the police officer brought her to his car to confirm her identity on the police computer. She gave her personal information (name, address, date of birth) and photographs of her appeared on the screen. She expected to see her official photographs – from her ID, passport, and driving license – but she also noticed photographs that she had published on her personal Facebook profile.

Why would the police store her Facebook photographs in their database? she wondered. Was that even legal?

The police declined to confirm or deny that they gathered pictures from social media profiles in their database. Their spokesperson then suggested to the journalist that the police officer may have also checked publicly available photographs on the Internet during the procedure, which could have given a wrong impression to the driver (that her Facebook photographs were stored in the police database). Instead, they claimed that everybody could access the information on social media and provided a lengthy legal argument on why citizens must always carry a valid ID to avoid such police procedures and paying a fine.

A similar example dates back even further – before Face Trace was officially introduced. A citizen told us that they filed a complaint to the Information Commissioner in September 2013. The complaint included a passage where a young woman reported an attempted sexual assault to the police. When she browsed through the police database to find and recognize the alleged attacker, she also saw a photo of her friend.

“He was never accused or convicted of any wrongdoings,” the complaint states. He did join some peaceful public protests, though. Is it possible that the police take photographs of protesters and store them in a database of photographed individuals, or in a database of “extremists”? she wondered. And on what legal basis could the police do such a thing?

The Information Commissioner replied to the complaint in February 2014. The police denied that a database of “extremist” protesters existed, the Commissioner wrote. But they explained that the database of photographed individuals could also contain a photograph of one’s friend or relative if this person has ever been suspected of a criminal act. The police can also record or photograph public events and keep “relevant pieces” of the material. At that time, the police database of photographed individuals contained around 30,000 people, according to the Commissioner.

From the examples above, it is possible to conclude that Slovenian police have been using face recognition software since 2014. Also, the police confirmed to me that they use face recognition systems to compare, for example, police sketches to pictures in their database of photographed individuals. This is important because the database could contain photographs from personal social profiles and mass public events.

We are using conditionals because we do not know for certain what photographs the police actually keep in their database. During our investigation, the official answers provided by the police remained ambiguous and the Information Commissioner has not yet decided to investigate the matter and evaluate the use and content of police databases.

Hacking the law

In addition, the police are often allowed to interpret the legislation as it best suits their needs – without much opposition from the public, politicians, or privacy experts. “The police always follow the same pattern. First, their representatives dramatically demonstrate all the new threats to public life: international terrorism and organized crime, migrant crisis, and online threats – from pedophilia to cyber- attacks. Next, they express their frustrations and concerns because they cannot efficiently fight the bad guys who have all the latest tools and no legal restrictions on how to use them. So, they wait for every opportunity to amend the existing legislation and expand their powers,” explained a public official who took part in several negotiation rounds with the police and who requested anonymity.

The police always present a very ambitious wish list, our source said. Some wishes are clearly not realistic and would never pass the legislative process: weakening the encryption on communication apps or using drones for mass surveillance. But they can be used in negotiations with the Information Commissioner and other privacy advocates.

“Fine, let us not touch the encryption,” say the police. But we still need to amend the outdated law on duties and powers of the police and include, for example, biometrics. After biometrics is mentioned in the legislation the police can interpret this as a legal basis for using face recognition.

“They are hacking the legislation this way. This is like install- ing a piece of malware to the operating system and then using the exploits to legalize whatever you are doing,” concluded the anonymous source.

Such hacking of the legislative process by the police is not merely hypothetical. When the Slovenian police bought its first IMSI catcher (a tool to snoop on cell phones) in 2004, there was no legal basis for its use, the Slovenian online medium Slo-Tech revealed. Nevertheless, the police secretly used the machine in more than 300 cases, from 2006 to 2012, and then bought another IMSI catcher in 2012.

When lawyers, journalists, and activists first challenged the allegedly illegal use of the IMSI catchers the police spokesperson denied the existence of such devices. However, they had to change their communications strategy after a senior police official said in an interview on public radio that they were using IMSI catchers “to find missing persons”. Instead of denying their purchase and use, the police tried – with a lot of help from the government – to retroactively legalize IMSI catchers.

The police approach to face recognition was very similar.

The Slovenian Information Commissioner confirmed that the police informed them about the use of the face recognition software, Face Trace, back in 2014. The police claimed that the Commissioner agreed to their suggested uses of the product. But the Commissioner’s office actually issued several critical comments regarding the use of biometric methods and face recognition between 2015 and 2019. The Commissioner’s office also appealed to the Slovenian Human Rights Ombudsman to file a formal complaint to the constitutional court when the Slovenian Ministry for Internal Affairs introduced a new law on the duties and powers of the police in 2017.

The new amendments allowed the police to use biometric and face recognition tools, and thus provided a legal framework for the use of the Face Trace software. Again, the police tried to retroactively legalize its use and the government ignored all the warnings from the Information Commissioner. How was this possible? There was no public pressure, argued the Information Commissioner. Nobody seemed to care about their warnings against potential police abuse of biometry and face recognition. As a consequence, the police are now allowed to automatically combine face recognition with other biometric data like fingerprints and DNA profiles.

No public pressure?

The police, nonetheless, did face some opposition. In the 2019 Automating Society report, we learned that the Slovenian Human Rights Ombudsman, and the Information Commissioner, filed a formal complaint to the constitutional court in 2017. They claimed that the new law on the duties and powers of the police legalized some excessive and inadmissible measures for gathering personal data without sufficient protection of citizens who have not been accused or suspected of any wrongdoing. In addition, they warned against the use of drones and automated systems for recognizing license plates, as well as the unregulated use of passenger name records at Slovenian airports. But for some reason, biometric systems (such as face recognition) were not included in the complaint.

How so? The Information Commissioner explained that they suggested to the Slovenian Human Rights Ombudsman that biometrics should also be included in the complaint. But that did not happen. The representatives of the Ombudsman explained (to us) that there had been no public debate or controversy surrounding the use of biometrics by the police at the time. They had a legal option to file another complaint to the constitutional court and address the use of face recognition but they did not take any action. Why? Because they did not want to over-use this exceptional legal measure, according to the Ombudsman.

This argument clearly illustrates some of the problematic aspects regarding the implementation and use of Artificial Intelligence (AI), algorithms, and automated decision-making systems (ADM) in Slovenia. During our research for the 2019 Automating Society report, very few civil society organizations or individuals addressed the social impacts of automation on society. In addition, conferences about AI are usually sponsored and organized by commercial companies and industry groups who want to promote their solutions. Furthermore, many of the specialized journalists who cover such issues have quit the media in recent years. As a result, there is no rigorous coverage on the topic and privacy experts or activists have lost access to the general public.

Civil society should not be blamed for this state of affairs. “Slovenian non-governmental organizations (NGOs) are small and understaffed. As far as I know, no NGO is monitoring the police and the use of their powers systematically,” said Katarina Bervar Sternad from the Legal Centre for the Protection of Human Rights and Environment. “We cannot afford to dedicate one staffer to this particular field. This is unfortunate because we receive many questions regarding potential abuses of police powers.”

Domen Savič, from the NGO Državljan D, makes a similar point. “The NGO sector is mostly funded by the government or international EU grants,” Savič said. If there are no tenders available for the information society, then NGOs cannot afford to cover this field. Furthermore, the media and politicians rarely understand how technology impacts human rights. “When AlgorithmWatch revealed that Slovenian police [were] using face recognition software to profile… citizens and some Slovenian media investigated the story, there was not a single voice of protest raised in the parliament.”

To make things worse, there is almost no collaboration between politics, the media, and civil society, warned Savič. “It takes an extraordinary amount of effort, time, and pressure to actually change a policy or a police practice. But most of the issues die out pretty quickly even if there is someone who tries to get answers from the people involved.”

Consequently, the police face almost no opposition when handing in their “wish-list” to the government.

Another worrying example is a recent attempt by the Ministry of Justice to use the GDPR – to which the necessary amendments to Slovenian legislation must comply – to exclude the police from civilian control. The Information Commissioner warned that the ministry wanted to redefine its role under the new (amended) law on personal data protection. Accord- ing to the ministry’s proposal, the Information Commissioner would no longer be able to oversee the police.

Authoritarian tendencies

Such safety mechanisms are critical and monitoring of the police should not be left only to NGOs and activists. Mr. Savič, Ms. Bervar Sternad, and the anonymous official all agreed that a common European regulatory approach would be necessary because national NGOs cannot effectively monitor how the police use the technology. EU Member States, on the other hand, can use the sense of emergency to adopt new pieces of legislation and bypass parliamentary debate. The current health crisis is the most recent example of such an attempt.

Soon after the COVID-19 pandemic started earlier this year, the Slovenian government began using the crisis as an excuse to significantly expand police powers.

The method they used was similar to the previous attempts at legislation hacking. In March 2020, the Slovenian government proposed a first draft of the “Act on the intervention measures to mitigate the consequences of the communicable disease SARS-CoV-2 (COVID-19) for citizens and the economy”. They submitted the text to the National Assembly for consideration and adoption under an emergency procedure. However, the draft also included two articles that dramatically increased the powers of the police.

Article 103 suggested that the police could use various methods to ensure that the citizens obey the quarantine and the Communicable Diseases Act. They could – among other measures – use face recognition when stopping and identifying individuals, enter their houses or apartments, limit their movement as well as collect and process protected personal information such as medical data provided by the National Institute of Public Health. Article 104 went even further. It suggested that the police could trace the location of the mobile phones of individuals without a court warrant.

All the suggested measures were introduced under an emergency procedure – without any consultations or public debates. The Information Commissioner thus described the anti-COVID-19 measures as an attempt to potentially “establish a police state”. They considered the new police powers to be too broad and potentially unconstitutional and undemocratic. The Human Rights Ombudsman wrote that it was hard to believe that such measures were really necessary and proportional (neither institutions were consulted during the process). A critical commentary was also published by the members of the Institute of Criminology who wrote that mass surveillance is not compatible with European legal culture.

Article 104 was eventually removed from the amended act because of strong criticisms from the public and the opposition political parties. However, article 103 on the powers of the police remained in the “COVID Act” that was adopted in April 2020. Furthermore, the government insisted that contact tracing applications were necessary to help health officials stop the pandemic. They also suggested that citizens would have to install such an application when traveling across the country. Data from the application would be collected and used by the National Institute of Public Health, but the police would also be allowed to access the database.

Public protests

When the first groups of citizens started protesting against the government in April 2020, Interior minister Aleš Hojs demanded on Twitter that the police use their new powers to identify and prosecute the protesters, such as collecting and analyzing all available images from traditional and social media. The government claimed that the protesters were violating laws on public health, and that any public gathering was illegal.

According to what we know, the police can collect and analyze photographs from social media. The police also routinely record public events, protests, and other mass gatherings. Furthermore, the new legislation could allow the police to access other kinds of personal information (contact tracing) to curb future anti-government protests.

It takes many small steps to build a – partly automated – surveillance infrastructure. The Slovenian police have been consistently taking such steps and the “COVID Act” has most likely legalized most of the items on their “wish-list”, from face recognition to contact tracing.

Author

Lenart J. Kučić

Lenart J. KučićLenart J. Kučić is a journalist, lecturer, and podcaster. He is a member of Pod črto, a non-profit and independent media collective specialized in investigative reporting, data journalism, and in-depth stories. He is a co-editor of the online magazine Disenz and a co-founder of the podcasting network, Marsowci. In addition, he is the author and co-author of several books, expert articles, and studies on the subjects of media ownership and new media. His recent work focuses on computational propaganda, the use of algorithms, and the social impact of ICT’s.