How Private AI can help comply with the EU AI Act

Mar 14, 2024
Share this post
Sharing to FacebookSharing to LinkedInSharing to XSharing to Email

On March 13, the European Parliament crossed another and (almost) the last hurdle for the EU AI Act to come into force. Anticipated to become law by May or June, it will see its provisions phased in: six months later, countries will be mandated to ban prohibited AI systems; one year later, regulations for general-purpose AI systems will commence; and two years later, the entire AI Act will be enforceable.It’s a good idea to prepare for compliance now, as there is a lot to do for AI developers and providers. This article focuses on the obligations imposed by the EU AI Act as they relate to data protection and how Private AI’s technology can facilitate compliance.

Data Protection Obligations under the EU AI Act

Recital 45 in the final text clarifies that “Practices that are prohibited by Union law, including data protection law [...] should not be affected by this Regulation.” This means, not surprisingly, that the General Data Protection Regulation (GDPR) applies in addition to the EU AI Act. Recital 69 expands on this:

The right to privacy and to protection of personal data must be guaranteed throughout the entire lifecycle of the AI system. In this regard, the principles of data minimisation and data protection by design and by default, as set out in Union data protection law, are applicable when personal data are processed. Measures taken by providers to ensure compliance with those principles may include not only anonymisation and encryption, but also the use of technology that permits algorithms to be brought to the data and allows training of AI systems without the transmission between parties or copying of the raw or structured data themselves, without prejudice to the requirements on data governance provided for in this Regulation.

The EU AI Act also tells us that it “should not be understood as providing for the legal ground for processing of personal data, including special categories of personal data, where relevant, unless it is specifically otherwise provided for in this Regulation.” This warrants an explanation.Under the GDPR, the processing of personal data and particularly of sensitive data called “special categories of data” can only be done if there is a legal basis for such processing, such as consent provided by the affected individual or a legitimate interest of the processor. The EU AI Act now clarifies that it does not constitute such a legal basis, but that a legal basis must instead be found in the GDPR, unless the Act specifically says otherwise.An exceptional legal basis for the processing of special categories of personal data is provided for in Art. 10(5) and the corresponding Recital 70. Art. 10(5) says for bias detection and correction special categories of personal data may be processed, subject to strict security safeguards and under the condition that bias detection and correction cannot be effectively carried out with synthetic or anonymized data. This exception to the prohibition of the GDPR around the processing of special categories of personal data is the only one in the Act and it does not apply to the development of AI systems that scrape data from the web for general training purposes.More generally, Art. 10(2)(b) requires developers of high-risk AI systems to implement risk mitigation and data governance practices that concern “data collection processes and the origin of data, and in the case of personal data, the original purpose of the data collection.” This somewhat cryptic provision seems to say that developers must consider the purpose for which personal data was originally collected and how, and ensure that the use for the development of the high-risk AI system is permitted. Note that Art. 5 of the GDPR requires the specification of the purpose for which personal data will be used and a legal basis for each purpose. Once collected, the data cannot be used for purposes incompatible with the original one. In other words, if an organization collects personal data for the purpose of providing services to a consumer, it is not a given that this data can then be used to train a high-risk AI system. Developers must also implement a risk mitigation system for high-risk AI systems. Art. 9 which imposes this obligation does not make reference to any specific types of risks, which allows the conclusion that risks to privacy must be covered.Providers of high-risk AI systems are required under Art. 17 to implement a quality management system which must include comprehensive systems and procedures for data management that cover pretty much every data-related operation performed before and for the purpose of the placing on the market or the putting into service of high-risk AI systems. As part of this quality management system, providers are obliged to include a risk mitigation system in accordance with Art. 9, addressed above.Deployers of high-risk AI systems must conduct a fundamental rights impact assessment under Art. 27. Art. 27(4) clarifies that if a data protection impact assessment has already been conducted pursuant to Art. 35 of the GDPR, the fundamental rights impact assessment shall complement that prior assessment. This means that the fundamental rights impact assessment must include privacy considerations that arise from the deployment of the high-risk AI system.Providers of general-purpose AI systems (GPAIs) (which are not automatically high-risk AI systems), including open source GPAIs, must “draw up and make publicly available a sufficiently detailed summary about the content used for training of the general-purpose AI model, according to a template provided by the AI Office.” (Art. 53(1)(d)). For details of GPAIs read our article EU AI Act Final Draft – Obligations of General-Purpose AI Systems relating to Data Privacy.Providers of general-purpose AI models with systemic risks have the additional obligation to assess and mitigate systemic risks that may arise from the development, the placing on the market, or the use of general-purpose AI models with systemic risk. A GPAI is presumed to have systemic risks when the cumulative amount of computation used for its training measured in FLOPs is greater than 10^25. The Act does not specify whether systemic risks that must be assessed and mitigated include privacy risks, but given that large language models of this magnitude are thus far trained on enormous amounts of data scraped from the internet, which includes personal information, and given that training data is regularly memorized by these models, it does not seem far fetched to conclude that systemic risks may include privacy risks. This interpretation is supported by the fact that the criteria for the determination of what constitutes systemic risks listed in Annex XIII include the quality and size of the data set as well as the specific types of in- and outputs.

How Private AI can Help with Compliance

All privacy-related responsibilities faced by developers, providers, and deployers of AI systems share a common challenge: they are exceedingly difficult to overcome without clear insight into the personal data within their training datasets. Absent such visibility, organizations will struggle to provide the necessary disclosures prior to gathering personal data, hindered by the inability to request specific consent or effectively implement data subject rights. This is particularly evident with the right to erasure, where worst-case scenarios may necessitate extensive retraining efforts, posing environmental concerns due to resource-intensive training processes and imposing financial burdens on businesses. Reporting obligations, fundamental right impact assessments, and systemic risks assessments are equally impossible without knowing what personal data is included in the training data sets.Despite various strategies employed by model developers to tackle privacy issues, their effectiveness is constrained. Some opt to exclude websites containing substantial personal data from data scraping efforts, while others engage independent privacy experts in Reinforcement Learning from Human Feedback (RLHF) endeavors to align the model with the objective of safeguarding personal information. However, these approaches still leave vulnerabilities in personal data protection, exposing entities to potential liabilities under the GDPR and the EU AI Act.Enter Private AI. Private AI’s technology is able to identify and report on personal identifiers included in large unstructured data sets and to replace them with synthetic data or placeholders. For many use cases, this approach that relies on context-aware algorithms trained by data experts in multiple languages is able to preserve data utility while maximizing data privacy.This technology is not only useful for model developers but also further down the value chain. When businesses are concerned about violating privacy rights when employees include personal data in their prompts sent to an external model, Private AI’s PrivateGPT can be deployed to intercept the prompt, filter out the personal data and re-inject it automatically into the response for a seamless user experience. Test PrivateGPT for free here.In addition, with the help of Private AI, privacy can be preserved during fine-tuning, creating embeddings for Retrieval Augmented Generation (RAG), and bias reduction.

Data Left Behind: AI Scribes’ Promises in Healthcare

Data Left Behind: Healthcare’s Untapped Goldmine

The Future of Health Data: How New Tech is Changing the Game

Why is linguistics essential when dealing with healthcare data?

Why Health Data Strategies Fail Before They Start

Private AI to Redefine Enterprise Data Privacy and Compliance with NVIDIA

EDPB’s Pseudonymization Guideline and the Challenge of Unstructured Data

HHS’ proposed HIPAA Amendment to Strengthen Cybersecurity in Healthcare and how Private AI can Support Compliance

Japan's Health Data Anonymization Act: Enabling Large-Scale Health Research

What the International AI Safety Report 2025 has to say about Privacy Risks from General Purpose AI

Private AI 4.0: Your Data’s Potential, Protected and Unlocked

How Private AI Facilitates GDPR Compliance for AI Models: Insights from the EDPB's Latest Opinion

Navigating the New Frontier of Data Privacy: Protecting Confidential Company Information in the Age of AI

Belgium’s Data Protection Authority on the Interplay of the EU AI Act and the GDPR

Enhancing Compliance with US Privacy Regulations for the Insurance Industry Using Private AI

Navigating Compliance with Quebec’s Act Respecting Health and Social Services Information Through Private AI’s De-identification Technology

Unlocking New Levels of Accuracy in Privacy-Preserving AI with Co-Reference Resolution

Strengthened Data Protection Enforcement on the Horizon in Japan

How Private AI Can Help to Comply with Thailand's PDPA

How Private AI Can Help Financial Institutions Comply with OSFI Guidelines

The American Privacy Rights Act – The Next Generation of Privacy Laws

How Private AI Can Help with Compliance under China’s Personal Information Protection Law (PIPL)

PII Redaction for Reviews Data: Ensuring Privacy Compliance when Using Review APIs

Independent Review Certifies Private AI’s PII Identification Model as Secure and Reliable

To Use or Not to Use AI: A Delicate Balance Between Productivity and Privacy

To Use or Not to Use AI: A Delicate Balance Between Productivity and Privacy

News from NIST: Dioptra, AI Risk Management Framework (AI RMF) Generative AI Profile, and How PII Identification and Redaction can Support Suggested Best Practices

Handling Personal Information by Financial Institutions in Japan – The Strict Requirements of the FSA Guidelines

日本における金融機関の個人情報の取り扱い - 金融庁ガイドラインの要件

Leveraging Private AI to Meet the EDPB’s AI Audit Checklist for GDPR-Compliant AI Systems

Who is Responsible for Protecting PII?

How Private AI can help the Public Sector to Comply with the Strengthening Cyber Security and Building Trust in the Public Sector Act, 2024

A Comparison of the Approaches to Generative AI in Japan and China

Updated OECD AI Principles to keep up with novel and increased risks from general purpose and generative AI

Is Consent Required for Processing Personal Data via LLMs?

The evolving landscape of data privacy legislation in healthcare in Germany

The CIO’s and CISO’s Guide for Proactive Reporting and DLP with Private AI and Elastic

The Evolving Landscape of Health Data Protection Laws in the United States

Comparing Privacy and Safety Concerns Around Llama 2, GPT4, and Gemini

How to Safely Redact PII from Segment Events using Destination Insert Functions and Private AI API

WHO’s AI Ethics and Governance Guidance for Large Multi-Modal Models operating in the Health Sector – Data Protection Considerations

How to Protect Confidential Corporate Information in the ChatGPT Era

Unlocking the Power of Retrieval Augmented Generation with Added Privacy: A Comprehensive Guide

Leveraging ChatGPT and other AI Tools for Legal Services

Leveraging ChatGPT and other AI tools for HR

Leveraging ChatGPT in the Banking Industry

Law 25 and Data Transfers Outside of Quebec

The Colorado and Connecticut Data Privacy Acts

Unlocking Compliance with the Japanese Data Privacy Act (APPI) using Private AI

Tokenization and Its Benefits for Data Protection

Private AI Launches Cloud API to Streamline Data Privacy

Processing of Special Categories of Data in Germany

End-to-end Privacy Management

Privacy Breach Reporting Requirements under Law25

Migrating Your Privacy Workflows from Amazon Comprehend to Private AI

A Comparison of the Approaches to Generative AI in the US and EU

Benefits of AI in Healthcare and Data Sources (Part 1)

Privacy Attacks against Data and AI Models (Part 3)

Risks of Noncompliance and Challenges around Privacy-Preserving Techniques (Part 2)

Enhancing Data Lake Security: A Guide to PII Scanning in S3 buckets

The Costs of a Data Breach in the Healthcare Sector and its Privacy Compliance Implications

Navigating GDPR Compliance in the Life Cycle of LLM-Based Solutions

What’s New in Version 3.8

How to Protect Your Business from Data Leaks: Lessons from Toyota and the Department of Home Affairs

New York's Acceptable Use of AI Policy: A Focus on Privacy Obligations

Safeguarding Personal Data in Sentiment Analysis: A Guide to PII Anonymization

Changes to South Korea’s Personal Information Protection Act to Take Effect on March 15, 2024

Australia’s Plan to Regulate High-Risk AI

How Private AI can help comply with the EU AI Act

Comment la Loi 25 Impacte l'Utilisation de ChatGPT et de l'IA en Général

Endgültiger Entwurf des Gesetzes über Künstliche Intelligenz – Datenschutzpflichten der KI-Modelle mit Allgemeinem Verwendungszweck

How Law25 Impacts the Use of ChatGPT and AI in General

Is Salesforce Law25 Compliant?

Creating De-Identified Embeddings

Exciting Updates in 3.7

EU AI Act Final Draft – Obligations of General-Purpose AI Systems relating to Data Privacy

FTC Privacy Enforcement Actions Against AI Companies

The CCPA, CPRA, and California's Evolving Data Protection Landscape

HIPAA Compliance – Expert Determination Aided by Private AI

Private AI Software As a Service Agreement

EU's Review of Canada's Data Protection Adequacy: Implications for Ongoing Privacy Reform

Acceptable Use Policy

ISO/IEC 42001: A New Standard for Ethical and Responsible AI Management

Reviewing OpenAI's 31st Jan 2024 Privacy and Business Terms Updates

Comparing OpenAI vs. Azure OpenAI Services

Quebec’s Draft Regulation Respecting the Anonymization of Personal Information

Version 3.6 Release: Enhanced Streaming, Auto Model Selection, and More in Our Data Privacy Platform

Brazil's LGPD: Anonymization, Pseudonymization, and Access Requests

LGPD do Brasil: Anonimização, Pseudonimização e Solicitações de Acesso à Informação

Canada’s Principles for Responsible, Trustworthy and Privacy-Protective Generative AI Technologies and How to Comply Using Private AI

Private AI Named One of The Most Innovative RegTech Companies by RegTech100

Data Integrity, Data Security, and the New NIST Cybersecurity Framework

Safeguarding Privacy with Commercial LLMs

Cybersecurity in the Public Sector: Protecting Vital Services

Privacy Impact Assessment (PIA) Requirements under Law25

Elevate Your Experience with Version 3.5

Fine-Tuning LLMs with a Focus on Privacy

GDPR in Germany: Challenges of German Data Privacy (Part 2)

Comply with US Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence using Private AI

How to Comply with EU AI Act using PrivateGPT