What the International AI Safety Report 2025 has to say about Privacy Risks from General Purpose AI

As the world gears up for the AI Action Summit in Paris in February 2025, global policymakers, researchers, and industry leaders are turning their attention to a landmark publication: The International AI Safety Report 2025. This report, a collaborative effort by 96 AI experts from around the world, represents the most comprehensive scientific assessment to date of the risks posed by general-purpose AI—a rapidly advancing form of AI with the ability to perform a wide range of tasks.

Building on the Interim Report released in May 2024 ahead of the AI Seoul Summit, this full edition reflects the latest advancements in AI capabilities, emerging risks, and evolving mitigation strategies. The report does not propose specific policies but aims to provide scientific clarity to inform international decision-making.

Why This Report Matters

Since the Bletchley Park AI Safety Summit in November 2023, the capabilities of general-purpose AI have continued to grow. Recent breakthroughs include models with superior performance in scientific reasoning, programming, and autonomous decision-making. Many companies are now investing in AI agents—systems that can plan, execute tasks, and delegate actions independently. While these advancements unlock significant opportunities, they also raise new concerns about safety, security, and privacy.

A key theme of the report is the uncertainty surrounding AI’s trajectory. While some experts believe risks such as AI-enabled cyberattacks, large-scale job displacement, and loss of human control are years away, others caution that these challenges may arise sooner than expected. Notably, the AI company OpenAI recently shared early test results from its latest model, o3, which has demonstrated unprecedented levels of reasoning and problem-solving abilities, outperforming many human experts in select tasks. This suggests that AI capabilities may be advancing even faster than anticipated, reinforcing the need for proactive risk management.

Privacy Risks in General-Purpose AI

One of the concerns outlined in the Report is the threat that general-purpose AI poses to privacy. The report categorizes these risks into three key areas, training risks, use risks, and intentional harm risks.

1. Training Risks: Personal Data Embedded in AI Models

General-purpose AI models are trained on vast datasets collected from public sources, proprietary databases, and user interactions. This data often includes personally identifiable information (PII) and sensitive data, often without the knowledge or consent of the individuals involved.Key concerns include:

• Memorization of training data: AI models can unintentionally retain and reproduce sensitive details, including health records, financial data, and private conversations.
• Lack of individual control: Many AI models are trained on publicly available data that may not have been intended for large-scale AI processing, disregarding a central pillar of privacy laws, namely that individuals are supposed to remain in control of their data, steering its use with their informed consent.
• Lack of clear deletion mechanisms: Once sensitive data is incorporated into an AI model, removing it is extremely difficult, raising questions about compliance with the right to have one’s data deleted.

The Report remarks that in sensitive fields like healthcare and finance, training on real-world data improves performance but increases the risk of privacy leaks. Efforts to reduce these risks include using anonymized or pseudonymized data, as seen with Google’s Gemini-Med, but further research is needed to assess their effectiveness. Privacy-preserving techniques and synthetic data offer potential solutions but remain technically challenging.

2. Use Risks: AI Handling Sensitive Information in Real Time

As for risks during the usage of general-purpose AI systems, the Report focuses on leaked information that is fed to the system to personalize the response. A common strategy to do so is Retrieval-Augmented Generation (RAG) that permits the access to current and personal data beyond the model’s initial training. While this enables more personalized responses in applications ranging from digital assistants to healthcare tools, it creates additional privacy risks. These risks are especially pronounced when data or insights move beyond local devices. Despite available cybersecurity measures, balancing privacy, transparency, and utility remains a key challenge, requiring both technical solutions and policy frameworks to protect user data while enabling innovation.

3. Intentional Harm Risks: AI as a Tool for Privacy Violations

Malicious actors can exploit general-purpose AI to compromise privacy at scale, using AI to enhance cyberattacks, automate surveillance, and generate fraudulent content.

Examples of misuse include:

• AI-enhanced cybercrime: Attackers can use AI to scan breached datasets, identify high-value targets, and automate phishing attacks.
• Deepfake and identity fraud risks: AI-generated synthetic media can be used for impersonation, misinformation, and harassment.
• Automated tracking and surveillance: Advanced pattern recognition and image analysis allow AI to infer sensitive personal details from seemingly unrelated data points, making mass privacy violations easier.

The Report cautions that privacy harms can remain hidden for a long time given the significant time lapse between collection and use for training and deployment. On the other hand, there is currently no known high-profile leak of PII or harmful use of confidential commercial information, despite mandatory breach reporting requirements.

Technological Solutions to Mitigate AI Privacy Risks

While policy and legal frameworks play a crucial role in addressing AI privacy concerns, technical solutions are equally essential. The Report highlights several key privacy-enhancing techniques that can be applied across the AI lifecycle to mitigate these risks, including:

1. Data Minimization and Privacy-Preserving Training

• Removing PII from training data: AI developers can pre-process datasets to eliminate PII before training. According to the Report, this is feasible but challenging, specifically highlighting the lack of clear standards. More on that later.
• Synthetic data generation: Creating artificial, non-identifiable training data can help train AI models without exposing real user information.
• Differential privacy: By adding mathematically calibrated noise to datasets, differential privacy techniques prevent models from memorizing individual data points while still allowing useful insights.

While such privacy techniques can protect individual privacy in AI models, they face real-world challenges. These methods often involve a "privacy-utility trade-off," where stronger privacy protections result in reduced model accuracy. This is particularly problematic for large, general-purpose AI models trained on text. Pre-training models on public data before adapting them for sensitive domains (like healthcare or finance) could provide strong privacy guarantees, but remains largely theoretical, the Report notes. Using synthetic data presents an alternative, but research shows it either maintains privacy risks when highly useful, or requires strong differential privacy—adding noise to limit learning about individual data while preserving overall patterns—leading back to the same utility trade-offs.

2. Privacy-Enhancing AI Deployment

• On-device processing: Running AI models locally on consumer devices (rather than the cloud) reduces exposure of sensitive data to third parties.
• Secure cloud deployment: End-to-end security solutions for data centers when handling sensitive data with larger models that can't run locally.
• Confidential computing: Hardware-based security solutions, such as secure enclaves and encrypted computation, ensure that AI operations occur in protected environments where even the cloud provider cannot access raw data.
• Cryptographic techniques: Methods like homomorphic encryption and zero-knowledge proofs allow AI systems to process data without revealing its contents, although these solutions remain computationally expensive.
• User-controlled data governance: Dashboards and interfaces for managing permissions, tracking data usage, and controlling PII could be helpful to give greater effect to individual’s ownership over their data, especially when paired with data provenance systems.

3. Strengthening Security Against AI-Enabled Attacks

• AI-driven cybersecurity tools: General-purpose AI can be leveraged to detect and neutralize threats, including phishing attacks, malware, and data breaches.
• Liability: Holding developers and distributors accountable for malicious use, like some legal frameworks, could disincentivize unsafe deployment.

Despite these advances, privacy-enhancing technologies and effective security measures are still evolving, and many AI privacy challenges remain unresolved.

This is where Private AI fits in.

Where Private AI Fits In: Practical, Scalable Privacy Solutions

At Private AI, we specialize in privacy-enhancing AI solutions that address some of the very challenges outlined in the Report. Our technology is designed to:

• Detect and redact PII before it enters AI training pipelines, helping developers build privacy-compliant AI models. Our high accuracy performance is capable of meeting strict de-identification standards such as those under HIPAA.
• Protect PII in AI interactions, ensuring that sensitive information is never exposed in AI-generated outputs.
• Enable secure AI deployment by applying automated privacy filters in real-time across structured and unstructured data.
• Support compliance with global privacy laws, including GDPR, Quebec’s Health Privacy Law, and Japan’s APPI.

To explore how Private AI can enhance privacy protection in AI systems, try our web demo or contact us to integrate privacy-preserving AI into your workflows.