Why is Privacy-Preserving NLP Important?

Jun 26, 2018
Share this post
Sharing to FacebookSharing to LinkedInSharing to XSharing to Email

I have had a number of people ask me why we should bother creating natural language processing (NLP) tools that preserve privacy. Apparently not everyone spends hours upon hours thinking about data breaches and data privacy infringements. Shocking!

The main argument for privacy-preserving NLP comes down to one fact: text and speech are our primary methods of communication. When interacting with web-based service providers and other users thereof, we often allow companies to store, use, and even sell the messages we have sent and received on their platforms.

So why do we even share this information with third parties in the first place? Frequently, the answer lies in our desire to get “free” services.

After the Cambridge Analytica/Facebook scandal, one can only hope that the general public has been made aware of how the personal data they have unknowingly been giving away in exchange for “free” services can be sold to third parties for any purpose at any time, without their knowledge or consent.

Sure, the thought of targeted ad campaigns siloing us into a political echo chamber is scary, but isn’t it even worse to think that the data being used to target us might not even be de-identified when sold to third-parties? Heck, even if the data are said to be de-identified, companies often attempt to do so poorly and without taking into account that cross-referencing them with other databases for the purpose of re-identification is far from unheard of.

Your name and number can be removed, but how about the locations you’ve visited, your preference for certain restaurants, and even your favourite tea flavours? Innocuous-seeming information becomes part of your digital fingerprint and can give your identity away, therefore we should be incredibly wary of what written and spoken data we allow onto the marketplace.

Why is Privacy-Preserving NLP Important?

We readily give away our data in exchange for convenience

Biometric authentication is an example of this. Simply put, it means using some unique part of your body/physiology as a password — think of using your fingerprint or face to unlock your computer. In the context of NLP, think of your voice being used by your bank to verify your identity during a customer service call (i.e., speaker authentication). There are two types of speaker authentication: text-dependent and text-independent. In the former, a spoken password is used to identify you in addition to comparing features extracted from your voice to that of previous recordings. In the latter, the system only uses features extracted from your voice while you speak to the automated or human representative without you having to say any particular keywords.

Securing user data within a speaker authentication system is by far the most researched area related to privacy-preserving NLP, though it certainly is not a solved problem

One common concern is the feature vectors associated with a user’s voice becoming compromised. Such data is therefore encrypted at rest. Another concern is that of replay attacks, where a user’s voice is recorded and played to the authentication system with the purpose of gaining access to particular files or locations. A band-aid solution to this is to use more than one mode of authentication (speech + fingerprint, speech + face recognition, speech + pin number, etc), such that using speech becomes a security enhancer rather than a system’s central component. Regardless of the system, a user’s privacy must be preserved from malicious outsiders in order for the system to remain reliable.

Speech authentication is one of the few examples where service providers have strong inherent incentive to put a lot of time and money into making sure our spoken data remains private from third parties. They are, however, still storing a biometric identifier associated with your personal information. An identifier that can be used to determine whether you are speaking in other audio or video recordings. Civil liberties groups argue against voice identifiers being stored without explicit consent (such as at Her Majesty’s Revenue and Customs in the UK) and without more information about how it is stored/shared/erased.

What about exchanging data privacy for physical security?

A concern that is often raised regarding data being encrypted without any backdoors is whether that makes it easier for nefarious communications (e.g., between terrorists and criminals) to go undetected. And, hey! If you’ve got nothing to hide you’ve got nothing to fear, right? That wonderfully nonsensical line is reserved for the fraction of us who have the good fortune of living in democratic societies where we won’t get thrown in jail and/or tortured for speaking our minds about the political party in power. Okay, let’s humour the people who think that line flies and that it’s a-okay for (some) governments and police to have quasi-unrestricted access to citizens’ private data via backdoors or requests for information from companies (predominantly without a warrant).

Data breaches

Suppose that for whatever reason there is some government or company that you sincerely trust with storing your written or spoken content. Splendid!

Well, I hate to break it to you, but over 2.6 billion records were breached in 2017 alone (76% due to accidental loss, 23% due to malicious outsiders). Here’s the crux: you might trust an organization’s intended use of your data without having a clue about how they protect it from being leaked.

Fine, then let’s not even bother sharing our data in the first place.

Why not just prevent people from accessing speech and text we produce altogether? Because we want to be provided with free or cheap services (Facebook, Twitter, Instagram, …). We also want training data for AI algorithms that are adaptable to our specific traits and preferences (speech recognition systems, personal assistants, search engines, …).

So how do we get what we want, give companies the data they need to profit/improve services AND maintain our privacy?

Research in privacy-preserving NLP is in its infancy, but it is likely to revolutionize the way companies and governments collect, store, process, and sell user data. With regulations like the GDPR coming into effect, public outcry over the Cambridge Analytica scandal, and the massive number of hacks that cost companies millions of dollars in reparations every year, the number of privacy-preserving data processing algorithms will snowball. I will be going into some detail on various (practical as well as still computationally intractable but promising) existing solutions in future blog posts, including privacy-preserving surveillance, federated learning, the application of differential privacy to neural networks in order to prevent reverse-engineering, homomorphically encrypted NLP, and so on.

Join us for more discussions about natural language processing on LinkedIn, Twitter, and Youtube

Acknowledgements. My deepest gratitude to Kelly Langlais, Dr. Siavash Kazemian, and Simon Emond for their invaluable feedback on this post.

Data Left Behind: AI Scribes’ Promises in Healthcare

Data Left Behind: Healthcare’s Untapped Goldmine

The Future of Health Data: How New Tech is Changing the Game

Why is linguistics essential when dealing with healthcare data?

Why Health Data Strategies Fail Before They Start

Private AI to Redefine Enterprise Data Privacy and Compliance with NVIDIA

EDPB’s Pseudonymization Guideline and the Challenge of Unstructured Data

HHS’ proposed HIPAA Amendment to Strengthen Cybersecurity in Healthcare and how Private AI can Support Compliance

Japan's Health Data Anonymization Act: Enabling Large-Scale Health Research

What the International AI Safety Report 2025 has to say about Privacy Risks from General Purpose AI

Private AI 4.0: Your Data’s Potential, Protected and Unlocked

How Private AI Facilitates GDPR Compliance for AI Models: Insights from the EDPB's Latest Opinion

Navigating the New Frontier of Data Privacy: Protecting Confidential Company Information in the Age of AI

Belgium’s Data Protection Authority on the Interplay of the EU AI Act and the GDPR

Enhancing Compliance with US Privacy Regulations for the Insurance Industry Using Private AI

Navigating Compliance with Quebec’s Act Respecting Health and Social Services Information Through Private AI’s De-identification Technology

Unlocking New Levels of Accuracy in Privacy-Preserving AI with Co-Reference Resolution

Strengthened Data Protection Enforcement on the Horizon in Japan

How Private AI Can Help to Comply with Thailand's PDPA

How Private AI Can Help Financial Institutions Comply with OSFI Guidelines

The American Privacy Rights Act – The Next Generation of Privacy Laws

How Private AI Can Help with Compliance under China’s Personal Information Protection Law (PIPL)

PII Redaction for Reviews Data: Ensuring Privacy Compliance when Using Review APIs

Independent Review Certifies Private AI’s PII Identification Model as Secure and Reliable

To Use or Not to Use AI: A Delicate Balance Between Productivity and Privacy

To Use or Not to Use AI: A Delicate Balance Between Productivity and Privacy

News from NIST: Dioptra, AI Risk Management Framework (AI RMF) Generative AI Profile, and How PII Identification and Redaction can Support Suggested Best Practices

Handling Personal Information by Financial Institutions in Japan – The Strict Requirements of the FSA Guidelines

日本における金融機関の個人情報の取り扱い - 金融庁ガイドラインの要件

Leveraging Private AI to Meet the EDPB’s AI Audit Checklist for GDPR-Compliant AI Systems

Who is Responsible for Protecting PII?

How Private AI can help the Public Sector to Comply with the Strengthening Cyber Security and Building Trust in the Public Sector Act, 2024

A Comparison of the Approaches to Generative AI in Japan and China

Updated OECD AI Principles to keep up with novel and increased risks from general purpose and generative AI

Is Consent Required for Processing Personal Data via LLMs?

The evolving landscape of data privacy legislation in healthcare in Germany

The CIO’s and CISO’s Guide for Proactive Reporting and DLP with Private AI and Elastic

The Evolving Landscape of Health Data Protection Laws in the United States

Comparing Privacy and Safety Concerns Around Llama 2, GPT4, and Gemini

How to Safely Redact PII from Segment Events using Destination Insert Functions and Private AI API

WHO’s AI Ethics and Governance Guidance for Large Multi-Modal Models operating in the Health Sector – Data Protection Considerations

How to Protect Confidential Corporate Information in the ChatGPT Era

Unlocking the Power of Retrieval Augmented Generation with Added Privacy: A Comprehensive Guide

Leveraging ChatGPT and other AI Tools for Legal Services

Leveraging ChatGPT and other AI tools for HR

Leveraging ChatGPT in the Banking Industry

Law 25 and Data Transfers Outside of Quebec

The Colorado and Connecticut Data Privacy Acts

Unlocking Compliance with the Japanese Data Privacy Act (APPI) using Private AI

Tokenization and Its Benefits for Data Protection

Private AI Launches Cloud API to Streamline Data Privacy

Processing of Special Categories of Data in Germany

End-to-end Privacy Management

Privacy Breach Reporting Requirements under Law25

Migrating Your Privacy Workflows from Amazon Comprehend to Private AI

A Comparison of the Approaches to Generative AI in the US and EU

Benefits of AI in Healthcare and Data Sources (Part 1)

Privacy Attacks against Data and AI Models (Part 3)

Risks of Noncompliance and Challenges around Privacy-Preserving Techniques (Part 2)

Enhancing Data Lake Security: A Guide to PII Scanning in S3 buckets

The Costs of a Data Breach in the Healthcare Sector and its Privacy Compliance Implications

Navigating GDPR Compliance in the Life Cycle of LLM-Based Solutions

What’s New in Version 3.8

How to Protect Your Business from Data Leaks: Lessons from Toyota and the Department of Home Affairs

New York's Acceptable Use of AI Policy: A Focus on Privacy Obligations

Safeguarding Personal Data in Sentiment Analysis: A Guide to PII Anonymization

Changes to South Korea’s Personal Information Protection Act to Take Effect on March 15, 2024

Australia’s Plan to Regulate High-Risk AI

How Private AI can help comply with the EU AI Act

Comment la Loi 25 Impacte l'Utilisation de ChatGPT et de l'IA en Général

Endgültiger Entwurf des Gesetzes über Künstliche Intelligenz – Datenschutzpflichten der KI-Modelle mit Allgemeinem Verwendungszweck

How Law25 Impacts the Use of ChatGPT and AI in General

Is Salesforce Law25 Compliant?

Creating De-Identified Embeddings

Exciting Updates in 3.7

EU AI Act Final Draft – Obligations of General-Purpose AI Systems relating to Data Privacy

FTC Privacy Enforcement Actions Against AI Companies

The CCPA, CPRA, and California's Evolving Data Protection Landscape

HIPAA Compliance – Expert Determination Aided by Private AI

Private AI Software As a Service Agreement

EU's Review of Canada's Data Protection Adequacy: Implications for Ongoing Privacy Reform

Acceptable Use Policy

ISO/IEC 42001: A New Standard for Ethical and Responsible AI Management

Reviewing OpenAI's 31st Jan 2024 Privacy and Business Terms Updates

Comparing OpenAI vs. Azure OpenAI Services

Quebec’s Draft Regulation Respecting the Anonymization of Personal Information

Version 3.6 Release: Enhanced Streaming, Auto Model Selection, and More in Our Data Privacy Platform

Brazil's LGPD: Anonymization, Pseudonymization, and Access Requests

LGPD do Brasil: Anonimização, Pseudonimização e Solicitações de Acesso à Informação

Canada’s Principles for Responsible, Trustworthy and Privacy-Protective Generative AI Technologies and How to Comply Using Private AI

Private AI Named One of The Most Innovative RegTech Companies by RegTech100

Data Integrity, Data Security, and the New NIST Cybersecurity Framework

Safeguarding Privacy with Commercial LLMs

Cybersecurity in the Public Sector: Protecting Vital Services

Privacy Impact Assessment (PIA) Requirements under Law25

Elevate Your Experience with Version 3.5

Fine-Tuning LLMs with a Focus on Privacy

GDPR in Germany: Challenges of German Data Privacy (Part 2)

Comply with US Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence using Private AI

How to Comply with EU AI Act using PrivateGPT