Adding Privacy to LangChain with Private AI

Oct 19, 2023
Share this post
Sharing to FacebookSharing to LinkedInSharing to XSharing to Email

LangChain is a powerful tool that allows you to setup a conversation with your favourite LLMs with ease. If you’re worried about what that LLM is doing with your information, Private AI makes it easy to integrate a privacy layer into your conversation, keeping your sensitive information safe.

Getting Started

If you don’t already have access to the Private AI de-identification service, see our guide on getting setup with AWS or request a free api key.

We’ll be coding this solution with Python. If you don’t have a python environment setup, see the official Python for Beginners guide to get setup quick and easy.

Once your environment is setup, LangChain can be installed with pip:



pip install LangChain

Now the coding can begin. First, a LangChain agent should be added to manage the conversation. This includes creating the object related to the LLM we want to chat with and a memory buffer to store the conversation’s history.



import os<br>from langchain.agents import AgentType, initialize_agent<br>from langchain.chat_models import ChatOpenAI<br>from langchain.memory import ConversationBufferMemory<br>OPENAI_API_KEY = os.environ['OPENAI_API_KEY']<br>def main():<br> # create our llm object<br> llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY, temperature=0)<br> # add a memory buffer<br> history = ConversationBufferMemory(memory_key="chat_history", return_messages=True)<br> # create a new agent to chat with<br> agent_chain = initialize_agent([], llm, agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION, memory=history, verbose=True)<br> while True: <br> message = input("Add message to send: ")<br> if message == "quit":<br> break<br> agent_chain.run(input=message)<br>if __name__ == "__main__":<br> main()



#Sample <br>Add message to send: hey!<br>> Entering new AgentExecutor chain...<br>Thought: Do I need to use a tool? No<br>AI: Hello! How can I assist you today?<br>> Finished chain.

Now the base conversation flow is created, it’s time to add a privacy layer!

Adding De-identification

NOTE: Make sure to grab Private AI’s client for seamless integration:



pip install privateai_client<br>

Let’s add a Private AI client to the flow and make sure it’s accessible for future de-identification requests.



...<br>from privateai_client import PAIClient<br>from privateai_client import request_objects as rq<br>def main():<br> # Initialize the private-ai client<br> pai_client = PAIClient(url="http://localhost:8080")<br> # Test the client is responsive<br> pai_client.ping()<br>....

Output



True

Once a connection to the client is established, text prompts can be de-identified before they are sent to the LLM. The conversation needs an update to add this new feature.



def chat_privately(message: str, agent_chain: AgentExecutor, pai_client: PAIClient):<br> deid_request = rq.process_text_obj(text=[message])<br> deid_response = pai_client.process_text(deid_request)<br> deid_message = deid_response.processed_text[0]<br> # Send the message<br> agent_chain.run(input=deid_message)<br>def main():<br>...<br> while True:<br> message = input("Add message to send: ")<br> if message == "quit":<br> break<br> chat_privately(message, agent_chain, pai_client, history)

Now the LLM is working with anonymized data!

Output



Add message to send: Hey! My name is Adam and I work at Private AI.<br>> Entering new AgentExecutor chain...<br>Thought: Do I need to use a tool? No<br>AI: Hi [NAME_GIVEN_1], it's nice to meet you! What kind of work do you do at [ORGANIZATION_1]?<br>> Finished chain.

While this conversation flow works, it isn’t really great for human interpretation. Keeping tabs on who NAME_1 is as the conversation progresses can become tedious.

To solve this, re-identification needs to be added to the conversation flow. By re-identifying the conversation history, the whole conversation (including the new prompt) can be run through de-identification as a single request.

Re-identification needs some extra information to work properly:

  • - The conversation history
  • - A collection of the entities de-identified so the service can replace them to their original state

A couple extra functions need to be added to handle this.



def get_conversation_history(history: ConversationBufferMemory):<br> return [row.content for row in history.chat_memory.messages]<br>def get_entities(processed_entities: List[Dict[str, str]]):<br> entities = {}<br> for entity in processed_entities:<br> for row in entity:<br> entities[row["processed_text"]] = row["text"]<br> return entities<br>

Now re-identification can be added! Let’s add a function to handle the request. The de-identification request should move to its own function as well, for readability, and the chat function needs updating so the entity list and conversation history can be managed.



def deidentify_conversation(pai_client: PAIClient, message: str, convo_history: List[str]=[]):<br> # Add the current message to the conversation history<br> full_conversation = convo_history + [message]<br> # Create a process_text_request object<br> req = rq.process_text_obj(text=full_conversation, link_batch=True)<br> # Return the response<br> return pai_client.process_text(req)<br>def reidentify_conversation(pai_client: PAIClient, convo_history: List[str], entities: Dict[str, str]):<br> if not convo_history:<br> return []<br> # Create a reidentify_text object<br> formatted_entities = [rq.entity(key, value) for key, value in entities.items()]<br> req = rq.reidentify_text_obj(convo_history, formatted_entities)<br> return pai_client.reidentify_text(req).body<br>def chat_privately(message: str, agent_chain: AgentExecutor, pai_client: PAIClient, history: ConversationBufferMemory, entities:Dict[str,str]):<br> convo_history = get_conversation_history(deid_history)<br> original_convo = reidentify_conversation(pai_client, convo_history, entities)<br> response = deidentify_conversation(pai_client, message, original_convo)<br> # Grab only the latest message from the deidentified text<br> deid_message = response.processed_text[-1]<br> agent_chain.run(input=deid_message)<br> return get_entities(response.entities)

Putting It All Together

Now we have a conversation flow that will de-identify any sensitive information, allowing us to safely pass the conversation history to the LLM! Let’s tie it all together with a nicer output so we can see what the LLM sees, as well as our re-identified version of the conversation:



import os<br>from typing import Dict, List<br>from langchain.agents import AgentExecutor, AgentType, initialize_agent<br>from langchain.chat_models import ChatOpenAI<br>from langchain.memory import ConversationBufferMemory<br>from privateai_client import PAIClient<br>from privateai_client import request_objects as rq<br>OPENAI_API_KEY = os.environ['OPENAI_API_KEY']<br>def get_conversation_history(history: ConversationBufferMemory):<br> return [row.content for row in history.chat_memory.messages]<br>def get_entities(processed_entities: List[Dict[str, str]]):<br> entities = {}<br> for entity in processed_entities:<br> for row in entity:<br> entities[row["processed_text"]] = row["text"]<br> return entities<br>def deidentify_conversation(pai_client: PAIClient, message: str, convo_history: List[str]=[]):<br> # Add the current message to the conversation history<br> full_conversation = convo_history + [message]<br> # Create a process_text_request object<br> req = rq.process_text_obj(text=full_conversation, link_batch=True)<br> # Return the response<br> return pai_client.process_text(req)<br>def reidentify_conversation(pai_client: PAIClient, convo_history: List[str], entities: Dict[str, str]):<br> if not convo_history:<br> return []<br> # Create a reidentify_text object<br> formatted_entities = [rq.entity(key, value) for key, value in entities.items()]<br> req = rq.reidentify_text_obj(convo_history, formatted_entities)<br> return pai_client.reidentify_text(req).body<br>def chat_privately(message: str, agent_chain: AgentExecutor, pai_client: PAIClient, history: ConversationBufferMemory, entities:Dict[str,str]):<br> convo_history = get_conversation_history(deid_history)<br> original_convo = reidentify_conversation(pai_client, convo_history, entities)<br> response = deidentify_conversation(pai_client, message, original_convo)<br> # Grab only the latest message from the deidentified text<br> deid_message = response.processed_text[-1]<br> agent_chain.run(input=deid_message)<br> return get_entities(response.entities)<br>def main():<br> # Initialize the private-ai client<br> pai_client = PAIClient(url="http://localhost:8080")<br> # Test the client is responsive<br> pai_client.ping()<br> # Create a memory buffer for the conversation history<br> deid_history = ConversationBufferMemory(<br> memory_key="chat_history", return_messages=True<br> )<br> # Add the llm to converse with<br> llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY, temperature=0)<br> # Setup a conversation chain<br> agent_chain = initialize_agent([], llm, agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION, memory=deid_history)<br> entity_list = {}<br> while True:<br> # Now start chating away with your llm!<br> message = input("Add message to send: ")<br> if message == "quit":<br> break<br> entity_list = chat_privately(<br> message, agent_chain, pai_client, deid_history, entity_list<br> )<br> # Get the llm's response<br> history = get_conversation_history(deid_history)<br> print("--Actual conversation --")<br> print(f"Human: {history[-2]}")<br> print(f"AI: {history[-1]}n")<br> # Print it in a readable format<br> print("--Readable conversation --")<br> print(f"Human: {message}")<br> print(<br> f"AI: {reidentify_conversation(pai_client, [history[-1]], entity_list)[0]}n"<br> )<br>if __name__ == "__main__":<br> main()

Output:



Add message to send: Hey! My name is Adam and I work at Private AI.<br>--Actual conversation --<br>Human: Hey! My name is [NAME_GIVEN_1] and I work at [ORGANIZATION_1].<br>AI: Hi [NAME_GIVEN_1], it's nice to meet you! What kind of work do you do at [ORGANIZATION_1]?<br>--Readable conversation --<br>Human: Hey! My name is Adam and I work at Private AI.<br>AI: Hi Adam, it's nice to meet you! What kind of work do you do at Private AI?<br>Add message to send: Can you tell me where I work?<br>--Actual conversation --<br>Human: Can you tell me where I work?<br>AI: Of course! You work at [ORGANIZATION_1].<br>--Readable conversation --<br>Human: Can you tell me where I work?<br>AI: Of course! You work at Private AI.<br>Add message to send: And my name?<br>--Actual conversation --<br>Human: And my name?<br>AI: Your name is [NAME_GIVEN_1].<br>--Readable conversation --<br>Human: And my name?<br>AI: Your name is Adam.

And that’s it! We can rest easy that our sensitive information is hidden, thanks to Private AI.

Sign up for our Community API

The “get to know us” plan. Our full product, but limited to 75 API calls per day and hosted by us.

Get Started Today

Data Left Behind: AI Scribes’ Promises in Healthcare

Data Left Behind: Healthcare’s Untapped Goldmine

The Future of Health Data: How New Tech is Changing the Game

Why is linguistics essential when dealing with healthcare data?

Why Health Data Strategies Fail Before They Start

Private AI to Redefine Enterprise Data Privacy and Compliance with NVIDIA

EDPB’s Pseudonymization Guideline and the Challenge of Unstructured Data

HHS’ proposed HIPAA Amendment to Strengthen Cybersecurity in Healthcare and how Private AI can Support Compliance

Japan's Health Data Anonymization Act: Enabling Large-Scale Health Research

What the International AI Safety Report 2025 has to say about Privacy Risks from General Purpose AI

Private AI 4.0: Your Data’s Potential, Protected and Unlocked

How Private AI Facilitates GDPR Compliance for AI Models: Insights from the EDPB's Latest Opinion

Navigating the New Frontier of Data Privacy: Protecting Confidential Company Information in the Age of AI

Belgium’s Data Protection Authority on the Interplay of the EU AI Act and the GDPR

Enhancing Compliance with US Privacy Regulations for the Insurance Industry Using Private AI

Navigating Compliance with Quebec’s Act Respecting Health and Social Services Information Through Private AI’s De-identification Technology

Unlocking New Levels of Accuracy in Privacy-Preserving AI with Co-Reference Resolution

Strengthened Data Protection Enforcement on the Horizon in Japan

How Private AI Can Help to Comply with Thailand's PDPA

How Private AI Can Help Financial Institutions Comply with OSFI Guidelines

The American Privacy Rights Act – The Next Generation of Privacy Laws

How Private AI Can Help with Compliance under China’s Personal Information Protection Law (PIPL)

PII Redaction for Reviews Data: Ensuring Privacy Compliance when Using Review APIs

Independent Review Certifies Private AI’s PII Identification Model as Secure and Reliable

To Use or Not to Use AI: A Delicate Balance Between Productivity and Privacy

To Use or Not to Use AI: A Delicate Balance Between Productivity and Privacy

News from NIST: Dioptra, AI Risk Management Framework (AI RMF) Generative AI Profile, and How PII Identification and Redaction can Support Suggested Best Practices

Handling Personal Information by Financial Institutions in Japan – The Strict Requirements of the FSA Guidelines

日本における金融機関の個人情報の取り扱い - 金融庁ガイドラインの要件

Leveraging Private AI to Meet the EDPB’s AI Audit Checklist for GDPR-Compliant AI Systems

Who is Responsible for Protecting PII?

How Private AI can help the Public Sector to Comply with the Strengthening Cyber Security and Building Trust in the Public Sector Act, 2024

A Comparison of the Approaches to Generative AI in Japan and China

Updated OECD AI Principles to keep up with novel and increased risks from general purpose and generative AI

Is Consent Required for Processing Personal Data via LLMs?

The evolving landscape of data privacy legislation in healthcare in Germany

The CIO’s and CISO’s Guide for Proactive Reporting and DLP with Private AI and Elastic

The Evolving Landscape of Health Data Protection Laws in the United States

Comparing Privacy and Safety Concerns Around Llama 2, GPT4, and Gemini

How to Safely Redact PII from Segment Events using Destination Insert Functions and Private AI API

WHO’s AI Ethics and Governance Guidance for Large Multi-Modal Models operating in the Health Sector – Data Protection Considerations

How to Protect Confidential Corporate Information in the ChatGPT Era

Unlocking the Power of Retrieval Augmented Generation with Added Privacy: A Comprehensive Guide

Leveraging ChatGPT and other AI Tools for Legal Services

Leveraging ChatGPT and other AI tools for HR

Leveraging ChatGPT in the Banking Industry

Law 25 and Data Transfers Outside of Quebec

The Colorado and Connecticut Data Privacy Acts

Unlocking Compliance with the Japanese Data Privacy Act (APPI) using Private AI

Tokenization and Its Benefits for Data Protection

Private AI Launches Cloud API to Streamline Data Privacy

Processing of Special Categories of Data in Germany

End-to-end Privacy Management

Privacy Breach Reporting Requirements under Law25

Migrating Your Privacy Workflows from Amazon Comprehend to Private AI

A Comparison of the Approaches to Generative AI in the US and EU

Benefits of AI in Healthcare and Data Sources (Part 1)

Privacy Attacks against Data and AI Models (Part 3)

Risks of Noncompliance and Challenges around Privacy-Preserving Techniques (Part 2)

Enhancing Data Lake Security: A Guide to PII Scanning in S3 buckets

The Costs of a Data Breach in the Healthcare Sector and its Privacy Compliance Implications

Navigating GDPR Compliance in the Life Cycle of LLM-Based Solutions

What’s New in Version 3.8

How to Protect Your Business from Data Leaks: Lessons from Toyota and the Department of Home Affairs

New York's Acceptable Use of AI Policy: A Focus on Privacy Obligations

Safeguarding Personal Data in Sentiment Analysis: A Guide to PII Anonymization

Changes to South Korea’s Personal Information Protection Act to Take Effect on March 15, 2024

Australia’s Plan to Regulate High-Risk AI

How Private AI can help comply with the EU AI Act

Comment la Loi 25 Impacte l'Utilisation de ChatGPT et de l'IA en Général

Endgültiger Entwurf des Gesetzes über Künstliche Intelligenz – Datenschutzpflichten der KI-Modelle mit Allgemeinem Verwendungszweck

How Law25 Impacts the Use of ChatGPT and AI in General

Is Salesforce Law25 Compliant?

Creating De-Identified Embeddings

Exciting Updates in 3.7

EU AI Act Final Draft – Obligations of General-Purpose AI Systems relating to Data Privacy

FTC Privacy Enforcement Actions Against AI Companies

The CCPA, CPRA, and California's Evolving Data Protection Landscape

HIPAA Compliance – Expert Determination Aided by Private AI

Private AI Software As a Service Agreement

EU's Review of Canada's Data Protection Adequacy: Implications for Ongoing Privacy Reform

Acceptable Use Policy

ISO/IEC 42001: A New Standard for Ethical and Responsible AI Management

Reviewing OpenAI's 31st Jan 2024 Privacy and Business Terms Updates

Comparing OpenAI vs. Azure OpenAI Services

Quebec’s Draft Regulation Respecting the Anonymization of Personal Information

Version 3.6 Release: Enhanced Streaming, Auto Model Selection, and More in Our Data Privacy Platform

Brazil's LGPD: Anonymization, Pseudonymization, and Access Requests

LGPD do Brasil: Anonimização, Pseudonimização e Solicitações de Acesso à Informação

Canada’s Principles for Responsible, Trustworthy and Privacy-Protective Generative AI Technologies and How to Comply Using Private AI

Private AI Named One of The Most Innovative RegTech Companies by RegTech100

Data Integrity, Data Security, and the New NIST Cybersecurity Framework

Safeguarding Privacy with Commercial LLMs

Cybersecurity in the Public Sector: Protecting Vital Services

Privacy Impact Assessment (PIA) Requirements under Law25

Elevate Your Experience with Version 3.5

Fine-Tuning LLMs with a Focus on Privacy

GDPR in Germany: Challenges of German Data Privacy (Part 2)

Comply with US Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence using Private AI

How to Comply with EU AI Act using PrivateGPT