On March 24, 2023, OpenAI reported a glitch that revealed 1.2% of ChatGPT Plus users’ personal data, including credit card details, potentially to other users. Later in July, South Korea’s Personal Information Protection Committee (PIPC) fined them for exposing Korean citizens’ personal data.
Bill Gates has hailed the emergence of GenAI as the start of the “Age of AI,” but apprehensions and risks with data security remain a “clear and present” danger, as exemplified by these incidents in his blog.
Balancing act-safeguarding privacy with GenAI applications
At one end of the AI spectrum lie far-reaching, life-changing applications that improve human productivity, health, society, governance, and industry, improving our world. Conversely, data privacy concerns mar the openness to innovate with it. Critical questions on ethics, bias, and personal data vulnerabilities must be addressed, along with exploring its full potential.
With the ramifications of this exemplary technology still fuzzy, it becomes a collective responsibility of developers, businesses, and policymakers to ensure that the benefits of GenAI are harnessed without compromising the fundamental right to privacy.
Personal data used with Applications of GenAI are already protected by privacy laws like the General Data Protection Regulation (GDPR) in Europe and the US’s California Privacy Rights Act (CPRA). With the enactment of the Digital Data Protection Bill, India has also created its own privacy protection framework. Further, the UK and the EU have taken strides in formulating AI-specific legislation like the Artificial Intelligence Act (EU AI Act).
The general consensus has been to take a “pro-innovation approach to AI regulation,” as aptly exemplified in the title of the UK’s policy paper. Regulators are being careful, encouraging progress and advancement while ensuring new and existing laws protect citizen’s rights.
Data exposure concerns
Let us look at some of the vulnerabilities that are particularly concerning for personal data privacy:
Data breaches: This type of hack involves unauthorized access to sensitive and personal information, which may be perpetuated by external or internal actors who exploit system vulnerabilities. The consequences can include identity theft, financial losses, and reputational damage.
Inadequate anonymization: When personally identifiable information (PII) is not satisfactorily anonymized, it can be reverse-engineered, leading to the unintended disclosure of personal information. Inadequate Anonymization undermines the trust individuals place in systems to safeguard their privacy.
Unauthorized sharing of personal data: This results in a violation of individuals’ privacy rights. The law requires explicit consent for collecting, sharing, and decimating all personal data.
Vulnerabilities with GenAI
The way LLMs are designed and used might lead to inadvertent privacy issues. Training data often carries PII records, which may stay memorized in the system, contingent on how the model is configured. Options to delete this data may not be available, causing sensitive data to be resurfaced and exposed. Similarly, prompts may carry private data with employment or business contract text, which stays within the stored chats.
GenAI systems are susceptible to hacking attacks with the malicious intent of stealing the confidential information contained within their models. Prompt engineering can make the system spill out PII along with content. LLMs are also exposed to exfiltration attacks where training data is accessed, changed, or removed.
Mitigation strategies
Some of the approaches that Enterprises are embracing to address data privacy while using AI are:
Running LLM models on-premise: Running LLMs on-premise allows enterprises to avoid potential risks associated with data privacy by ensuring that sensitive information remains within their controlled environment. Organizations can implement and customize security measures, maintaining complete control over the infrastructure and data.
Using private instances of LLMs on cloud providers: Hosting private instances of LLMs is another mitigation strategy. This approach combines the scalability and flexibility of cloud computing with the security benefits of private cases.
Anonymization and de-identification of data: Strong anonymization and de-identification techniques strip sensitive PII information from datasets used to train or fine-tune LLMs.
Storing sensitive data in vaults: By isolating sensitive data in secure vaults, organizations add a layer of protection against unauthorized access, enhancing overall data privacy.
Privacy firewalls for LLMs: Implementing privacy firewalls around LLMs is a proactive measure to control and monitor data access. These firewalls regulate incoming and outgoing data, ensuring that sensitive information is only accessed or shared according to predefined rules and policies.
Best practices for data privacy protection
A good place to start robust data protection practices is by prioritizing regulatory compliance and adhering to relevant data protection laws and guidelines.
Obtaining explicit user consent before collecting or processing personal information is necessary. Practicing data minimization by only collecting and storing what is essential information for specific purposes is another tactical step. Publishing privacy notices informing users about data collection and processing practices is integral to good data governance standards.
Regular assessments of AI Applications to identify and mitigate potential privacy risks are essential with continuously evolving models and algorithms. These audits should include a review of ethics and detect any bias, keeping them in line with evolving privacy standards.
Securing data storage and data in motion through encryption and other protective measures are equally vital to prevent unauthorized access. Stringent access controls that restrict data access only to authorized personnel should be implemented.
The transparency of AI algorithms provides a view into decision-making processes, which goes a long way in increasing users’ trust. By incorporating the best practices outlined here, organizations can establish a comprehensive framework for safeguarding data privacy protection while leveraging the technological prowess of Generative AI.
Saurabh Dutta is the Senior Solution Architect at Gathr Data Inc.
Image Source: Freepik
