Why GenAI puts privacy officers on legal ground zero

As regulators and courts catch up with generative AI, Data Protection Officers (DPOs) are facing new challenges. 

Earlier this week, we asked Betania Allo (Cybersecurity Lawyer and Policy Strategist) what the recent landmark preservation order issued against OpenAI means for CISOs. 

Today, we’re talking with Allo about how the New York Times vs. OpenAI case impacts GDPR compliance within the European Union, as well as privacy policy updates worldwide – and ultimately, trust in AI. 

Can you briefly explain the case for those who haven’t been following it? 

“In December 2023, The New York Times filed a high-profile lawsuit against OpenAI and Microsoft, alleging that millions of its articles were used without authorisation to train large language models such as GPT-4. While the case centers on copyright infringement, the legal proceedings have introduced unexpected consequences for privacy and data governance. 

“To substantiate its claims, the NYT requested that OpenAI preserve all user-generated content – including chats users had previously deleted. A U.S. federal magistrate granted this request, and in May 2025 issued a sweeping preservation order. It required OpenAI to retain all inputs, completions, and associated metadata from ChatGPT users across its Free, Plus, Pro, Team, and API services, with the exception of accounts governed by Zero Data Retention (ZDR) agreements. 

“This is not a matter of surveillance, but of discovery – the legal mechanism that allows parties in litigation to obtain relevant evidence. Yet the ripple effects of this process extend well beyond the courtroom, raising critical challenges for cybersecurity, governance, and trust in AI systems.”  

How does the OpenAI legal hold challenge GDPR principles?

“This directly challenges GDPR principles such as data minimisation and the right to erasure. OpenAI has placed the preserved data in a segregated legal hold system accessible only to a small legal and security team, but this technical safeguard does not negate the broader conflict between jurisdictional privacy norms and extraterritorial legal mandates.” 

What does this mean for DPOs in the EU and beyond?

“For security and privacy leaders operating within the EU, this creates a significant compliance dilemma. Until the legal hold is lifted or narrowed, the only reliable way to shield sensitive inputs from litigation-based preservation is to use ChatGPT Enterprise, Edu, or API tiers with Zero Data Retention (ZDR) enabled.” 

What should privacy teams be doing now?

“DPOs should revisit vendor risk assessments, revise privacy notices to reflect potential cross-border retention, and prepare to address gaps in data subject rights enforcement—especially where AI output logs can now be classified as legal evidence.”

How should organisations adapt their governance policies?

“Update your privacy policies. If your organisation promises data minimisation or deletion, those statements must now clearly disclose legal exceptions such as court-ordered data holds. Failing to do so could result in misrepresentation or noncompliance under frameworks like GDPR.” 

What's your bottom line for leadership teams dealing with generative AI?

“Across the organisation, leadership must take proactive steps to address the emerging legal and regulatory risks introduced by AI data retention. This case is about much more than copyright. It illustrates what happens when highly complex, probabilistic technologies collide with legal systems designed around traceability, accountability, and evidence preservation.”

Data privacy is a moving target 

Generative AI has pushed regulators to look at data privacy from new angles. Allo added, "We once thought of generative AI as experimental or even ephemeral. Now, it must also be viewed as legally actionable."

For security and privacy leaders across organisations, this is a pivotal moment – and staying ahead of shifting legal requirements is critical. 

Connect with Betania Allo on LinkedIn. Join us at Black Hat MEA 2025 to stay ahead of the security curve.