OpenAI's Filter Cannot Make Your Chats Anonymous

A Filter With a Published Miss Rate

OpenAI's own release notes for Privacy Filter, published April 22, 2026, state the exact rate at which the model fails to catch personal data, and that rate is not zero. The model labels text for eight categories of personal information, including names, addresses, account numbers and secrets such as passwords, then masks what it finds.

On the corrected version of the public PII-Masking-300k benchmark, the model still misses roughly 2 percent of the personal data spans it was tested against. OpenAI's own documentation calls Privacy Filter 'a small model with frontier personal data detection capability,' not a guarantee, and separately warns that fine-tuning on a new domain can lift accuracy from an F1 score of 54 percent to 96 percent, meaning the same base model performs very differently depending on what kind of text it sees.

Project Lily Puts Real Chats Behind That Filter

A September 14, 2026 investigation by 404 Media named the review program Project Lily and reported that OpenAI pays hundreds of outside contractors, often more than 50 dollars an hour through third-party staffing firms, to read real ChatGPT prompts and score the model's replies on a numeric scale.

Reviewers do not see a user's name, but 404 Media reports they can see a memory summary above the prompt that sometimes shows what the person has used the chatbot for before and where in the world they live. OpenAI told the outlet that a Privacy Filter model strips personal information first, and its own page for that model concedes it 'can make mistakes, miss uncommon identifiers, and under-redact when context is limited.' One contractor told 404 Media plainly that a typical user would not expect a stranger to be reading their chat at all.

Redaction Is Not the Same Word as Anonymous

OpenAI's own documentation states plainly that Privacy Filter 'is not an anonymization tool, a compliance certification, or a substitute for policy review in high-stakes settings,' and that distinction carries specific legal weight inside the EU that a product page does not have to spell out.

Under GDPR Article 4(5) and Recital 26, data that has been pseudonymized or masked but could still be linked back to a person remains personal data in full, not anonymous data outside the regulation's scope. A conversation with a name swapped for a placeholder is pseudonymized, not anonymous, the moment the original text or the account behind it still exists anywhere on OpenAI's systems. That keeps every EU user's reviewed conversation inside GDPR: a lawful basis is still required to let a contractor read it, the user's access and erasure rights still apply to it, and a breach involving it is still a reportable breach.

The Enterprise Question Nobody Has Answered

Neither OpenAI's privacy documentation nor 404 Media's report states where Project Lily's contractors are physically located, and that single fact decides whether an EU user's reviewed conversation ever left the European Economic Area, and under which transfer mechanism.

For a business running ChatGPT Enterprise or the API to keep customer conversations away from human reviewers, the consumer-tier disclosures in this report do not automatically extend to that traffic. OpenAI's enterprise terms describe data-use defaults separately from the consumer product, but the company has not stated publicly whether Enterprise or API conversations sit inside or outside the same review pool that Project Lily draws from. A business that assumed the two tiers were reviewed under identical rules should ask OpenAI for that answer in writing rather than assume it.

Read next: Meta's AI Built A Profile Of A Child Nobody Asked For  |  France Just Priced Missing MFA at EUR 500,000