404 Media reported this week that OpenAI pays hundreds of contractors to read ChatGPT conversations through an internal program called Project Lily. Reviewers summarize what each user was trying to do and score the model’s answer on a seven-point scale. They see the complete exchange, including chats that contain medical histories, legal problems or a request to “keep this between us.”
Outside recruiters place the contractors, whose pay can exceed fifty dollars an hour. Their dashboard shows the user’s prompt alongside a summary of earlier interactions that may reveal a rough location or profession. A privacy filter is meant to remove identifying details before a conversation reaches a reviewer. OpenAI’s documentation acknowledges that the filter can miss uncommon identifiers or under-redact when context is thin. Training on consumer accounts is enabled by default, and turning it off affects only new conversations. Anthropic has confirmed that it uses human review for users who opt in. Google also tells users that people may review some saved chats.
Human review helped make chat models usable and remains part of how major labs improve them. People read transcripts and rank outputs. The failure lies in what users are told when they begin typing. The interface does not clearly warn them that another person may later read their words. Anyone who has worked in audit will recognize this as a basic failure of notice.
Privacy standards already cover this situation. NIST SP 800-53 and Canada’s ITSP.10.033 include the PT family, Personally Identifiable Information Processing and Transparency. PT-5 requires notice when information is collected. PT-4 requires consent mechanisms linked to the stated processing, and PT-3 requires organizations to state their purposes before processing begins. These are routine controls for government systems that handle personal information.
When 404 Media asked where users are told that people may read their chats, OpenAI did not answer. After publication, the company pointed to a help page. That page does not provide notice when the information is entered. Europe’s highest court has held that the duty to inform begins when data is collected, even if a later recipient cannot identify the person. Italy’s privacy regulator has also fined OpenAI fifteen million euros, partly for processing without an adequate legal basis, and ordered a public information campaign.
The interface encourages the misunderstanding. A chat window feels private because it shows no audience and no stranger can reply. Behind that quiet screen, conversations may be retained and reviewed. That gap helps explain why people tell a model things they would hesitate to say with another person at the table. They may use it as a therapist or confidant because the room appears empty.
Providers could close this gap without slowing model training in four practical ways.
Place a plain notice beside the input box stating that people may read conversations. Users would see the warning before they type. A help page or regulator-ordered advertising campaign reaches them too late.
Make human review opt-in for consumer accounts and retain the consent record like any other system artifact. Enterprise customers that negotiated contracts received access to zero-retention previews, while consumers had human review enabled by default.
Publish the redaction error rate. OpenAI’s documentation acknowledges that its privacy filter can fail, but the company does not report how often. It should sample the filtered stream, measure what identifying information passed through, and publish the result each quarter. A control cannot be assessed when its failure rate is unknown.
Manage reviewers as a defined access group. Give each reviewer only the dashboard functions required for the task. Hide the prior-interactions summary unless it is needed, and log every conversation opened. Those access records would make the first three safeguards auditable.
Readers of the Mobius Nexus Cycle will recognize this arrangement. Its users treat the Uplink as a point-to-point channel, but the books keep returning to the records it retains. The Fragments Operation begins with records nobody remembers agreeing to create. In the archive, a second system reads what the first one said according to a schedule and a purpose buried in a document no one opened. The 404 Media report shows that this asymmetry is already part of ordinary life. The person speaking may be the only one who believes the room is empty.
RECORD RETAINED
SOURCE INTEGRITY UNCONFIRMED