Cybersecurity

Blocking the Crawler Is What Published Your Chats

Shared Claude conversations reached Google search on 27 July. The robots.txt rule meant to keep crawlers out is what stopped the noindex fix from ever being read.

CybersecurityBy Servola Tech Desk2026-07-284 min read

AI-assisted, edited by humans. | Editor-in-Chief: Leon Soliman | Editorial standards

Blocking the Crawler Is What Published Your Chats

A search operator returned other people's work

On the weekend of 25 July a user typed a single search operator into Google, site:claude.ai/share, and got back hundreds of other people's conversations with Claude. The results were not previews or titles. They were full transcripts, sitting on the open web, readable by anyone who ran the same query. Reporting through 27 July described legal strategy written by practising lawyers, engineering and troubleshooting sessions, resumes and financial detail indexed alongside ordinary questions.

By 27 July the Google results had largely gone, after Anthropic changed the configuration on its share pages. Bing was still returning some of them. That gap between two engines is the first useful signal here: the exposure was never governed by the vendor's intention, it was governed by what each crawler had already been permitted to do, and permission had been granted long before anyone looked.

The disallow is what stopped the fix

The mechanism matters more than the incident. Anthropic's robots.txt already disallowed crawling of its share URLs. That sounds like the stronger protection and it is the weaker one. Google's own Search Central documentation is explicit about the interaction: for the noindex rule to be effective, the page must not be blocked by a robots.txt file, because a crawler refused access never reads the rule telling it to stay out, and the address can still appear in results when other pages link to it.

So the disallow did not merely fail to help. It removed the only channel through which the correct instruction could arrive. A share link posted once in a public forum, a support thread or an exported chat log is enough to put the address in Google's hands, and from that moment a blocked page is an indexable page. The safe configuration is the counterintuitive one. Let the crawler reach the page, so that it can be told, in a header or a meta tag, not to list what it finds.

Two vendors, one archive, the same wiring

A public GitHub repository named Shared-Claude-Chats had already collected 453 Claude conversations and 519 Grok chats pulled from exposed links. The second number is the one to sit with. Two unrelated vendors, two separately built share features, one identical failure. That is not a defect in a single company's release. It is a design pattern in how conversational products hand out addresses.

The pattern reproduces easily. A team adds share-by-link, treats the resulting URL as private because it is long and unguessable, and never writes an indexing rule because nobody expects the address to travel. The address always travels. If your own product, your document portal, your reporting dashboard or your client delivery system mints links this way, you have the same wiring and very likely the same assumption sitting underneath it.

Audit every surface that mints a link

Deindexing is not remediation. Removing a result stops future discovery through that one engine. It recalls nothing that was already fetched, cached, archived or reposted, and the GitHub collection is the proof that fetching happened. For a European operator the legal reading follows directly: under the GDPR the platform is the controller for what its own service publishes to the open web, and a user clicking share is authorising disclosure to a chosen recipient, not publication and indexing. Those are two different processing operations, and agreeing to the first does not cover the second.

The work this week is not to wait for a vendor statement. List every surface in your business that turns content into a link a stranger could hold: shared chats your staff create in AI tools, your own product's share feature, exports from your ticketing system, generated client reports. For each one, confirm the page is reachable by crawlers and carries an explicit noindex, rather than being hidden behind a disallow that quietly guarantees the opposite. Then run the site: operator against your own domains and read what comes back.

Frequently asked questions

We already block crawlers in robots.txt. Is that enough?

No, and it is the specific configuration that failed here. Google's documentation states that a noindex rule only works if the page is not blocked in robots.txt, because a crawler that cannot fetch the page cannot read the rule. A disallowed address that someone links to publicly can still be listed. Allow the crawl and serve noindex, either as an X-Robots-Tag header or a meta tag.

The links were removed from Google. Is the exposure finished?

No. Removal ends discovery through that engine and nothing more. Bing was still surfacing some of the same pages, and a public archive of 453 Claude and 519 Grok conversations had already been built from the exposed links. Treat any content that reached an index as disclosed, and run the incident assessment on that basis rather than on the current state of the search results.

The instinct to slam a door is the wrong instinct when the lock only works from the inside. Blocking the crawler felt like protection and functioned as publication, and it will keep functioning that way in every product that copies the pattern. Go and look at what your own share links do before someone else runs the query for you.

AnthropicClaudexAIData protectionGDPRSearch indexingShadow AI

Microsoft Trained Its Sellers to Call Claude Slower

Microsoft trained its sellers to call Claude slower and less accurate inside Office. A separate report says it is swapping the model out to cut cost. Read the ranking accordingly.

4 min read

AI Policy

No AI Lab Scored Better Than a C+ on Safety

The Future of Life Institute graded nine frontier AI companies on safety across six domains. The best score was Anthropic's C+, Europe's Mistral came last, and several firms have walked back their own risk pledges. Here is what an independent scorecard changes for anyone choosing an AI vendor.

3 min read

Law & Regulation

The EA Buyout Now Hinges on Brussels

The largest take-private in history has cleared EA shareholders and US antitrust. The last real gate is the EU - and not classic merger review, but the Foreign Subsidies Regulation, facing its first blockbuster test.