economist.com · United Kingdom · measured 8 Oct 2026
As read on 8 Oct 2026, 2,664 bytes. 12 rule groups name AI crawlers; shown first.
User-agent: ChatGPT-User Allow: /pro$ Allow: /pro/ Allow: /syndication$ Allow: /syndication/ Disallow: / User-agent: Claude-Web Allow: /pro$ Allow: /pro/ Allow: /syndication$ Allow: /syndication/ Disallow: / User-agent: PerplexityBot Allow: /pro$ Allow: /pro/ Allow: /syndication$ Allow: /syndication/ Disallow: / User-agent: OAI-SearchBot Allow: /pro$ Allow: /pro/ Allow: /syndication$ Allow: /syndication/ Disallow: / User-agent: GPTBot Disallow: / User-agent: Google-Extended Allow: /pro$ Allow: /pro/ Allow: /syndication$ Allow: /syndication/ Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Applebot-Extended Disallow: / User-Agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: PetalBot Disallow: / Sitemap: https://www.economist.com/sitemap.xml Sitemap: https://www.economist.com/googlenews.xml User-agent: Claude-SearchBot Allow: /pro$ Allow: /pro/ Allow: /syndication$ Allow: /syndication/ Disallow: / User-agent: Claude-User Allow: /pro$ Allow: /pro/ Allow: /syndication$ Allow: /syndication/ Disallow: / User-agent: Perplexity-User Allow: /pro$ Allow: /pro/ Allow: /syndication$ Allow: /syndication/ Disallow: / User-agent: PiplBot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: MoodleBot Disallow: / User-agent: magpie-crawler Disallow: / User-agent: ia_archiver Disallow: / User-Agent: Mozilla/5.0 (compatible; parse.ly scraper/0.16; +http://parsely.com) Disallow: /checkout User-agent: * Disallow: /5605/ Disallow: /pubads.g.doubleclick.net/ Disallow: /assets/infographic/ Disallow: /search/ Disallow: /search?q= Disallow: /graphql Disallow: /api Disallow: /subscribe/getstarted/ Disallow: /checkout Disallow: /switch* Disallow: /audio-edition-podcast/*/index.xml
Changes recorded since June 2026.
No change recorded.