CrawlerPulse
CrawlerPulse / North America / Mexico / bloomberglinea.com

Bloomberg

bloomberglinea.com · Mexico · Bloomberg Línea Mexico · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,036 bytes. 1 rule group name AI crawlers; shown first.

User-agent: PetalBot
Allow:

User-agent: *
User-agent: Googlebot
User-agent: Googlebot-News
User-agent: Googlebot-Image
User-agent: Googlebot-Video
User-agent: Googlebot-Mobile
User-agent: Twitterbot
Allow: /
Disallow: /test/
Disallow: /zzz/*
Disallow: /composer-preview/*
Disallow: /test
Disallow: /anuncios/
Disallow: /anuncios
Disallow: /amp/logout/
Disallow: /account-bl/
Disallow: /queryly-advanced-search/?query=*
Disallow: /pf/api/v3/*
Disallow: /tags/*bz
Disallow: /tags/*bb
Disallow: /2023/04/28/ola-socialista-en-america-latina-desata-gran-fuga-de-capitales/
Disallow: /latinoamerica/ecuador/jan-topic-lo-que-debe-saber-del-posible-candidato-presidencial-2023/
Sitemap: https://www.bloomberglinea.com/arc/outboundfeeds/google-news-feed-latam/?outputType=xml
Sitemap: https://www.bloomberglinea.com/arc/outboundfeeds/google-news-feed-english/?outputType=xml
Sitemap: https://www.bloomberglinea.com/arc/outboundfeeds/sitemap-index/
Sitemap: https://www.bloomberglinea.com/arc/outboundfeeds/full-sitemap-index/?outputType=xml

History

Changes recorded since June 2026.

No change recorded.

In context

Mexico
11.1% block at least one AI crawler
Behind Cloudflare
No