CrawlerPulse
CrawlerPulse / North America / Mexico / estadiodeportes.mx

estadiodeportes.mx

estadiodeportes.mx · Mexico · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 2,218 bytes. 3 rule groups name AI crawlers; shown first.

User-agent: Applebot
User-agent: baiduspider
User-agent: Bingbot
User-agent: Discordbot
User-agent: facebookexternalhit
User-agent: FacebookBot
User-agent: Googlebot
User-agent: Googlebot-News
User-agent: Googlebot-Mobile
User-agent: Mediapartners-Google
User-agent: Adsbot-Google
User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google-Mobile-Apps
User-agent: Googlebot-Image
User-agent: Googlebot-Video
User-agent: Storebot-Google
User-agent: APIs-Google
User-agent: LinkedInBot
User-agent: msnbot
User-agent: Naverbot
User-agent: Pinterestbot
User-agent: PetalBot
User-agent: seznambot
User-agent: Slurp
User-agent: teoma
User-agent: TelegramBot
User-agent: Twitterbot
User-agent: Yeti
User-agent: Exabot
User-agent: CriteoBot
User-agent: DuckDuckBot
User-agent: AmazonAdBot
User-agent: AdIdxBot
User-agent: BingPreview
User-agent: MicrosoftPreview
User-agent: GoogleOther
User-agent: GoogleOther-Image
User-agent: GoogleOther-Video
User-agent: Google-InspectionTool
User-agent: YouBot
User-agent: Bytespider
User-agent: Timpibot
User-agent: Seekr
User-agent: BraveBot
Disallow: /_req/
Disallow: /*contenidos/
Disallow: /vipcontent/
Disallow: /buscar.html
Disallow: /22422907837/
Allow: /

User-agent: OAI-Searchbot
User-agent: ChatGPT-User
User-agent: ChatGPT-User/2.0
User-agent: PerplexityBot
User-agent: Claude-SearchBot
User-agent: Meta-WebIndexer
User-agent: Meta-ExternalAds
User-agent: cohere-ai
User-agent: iaskSpider
User-agent: BraveBot
User-agent: Diffbot
User-agent: wpbot
Disallow: /_req/
Disallow: /*contenidos/
Disallow: /vipcontent/
Disallow: /buscar.html
Disallow: /22422907837/
Allow: /

User-agent: anthropic-ai
User-agent: Applebot-Extended
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: DuckAssistBot
User-agent: Google-Extended
User-agent: Meta-ExternalAgent
User-agent: CCBot
User-agent: DataForSeoBot
User-agent: ImagesiftBot
User-agent: Omgilibot
User-agent: peer39_crawler
User-agent: Amazonbot
Disallow: /_req/
Disallow: /*contenidos/
Disallow: /vipcontent/
Disallow: /buscar.html
Disallow: /22422907837/
Allow: /futbol/
Allow: /deportes/

User-agent: *
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Mexico
11.1% block at least one AI crawler
Behind Cloudflare
No