CrawlerPulse
CrawlerPulse / North America / Mexico / tribuna.com.mx

TRIBUNA DEL YAQUI

tribuna.com.mx · Mexico · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Blocked
Common Crawl
Allowed
ByteDance
Blocked
Apple
Allowed
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 2,917 bytes. 10 rule groups name AI crawlers; shown first.

User-agent: PetalBot
Disallow: /

User-agent: ClaudeBot
Allow: /

User-agent: GPTBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: CCBot
Allow: /

User-agent: Bytespider
Disallow: /

User-agent: Amazonbot
Disallow: /

User-agent: meta-externalagent
Disallow: /
Sitemap: https://tribuna.com.mx/sitemap.xml
Sitemap: https://tribuna.com.mx/sitemap-news.xml

User-agent: *
Allow: /
Allow: /wp-content/uploads/
Allow: /wp-content/themes/*/assets/
Allow: /wp-content/*.js
Allow: /wp-content/*.css
Allow: /wp-content/*.svg
Allow: /wp-content/*.jpg
Allow: /wp-content/*.jpeg
Allow: /wp-content/*.png
Allow: /wp-content/*.gif
Allow: /wp-content/*.webp
Allow: /wp-includes/*.js
Allow: /wp-includes/*.css
Allow: /ads.txt
Allow: /feed/$
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /xmlrpc.php
Disallow: /cgi-bin/
Disallow: /trackback/
Disallow: /*/trackback/
Disallow: /comments/
Disallow: /comments/feed/
Disallow: /?s=
Disallow: /search
Disallow: /?buscar=
Disallow: /*?attachment_id=
Disallow: /*?preview_id=
Disallow: /*?preview=
Disallow: /*?preview_nonce=
Disallow: /*/attachment/
Disallow: /tag/*/page/
Disallow: /tag/*/feed/
Disallow: /feed/
Disallow: /*/feed/
Disallow: /*/feed/rss/

User-agent: SemrushBot
Crawl-delay: 10

User-agent: AhrefsBot
Crawl-delay: 10

User-agent: MJ12bot
Crawl-delay: 10

User-agent: DataForSeoBot
Disallow: /

User-agent: SeekportBot
Disallow: /

User-agent: SiteAuditBot
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Mexico
11.1% block at least one AI crawler
Behind Cloudflare
Yes