CrawlerPulse
CrawlerPulse / North America / Mexico / lajornadadeoriente.com.mx

La Jornada de Oriente

lajornadadeoriente.com.mx · Mexico · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 2,838 bytes. 4 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Amazonbot
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: meta-webindexer
Disallow: /

User-agent: meta-webindexer/1.1
Disallow: /

User-agent: CorporateResearch
Disallow: /

User-agent: AionBot
Disallow: /

User-agent: KeenableBot
Disallow: /

User-agent: ShapBot
Disallow: /

User-agent: *
Disallow: /columnas/tlaxcala/opinion.html?
Disallow: /columnas/puebla/opinion.html?

User-agent: *
Disallow: /?s=
Disallow: /search/
Disallow: /wp-json/wp/v2/users
Disallow: /xmlrpc.php
Disallow: /page/
Disallow: /author/
Disallow: /temas/

User-agent: ExaSearchBot
Disallow: /

User-agent: BLEXBot
Disallow: /

User-agent: SeekportBot
Disallow: /

User-agent: msnbot
Disallow: /

User-agent: OpenindexSpider
Disallow: /

User-agent: Arquivo-web-crawler
Disallow: /

User-agent: bingbot
Disallow: /

User-agent: Bingbot
Disallow: /

User-agent: bingbot/2.0
Disallow: /

User-agent: dotbot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: baiduspider
Disallow: /

User-agent: Baiduspider+
Disallow: /

User-agent: Baiduspider-video
Disallow: /

User-agent: Baiduspider-image
Disallow: /

User-agent: SentiBot
Disallow: /

User-agent: Semrushbot-SA
Disallow: /

User-agent: Semrushbot-SI
Disallow: /

User-agent: trendictionbot
Disallow: /

User-agent: CriteoBot/0.1
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: grapeshot
Disallow: /

User-agent: proximic
Disallow: /

User-agent: GnowitNewsbot
Disallow: /

User-agent: *
Allow: /ads.txt
Disallow: /foro/
Disallow: /rss/
Disallow: /publicidad/
Disallow: /encuestas/
Disallow: /gopoll/
Disallow: /poll/
Disallow: /imprimir*
Disallow: /suplementos/anuario.html
Disallow: /publicidad.php
Disallow: /suscripcion.php
Disallow: /enviar*
Disallow: /contacto*
Disallow: /piepag.php
Disallow: /2008/12/20/
Disallow: /foro/*
Disallow: /eyeblaster
Disallow: /addineyeV2.html
Disallow: /wp/
Disallow: /remen/?tid=2053
Disallow: /bowen/?tid=6717
Disallow: /zixun/index.php?tid=8751
Disallow: /YTT9k/20150803053521/index.html
Disallow: /8I7p3/20150803080308/index.html
Disallow: /YTT9k/20150803035247/index.html
Disallow: /7nb98/20150803103925/index.html
Disallow: /iATvD/20150802211117/index.html
Disallow: /7nb98/20150803121157/index.html
Disallow: /q6nYB/20150803113342/index.html
Disallow: /3Ql7O/20150803153517/index.html
Disallow: /YTT9k/20150802224841/index.html
Disallow: /q6nYB/20150802235745/index.html
Disallow: /3Ql7O/20150803060319/index.html
Disallow: /iATvD/20150803203331/index.html
Disallow: /q6nYB/20150802221324/index.html
Disallow: /YTT9k/20150803202349/index.html
Disallow: /3Ql7O/20150803104344/huarenbocaishequ.html
Disallow: /8I7p3/20150803055935/index.html
Disallow: /TZ9Ic/mxl.php?keywords=%CE%E5%D0%C7%D3%E9%C0%D6%B3%C7%BF%AA%BB%A7
Disallow: /xml.php

History

Changes recorded since June 2026.

No change recorded.

In context

Mexico
11.1% block at least one AI crawler
Behind Cloudflare
Yes