CrawlerPulse
CrawlerPulse / South America / Peru / larepublica.pe

La República

larepublica.pe · Peru · Grupo La República · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 3,864 bytes. 1 rule group name AI crawlers; shown first.

User-agent: *
User-agent: Googlebot
User-agent: Googlebot-News
User-agent: Googlebot-Image
User-agent: Googlebot-Video
User-agent: Googlebot-Mobile
User-agent: PetalBot
User-agent: Bingbot
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: GPTBot
User-agent: Amazonbot
User-agent: anthropic-ai
User-agent: AwarioRssBot
User-agent: AwarioSmartBot
User-agent: Bytespider
User-agent: CCBot
User-agent: ClaudeBot
User-agent: Claude-Web
User-agent: cohere-ai
User-agent: DataForSeoBot
User-agent: Diffbot
User-agent: ImagesiftBot
User-agent: FacebookBot
User-agent: Google-Extended
User-agent: magpie-crawler
User-agent: NewsNow
User-agent: news-please
User-agent: omgili
User-agent: omgilibot
User-agent: PerplexityBot
User-agent: PiplBot
User-agent: Scrapy
User-agent: TurnitinBot
User-agent: YouBot
Disallow: /buscador*
Disallow: /data
Disallow: /data/*
Disallow: /contactenos
Disallow: /*/pag/
Disallow: /*/pag/*
Disallow: /*/*/page/*
Disallow: /archive/*
Disallow: /archivo/*
Disallow: /node/*
Disallow: /taxonomy/term/*
Disallow: /assets/*
Disallow: /files/*
Disallow: /comment/reply/*
Disallow: /envivo/*
Disallow: /*/tag/*
Disallow: */*Perú
Disallow: */Encuesta*
Disallow: */EMPRESA/*
Disallow: *ref=photogallery*
Disallow: */sitemaps/*
Disallow: */pf/*
Allow: /sitemap/
Allow: /sitemap/*
Allow: /sitemaps/*
Allow: /rss/
Allow: /rss/*
Sitemap: https://larepublica.pe/sitemap/sociedad.xml
Sitemap: https://larepublica.pe/sitemap/noticias.xml
Sitemap: https://larepublica.pe/sitemap/politica.xml
Sitemap: https://larepublica.pe/sitemap/opinion.xml
Sitemap: https://larepublica.pe/sitemap/mundo.xml
Sitemap: https://larepublica.pe/sitemap/mastergrama.xml
Sitemap: https://larepublica.pe/sitemap/espectaculos.xml
Sitemap: https://larepublica.pe/sitemap/economia.xml
Sitemap: https://larepublica.pe/sitemap/deportes.xml
Sitemap: https://larepublica.pe/sitemap/cine-series.xml
Sitemap: https://larepublica.pe/sitemap/carlincatura.xml
Sitemap: https://larepublica.pe/sitemap/ciencia.xml
Sitemap: https://larepublica.pe/sitemap/horoscopo.xml
Sitemap: https://larepublica.pe/sitemap/servicios.xml
Sitemap: https://larepublica.pe/sitemap/latinos-en-usa.xml
Sitemap: https://larepublica.pe/sitemap/web/horoscopo.xml
Sitemap: https://larepublica.pe/sitemap/web/sociedad.xml
Sitemap: https://larepublica.pe/sitemap/web/politica.xml
Sitemap: https://larepublica.pe/sitemap/web/opinion.xml
Sitemap: https://larepublica.pe/sitemap/web/servicios.xml
Sitemap: https://larepublica.pe/sitemap/web/mundo.xml
Sitemap: https://larepublica.pe/sitemap/web/mastergrama.xml
Sitemap: https://larepublica.pe/sitemap/web/espectaculos.xml
Sitemap: https://larepublica.pe/sitemap/web/economia.xml
Sitemap: https://larepublica.pe/sitemap/web/deportes.xml
Sitemap: https://larepublica.pe/sitemap/web/cine-series.xml
Sitemap: https://larepublica.pe/sitemap/web/carlincatura.xml
Sitemap: https://larepublica.pe/sitemap/web/ciencia.xml
Sitemap: https://larepublica.pe/sitemap/web/noticias.xml
Sitemap: https://larepublica.pe/sitemap/web/latinos-en-usa.xml
Sitemap: https://larepublica.pe/rss/sociedad.xml
Sitemap: https://larepublica.pe/rss/latinos-en-usa.xml
Sitemap: https://larepublica.pe/rss/politica.xml
Sitemap: https://larepublica.pe/rss/opinion.xml
Sitemap: https://larepublica.pe/rss/mundo.xml
Sitemap: https://larepublica.pe/rss/mastergrama.xml
Sitemap: https://larepublica.pe/rss/espectaculos.xml
Sitemap: https://larepublica.pe/rss/economia.xml
Sitemap: https://larepublica.pe/rss/deportes.xml
Sitemap: https://larepublica.pe/rss/cine-series.xml
Sitemap: https://larepublica.pe/rss/carlincatura.xml
Sitemap: https://larepublica.pe/rss/ciencia.xml
Sitemap: https://larepublica.pe/rss/home.xml
Sitemap: https://larepublica.pe/rss/servicios.xml
Sitemap: https://larepublica.pe/rss/horoscopo.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Peru
10.3% block at least one AI crawler
Behind Cloudflare
No
Same group