O Povo

opovo.com.br · Brazil · measured 8 Oct 2026

Google Search

Allowed

Google-Extended

Blocked

OpenAI

Blocked

Anthropic

Blocked

Perplexity

Allowed

robots.txt

As read on 8 Oct 2026, 2,205 bytes. 2 rule groups name AI crawlers; shown first.

User-agent: Google-Extended
User-agent: Meta-ExternalAgent
User-agent: FacebookBot
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Applebot-Extended
User-agent: Grok
User-agent: GrokBot
User-agent: xAI-Grok
User-agent: xAI-Bot
User-agent: Grok-DeepSearch
User-agent: CCBot
User-agent: Bytespider
User-agent: DeepSeek
User-agent: DeepSeekBot
User-agent: Qwen
User-agent: Cohere-ai
User-agent: cohere-training-data-crawler
User-agent: AI2Bot
User-agent: ai2bot-dolma
User-agent: Webzio-Extended
User-agent: SemrushBot-OCOB
User-agent: DataForSeoBot
User-agent: Amazonbot
User-agent: SabiáBot
User-agent: Diffbot
User-agent: Factset_spyderbot
User-agent: AwarioRssBot
User-agent: AwarioSmartBot
User-agent: Peer39_crawler/1.0
User-agent: omgili
User-agent: Omgilibot
User-agent: ICC-Crawler
User-agent: ImagesiftBot
User-agent: Kangaroo Bot
User-agent: NovaAct
User-agent: netEstate Imprint Crawler
User-agent: Cotoyogi
User-agent: Timpibot
User-agent: YouBot
User-agent: Operator
User-agent: PanguBot
User-agent: Devin
User-agent: Datenbank Crawler
User-agent: Copilot-LLM
Disallow: /

User-agent: Googlebot
User-agent: Googlebot-News
User-agent: Applebot
User-agent: PetalBot
User-agent: facebookexternalhit
User-agent: Twitterbot
User-agent: Bingbot
User-agent: BingPreview
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: Gemini-Deep-Research
User-agent: GoogleAgent-Mariner
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: MistralAI-User
User-agent: Meta-ExternalFetcher
User-agent: DuckAssistBot
Disallow: /_templates/
Disallow: /_temp/
Disallow: /static/
Disallow: /src/
Disallow: /imagens/
Disallow: /cdn/
Disallow: /teste-*.html$
Disallow: /goto/
Disallow: /virtual/
Sitemap: https://www.opovo.com.br/sitemap-index.xml
Sitemap: https://www.opovo.com.br/sitemap-news-day.xml
Sitemap: https://opovo.com.br/trends/sitemap_index.xml

User-agent: *
Disallow: /_templates/
Disallow: /_temp/
Disallow: /static/
Disallow: /src/
Disallow: /imagens/
Disallow: /cdn/
Disallow: /teste-*.html$
Disallow: /goto/
Disallow: /virtual/

History

Changes recorded since June 2026.

No change recorded.

In context

Brazil

14.5% block at least one AI crawler

Behind Cloudflare

Yes