CrawlerPulse
CrawlerPulse / South America / Brazil / uol.com.br

UOL

uol.com.br · Brazil · Universo Online · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Allowed
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 2,515 bytes. 20 rule groups name AI crawlers; shown first.

User-agent: Google-Extended
Disallow: /
Allow: /*.ghtm$

User-agent: Amazonbot
Disallow: /

User-agent: Anthropic-ai
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Cohere-ai
Disallow: /

User-agent: Diffbot
Disallow: /

User-agent: FacebookBot
Disallow: /

User-agent: ImagesiftBot
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: MistralAI-User
Disallow: /

User-agent: omgili
Disallow: /

User-agent: Omgilibot
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: Timpibot
Disallow: /

User-agent: YouBot
Disallow: /

User-agent: *
Sitemap: https://www.uol.com.br/carros/sitemap/v2/news-01.xml
Sitemap: https://www.uol.com.br/ecoa/sitemap/news-01.xml
Sitemap: https://www.uol.com.br/esporte/sitemap/v2/news-01.xml
Sitemap: https://www.uol.com.br/nossa/sitemap/news-01.xml
Sitemap: https://www.uol.com.br/splash/sitemap/news-01.xml
Sitemap: https://www.uol.com.br/tilt/sitemap/news-01.xml
Sitemap: https://www.uol.com.br/universa/sitemap/v2/news-01.xml
Sitemap: https://www.uol.com.br/vivabem/sitemap/v2/news-01.xml
Sitemap: https://c.jsuol.com.br/assets/jupiter-news/?resource-id=sitemap&source=toca/index.xml
Allow: /
Disallow: /carros/dev/
Disallow: /*?uol_app
Disallow: /*?*&uol_app

User-agent: AI2Bot
Disallow: /

User-agent: Applebot
Disallow: /

User-agent: AwarioRssBot
Disallow: /

User-agent: AwarioSmartBot
Disallow: /

User-agent: Claude-SearchBot
Disallow: /

User-agent: Claude-User
Disallow: /

User-agent: cohere-training-data-crawler
Disallow: /

User-agent: Cotoyogi
Disallow: /

User-agent: DataForSeoBot
Disallow: /

User-agent: Datenbank Crawler
Disallow: /

User-agent: DeepSeek
Disallow: /

User-agent: Devin
Disallow: /

User-agent: DuckAssistBot
Disallow: /

User-agent: Factset_spyderbot
Disallow: /

User-agent: Gemini-Deep-Research
Disallow: /

User-agent: GoogleAgent-Mariner
Disallow: /

User-agent: ICC-Crawler
Disallow: /

User-agent: Kangaroo Bot
Disallow: /

User-agent: Meta-ExternalFetcher
Disallow: /

User-agent: netEstate Imprint Crawler
Disallow: /

User-agent: NovaAct
Disallow: /

User-agent: Operator
Disallow: /

User-agent: PanguBot
Disallow: /

User-agent: Peer39_crawler/1.0
Disallow: /

User-agent: Perplexity-User
Disallow: /

User-agent: Qwen
Disallow: /

User-agent: SemrushBot-OCOB
Disallow: /

User-agent: Twitterbot
Disallow: /

User-agent: Webzio-Extended
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Brazil
14.5% block at least one AI crawler
Behind Cloudflare
No