CrawlerPulse
CrawlerPulse / South America / Brazil / g1.globo.com

G1

g1.globo.com · Brazil · Globo · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 608 bytes. No rule names an AI crawler.

User-Agent: *
Disallow: /jornalismo/g1/
Disallow: /_ssi/
Disallow: /teste-*.html$
Disallow: /beta/
Disallow: /componentes/
Disallow: /busca/*
Disallow: /globo-news/jornal-globo-news/videos/v/globo-news-ao-vivo/61910/
Disallow: /globonews/playlist/globonews-ao-vivo.ghtml
Disallow: *globo-cdn-src/*
Disallow: /zeta/
Disallow: /content-aggregator/
Disallow: /jogos-app/
Disallow: /espiar/
Disallow: /especiais-publicitarios/*/check/
Sitemap: https://g1.globo.com/sitemap/g1/sitemap.xml
Sitemap: https://g1.globo.com/sitemap/Apuração/g1/sitemap.xml
Sitemap: https://g1.globo.com/sitemap/videos/g1/sitemap.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Brazil
14.5% block at least one AI crawler
Behind Cloudflare
No
Same group