rainews.it · Italy · Rai · measured 8 Oct 2026
As read on 8 Oct 2026, 3,185 bytes. 2 rule groups name AI crawlers; shown first.
User-agent: Twitterbot User-Agent: FacebookBot Allow: /*?nxtep User-agent: ChatGPT-User User-agent: Google-Extended User-agent: GPTBot User-agent: PerplexityBot User-agent: Amazonbot User-agent: anthropic-ai User-agent: AwarioRssBot User-agent: AwarioSmartBot User-agent: Bytespider User-agent: CCBot User-agent: ClaudeBot User-agent: Claude-Web User-agent: cohere-ai User-agent: DataForSeoBot User-agent: Diffbot User-agent: FacebookBot User-agent: magpie-crawler User-agent: NewsNow User-agent: news-please User-agent: omgili User-agent: omgilibot User-agent: peer39_crawler User-agent: peer39_crawler/1.0 User-agent: Scrapy User-agent: TurnitinBot User-agent: Botify Disallow: / User-agent: * Allow: / Disallow: /app/ Disallow: /webview/ Disallow: /programmi/ Disallow: /collezioni/ Disallow: /atomatic/ Disallow: /StatisticheProxy/ Disallow: /live/ Disallow: /iframe/ Disallow: /foto/ Allow: /tgr/*/foto/ Disallow: /*index.html? Allow: /dl/raiSport/speciali/*index.html Disallow: /*ricerca.html? Disallow: /*chi-siamo.html? Disallow: /*live_frame.html Disallow: /*?print$ Disallow: */related-tag-iframe.html Disallow: /*?refresh_ce=$ Disallow: /*?app_name=$ Disallow: /*?wt_mc=$ Disallow: /*?nxtep Disallow: /*.json$ Disallow: /tgr/?set Disallow: /tgr/amp/ Disallow: /ran24/speciali/ Disallow: /hit-service/ Disallow: /akam/ Disallow: /dl/RaiTV/ Disallow: /dl/rainews/redazioni/ Disallow: /dl/rainews/ricerca.html Disallow: /dl/rainews/articoli/ Disallow: /dl/rainews/media/ Disallow: /dl/rainews/speciali/ Disallow: /dl/rainews/live/ Disallow: /dl/rainews/elezioni2021/ Disallow: /dl/rainews/elezioni2020/ Disallow: /dl/rainews/includes/ Disallow: /dl/rainews/TGR/ Disallow: /dl/rainews/json/ Disallow: /dl/rai24/ Allow: /dl/rai24/assets/images/*.jpg$ Allow: /dl/rai24/assets/images/*.png$ Disallow: /dl/Report/ Disallow: /dl/grr/ Disallow: /dl/bilancio Disallow: /dl/portaleRadio Disallow: /dl/portali/site/ Disallow: /dl/js/ Disallow: /dl/objects/ Disallow: /dl/raisport/ Disallow: /dl/tg3/ Disallow: /dl/video/ Disallow: /dl/portale/ Disallow: /dl/docs/ Disallow: /dl/doc/ Disallow: /dl/analytics/ Disallow: /dl/siti/ Disallow: /dl/test/ Sitemap: https://www.rainews.it/sitemap.xml
Changes recorded since June 2026.
No change recorded.