CrawlerPulse
CrawlerPulse / Europe / Italy / tg24.sky.it

Sky TG24

tg24.sky.it · Italy · Sky Italia · measured 8 Oct 2026

Google Search
Partial
Google-Extended
Partial
OpenAI
Partial
Anthropic
Partial
Perplexity
Partial
Meta
Partial
Common Crawl
Partial
ByteDance
Partial
Apple
Partial
Amazon
Partial

robots.txt

As read on 8 Oct 2026, 2,948 bytes. No rule names an AI crawler.

User-agent: bingbot
Crawl-delay: 10

User-agent: Baiduspider
Disallow: /

User-agent: Yandex
Disallow: /

User-agent: *
Disallow: /AdvTest/*
Disallow: /video/test/*
Disallow: /content/*
Disallow: /*jcr_content*/
Disallow: /cronaca/2012/01/10/carlo_malinconico_sottosegretario_governo_monti_vacanze_pagate_inchiesta_g8*
Disallow: /politica/2012/01/10/carlo_malinconico_dimissioni_governo_monti_vacanze_pagate_inchiesta_g8*
Disallow: /cronaca/2016/01/18/monsignor-scarano-condanna-calunnia-assolto-da-corruzione*
Disallow: /cronaca/2013/07/01/monsignor_scarano_ior_procura_roma_regina_coeli*
Disallow: /cronaca/2013/06/28/ior_arresti_truffa_corruzione_gdf_procura_roma*
Disallow: /cronaca/2017/03/14/stalking-moglie-arresto-milano*
Disallow: /cronaca/2014/06/20/vaccini_novartis_h1n1_influenza_a_ipotesi_truffa_prezzi_gonfiati_indagini*
Disallow: /cronaca/2010/08/29/lite_in_sala_parto_messina_migliorano_condizioni_donna_bimbo_medici_sospesi*
Disallow: /cronaca/2011/02/24/muammar_gheddafi_ragazze_italiane_viaggio_libia_lezione_islam_roma_tripoli_hostess*
Disallow: /cronaca/2009/02/09/Fallimento_Micop_sei_anni_a_Danilo_Coppola*
Disallow: /speciali/elezioni/italia/politiche/camera/collegio_uni*
Disallow: /speciali/elezioni/italia/politiche/camera/collegio_pluri*
Disallow: /speciali/elezioni/italia/politiche/senato/collegio_uni*
Disallow: /speciali/elezioni/italia/politiche/senato/collegio_pluri*
Disallow: /politica/2012/05/15/andrea_zoppini_sottosegretario_giustizia_indagato*
Disallow: /content/skytg24-responsive/it/xkwl9udf7hz25bj0fq5drzb6rgcb2n
Disallow: /tg24/
Disallow: /glisgommati/
Disallow: /cinema/
Disallow: /mag/
Disallow: /servizi/
Disallow: /eco_style/
Disallow: /SkyItVideoportalUtility/
Disallow: /assistenza/
Disallow: /assistenza-e-supporto/
Disallow: /data/
Disallow: /speciali_tg24/
Disallow: /common/gsa_search.html
Disallow: /*.print.html$
Disallow: /*/null$
Disallow: /*/news@sky.it$
Disallow: /inclusion-pages/*
Disallow: /liveblog.html$
Disallow: /spettacolo/serie-tv/2018/04/19/westworld-2-episodio-1-analisi-commenti.*.amp*
Disallow: /speciale_elezioni/
Disallow: /news/
Disallow: /home.listnews$
Disallow: /.html$
Disallow: /p.js$
Disallow: /ReSky/
Disallow: /skyuno/
Disallow: /skyatlantic/
Disallow: /gossip/
Disallow: /life_style/
Disallow: /web_style/
Disallow: /serie_tv/
Disallow: /calcio/
Disallow: /musica/
Disallow: /social/
Disallow: /tags/
Disallow: *?+SkyitTg24
Disallow: /insider-registrazione
Disallow: /stories/
Allow: /stories/*/*.html$
Allow: /stories/*/*/*.html$
Allow: /spettacolo/serie-tv/2018/04/19/westworld-2-episodio-1-analisi-commenti.amp$
Allow: /content/dam/static/contentimages/
Allow: /content/skytg24-responsive/it/inclusion-pages/inclusion-list.listnews
Sitemap:  https://tg24.sky.it/sitemap-index.xml
Sitemap:  https://tg24.sky.it/sitemap-static.index.xml
Sitemap:  https://tg24.sky.it/sitemap.elections.index.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Italy
54.5% block at least one AI crawler
Behind Cloudflare
No