CrawlerPulse
CrawlerPulse / Europe / Italy / lagazzettadelmezzogiorno.it

lagazzettadelmezzogiorno

lagazzettadelmezzogiorno.it · Italy · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Blocked
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 21,496 bytes. 2 rule groups name AI crawlers; shown first.

User-agent: facebookexternalhit
User-agent: Facebot
User-agent: FacebookBot
Allow: /

User-agent: grapeshot
User-agent: Oggettivolanti.it
User-agent: 01h4x.com
User-agent: 360Spider
User-agent: 404checker
User-agent: 404enemy
User-agent: 80legs
User-agent: Abonti
User-agent: Aboundex
User-agent: Aboundexbot
User-agent: Acunetix
User-agent: ADmantX
User-agent: AfD-Verbotsverfahren
User-agent: AIBOT
User-agent: AiHitBot
User-agent: Aipbot
User-agent: Alexibot
User-agent: Alligator
User-agent: AllSubmitter
User-agent: AlphaBot
User-agent: Anarchie
User-agent: Anarchy
User-agent: Anarchy99
User-agent: Ankit
User-agent: Anthill
User-agent: Apexoo
User-agent: arquivo.pt
User-agent: arquivo-web-crawler
User-agent: Aspiegel
User-agent: ASPSeek
User-agent: Asterias
User-agent: Attach
User-agent: autoemailspider
User-agent: AwarioRssBot
User-agent: AwarioSmartBot
User-agent: BackDoorBot
User-agent: Backlink-Ceck
User-agent: backlink-check
User-agent: BacklinkCrawler
User-agent: BackStreet
User-agent: BackWeb
User-agent: Badass
User-agent: Bandit
User-agent: Barkrowler
User-agent: BatchFTP
User-agent: Battleztar Bazinga
User-agent: BBBike
User-agent: BDCbot
User-agent: BDFetch
User-agent: BetaBot
User-agent: Bigfoot
User-agent: Bitacle
User-agent: Blackboard
User-agent: Black Hole
User-agent: BlackWidow
User-agent: BLEXBot
User-agent: Blow
User-agent: BlowFish
User-agent: Boardreader
User-agent: Bolt
User-agent: BotALot
User-agent: Brandprotect
User-agent: Brandwatch
User-agent: Buck
User-agent: Buddy
User-agent: BuiltBotTough
User-agent: BuiltWith
User-agent: Bullseye
User-agent: BunnySlippers
User-agent: BuzzSumo
User-agent: Calculon
User-agent: CATExplorador
User-agent: CazoodleBot
User-agent: CCBot
User-agent: Cegbfeieh
User-agent: CensysInspect
User-agent: check1.exe
User-agent: CheeseBot
User-agent: CherryPicker
User-agent: CheTeam
User-agent: ChinaClaw
User-agent: Chlooe
User-agent: Claritybot
User-agent: Cliqzbot
User-agent: Cloud mapping
User-agent: coccocbot-web
User-agent: Cocolyzebot
User-agent: CODE87
User-agent: Cogentbot
User-agent: cognitiveseo
User-agent: Collector
User-agent: com.plumanalytics
User-agent: Copier
User-agent: CopyRightCheck
User-agent: Copyscape
User-agent: Cosmos
User-agent: Craftbot
User-agent: crawler4j
User-agent: crawler.feedback
User-agent: crawl.sogou.com
User-agent: CrazyWebCrawler
User-agent: Crescent
User-agent: CrunchBot
User-agent: CSHttp
User-agent: Curious
User-agent: Custo
User-agent: CyotekWebCopy
User-agent: DatabaseDriverMysqli
User-agent: DataCha0s
User-agent: DBLBot
User-agent: demandbase-bot
User-agent: Demon
User-agent: Deusu
User-agent: Devil
User-agent: Digincore
User-agent: DigitalPebble
User-agent: DIIbot
User-agent: Dirbuster
User-agent: Disco
User-agent: Discobot
User-agent: Discoverybot
User-agent: Dispatch
User-agent: DittoSpyder
User-agent: DnyzBot
User-agent: DomainAppender
User-agent: DomainCrawler
User-agent: DomainSigmaCrawler
User-agent: Domains Project
User-agent: domainsproject.org
User-agent: DomainStatsBot
User-agent: Dotbot
User-agent: Download Wonder
User-agent: Dragonfly
User-agent: Drip
User-agent: DSearch
User-agent: DTS Agent
User-agent: EasyDL
User-agent: Ebingbong
User-agent: eCatch
User-agent: ECCP/1.0
User-agent: Ecxi
User-agent: EirGrabber
User-agent: EMail Siphon
User-agent: EMail Wolf
User-agent: EroCrawler
User-agent: evc-batch
User-agent: Evil
User-agent: Exabot
User-agent: Express WebPictures
User-agent: ExtLinksBot
User-agent: Extractor
User-agent: ExtractorPro
User-agent: Extreme Picture Finder
User-agent: EyeNetIE
User-agent: Ezooms
User-agent: FDM
User-agent: FemtosearchBot
User-agent: FHscan
User-agent: Fimap
User-agent: Firefox/7.0
User-agent: FlashGet
User-agent: Flunky
User-agent: Foobot
User-agent: Freeuploader
User-agent: FrontPage
User-agent: Fuzz
User-agent: FyberSpider
User-agent: Fyrebot
User-agent: GalaxyBot
User-agent: Genieo
User-agent: GermCrawler
User-agent: Getintent
User-agent: GetRight
User-agent: GetWeb
User-agent: Gigablast
User-agent: Gigabot
User-agent: G-i-g-a-b-o-t
User-agent: Go-Ahead-Got-It
User-agent: gopher
User-agent: Gotit
User-agent: GoZilla
User-agent: Go!Zilla
User-agent: Grabber
User-agent: GrabNet
User-agent: Grafula
User-agent: GrapeFX
User-agent: GrapeshotCrawler
User-agent: GridBot
User-agent: GT::WWW
User-agent: Haansoft
User-agent: HaosouSpider
User-agent: Harvest
User-agent: Havij
User-agent: HEADMasterSEO
User-agent: heritrix
User-agent: Heritrix
User-agent: Hloader
User-agent: HMView
User-agent: HTMLparser
User-agent: HTTP::Lite
User-agent: HTTrack
User-agent: Humanlinks
User-agent: HybridBot
User-agent: Iblog
User-agent: IDBot
User-agent: IDBTE4M
User-agent: Id-search
User-agent: IlseBot
User-agent: Image Fetch
User-agent: Image Sucker
User-agent: IndeedBot
User-agent: Indy Library
User-agent: InfoNaviRobot
User-agent: InfoTekies
User-agent: instabid
User-agent: Intelliseek
User-agent: InterGET
User-agent: Internet Ninja
User-agent: InternetSeer
User-agent: internetVista monitor
User-agent: ips-agent
User-agent: Iria
User-agent: IRLbot
User-agent: isitwp.com
User-agent: Iskanie
User-agent: IstellaBot
User-agent: JamesBOT
User-agent: Jbrofuzz
User-agent: JennyBot
User-agent: JetCar
User-agent: Jetty
User-agent: JikeSpider
User-agent: JOC Web Spider
User-agent: Joomla
User-agent: Jorgee
User-agent: JustView
User-agent: Jyxobot
User-agent: Kenjin Spider
User-agent: Keyword Density
User-agent: Kinza
User-agent: Kozmosbot
User-agent: Lanshanbot
User-agent: Larbin
User-agent: LeechFTP
User-agent: LeechGet
User-agent: LexiBot
User-agent: Lftp
User-agent: LibWeb
User-agent: Libwhisker
User-agent: LieBaoFast
User-agent: Lightspeedsystems
User-agent: Likse
User-agent: Linkbot
User-agent: Linkdexbot
User-agent: LinkextractorPro
User-agent: LinkpadBot
User-agent: LinkScan
User-agent: LinksManager
User-agent: LinkWalker
User-agent: LinqiaMetadataDownloaderBot
User-agent: LinqiaRSSBot
User-agent: LinqiaScrapeBot
User-agent: Lipperhey
User-agent: Lipperhey Spider
User-agent: Litemage_walker
User-agent: Lmspider
User-agent: LNSpiderguy
User-agent: Ltx71
User-agent: lwp-request
User-agent: LWP::Simple
User-agent: lwp-trivial
User-agent: Magnet
User-agent: Mag-Net
User-agent: magpie-crawler
User-agent: Mail.RU_Bot
User-agent: Majestic12
User-agent: Majestic SEO
User-agent: Majestic-SEO
User-agent: MarkMonitor
User-agent: MarkWatch
User-agent: Masscan
User-agent: Mass Downloader
User-agent: Mata Hari
User-agent: MauiBot
User-agent: Mb2345Browser
User-agent: meanpathbot
User-agent: Meanpathbot
User-agent: MeanPath Bot
User-agent: Mediatoolkitbot
User-agent: mediawords
User-agent: MegaIndex.ru
User-agent: Metauri
User-agent: MFC_Tear_Sample
User-agent: MicroMessenger
User-agent: Microsoft Data Access
User-agent: Microsoft URL Control
User-agent: MIDown tool
User-agent: MIIxpc
User-agent: Mister PiX
User-agent: Mojeek
User-agent: Mojolicious
User-agent: Morfeus Fucking Scanner
User-agent: Mozlila
User-agent: MQQBrowser
User-agent: Mr.4x3
User-agent: MSFrontPage
User-agent: MSIECrawler
User-agent: Msrabot
User-agent: MTRobot
User-agent: muhstik-scan
User-agent: Musobot
User-agent: Name Intelligence
User-agent: Nameprotect
User-agent: Navroad
User-agent: NearSite
User-agent: Needle
User-agent: Nessus
User-agent: NetAnts
User-agent: Netcraft
User-agent: netEstate NE Crawler
User-agent: NetLyzer
User-agent: NetMechanic
User-agent: NetSpider
User-agent: Nettrack
User-agent: Net Vampire
User-agent: Netvibes
User-agent: NetZIP
User-agent: NextGenSearchBot
User-agent: Nibbler
User-agent: NICErsPRO
User-agent: Niki-bot
User-agent: Nikto
User-agent: NimbleCrawler
User-agent: Nimbostratus
User-agent: Ninja
User-agent: Nmap
User-agent: NPbot
User-agent: Nuclei
User-agent: Nutch
User-agent: oBot
User-agent: Octopus
User-agent: Offline Explorer
User-agent: Offline Navigator
User-agent: OnCrawl
User-agent: Openfind
User-agent: OpenLinkProfiler
User-agent: Openvas
User-agent: OpenVAS
User-agent: OPPO A33
User-agent: OrangeBot
User-agent: OrangeSpider
User-agent: OutclicksBot
User-agent: OutfoxBot
User-agent: PageAnalyzer
User-agent: Page Analyzer
User-agent: PageGrabber
User-agent: page scorer
User-agent: PageScorer
User-agent: PageThing.com
User-agent: Pandalytics
User-agent: Panscient
User-agent: Papa Foto
User-agent: Pavuk
User-agent: pcBrowser
User-agent: PECL::HTTP
User-agent: PeoplePal
User-agent: Petalbot
User-agent: PHPCrawl
User-agent: Picscout
User-agent: Picsearch
User-agent: PictureFinder
User-agent: Piepmatz
User-agent: Pimonster
User-agent: Pi-Monster
User-agent: Pixray
User-agent: PleaseCrawl
User-agent: plumanalytics
User-agent: Pockey
User-agent: POE-Component-Client-HTTP
User-agent: polaris version
User-agent: probe-image-size
User-agent: Probethenet
User-agent: ProPowerBot
User-agent: ProWebWalker
User-agent: Psbot
User-agent: Pu_iN
User-agent: Pump
User-agent: PxBroker
User-agent: PyCurl
User-agent: QueryN Metasearch
User-agent: Quick-Crawler
User-agent: RankActive
User-agent: RankActiveLinkBot
User-agent: RankFlex
User-agent: RankingBot
User-agent: RankingBot2
User-agent: Rankivabot
User-agent: RankurBot
User-agent: RealDownload
User-agent: Reaper
User-agent: RebelMouse
User-agent: Recorder
User-agent: RedesScrapy
User-agent: ReGet

User-agent: *
Disallow: /!panel/
Disallow: /tempcache/
Disallow: /webservices/
Disallow: /widget/head/login
Disallow: /clickhere.jsp
Disallow: /sezioni/122/english
Disallow: /sezioni/127/battaglia-treni
Disallow: /sezioni/128/newspapergame
Disallow: /sezioni/129/test-primo-piano
Disallow: /sezioni/130/quadretti-selvaggi
Disallow: /sezioni/134/servizi
Disallow: /sezioni/135/vivi-citta
Disallow: /sezioni/143/foto-video
Disallow: /sezioni/146/lettere-al-direttore
Disallow: /sezioni/147/scrivi-alla-gazzetta
Disallow: /sezioni/151/appuntamenti
Disallow: /sezioni/152/musica-e-concerti
Disallow: /sezioni/153/teatri-e-danza
Disallow: /sezioni/154/cinema
Disallow: /sezioni/155/speciali-test
Disallow: /sezioni/164/puglia-tv
Disallow: /sezioni/165/basilicata-tv
Disallow: /sezioni/171/calcio-tv
Disallow: /sezioni/172/meteo-tv
Disallow: /sezioni/184/test-section-primo-piano
Disallow: /sezioni/185/primi-piatti
Disallow: /sezioni/186/secondi-piatti-e-contorni
Disallow: /sezioni/187/dolci-e-dessert
Disallow: /sezioni/188/speciale-giubileo
Disallow: /sezioni/191/mario-rossi
Disallow: /sezioni/198/sport-news-24
Disallow: /sezioni/203/autopromo-1
Disallow: /sezioni/213/moda
Disallow: /sezioni/214/news-sport
Disallow: /sezioni/216/autopromo-2
Disallow: /sezioni/220/sanita
Disallow: /sezioni/222/master-e-formazione
Disallow: /sezioni/223/giovanni-panza
Disallow: /sezioni/225/le-tue-emozioni-in-uno-scatto
Disallow: /sezioni/226/giochi
Disallow: /sezioni/227/big-farm-goodgame
Disallow: /sezioni/228/goodgame-empire
Disallow: /sezioni/229/sport-new-site
Disallow: /sezioni/231/cinema-bari231
Disallow: /sezioni/232/cinema-bat
Disallow: /sezioni/233/cinema-brindisi
Disallow: /sezioni/234/cinema-foggia
Disallow: /sezioni/235/cinema-lecce
Disallow: /sezioni/236/cinema-taranto
Disallow: /sezioni/237/cinema-matera
Disallow: /sezioni/238/cinema-potenza
Disallow: /sezioni/242/cinema242
Disallow: /sezioni/243/quell-estate-fa
Disallow: /sezioni/244/famiglie-di-puglia
Disallow: /sezioni/245/viaggi-crociere
Disallow: /sezioni/246/elezioni-politiche-2022
Disallow: /sezioni/248/dossier-ilva
Disallow: /sezioni/250/raccolta-differenziata-puglia
Disallow: /sezioni/251/medimex
Disallow: /sezioni/253/la-gazzetta-tv
Disallow: /sezioni/258/test-hp-svago-e-arte
Disallow: /sezioni/263/gazzetta-music-contest
Disallow: /sezioni/264/sondaggi-new-site
Disallow: /sezioni/266/testtt
Disallow: /sezioni/267/cronaca
Disallow: /sezioni/269/section-della-cultura
Disallow: /sezioni/270/andrea-del-monaco
Disallow: /sezioni/276/new-site-new
Disallow: /sezioni/278/che-bella-giornata
Disallow: /sezioni/279/buongusto
Disallow: /sezioni/280/ricette-buongusto
Disallow: /sezioni/281/chef-buongusto
Disallow: /sezioni/282/news-agroalimentari-buongusto
Disallow: /sezioni/284/viaggi-del-buongusto
Disallow: /sezioni/288/fritto-e-mangiato
Disallow: /sezioni/289/giorgio-assennato
Disallow: /sezioni/290/francesco-chiarello
Disallow: /sezioni/291/il-nuovo-ponte-di-bari
Disallow: /sezioni/292/fiera-del-levante
Disallow: /sezioni/294/incontri-d-autore
Disallow: /sezioni/295/section-new-site
Disallow: /sezioni/296/le-tue-emozioni-in-una-immagine-calendario-2017
Disallow: /sezioni/301/fruit-logistica
Disallow: /sezioni/302/pagine-storiche-gazzetta
Disallow: /sezioni/306/mezzogiorno-di-valore
Disallow: /sezioni/312/midi-fumetti-per-il-sud
Disallow: /sezioni/313/gazzetta-music-contest-2018
Disallow: /sezioni/315/gdm-tv-new-site
Disallow: /sezioni/316/test-new-site
Disallow: /sezioni/317/130-anni-eventi
Disallow: /sezioni/322/pierluigi-di-palma
Disallow: /sezioni/324/europa
Disallow: /sezioni/325/europa-primo-piano-attualita
Disallow: /sezioni/326/europa-primo-piano-intorno-a-te
Disallow: /sezioni/327/europa-altre-news
Disallow: /sezioni/331/asya-argentieri
Disallow: /sezioni/332/ed-locali
Disallow: /sezioni/337/social
Disallow: /sezioni/338/puglia-e-basilicata-tv
Disallow: /sezioni/342/redazionali
Disallow: /sezioni/344/newsletter-feed
Disallow: /sezioni/347/test-hp-multimedia
Disallow: /sezioni/348/130-anni
Disallow: /sezioni/349/130-brindisi
Disallow: /sezioni/350/130-copertine
Disallow: /sezioni/351/130-bari
Disallow: /sezioni/353/130-foggia
Disallow: /sezioni/354/130-lecce
Disallow: /sezioni/355/130-taranto
Disallow: /sezioni/356/130-matera
Disallow: /sezioni/357/130-potenza
Disallow: /sezioni/358/invia-un-contenuto
Disallow: /sezioni/359/canta-sting-e-gazzetta-ti-porta-al-concerto
Disallow: /sezioni/360/test-section-rubriche
Disallow: /sezioni/361/test-hp-focus
Disallow: /sezioni/362/test-section-focus
Disallow: /sezioni/363/test-section-speciali
Disallow: /sezioni/364/vini-buongusto
Disallow: /sezioni/365/birre-buongusto
Disallow: /sezioni/367/pausini-antonacci
Disallow: /sezioni/368/registrazione-promozioni-gazzetta
Disallow: /sezioni/369/gazzetta-beach
Disallow: /sezioni/371/turismo-in-grecia
Disallow: /sezioni/378/live
Disallow: /sezioni/381/sezioni
Disallow: /sezioni/382/edizioni-locali
Disallow: /sezioni/384/puglia-con-le-stellette
Disallow: /sezioni/387/share
Disallow: /sezioni/388/sezione-a-pagamento
Disallow: /sezioni/389/settimanale-gazzettaffari
Disallow: /sezioni/390/federica-marangio
Disallow: /sezioni/392/speciale-elezioni-2019
Disallow: /sezioni/393/amministrative-bari
Disallow: /sezioni/394/amministrative-brindisi
Disallow: /sezioni/395/amministrative-foggia
Disallow: /sezioni/396/amministrative-lecce
Disallow: /sezioni/397/amministrative-taranto
Disallow: /sezioni/398/amministrative-matera
Disallow: /sezioni/399/amministrative-potenza
Disallow: /sezioni/406/auto-storiche-da-collezione
Disallow: /sezioni/407/salento
Disallow: /sezioni/408/roberto-test
Disallow: /sezioni/409/monica-test
Disallow: /sezioni/410/sezioni
Disallow: /sezioni/424/novelle-contro-la-paura
Disallow: /sezioni/426/fiera-del-levante-2020
Disallow: /sezioni/428/cronache-di-distanziamento-socievole
Disallow: /sezioni/429/estrazione-del-lotto
Disallow: /sezioni/430/la-bilancia-e-il-bilancio
Disallow: /sezioni/431/la-tua-opinione-i-sondaggi
Disallow: /sezioni/432/diritto-economia
Disallow: /sezioni/433/museo-virtuale
Disallow: /sezioni/435/dalla-gazzetta-in-edicola
Disallow: /sezioni/436/test-rubriche-newsite
Disallow: /sezioni/439/legale-avvisi-giudiziari
Disallow: /sezioni/440/tribunale-di-bari
Disallow: /sezioni/441/tribunale-di-trani
Disallow: /sezioni/442/tribunale-di-brindisi
Disallow: /sezioni/443/tribunale-di-foggia
Disallow: /sezioni/444/tribunale-di-lecce
Disallow: /sezioni/445/tribunale-di-taranto
Disallow: /sezioni/446/tribunale-di-matera
Disallow: /sezioni/447/tribunale-di-potenza
Disallow: /sezioni/448/emozioni-in-gazzetta
Disallow: /sezioni/449/ministero-dello-sviluppo-economico
Disallow: /sezioni/450/blog-newsite
Disallow: /sezioni/459/test-123
Disallow: /sezioni/463/test-nuovo
Disallow: /sezioni/480/pubblicita-hp
Disallow: /sezioni/481/pubblicita-section
Disallow: /sezioni/524/pagine-bari
Disallow: /sezioni/525/pagine-bat
Disallow: /sezioni/526/pagine-brindisi
Disallow: /sezioni/527/pagine-foggia
Disallow: /sezioni/528/pagine-lecce
Disallow: /sezioni/529/pagine-taranto
Disallow: /sezioni/530/pagine-matera
Disallow: /sezioni/531/pagine-potenza
Disallow: /sezioni/536/pagina-no-cookie
Disallow: /sezioni/543/risultati-calcio
Disallow: /sezioni/544/geolocation-search
Disallow: /sezioni/545/elezioni
Disallow: /sezioni/547/elezioni-comunali-puglia
Disallow: /sezioni/549/tipologia-abbonamenti-app
Disallow: /sezioni/550/checkout-app
Disallow: /sezioni/551/thank-you-page-app
Disallow: /sezioni/567/live-football-gdm
Sitemap: https://www.lagazzettadelmezzogiorno.it/siteMap.jsp
Sitemap: https://www.lagazzettadelmezzogiorno.it/siteMapNews.jsp

User-agent: AmazonAdBot
Allow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Italy
54.5% block at least one AI crawler
Behind Cloudflare
No