CrawlerPulse
CrawlerPulse / Europe / France / em-consulte.com

EM-CONSULTE

em-consulte.com · France · ELSEVIER MASSON · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 5,175 bytes. 3 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: teleport
Disallow: /

User-agent: wget
Disallow: /

User-agent: webzip
Disallow: /

User-agent: webcopier
Disallow: /

User-agent: offline
Disallow: /

User-agent: webbandit
Disallow: /

User-agent: webstripper
Disallow: /

User-agent: sitesnagger
Disallow: /

User-agent: website
Disallow: /

User-agent: teleportpro
Disallow: /

User-agent: webstripper
Disallow: /

User-agent: trendkite-akashic-crawler
Disallow: /

User-agent: Seekport Crawler
Disallow: /

User-agent: crescent
Disallow: /

User-agent: emailsiphon
Disallow: /

User-agent: extractorpro
Disallow: /

User-agent: emailcollector
Disallow: /

User-agent: emailwolf
Disallow: /

User-agent: nicerspro
Disallow: /

User-agent: scooter	 	# Robot d'Astalavista
Disallow: /

User-agent: Yandex
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: sogou spider
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: SEOkicks-Robot
Disallow: /

User-agent: * 			# match all other bots.
Crawl-delay: 10			# number of seconds to wait between successive requests to the same server, by crawlers.
Visit-time: 0000-1200	# only visit between 00:00 and 12:00 UTC (GMT)
Disallow: /META-INF
Disallow: /WEB-INF
Disallow: /robots.txt
Disallow: /backoffice
Disallow: /templates/common/css
Disallow: /templates/common/js
Disallow: /templates/common/framworks
Disallow: /redirectIdpServlet
Disallow: /it/redirectIdpServlet
Disallow: /es/redirectIdpServlet
Disallow: /neurochirurgie/redirectIdpServlet
Disallow: /clinicalkeynowsub/redirectIdpServlet
Disallow: /resetpwd
Disallow: /it/resetpwd
Disallow: /es/resetpwd
Disallow: /ajax
Disallow: /emailing/redirection
Disallow: /showarticlefile
Disallow: /module/addtobookmarks 	# Pour éviter l'indexation de /module/addtobookmarks mais aussi /module/addtobookmarks/ et tout ce qui pourrait suivre dans l'URL...
Disallow: /module/addtomyalerts
Disallow: /module/impression
Disallow: /module/lastviewedarticles
Disallow: /module/recommendthispublication
Disallow: /module/sendtoafriend
Disallow: /mg-consulte/
Disallow: /mgconsulte/
Disallow: /monpanier
Disallow: /es/backoffice
Disallow: /es/templates/common/css
Disallow: /es/templates/common/js
Disallow: /es/templates/common/framworks
Disallow: /es/ajax
Disallow: /es/emailing/redirection
Disallow: /es/showarticlefile
Disallow: /es/module/addtobookmarks 	# Pour éviter l'indexation de /module/addtobookmarks mais aussi /module/addtobookmarks/ et tout ce qui pourrait suivre dans l'URL...
Disallow: /es/module/addtomyalerts
Disallow: /es/module/impression
Disallow: /es/module/lastviewedarticles
Disallow: /es/module/recommendthispublication
Disallow: /es/module/sendtoafriend
Disallow: /es/mg-consulte/
Disallow: /es/mgconsulte/
Disallow: /es/monpanier
Disallow: /en/templates/common/css
Disallow: /en/templates/common/js
Disallow: /en/templates/common/framworks
Disallow: /en/ajax
Disallow: /en/emailing/redirection
Disallow: /en/showarticlefile
Disallow: /en/module/addtobookmarks 	# Pour éviter l'indexation de /module/addtobookmarks mais aussi /module/addtobookmarks/ et tout ce qui pourrait suivre dans l'URL...
Disallow: /en/module/addtomyalerts
Disallow: /en/module/impression
Disallow: /en/module/lastviewedarticles
Disallow: /en/module/recommendthispublication
Disallow: /en/module/sendtoafriend
Disallow: /en/mg-consulte/
Disallow: /en/mgconsulte/
Disallow: /en/monpanier
Disallow: /it/backoffice
Disallow: /it/templates/common/css
Disallow: /it/templates/common/js
Disallow: /it/templates/common/framworks
Disallow: /it/ajax
Disallow: /it/emailing/redirection
Disallow: /it/showarticlefile
Disallow: /it/module/addtobookmarks 	# Pour éviter l'indexation de /module/addtobookmarks mais aussi /module/addtobookmarks/ et tout ce qui pourrait suivre dans l'URL...
Disallow: /it/module/addtomyalerts
Disallow: /it/module/impression
Disallow: /it/module/lastviewedarticles
Disallow: /it/module/recommendthispublication
Disallow: /it/module/sendtoafriend
Disallow: /it/mg-consulte/
Disallow: /it/mgconsulte/
Disallow: /it/monpanier

History

Changes recorded since June 2026.

No change recorded.

In context

France
42.3% block at least one AI crawler
Behind Cloudflare
Yes