CrawlerPulse
CrawlerPulse / Europe / United Kingdom / autosport.com

Autosport

autosport.com · United Kingdom · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 1,610 bytes. No rule names an AI crawler.

User-agent: Mediapartners-Google
Allow: /

User-agent: Yandex
Allow: /rss/yandex-zen/news/
Disallow: *?q=
Disallow: */?filters
Disallow: */videos/add-your-video/
Disallow: */photos/user-uploads-albums/
Disallow: */?edition=
Disallow: /srv.php
Disallow: /index.php
Disallow: /disqus
Allow: /f1/news/?q=giorgio%20piola
Host: www.autosport.com
Sitemap: https://www.autosport.com/sitemap.xml
Disallow: /driver/*/bio/
Disallow: /v/*
Disallow: /lt/*
Disallow: /dashboard/*
Disallow: /search/*
Allow:    /*/photos/
Allow:    /*/photos/*
Disallow: /*/photos/*/*

User-agent: Googlebot
Disallow: *?q=
Disallow: */videos/add-your-video/
Disallow: */photos/user-uploads-albums/
Disallow: */?edition=
Disallow: *?_openstat=
Disallow: /thewall2
Disallow: /srv.php
Disallow: /index.php
Disallow: /disqus
Allow: /f1/news/?q=giorgio%20piola
Allow: /design/*
Disallow: /driver/*/bio/
Sitemap: https://www.autosport.com/sitemap.xml
Disallow: /v/*
Disallow: /lt/*
Disallow: /dashboard/*
Disallow: /search/*
Disallow: /search/photos/*
Disallow: */party-poker/get-odds/*
Disallow: */first-article/*
Disallow: */coral/*
Disallow: /info/about-us/all/*
Disallow: /live-timing/widget/*
Disallow: /live-tv/*

User-agent: proximic
Disallow: /

User-agent: AmazonAdBot
Allow: /

User-Agent: Amazon-Advertising-ad-standards-bot/1.0
User-Agent: APS-ad-standards-bot/1.0
Allow: /ads.txt

User-agent: *
Disallow: /videos/embed/
Disallow: /driver/*/bio/
Disallow: /v/*
Disallow: /lt/*
Disallow: /dashboard/*
Disallow: /search/*
Disallow: */party-poker/get-odds/*
Disallow: */first-article/*
Disallow: /live-timing/widget/*
Disallow: /live-tv/*

History

Changes recorded since June 2026.

No change recorded.

In context

United Kingdom
73.8% block at least one AI crawler
Behind Cloudflare
No