CrawlerPulse
CrawlerPulse / Europe / United Kingdom / thestage.co.uk

The Stage

thestage.co.uk · United Kingdom · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 2,663 bytes. No rule names an AI crawler.

User-agent: *
Crawl-delay: 10
Sitemap: https://www.thestage.co.uk/SiteMap_TheStage_index.xml
Sitemap: https://www.thestage.co.uk/TheStageNewsSitemap.xml
Disallow: /AcuCustom/Sitename/Fonts/
Disallow: /AfcTool/fontawesome/webfonts/
Disallow: /AfcTemp/DesignCentre/
Disallow: /*?utm_source=*
Disallow: /*?fbclid=*
Disallow: /*?Preview=
Disallow: /*?preview=
Disallow: /dynamic-forms
Disallow: /online-forms
Disallow: /afotrack
Disallow: /personal-message-campaign-channel
Disallow: /search?
Disallow: /login
Disallow: /ao-d-f-t-link
Disallow: /preview/
Disallow: /AcuCustom/Sitename/Applications/
Disallow: /AcuCustom/Sitename/Assets/
Disallow: /AcuCustom/Sitename/Documents/
Disallow: /AcuCustom/Sitename/Fonts/
Disallow: /AcuCustom/Sitename/Skins/
Disallow: /AcuCustom/Sitename/Styling/
Disallow: /AcuCustom/Sitename/Templates/
Disallow: /AcuCustom/Sitename/Users/
Disallow: /AcuGeoIP/
Disallow: /AcuLicence/
Disallow: /AcuPatches/
Disallow: /AcuStorageMonitor/
Disallow: /AcuSystem/
Disallow: /AcuUpdate/
Disallow: /AfcAutomation/
Disallow: /AfcChannel/
Disallow: /AfcCommunityMgr/
Disallow: /AfcControl/
Disallow: /AfcCRM/
Disallow: /AfcDesignCentre/
Disallow: /AfcEdition/
Disallow: /AfcEmail/
Disallow: /AfcEngine/
Disallow: /AfcForm/
Disallow: /AfcLicence/
Disallow: /AfcLiveEdit/
Disallow: /AfcLogin/
Disallow: /AfcMediaLibrary/
Disallow: /AfcMessage/
Disallow: /AfcMyMessages/
Disallow: /AfcRecruitment/
Disallow: /AfcRegistration/
Disallow: /AfcRelated/
Disallow: /AfcRest/
Disallow: /AfcScript/
Disallow: /AfcSearch/
Disallow: /AfcSecurity/
Disallow: /AfcSiteMap/
Disallow: /AfcStandard/
Disallow: /AfcStorageMonitor/
Disallow: /AfcStyle/
Disallow: /AfcTemp/DesignCentre/cus.cfm
Disallow: /AfcTool/fontawesome/webfonts/
Disallow: /AfcTool/fontawesome/webfonts5.5.0/
Disallow: /AfcUpdate/
Disallow: /AfcUpdater/
Disallow: /AfoCampaign/
Disallow: /AfoCommentAndRating/
Disallow: /AfoCommunity/
Disallow: /AfoComparison/
Disallow: /AfoContact/
Disallow: /AfoContentSubscription/
Disallow: /AfoDynamicForm/
Disallow: /AfoECard/
Disallow: /AfoECommerce/
Disallow: /AfoExternalIntegration/
Disallow: /AfoForum/
Disallow: /AfoMediaWave/
Disallow: /AfoMyInterests/
Disallow: /AfoMyLibrary/
Disallow: /AfoNewsletter/
Disallow: /AfoOnlineForm/
Disallow: /AfoPromotion/
Disallow: /AfoSiteAnalysis/
Disallow: /AfoSocialBookMark/
Disallow: /AfoSyndicationAndAggregation/
Disallow: /AfoTaxonomyMgr/
Disallow: /AfoTeamTime/
Disallow: /AfoWave/
Disallow: /afotrack/
Disallow: /CampaignStatistics/
Disallow: /com/
Disallow: /control/
Disallow: /Control/
Disallow: /AfcFormWidgetJS.js
Disallow: /AfcUser/cfc/CrossDomainLogin.cfc
Disallow: /AfcUser/cfc/US_LoginDisplay.cfc

History

Changes recorded since June 2026.

No change recorded.

In context

United Kingdom
73.8% block at least one AI crawler
Behind Cloudflare
No