space.com · United States · measured 9 Oct 2026
As read on 9 Oct 2026, 1,679 bytes. 1 rule group name AI crawlers; shown first.
User-agent: bytespider User-agent: mistralai User-agent: cohere User-agent: ai2bot User-agent: youbot User-agent: omgili User-agent: diffbot User-agent: kangaroo User-agent: img2dataset User-agent: amazonbot User-agent: amazon-qbusiness User-Agent: meta-externalagent User-Agent: meta-webindexer Disallow: / User-agent: * Disallow: */deals/compare Disallow: */html/ Disallow: */p/*/embed/captioned Disallow: */outlink* Disallow: *.freyr.* Disallow: */comments/* Disallow: */.events-handler* Disallow: *CoreWebVitalsThresholdEvent* Disallow: */partials/* Disallow: */future.adtech.freyr.v1.PageViewEvent Disallow: *searchTerm=* Disallow: *sortBy=* Disallow: *productBrand=* Disallow: *%7B*%7D* Disallow: *seenMatchId=* Disallow: /infinite-scroll-article/* Disallow: /infinite-scroll-review/* Disallow: /infinite-scroll-recipe/* User-agent: AmazonAdBot Disallow: User-Agent: * Disallow: /search.php Disallow: /social.php Disallow: /newsletter-signup Disallow: /_proxy* User-agent: * Sitemap: https://www.space.com/sitemap.xml Sitemap: https://www.space.com/sitemap-news.xml
Changes recorded since June 2026.
No change recorded.