cntraveler.com · United States · Conde Nast US · measured 8 Oct 2026
As read on 8 Oct 2026, 2,285 bytes. 1 rule group name AI crawlers; shown first.
User-agent: Amazonbot User-agent: Applebot-Extended User-agent: archive.org_bot User-agent: Bravebot User-agent: Brightbot 1.0 User-agent: Bytespider User-agent: CCBot User-agent: Claude-SearchBot User-agent: Claude-User User-agent: ClaudeBot User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: Diffbot User-agent: DuckAssistBot User-agent: factset_spyderbot User-agent: FirecrawlAgent User-agent: Google-CloudVertexBot User-agent: Google-Extended User-agent: GoogleOther User-agent: heritrix User-agent: ia_archiver User-agent: ia_archiver-web.archive.org User-agent: meta-externalagent User-agent: meta-externalfetcher User-agent: meta-webindexer User-agent: MistralAI-User User-agent: PanguBot User-agent: Perplexity-User User-agent: PerplexityBot User-agent: PetalBot User-agent: Scrapy User-agent: Shap-User User-agent: ShapBot User-agent: Timpibot User-agent: Yisouspider User-agent: YouBot User-agent: Webzio User-agent: Webzio-Extended Disallow: / Sitemap: https://www.cntraveler.com/sitemap.xml Sitemap: https://www.cntraveler.com/tagpages-sitemap.xml Sitemap: https://www.cntraveler.com/categories-sitemap.xml?hierarchy=location Sitemap: https://www.cntraveler.com/contributors-sitemap.xml Sitemap: https://www.cntraveler.com/bundles-sitemap.xml Sitemap: https://www.cntraveler.com/branded-sitemap.xml Sitemap: https://www.cntraveler.com/feed/google-latest-news/sitemap-google-news Sitemap: https://www.cntraveler.com/feed/rss User-agent: * Disallow: /*? Allow: /*?page Allow: /*.xml? Allow: /*.js Allow: /*.css Allow: /*.ico Allow: /verso/static Allow: /*.svg Allow: /*.png Allow: /*.jpg Allow: /*.jpeg Allow: /*rss? Allow: /*?id= Disallow: /auth/ Disallow: /account/ Disallow: /user/ Disallow: /user-context Disallow: /preview/ Disallow: /search Disallow: /product/ Disallow: /cdn-cgi/ Disallow: /services.min.js Disallow: /com.condenast/yv8 Disallow: /reject-all Disallow: */c_limit/ Allow: /v2/offers/cnta Allow: /*?*utm_medium=social Allow: /*?*utm_medium=syndication Disallow: /*?filter Disallow: /itineraries/ User-agent: AmazonAdBot User-agent: FacebookExternalHit User-agent: Google-InspectionTool User-agent: LinkedInBot User-agent: Pinterestbot User-agent: Storebot-Google User-agent: TikTokSpider User-agent: Twitterbot Allow: /
Changes recorded since June 2026.
No change recorded.