segye.com · South Korea · measured 8 Oct 2026
As read on 8 Oct 2026, 1,213 bytes. 3 rule groups name AI crawlers; shown first.
User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Googlebot Allow: / User-agent: Googlebot-News Allow: / User-agent: Googlebot-Image Allow: / User-agent: Googlebot-Mobile Allow: / User-agent: Mediapartners-Google Allow: / User-agent: Googlebot-News Allow: / User-agent: Facebot Allow: / User-agent: Bingbot Allow: / User-agent: facebookexternalhit/1.1 Allow: / User-agent: Twitterbot Allow: / User-agent: MSNBot Allow: / User-agent: MSNBot-Media Allow: / User-agent: Yeti Allow: / User-agent: Daumoa Allow: / User-agent: popIn_Agent Allow: / User-agent: Google Search Console Allow: / User-agent: Googlebot/2.1 Allow: / User-agent: Googlebot-Smartphone Allow: / User-agent: Google- Extended Allow: / User-agent: Applebot- Extended Allow: / User-agent: * Allow: / User-agent: Sogou web spider Disallow: / User-agent: Sogou inst spider Disallow: / User-agent: Sogou spider2 Disallow: / User-agent: Baiduspider Disallow: / User-agent: Baiduspider-image Disallow: / User-agent: Baiduspider-video Disallow: / User-agent: Baiduspider-news Disallow: / sitemap: https://www.segye.com/Sitemap_index.xml sitemap: https://www.segye.com/sitemap_img_index.xml
Changes recorded since June 2026.
No change recorded.