CrawlerPulse
CrawlerPulse / Asia / South Korea / ajunews.com

ajunews.com

ajunews.com · South Korea · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 633 bytes. 1 rule group name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /
Sitemap: https://www.ajunews.com/sitemap_section.xml
Sitemap: https://www.ajunews.com/sitemap.php
Sitemap: https://www.ajunews.com/image_sitemap.php
Sitemap: https://www.ajunews.com/sitemap/xml/sitemap_news_index_ym.xml
Sitemap: https://www.ajunews.com/sitemap/xml/sitemap_image_index_ym.xml

User-agent: *
Disallow: /search
Disallow: /keyword
Disallow: /gen/
Disallow: /util/
Disallow: /ajax/
Disallow: /ext/
Disallow: /info/
Disallow: /log/
Disallow: /img/ads/
Disallow: /election2020/
Disallow: /view_expand/
Disallow: /view/20200404130938913
Disallow: /discovery/

History

Changes recorded since June 2026.

No change recorded.

In context

South Korea
34.6% block at least one AI crawler
Behind Cloudflare
No