mingpao.com · Hong Kong · measured 8 Oct 2026
As read on 8 Oct 2026, 1,293 bytes. 2 rule groups name AI crawlers; shown first.
User-Agent: ImagesiftBot Disallow: / User-agent: PetalBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: IndeedBot Disallow: / User-agent: LinkedInBot Disallow: / User-agent: 008 Disallow: / User-agent: JobdiggerSpider Disallow: / User-agent: Cliqzbot Disallow: / User-agent: Slurp Disallow: / User-agent: Baiduspider Disallow: / User-agent: Yandex Disallow: / User-agent: Teoma Disallow: / User-agent: FAST-WebCrawler Disallow: / User-agent: GurujiBot Disallow: / User-agent: ia_archiver Disallow: / User-agent: Exabot Disallow: / User-agent: Soso Spider Disallow: / User-agent: DotBot Disallow: User-agent: DuckDuckBot Disallow: / User-agent: Siteliner Disallow: / User-agent: Curious George Disallow: / User-agent: * Allow: / User-agent: TTD-Content Allow: / User-agent: grapeshot Disallow: User-agent: AmazonAdBot Disallow: Crawl-Delay: 1 User-agent: AdsTxtCrawler/1.0 Allow: /ads.txt User-agent: * Disallow: *dummy=*$ Disallow: submit= Disallow: /php/manage/ Disallow: /php/class/ Disallow: /dat/ Disallow: /htm/dummy/ Disallow: /*?*addetails=*
Changes recorded since June 2026.
No change recorded.