yabiladi.com · Morocco · measured 8 Oct 2026
As read on 8 Oct 2026, 1,145 bytes. 4 rule groups name AI crawlers; shown first.
User-agent: GPTBot Disallow: / User-agent: Bytespider Disallow: / User-agent: PetalBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: 008 Disallow: / User-agent: proximic Disallow: / User-agent: Exabot Disallow: / User-agent: MJ12bot Disallow: / User-agent: bingbot Crawl-delay: 1 User-agent: * Allow: / Disallow: /articles/printview/ Disallow: /mobile/ Disallow: /prieres2/ Disallow: /account/ Disallow: /comments/ Disallow: /forum/control/ Disallow: /rencontres/account/ Disallow: /old/ Disallow: /es/ Disallow: /ar/ Disallow: /pub.php Disallow: /adsense.php Disallow: /forum/search.php Disallow: /m/ Disallow: /articles/search/ Disallow: /articles/advanced/ Disallow: /rencontres/profile.php Disallow: /files/radio Disallow: /mp Disallow: /nav Disallow: /annonces/redirection/ User-agent: Claude-SearchBot Disallow: /*.html.html User-agent: * Disallow: /*.html.html User-agent: Sogou Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: SofyaBot Disallow: / User-agent: Mediapartners-Google Allow: / User-agent: Claude-SearchBot Allow: /
Changes recorded since June 2026.
No change recorded.