CrawlerPulse
CrawlerPulse / Asia / Lebanon / lebanon24.com

lebanon24.com

lebanon24.com · Lebanon · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 2,080 bytes. No rule names an AI crawler.

User-agent: *
Allow: /
Disallow: /thewall/
Disallow: /admin/
Disallow: /backend/
Disallow: /Handler/
Disallow: /webservices/
Disallow: /App_Data/
Disallow: /bin/
Disallow: /obj/
Disallow: /packages/
Disallow: /uploadImages/temp/
Disallow: /StyleGuide.aspx
Disallow: /WebForm1.aspx
Disallow: /OldLinks.aspx
Disallow: /controls/
Disallow: /*?*utm_source=*
Disallow: /*?*utm_medium=*
Disallow: /*?*utm_campaign=*
Disallow: /*?*LoadIndex=*
Disallow: /*?*preview=*
Allow: /news/
Allow: /article/
Allow: /category/
Allow: /images/
Allow: /Style/
Allow: /JS/
Allow: /YoutubePlayer/
Allow: /ads.txt
Sitemap: https://www.lebanon24.com/sitemap.xml
Sitemap: https://www.lebanon24.com/newssitemap.xml
Sitemap: https://www.lebanon24.com/imagessitemap.xml
Sitemap: https://www.lebanon24.com/videossitemap.xml
Sitemap: https://www.lebanon24.com/sitemap-archive.xml
Sitemap: https://www.lebanon24.com/sitemaps/sitemap-495001-500000.xml
Sitemap: https://www.lebanon24.com/sitemaps/newssitemap-495001-500000.xml
Sitemap: https://www.lebanon24.com/sitemaps/sitemap-490001-495000.xml
Sitemap: https://www.lebanon24.com/sitemaps/newssitemap-490001-495000.xml
Sitemap: https://www.lebanon24.com/sitemaps/sitemap-485001-490000.xml
Sitemap: https://www.lebanon24.com/sitemaps/newssitemap-485001-490000.xml

User-agent: Googlebot
Allow: /
Crawl-delay: 0

User-agent: Googlebot-News
Allow: /
Crawl-delay: 0

User-agent: Bingbot
Allow: /
Crawl-delay: 1

User-agent: Slurp
Allow: /
Crawl-delay: 1

User-agent: facebookexternalhit
Allow: /
Crawl-delay: 0

User-agent: Facebot
Allow: /
Crawl-delay: 0

User-agent: Twitterbot
Allow: /
Crawl-delay: 0

User-agent: *
Disallow: https://cdn.lebanon24.com/

History

Changes recorded since June 2026.

No change recorded.

In context

Lebanon
8.7% block at least one AI crawler
Behind Cloudflare
Yes