CrawlerPulse
CrawlerPulse / North America / United States / benzinga.com

Benzinga

benzinga.com · United States · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,472 bytes. No rule names an AI crawler.

Sitemap: https://www.benzinga.com/sitemap.xml
Sitemap: https://www.benzinga.com/sec/insider-trades/sitemap.xml
Sitemap: https://www.benzinga.com/sitemap_index.xml
Sitemap: https://www.benzinga.com/sitemap-benzinga.com/www/archive.xml
Sitemap: https://www.benzinga.com/sitemap-benzinga.com/www/latest.xml
Sitemap: https://www.benzinga.com/sitemap-benzinga.com/www/quotes.xml
Sitemap: https://www.benzinga.com/sitemap-benzinga.com/www/authors.xml
Sitemap: https://www.benzinga.com/sitemap-benzinga.com/www/googlenews.xml

User-agent: *
Disallow: /*?q=
Disallow: /*&q=
Allow: /ads.txt
Allow: /app-ads.txt
Disallow: /ads.
Disallow: /admin/
Disallow: /bzfeed
Disallow: /comment/reply/
Disallow: /filter/tips/
Disallow: /go/
Disallow: /i/
Disallow: /includes/
Disallow: /logout/
Disallow: /misc/
Disallow: /modules/
Disallow: /money/attachments/
Disallow: /money/products/
Disallow: /money/widgets/
Disallow: /node/add/
Disallow: /profiles/
Disallow: /scripts/
Disallow: /themes/
Disallow: /user/
Disallow: /web-stories/
Disallow: /pressreleases
Disallow: /secfilings
Disallow: /boost_stats.php
Disallow: /CHANGELOG.txt
Disallow: /cron.php
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /install.php
Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
Disallow: /MAINTAINERS.txt
Disallow: /update.php
Disallow: /UPGRADE.txt
Disallow: /xmlrpc.php
Disallow: /wp-admin/
Disallow: /wp-login.php

User-agent: atlassian-bot
Allow: /apis/
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

United States
63.2% block at least one AI crawler
Behind Cloudflare
Yes