CrawlerPulse
CrawlerPulse / Asia / India / prajavani.net

Prajavani

prajavani.net · India · Deccan Herald · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Allowed
Common Crawl
Blocked
ByteDance
Blocked
Apple
Allowed
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 3,957 bytes. 11 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: Bytespider-webpage
Disallow: /

User-agent: Amazonbot
Disallow: /

Sitemap: https://www.prajavani.net/sitemap.xml
Sitemap: https://www.prajavani.net/news_sitemap.xml

User-agent: Mediapartners-Google
Allow: /

User-agent: SeekportBot
Disallow: /

User-agent: Yandex
Disallow: /

User-agent: GPTCrawler
Disallow: /

User-agent: Perplexity-User
Disallow: /

User-agent: *
Allow: /
Disallow: /*.woff$
Disallow: /includes/
Disallow: /modules/
Disallow: /profiles/
Disallow: /scripts/
Disallow: /themes/
Disallow: /CHANGELOG.txt
Disallow: /cron.php
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /INSTALL.sqlite.txt
Disallow: /install.php
Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
Disallow: /MAINTAINERS.txt
Disallow: /update.php
Disallow: /UPGRADE.txt
Disallow: /xmlrpc.php
Disallow: /newsletter-ads.html
Disallow: /admin/
Disallow: /comment/reply/
Disallow: /filter/tips/
Disallow: /node/add/
Disallow: /search/
Disallow: /user/register/
Disallow: /user/password/
Disallow: /user/login/
Disallow: /user/logout/
Disallow: /taxonomy/*
Disallow: /node/*
Disallow: /?q=admin/
Disallow: /?q=comment/reply/
Disallow: /?q=filter/tips/
Disallow: /?q=node/add/
Disallow: /?q=search/
Disallow: /?q=user/password/
Disallow: /?q=user/register/
Disallow: /?q=user/login/
Disallow: /?q=user/logout/
Disallow: /updateemoji
Disallow: /updatestats
Disallow: /updateemoji*
Disallow: /updatestats*
Disallow: /crosswords
Disallow: /app-sports-lp
Disallow: /app-home-lp
Disallow: /app-business-lp
Disallow: /app-abhimata-lp
Disallow: /app-art-literature-lp
Disallow: /app-farming-lp
Disallow: /app-suddhi-lp
Disallow: /app-tech-lp
Disallow: /app-food-lp
Disallow: /app-pv-special-lp
Disallow: /app-photo-lp
Disallow: /app-video-lp
Disallow: /app-monoranjane-home-lp
Disallow: /izooto.html
Disallow: /tags/app-photo
Disallow: /tags/app-video
Disallow: /tags/app-notification
Disallow: /tags/app-settings
Disallow: /tags/app-feedback
Disallow: /tags/app-ratting
Disallow: /tags/app-version
Disallow: /tags/app-hamburger-more-option
Disallow: /app-home-sports-top-slider
Disallow: /app-home-business-top-slider
Disallow: /app-home-topslider-abhimata
Disallow: /app-home-topslider-art-literature
Disallow: /app-home-top-slider-farming
Disallow: /app-home-top-slider-sudhi
Disallow: /app-manoranjane-top-slider
Disallow: /latest-album
Disallow: /karnataka-album
Disallow: /bharata-video
Disallow: /cinema-photos
Disallow: /delete-account
Disallow: /.well-known/assetlinks.json
Disallow: /banner/live
Disallow: /app-home-food-topslider
Disallow: /app-arts-lp-video-slider
Disallow: /app-business-lp-video
Disallow: /app-business-lp-photo
Disallow: /app-abhimata-lp-photo
Disallow: /app-abhimata-lp-video
Disallow: /app-sports-lp-photo
Disallow: /app-sports-lp-video
Disallow: /app-home-tech-top-slider
Disallow: /app-tech-lp-video
Disallow: /app-tech-lp-photo
Disallow: /app-home-food-top-slider
Disallow: /app-food-lp-video
Disallow: /app-food-lp-photo
Disallow: /app-election-home
Disallow: /app-karunaada-vaibhava
Disallow: /apple-app-site-association
Disallow: /appelections-2024
Disallow: /limited_time_999
Disallow: /pv-faq
Disallow: /pv-testimonial
Disallow: /limited_time_149
Disallow: /sub-testimonial
Disallow: /sub-faq
Disallow: /campaign.json
Disallow: /subscription-banner
Disallow: /subscription-paywall

History

Changes recorded since June 2026.

No change recorded.

In context

India
25.0% block at least one AI crawler
Behind Cloudflare
Yes
Same group