CrawlerPulse
CrawlerPulse / Asia / India / filmfare.com

FilmFare.com

filmfare.com · India · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Blocked
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,823 bytes. 6 rule groups name AI crawlers; shown first.

User-Agent: GPTBot
Allow: /

User-Agent: ChatGPT-User
Allow: /

User-Agent: OAI-SearchBot
Allow: /

User-Agent: ClaudeBot
Allow: /

User-Agent: Google-Extended
Disallow: /

User-Agent: CCBot
Disallow: /
Sitemap: https://www.filmfare.com/sitemap.xml
Sitemap: https://www.filmfare.com/gImageSiteMap.xml
Sitemap: https://www.filmfare.com/google-news-sitemap.xml
Sitemap: https://www.filmfare.com/news-sitemap.xml
Sitemap: https://www.filmfare.com/news-sitemap-2.xml
Sitemap: https://www.filmfare.com/news-sitemap-3.xml
Sitemap: https://www.filmfare.com/news-sitemap-4.xml
Sitemap: https://www.filmfare.com/news-sitemap-5.xml
Sitemap: https://www.filmfare.com/amp-stories-sitemap.xml
Sitemap: https://www.filmfare.com/features-sitemap.xml
Sitemap: https://www.filmfare.com/photos-sitemap.xml
Sitemap: https://www.filmfare.com/photos-sitemap-2.xml
Sitemap: https://www.filmfare.com/reviews-sitemap.xml
Sitemap: https://www.filmfare.com/interviews-sitemap.xml
Sitemap: https://www.filmfare.com/celeb-lifestyle-and-fashion-sitemap.xml
Sitemap: https://www.filmfare.com/nostalgia-sitemap.xml
Sitemap: https://www.filmfare.com/south-sitemap.xml
Sitemap: https://www.filmfare.com/videos-sitemap.xml
Sitemap: https://www.filmfare.com/filmfare-awards-sitemap.xml
Sitemap: https://www.filmfare.com/author-sitemap.xml
Sitemap: https://www.filmfare.com/what-to-watch-sitemap.xml
Sitemap: https://www.filmfare.com/upcoming-movies-sitemap.xml

User-Agent: *
Allow: /
Disallow: /21806551354/*
Disallow: /SolrApi/
Disallow: /crons/
Disallow: /inc/
Disallow: /tpl/
Disallow: /config/
Disallow: /api/
Disallow: /searchlogs/
Disallow: /classes/
Disallow: /CommentModerator/
Disallow: /fonts/
Disallow: /2db/
Disallow: /ssonew/
Disallow: /sso/
Disallow: /search/*

User-Agent: Bingbot
Allow: /

User-Agent: Screaming Frog SEO Spider
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

India
25.0% block at least one AI crawler
Behind Cloudflare
No