vg247.com · United Kingdom · measured 9 Oct 2026
As read on 9 Oct 2026, 5,606 bytes. 1 rule group name AI crawlers; shown first.
User-agent: "008" User-agent: AddSearchBot User-agent: AgentDataBot User-agent: AI2Bot User-agent: Ai2Bot-DeepResearchEval User-agent: Ai2Bot-Dolma User-agent: AIWebIndex User-agent: AIWebIndex-Agent User-agent: amazon-kendra User-agent: amazon-QBusiness User-agent: Amazonbot User-agent: AmazonBuyForMe User-agent: Amzn-SearchBot User-agent: Amzn-User User-agent: Anchor Browser User-agent: AndiAgent User-agent: anthropic-ai User-agent: ApifyBot User-agent: ApifyWebsiteContentCrawler User-agent: Applebot User-agent: Applebot-extended User-agent: atlassian-bot User-agent: AutoRAG User-agent: AwarioSmartBot User-agent: AzureAI-SearchBot User-agent: BeansLLM-CorpusBot User-agent: bedrockbot User-agent: Big Sur AI User-agent: bigsur.ai User-agent: BixelBot User-agent: Brandwatch User-agent: Bravebot User-agent: Brightbot User-agent: Bytespider User-agent: CCBot User-agent: Channel3Bot User-agent: ChatGLM-Spider User-agent: ChatGPT-User User-agent: Claude-Code User-agent: Claude-SearchBot User-agent: Claude-User User-agent: Claude-Web User-agent: ClaudeBot User-agent: Cloudflare-AutoRAG User-agent: CloudflareBrowserRenderingCrawler User-agent: Code User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: CohereBot User-agent: Cotoyogi User-agent: CragCrawler User-agent: Crawl4AI User-agent: Crawlspace User-agent: Cursor User-agent: Datenbank Crawler User-agent: Datenbank-Crawler User-agent: DDM-DCipher* User-agent: DDM-DCipher/1.0.7 User-agent: DDM* User-agent: DeepSeekBot User-agent: DesearchBot User-agent: Devin User-agent: Diffbot User-agent: Direqt Anomura User-agent: Dola-SearchBot User-agent: Doubaobot User-agent: DuckAssistBot User-agent: Echobot Bot User-agent: ERNIEBot User-agent: ExaBot User-agent: ExaSearchBot User-agent: FacebookBot User-agent: Factset_spyderbot User-agent: FirecrawlAgent User-agent: GeistHaus-PageFetcher User-agent: GPTBot User-agent: grok-agent User-agent: GrokBot User-agent: HenkBot User-agent: HTTrack User-agent: Hunyuan User-agent: iAsk User-agent: iAskBot User-agent: iaskspider User-agent: ICC Crawler User-agent: ICC-Crawler User-agent: ImageBot User-agent: ImageSiftBot User-agent: imageSpider User-agent: IsonSearchBot User-agent: kagi-fetcher User-agent: Kangaroo-Bot User-agent: Keenable-User User-agent: KeenableBot User-agent: Kimi-Agent User-agent: Kimi-SearchBot User-agent: Kimi-User User-agent: KimiBot User-agent: KimiCrawler User-agent: KlaviyoAIBot User-agent: Kunato User-agent: KunatoCrawler User-agent: laion-huggingface-processor User-agent: LCC User-agent: LINER Bot User-agent: LinkupBot User-agent: Manus-User User-agent: meta-externalagent User-agent: meta-externalfetcher User-agent: meta-webindexer User-agent: micro-crawl User-agent: mistral.ai User-agent: MistralAI-Index User-agent: MistralAI-User User-agent: MistralBot User-agent: MoonshotBot User-agent: MoonSpider User-agent: Mozilla-Tabstack User-agent: netEstate Imprint Crawler User-agent: NovaAct User-agent: Novellum AI Crawl User-agent: Nutch User-agent: Novellum User-agent: OAI-SearchBot User-agent: Offline Explorer User-agent: omgili User-agent: opencode User-agent: PanguBot User-agent: PeopleInc-DCipher-Scraper/1.1.0 User-agent: Perplexity-User User-agent: PerplexityBot User-agent: PetalBot User-agent: PhindBot User-agent: Poggio-Citations User-agent: qodercli User-agent: QualifiedBot User-agent: Querit-SearchBot User-agent: QueritBot User-agent: QwenBot User-agent: recraft-image-dataset User-agent: Reflectionbot User-agent: SalamandraVLM User-agent: SBIntuitionsBot User-agent: ScryBot User-agent: Scrapy User-agent: SeekrBot User-agent: SemrushBot-OCOB User-agent: SemrushBotSwa User-agent: Shap-User User-agent: ShapBot User-agent: SofyaBot User-agent: SomethinggBot User-agent: Spider User-agent: TalarionFrontierCrawler User-agent: TalarionSearchBot User-agent: TavilyBot User-agent: TerraCotta User-agent: Timpibot User-agent: TongyiBot User-agent: Trae User-agent: TwinAgent User-agent: UseAI User-agent: Velen Crawler User-agent: VelenPublicWebCrawler User-agent: Webzio-Extended User-agent: Wrtn User-agent: WRTNBot User-agent: xAI-Grok User-agent: xAI-SearchBot User-agent: YandexAdditionalBot User-agent: YepBot User-agent: YiBot User-agent: YiyanBot User-agent: YouBot User-agent: ZanistaBot Allow: /how-to-achieve-the-best-graphics-on-samsung-galaxy-s26 Disallow: / User-agent: * Disallow: /search?q= Disallow: /search/ Disallow: /community/ Disallow: /*?view=comments Disallow: /*?ajax=1 Disallow: /*?preview_key= Disallow: /mypop/ Disallow: /for-you/ Disallow: /competition/ Sitemap: https://www.vg247.com/sitemap.xml Sitemap: https://www.vg247.com/sitemap-news.xml Sitemap: https://www.vg247.com/sitemap-videogames.xml Sitemap: https://www.vg247.com/sitemap-authors.xml
Changes recorded since June 2026.
No change recorded.