CrawlerPulse
CrawlerPulse / Europe / Netherlands / babybytes.nl

Babybytes

babybytes.nl · Netherlands · PXR · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,416 bytes. No rule names an AI crawler.

User-agent: *
Disallow: /includes/
Disallow: /vip/
Disallow: /vip-kids/
Disallow: /vip-photo/
Disallow: /vip-reacties/
Disallow: /images/*.jpg
Disallow: /poll/
Disallow: /areaedit/
Disallow: /autoComplete/
Disallow: /Avatars/
Disallow: /cron/
Disallow: /Forum/box_new_comments_in_my_groups-compact.php
Disallow: /Forum/box_new_comments_in_my_groups-new.php
Disallow: /Forum/box_sidetabs.php
Disallow: /Forum/box_topic_comments.php
Disallow: /kaartje2go/
Disallow: /vlinderpagina/
Disallow: *?*mnf=*
Disallow: *?*mygroups=*
Disallow: *&jumpto=*
Disallow: /Forum/delete_topic.php
Disallow: /Forum/follow_category.php
Disallow: /Forum/defollow_category.php
Disallow: /pages/newmembers.php
Disallow: /pages/log-in.php
Disallow: /pages/sign-up.php
Disallow: /pages/zoeken.php
Disallow: /pages/member-photos.php
Disallow: /pages/members.php
Disallow: /Nieuws/tag/
Disallow: /Zwangerschap/*page=*
Sitemap: https://www.babybytes.nl/namen/namen-sitemap.php
Sitemap: https://www.babybytes.nl/Forum/sitemap.php
Sitemap: https://www.babybytes.nl/encyclopedie/sitemap.php
Sitemap: https://www.babybytes.nl/Zwangerschap/sitemap.php
Sitemap: https://www.babybytes.nl/Baby`s/sitemap.php
Sitemap: https://www.babybytes.nl/Nieuws/sitemap.php
Sitemap: https://www.babybytes.nl/google-news-sitemap.php

History

Changes recorded since June 2026.

No change recorded.

In context

Netherlands
49.4% block at least one AI crawler
Behind Cloudflare
No
Same group