CrawlerPulse
CrawlerPulse / Oceania / Papua New Guinea / pnggossip.com

Papua New Guinea Gossip Newsletter Home Page

pnggossip.com · Papua New Guinea · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,005 bytes. No rule names an AI crawler.

User-agent: *
Disallow: /autoresponder/
Disallow: /bak/
Disallow: /cgi-bin/
Disallow: /hitmat/
Disallow: /images/
Disallow: /default.htm
Disallow: /test4.html
Disallow: /lcl99xx.html
Disallow: /balus/oops3.html
Disallow: /balus/template.html
Disallow: /balus/thanks3.html
Disallow: /gadoeno/gadoform.html
Disallow: /gadoeno/oops1.html
Disallow: /gadoeno/thanks.txt
Disallow: /gadoeno/thanks1.html
Disallow: /misc/bnbform.html
Disallow: /misc/bnbtest.html
Disallow: /misc/oops1.html
Disallow: /misc/pomtime.html
Disallow: /misc/summaries.html
Disallow: /misc/survey.html
Disallow: /misc/survey1.html
Disallow: /misc/survey1.txt
Disallow: /misc/surveybnb.html
Disallow: /misc/thanks1.html
Disallow: /misc/index.txt
Disallow: /tasiana/s.html
Disallow: /pnggoss/apply.html
Disallow: /404.html
Disallow: /error.html
Disallow: /index_css.html

History

Changes recorded since June 2026.

No change recorded.

In context

Papua New Guinea
0.0% block at least one AI crawler
Behind Cloudflare
No