lancasteronline.com · United States · measured 9 Oct 2026
As read on 9 Oct 2026, 9,100 bytes. 22 rule groups name AI crawlers; shown first.
User-agent: GPTBot Disallow: / User-agent: CCBOT Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: Google-Extended Disallow: / User-agent: Bytespider Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Amazonbot Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Claude-Web Disallow: / User-agent: Diffbot Disallow: / User-agent: FacebookBot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: PetalBot Disallow: / User-agent: Timpibot Disallow: / User-agent: YouBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: cohere-ai Disallow: / User-agent: omgili Disallow: / User-agent: omgilibot Disallow: / User-agent: ChatGPTUser Disallow: / User-agent: AI2Bot Disallow: / User-agent: Ai2Bot-Dolma Disallow: / User-agent: FriendlyCrawler Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: Scrapy Disallow: / User-agent: VelenPublicWebCrawler Disallow: / User-agent: Webzio-Extended Disallow: / User-agent: iaskspider/2.0 Disallow: / User-agent: img2dataset Disallow: / User-agent: * Disallow: /tncms/tracking/ Disallow: /_services/ Disallow: /advertisingproof/ Disallow: /advertisingproof2/ Disallow: /business/consumer/ Disallow: /business/nation/ Disallow: /business/technology/ Disallow: /contests/cutest-pet/ Disallow: /contests/cutest-pet/winners/ Disallow: /contests/firefighters/ Disallow: /contests/firefighters/winners/ Disallow: /contests/garden/winners/ Disallow: /contests/keepourrepublic/submit/ Disallow: /jobs/now-hiring/ Disallow: /news/national/ Disallow: /news/pennsylvania/ Disallow: /news/world/ Disallow: /newsletters/entertainment_lancaster/ Disallow: /newsletters/just-township/ Disallow: /newsletters/llfootball/ Disallow: /nowapp/ Disallow: /nowapp/android/ Disallow: /nowapp/business/ Disallow: /nowapp/ios/ Disallow: /nowapp/news/ Disallow: /nowapp/sports/ Disallow: /nowapp/weather/ Disallow: /obituaries/place/ Disallow: /offers/ Disallow: /pocket/ Disallow: /publicnotices/sheriffsales/ Disallow: /schools/adcontest/advertise/ Disallow: /schools/adcontest/submit/ Disallow: /schools/adcontest/teachers/ Disallow: /site/birthday/ Disallow: /site/forms/women-in-leadership/application/ Disallow: /site/forms/women-in-leadership/application-thank-you/ Disallow: /site/forms/women-in-leadership/nomination-thank-you/ Disallow: /site/subscribe/digital/ Disallow: /site/subscribe/direct-mail/ Disallow: /site/subscribe/direct-mail-sports/ Disallow: /site/subscribe/freetrial/ Disallow: /site/subscribe/noscript/ Disallow: /site/subscribe/obits/ Disallow: /site/subscribe/pocket/ Disallow: /site/subscribe/power/ Disallow: /site/subscribe/sports/ Disallow: /site/subscribe/upgrade/ Disallow: /start/ Disallow: /style_guide/ Disallow: /test/ Disallow: /test/activate/ Disallow: /test/blox/ Disallow: /test/grid/ Disallow: /test/info/ Disallow: /test/print/ Disallow: /whiz-app/ Disallow: /whiz-app/archives/ Disallow: /whiz-app/closings/ Disallow: /whiz-app/contact/ Disallow: /whiz-app/contests/ Disallow: /whiz-app/feeds/ Disallow: /whiz-app/feeds/features/ Disallow: /whiz-app/feeds/news/ Disallow: /whiz-app/feeds/opinion/ Disallow: /whiz-app/feeds/sports/ Disallow: /whiz-app/marketplace/ Disallow: /whiz-app/marketplace/award/ Disallow: /whiz-app/marketplace/blog/ Disallow: /whiz-app/marketplace/coupon/ Disallow: /whiz-app/marketplace/menu/ Disallow: /whiz-app/marketplace/photo/ Disallow: /whiz-app/marketplace/press-release/ Disallow: /whiz-app/marketplace/product/ Disallow: /whiz-app/marketplace/service/ Disallow: /whiz-app/marketplace/staff/ Disallow: /whiz-app/marketplace/testimonial/ Disallow: /whiz-app/newsletters/ Disallow: /whiz-app/obituaries/ Disallow: /whiz-app/olympics-events/ Disallow: /whiz-app/olympics-medals/ Disallow: /whiz-app/stats/ Disallow: /whiz-app/traffic/ Disallow: /whiz-app/weather/ Disallow: /wrapper/ Disallow: /wrapper/arkadium/ Disallow: /wrapper/basic/ Disallow: /wrapper/secondstreet/ Disallow: /wrappers/ Disallow: /wrappers/noads/ Disallow: /tncms/weather/ Disallow: /tncms/webservice/ Disallow: /tncms/auth/ Disallow: /tncms/openid2/ Disallow: /tncms/media/ Disallow: /tncms/messaging/ Disallow: /tncms/search/ Disallow: /tncms/counter/ Disallow: /tncms/block/ Disallow: /tncms/tax/ Disallow: /tncms/dmp/ Disallow: /tncms/csrf/ Disallow: /tncms/gtm/ Disallow: /tncms/admin/ Disallow: /tncms/track/ Disallow: /tncms/openweb/* Disallow: /content/tncms/ Disallow: /tncms/calendar/ Disallow: /content/tncms/assets/v3/form/ Disallow: /content/tncms/form/ Disallow: /classifieds/*? Disallow: /classifieds/community/*? Disallow: /classifieds/job/*? Disallow: /classifieds/personals/*? Disallow: /classifieds/sale/*? Disallow: /classifieds/service/*? Disallow: /classifieds/vehicle/*? Disallow: /jobs/engineering/*? Disallow: /jobs/healthcare/*? Disallow: /jobs/newcareer/*? Disallow: /jobs/now-hiring/*? Disallow: /jobs/skilled-trades/*? Disallow: /jobs/transportation/*? Disallow: /auctions/*? Disallow: /offers/nickoftime/*? Disallow: /newsletters/entertainment_lancaster Disallow: /newsletters/test Disallow: /newsletters/gardening Disallow: /newsletters/llfootball Disallow: /newsletters/just-township Disallow: /tncms/user/ Disallow: /users/admin/ Disallow: /users/*/? Disallow: /marketplace/*action=srch Disallow: /marketplace/retail/*action=srch Disallow: /visitlancaster/marketplace/*action=srch Disallow: /whiz-app/marketplace/*action=srch Disallow: /whiz-app/marketplace/award/*action=srch Disallow: /whiz-app/marketplace/blog/*action=srch Disallow: /whiz-app/marketplace/coupon/*action=srch Disallow: /whiz-app/marketplace/menu/*action=srch Disallow: /whiz-app/marketplace/photo/*action=srch Disallow: /whiz-app/marketplace/press-release/*action=srch Disallow: /whiz-app/marketplace/product/*action=srch Disallow: /whiz-app/marketplace/service/*action=srch Disallow: /whiz-app/marketplace/staff/*action=srch Disallow: /whiz-app/marketplace/testimonial/*action=srch Disallow: /tncms/disqus/ Sitemap: https://lancasteronline.com/sitemap.xml Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1970 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1992 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1993 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1994 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1995 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1996 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1997 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1998 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=1999 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2000 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2001 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2002 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2003 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2004 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2005 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2006 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2007 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2008 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2009 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2010 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2011 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2012 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2013 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2014 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2015 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2016 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2017 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2018 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2019 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2020 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2021 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2022 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2023 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2024 Sitemap: https://lancasteronline.com/tncms/sitemap/editorial.xml?year=2025
Changes recorded since June 2026.
No change recorded.