CrawlerPulse
CrawlerPulse / South America / Colombia / vanguardia.com

Vanguardia

vanguardia.com · Colombia · Vanguardia Liberal · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Blocked
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 4,180 bytes. 6 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: CCbot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: ChatGPT-user
Disallow: /

User-agent: *
Disallow: /pf/api/v3/*
Disallow: /*?fb_comment_id=
Disallow: /editor/public/*
Disallow: /sites/default/files/pdf/*
Disallow: /sites/default/files/imagecache/*
Disallow: /widgets/*
Disallow: /area-metropolitana/piedecuesta/2014/05/07/venden-ilegalmente-predios-en-areas-de-proteccion-y-parques/
Disallow: /colombia/2012/12/13/desmantelada-red-que-falsificaba-medicamentos-para-el-cancer-y-vih/
Disallow: /entretenimiento/tendencias/2024/09/06/fallecio-el-iconico-reportero-del-transito-renato-yanez/
Disallow: /judicial/2014/02/07/por-el-crimen-de-un-taxista-policia-capturo-a-un-estudiante-de-derecho/
Disallow: /judicial/2024/06/20/hombre-fue-capturado-para-cumplir-una-condena-por-comercializar-estupefaciente-en-bucaramanga/
Disallow: /judicial/2017/12/19/cayeron-dos-hombres-que-harian-parte-de-una-banda-de-estafadores/
Disallow: /colombia/2012/12/12/desmantelan-red-que-falsificaba-medicamentos-para-cancer-y-vih/
Disallow: /colombia/2012/12/13/es-una-falacia-que-reforma-tributaria-afecte-clase-media-presidente-santos/
Disallow: /area-metropolitana/piedecuesta/2014/05/07/venden-ilegalmente-predios-en-areas-de-proteccion-y-parques/
Disallow: /colombia/2015/03/07/exdirectivos-de-la-san-martin-habrian-desviado-cerca-de-un-billon-de-pesos/
Disallow: /colombia/2015/03/04/ex-directivos-de-la-san-martin-seran-presentados-ante-un-juez-en-girardot/
Disallow: /deportes/mundial-de-futbol/2009/07/12/denuncian-supuesta-captadora-ilegal-de-dinero-denominada-fox-ub-best-men/
Disallow: /judicial/2018/09/12/por-el-escandalo-de-las-chuzadas-capturan-a-un-hombre-en-bucaramanga/
Disallow: /judicial/2024/03/13/dramatico-testimonio-de-adulta-mayor-victima-de-estafa-inmobiliaria-en-piedecuesta/
Disallow: /judicial/2024/03/12/a-prision-tres-representantes-y-el-asesor-juridico-de-constructora-involucrada-en-millonaria-estafa-en-piedecuesta/
Disallow: /area-metropolitana/piedecuesta/2018/04/17/qbica-aclaro-que-proyecto-en-piedecuesta-no-es-una-estafa/
Disallow: /area-metropolitana/bucaramanga/2026/01/25/condenan-a-contrabandistas-de-oro-cual-es-la-relacion-con-bucaramanga/
Disallow: /area-metropolitana/bucaramanga/denuncian-caso-de-presunto-acoso-laboral-y-sexual-en-el-hus-MB4982394
Disallow: /area-metropolitana/bucaramanga/2022/03/16/denuncian-caso-de-presunto-acoso-laboral-y-sexual-en-el-hus/
Disallow: /judicial/2023/12/20/un-abogado-y-una-mujer-las-victimas-de-un-ataque-sicarial-dentro-de-un-vehiculo/
Disallow: /judicial/2012/08/15/capo-de-presunta-banda-que-traficaba-coca-se-refugiaba-en-floridablanca/
Disallow: /entretenimiento/galeria/2012/09/14/maryuri-gonzalez-modelo-de-la-semana/
Disallow: /politica/2018/02/18/subdirectora-de-la-cdmb-no-acepto-cargos-por-cartel-de-la-madera/
Allow: /
Sitemap: https://www.vanguardia.com/arc/outboundfeeds/sitemap-index/
Sitemap: https://www.vanguardia.com/arc/outboundfeeds/sitemap-news-index/
Sitemap: https://www.vanguardia.com/arc/outboundfeeds/sitemap-section-index/

User-agent: sitecheck.internetseer.com
Disallow: /

User-agent: Zealbot
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: Fetch
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: WebZIP
Disallow: /

User-agent: linko
Disallow: /

User-agent: HTTrack
Disallow: /

User-agent: Microsoft.URL.Control
Disallow: /

User-agent: Xenu
Disallow: /

User-agent: larbin
Disallow: /

User-agent: libwww
Disallow: /

User-agent: ZyBORG
Disallow: /

User-agent: Download Ninja
Disallow: /

User-agent: UbiCrawler
Disallow: /

User-agent: DOC
Disallow: /

User-agent: Zao
Disallow: /

User-agent: Slurp
Disallow: /

User-agent: Maxthon
Disallow: /

User-agent: CNCDialer
Disallow: /

User-agent: newsproxy.app
Disallow: /

User-agent: ia_archiver
Disallow: /

User-agent: netseer
Disallow: /

User-agent: Clickagy Intelligence Bot v2
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Colombia
11.9% block at least one AI crawler
Behind Cloudflare
No
Same group