CrawlerPulse
CrawlerPulse / North America / Mexico / surtidor.com

El Surtidor

surtidor.com · Mexico · El Surtidor (MX) · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 5,018 bytes. 1 rule group name AI crawlers; shown first.

User-agent: meta-externalagent
Disallow: /busqueda

User-agent: *
Allow: */modules/*.css
Allow: */modules/*.js
Allow: */modules/*.png
Allow: */modules/*.jpg
Allow: /js/jquery/*
Disallow: /*?order=
Disallow: /*?tag=
Disallow: /*?id_currency=
Disallow: /*?search_query=
Disallow: /*?back=
Disallow: /*?n=
Disallow: /*&order=
Disallow: /*&tag=
Disallow: /*&id_currency=
Disallow: /*&search_query=
Disallow: /*&back=
Disallow: /*&n=
Disallow: /*controller=addresses
Disallow: /*controller=address
Disallow: /*controller=authentication
Disallow: /*controller=cart
Disallow: /*controller=discount
Disallow: /*controller=footer
Disallow: /*controller=get-file
Disallow: /*controller=header
Disallow: /*controller=history
Disallow: /*controller=identity
Disallow: /*controller=images.inc
Disallow: /*controller=init
Disallow: /*controller=my-account
Disallow: /*controller=order
Disallow: /*controller=order-slip
Disallow: /*controller=order-detail
Disallow: /*controller=order-follow
Disallow: /*controller=order-return
Disallow: /*controller=order-confirmation
Disallow: /*controller=pagination
Disallow: /*controller=password
Disallow: /*controller=pdf-invoice
Disallow: /*controller=pdf-order-return
Disallow: /*controller=pdf-order-slip
Disallow: /*controller=product-sort
Disallow: /*controller=search
Disallow: /*controller=statistics
Disallow: /*controller=attachment
Disallow: /*controller=guest-tracking
Disallow: /app/
Disallow: /cache/
Disallow: /classes/
Disallow: /config/
Disallow: /controllers/
Disallow: /download/
Disallow: /js/
Disallow: /localization/
Disallow: /log/
Disallow: /mails/
Disallow: /modules/
Disallow: /override/
Disallow: /pdf/
Disallow: /src/
Disallow: /tools/
Disallow: /translations/
Disallow: /upload/
Disallow: /var/
Disallow: /vendor/
Disallow: /webservice/
Disallow: /interior/app/
Disallow: /interior/cache/
Disallow: /interior/classes/
Disallow: /interior/config/
Disallow: /interior/controllers/
Disallow: /interior/download/
Disallow: /interior/js/
Disallow: /interior/localization/
Disallow: /interior/log/
Disallow: /interior/mails/
Disallow: /interior/modules/
Disallow: /interior/override/
Disallow: /interior/pdf/
Disallow: /interior/src/
Disallow: /interior/tools/
Disallow: /interior/translations/
Disallow: /interior/upload/
Disallow: /interior/var/
Disallow: /interior/vendor/
Disallow: /interior/webservice/
Disallow: /guadalajara/app/
Disallow: /guadalajara/cache/
Disallow: /guadalajara/classes/
Disallow: /guadalajara/config/
Disallow: /guadalajara/controllers/
Disallow: /guadalajara/download/
Disallow: /guadalajara/js/
Disallow: /guadalajara/localization/
Disallow: /guadalajara/log/
Disallow: /guadalajara/mails/
Disallow: /guadalajara/modules/
Disallow: /guadalajara/override/
Disallow: /guadalajara/pdf/
Disallow: /guadalajara/src/
Disallow: /guadalajara/tools/
Disallow: /guadalajara/translations/
Disallow: /guadalajara/upload/
Disallow: /guadalajara/var/
Disallow: /guadalajara/vendor/
Disallow: /guadalajara/webservice/
Disallow: /mx/app/
Disallow: /mx/cache/
Disallow: /mx/classes/
Disallow: /mx/config/
Disallow: /mx/controllers/
Disallow: /mx/download/
Disallow: /mx/js/
Disallow: /mx/localization/
Disallow: /mx/log/
Disallow: /mx/mails/
Disallow: /mx/modules/
Disallow: /mx/override/
Disallow: /mx/pdf/
Disallow: /mx/src/
Disallow: /mx/tools/
Disallow: /mx/translations/
Disallow: /mx/upload/
Disallow: /mx/var/
Disallow: /mx/vendor/
Disallow: /mx/webservice/
Disallow: /*mx/direccion
Disallow: /*mx/direcciones
Disallow: /*mx/iniciar-sesion
Disallow: /*mx/carrito
Disallow: /*mx/descuento
Disallow: /*mx/seguimiento-pedido-invitado
Disallow: /*mx/historial-compra
Disallow: /*mx/datos-personales
Disallow: /*mx/mi-cuenta
Disallow: /*mx/pedido
Disallow: /*mx/confirmacion-pedido
Disallow: /*mx/seguimiento-pedido
Disallow: /*mx/nota-credito
Disallow: /*mx/recuperar-contrasena
Disallow: /*mx/busqueda
Sitemap: https://www.surtidor.com/1_index_sitemap.xml

User-agent: facebookexternalhit
Disallow: /busqueda

User-agent: meta-externalads
Disallow: /busqueda

User-agent: meta-webindexer
Disallow: /busqueda

User-agent: meta-externalfetcher
Disallow: /busqueda
Sitemap: https://www.surtidor.com/guadalajara/sitemap_shop_1.xml
Sitemap: https://www.surtidor.com/guadalajara/sitemap_shop_2.xml
Sitemap: https://www.surtidor.com/guadalajara/sitemap_shop_3.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Mexico
11.1% block at least one AI crawler
Behind Cloudflare
Yes