view browser/public/robots.txt @ 1168:ea0caa7b1bcc

pages (i18n): check presence of u"/" in locale: if u"/" is present in local, a warning message is logged, and locale is ignored. This is done because in some cases locale can be used with a path (e.g. if several localised version of a document are used).
author Goffi <goffi@goffi.org>
date Fri, 12 Apr 2019 14:06:50 +0200
parents 28e3eb3bb217
children
line wrap: on
line source

User-agent: *
Allow: /blog/
Allow: /u/
Allow: /b/
Disallow: /libervia.html
Disallow: /mr
Disallow: /login

User-agent: Mediapartners-*
Disallow: /