view src/browser/public/robots.txt @ 1053:a54882348ecc

browser: updated robots.txt
author Goffi <goffi@goffi.org>
date Thu, 25 Jan 2018 09:06:36 +0100
parents 0f309eeef26d
children
line wrap: on
line source

User-agent: *
Allow: /blog/
Allow: /u/
Allow: /b/
Disallow: /libervia.html
Disallow: /mr
Disallow: /login

User-agent: Mediapartners-*
Disallow: /