view src/browser/public/robots.txt @ 1069:2dab7692eae7

pages (files/list): comments handling + size parameters: comments are now handled if comments_url is set. 2 new parameters allows some customization: data['size'] is the size hint to retrieve thumbnails data['retrieve_comments'] indicate if comments must be fetched if the count is > 0.
author Goffi <goffi@goffi.org>
date Sun, 18 Mar 2018 11:36:46 +0100
parents a54882348ecc
children
line wrap: on
line source

User-agent: *
Allow: /blog/
Allow: /u/
Allow: /b/
Disallow: /libervia.html
Disallow: /mr
Disallow: /login

User-agent: Mediapartners-*
Disallow: /