Mercurial > libervia-web

view src/browser/public/robots.txt @ 1019:34240d08f682

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
pages: HTTP cache headers handling: when checkCache is used, HTTP headers handling cache are now used: - ETag is first checked, using a hash of the rendered content - Last-Modified is used as a fallback is client is not handling ETag When suitable, a HTTP 304 code (Not Modified) wihtout body is returned instead of the whole page.
author Goffi <goffi@goffi.org>
date Sun, 21 Jan 2018 13:14:06 +0100
parents 0f309eeef26d
children a54882348ecc
line wrap: on
line source

User-agent: *
Allow: /blog/
Allow: /u/
Disallow: /

User-agent: Mediapartners-*
Disallow: /