# This robots.txt is to avoid to crawl https pages. It is served via: # RewriteCond %{HTTPS} =on # RewriteRule ^robots.txt$ robots-deny-all.txt [L] User-agent: * Disallow: /