Re: [w3c/manifest] Describe how developers can block web crawlers (#498)

> If a web app developer wants to block traffic from web crawlers, the developer MAY do so by including the web app manifest in a robots.txt file. 

Note: One can also use the [`X-Robots-Tag`](https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag#using-the-x-robots-tag-http-header) HTTP header.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/w3c/manifest/issues/498#issuecomment-246104010