W3C home > Mailing lists > Public > whatwg@whatwg.org > November 2011

[whatwg] Proposal: Exclude robots on a per-link basis

From: Karl Dubost <karld@opera.com>
Date: Sat, 26 Nov 2011 09:34:00 -0500
Message-ID: <E2C52AE1-472B-4498-A214-B00E252FE0BC@opera.com>

Le 26 nov. 2011 ? 07:20, Markus Ernst a ?crit :
> Viewing the logs of applications I wrote, I noticed that a considerable number of requests are from robots following links of types such as "Add to shopping cart" or "Remember this item" - links that typically point to the same page they are clicked on, with some GET variable that triggers an action on the server.


http://www.w3.org/2001/tag/doc/whenToUseGet.html
http://blog.whatfettle.com/2005/10/31/is-it-safe/

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations & Tools, Opera Software
Received on Saturday, 26 November 2011 06:34:00 UTC

This archive was generated by hypermail 2.3.1 : Monday, 13 April 2015 23:09:09 UTC