W3C home > Mailing lists > Public > whatwg@whatwg.org > November 2011

[whatwg] Proposal: Exclude robots on a per-link basis

From: Karl Dubost <karld@opera.com>
Date: Sat, 26 Nov 2011 09:34:00 -0500
Message-ID: <E2C52AE1-472B-4498-A214-B00E252FE0BC@opera.com>

Le 26 nov. 2011 ? 07:20, Markus Ernst a ?crit :
> Viewing the logs of applications I wrote, I noticed that a considerable number of requests are from robots following links of types such as "Add to shopping cart" or "Remember this item" - links that typically point to the same page they are clicked on, with some GET variable that triggers an action on the server.


Karl Dubost - http://dev.opera.com/
Developer Relations & Tools, Opera Software
Received on Saturday, 26 November 2011 06:34:00 UTC

This archive was generated by hypermail 2.4.0 : Wednesday, 22 January 2020 16:59:38 UTC