W3C home > Mailing lists > Public > whatwg@whatwg.org > January 2012

[whatwg] Proposal: Exclude robots on a per-link basis

From: Ian Hickson <ian@hixie.ch>
Date: Tue, 31 Jan 2012 20:27:24 +0000 (UTC)
Message-ID: <Pine.LNX.4.64.1201312024550.3868@ps20323.dreamhostps.com>
On Sat, 26 Nov 2011, Markus Ernst wrote:
> Viewing the logs of applications I wrote, I noticed that a considerable 
> number of requests are from robots following links of types such as "Add 
> to shopping cart" or "Remember this item" - links that typically point 
> to the same page they are clicked on, with some GET variable that 
> triggers an action on the server.

That's mis-use of GET. The right solution is to not use a link.

Because of this, I haven't added a new feature for this request.

On Sat, 26 Nov 2011, Eric Shtivelberg wrote:
> The rel="nofollow" does serve a purpose and it is even supported by all 
> major search engines, but there's a big "but" - some of them do follow 
> it but don't use it, see: 
> http://en.wikipedia.org/wiki/Nofollow#Interpretation_by_the_individual_search_engines

Technically rel=nofollow doesn't mean that robots shouldn't follow the 
link, it means that authority shouldn't flow through the link in systems 
that attempt to rank pages (e.g. search engines).

Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'
Received on Tuesday, 31 January 2012 12:27:24 UTC

This archive was generated by hypermail 2.4.0 : Wednesday, 22 January 2020 16:59:39 UTC