Re: Question about web spiders...

Lachlan Hunt wrote:
>> That's perfectly standards compliant, and Googlebot obeys that, as well
>> as several other major spiders AFAIK.
> 
> It is not standards compliant at all.  It's a proprietary extension that
> just happens to pass DTD based validation.  nofollow was discussed quite
> extensively on this list when Google introduced it and the vast majority
> of this community rejected it.

Well, according to the relevant standard[1]:

"Authors may wish to define additional link types not described in this
specification."

I don't see how it isn't compliant, but feel free to clarify things for me.

Wasn't aware that the robot still followed the link though, perhaps they
should have given it a more descriptive name...

Jasper

[1] http://www.w3.org/TR/html401/types.html#type-links