Re: Just what *does* robots.txt mean for a LOD site?

--------------------------------------------
On Sat, 7/26/14, ahogan@dcc.uchile.cl <ahogan@dcc.uchile.cl> wrote:

 The difference in opinion remains to what extent Linked Data
 agents need to pay attention to the robots.txt file.
 
 As many others have suggested, I buy into the idea of any
 agent not relying document-wise on user input being subject to
 robots.txt.

=====================
+1
Just a comment.

Somewhere, sometime, somebody with Yahoo Mail decided that public-lod mail was spam, so every morning I dig it out because I value the content.

Of course, I could wish for a Linked Data Agent which does that for me, but that would be to complete a banal or vicious cycle, depending on the circle classification scheme in use.  I'm looking gor virtuous cycles and in the case of robots.txt, "The lady doth protest too much, methinks".
--Gannon

Received on Sunday, 27 July 2014 18:27:18 UTC