Assuming this post is accurate, Mr. Warden was threatened with lawsuits from Facebook. He built a more or less garden variety crawler, he claims, and he observed the limitations in Facebook's robots.txt. Nonetheless, they apparently made him destroy the data. See [1]. Anyone know more about this? Noah [1] http://petewarden.typepad.com/searchbrowser/2010/04/how-i-got-sued-by-facebook.html -------------------------------------- Noah Mendelsohn IBM Corporation One Rogers Street Cambridge, MA 02142 1-617-693-4036 --------------------------------------Received on Tuesday, 6 April 2010 14:09:01 GMT
This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 26 April 2012 12:48:20 GMT