W3C home > Mailing lists > Public > www-rdf-interest@w3.org > January 2003

Re: RDFIG chats in ntriples

From: James Michael DuPont <mdupont777@yahoo.com>
Date: Wed, 15 Jan 2003 09:07:02 -0800 (PST)
Message-ID: <20030115170702.79828.qmail@web13305.mail.yahoo.com>
To: Dave Beckett <dave.beckett@bristol.ac.uk>
Cc: www-rdf-interest@w3.org


--- Dave Beckett <dave.beckett@bristol.ac.uk> wrote:
> >>>James Michael DuPont said:
> > 
> 
> > I will look into setting up a daily snapshot of the rdf logs in the
> > next months. At least to keep a weeks worth of rdf and rotate it
> should
> > be fine.
> 
> I'm puzzled why you need to do that.  The RDF/XML logs are live on
> the web.  If you want to copy them, take the one new file each day
> and do something with it.  In fact, I'd rather you didn't try to make
> an alternative site.

Ok, i just wanted to put them onto a spot i could get at them from
work.
but as I mailed this afternoon : someone already did this.


> > to be honest, i think the best would be to push/mirror them onto a
> > server that does not have this restriction of robots, i dont
> understand
> > why it works at home but not at work. (to be honest, i dont
> understand
> > this error message at all)
> 
> Because the site was hit by agressive web robot client that tried to
> download all the chatlogs every hour, several requests per second.
> When it reached 70% of our entire site's requests and refused to
> respect robots.txt, I banned the client and the IP addresses.  We are
> installing a new server, but I'm sure I don't want it sucked up with
> these requests either.
> 
> There is only one new file each day in RDF, text and HTML
> respectively.
> Mirroring would take a few seconds.

Ok, thanks,

mike

=====
James Michael DuPont
http://introspector.sourceforge.net/

__________________________________________________
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com
Received on Wednesday, 15 January 2003 12:07:12 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Monday, 7 December 2009 10:51:57 GMT