- From: James Michael DuPont <mdupont777@yahoo.com>
- Date: Wed, 15 Jan 2003 09:07:02 -0800 (PST)
- To: Dave Beckett <dave.beckett@bristol.ac.uk>
- Cc: www-rdf-interest@w3.org
--- Dave Beckett <dave.beckett@bristol.ac.uk> wrote: > >>>James Michael DuPont said: > > > > > I will look into setting up a daily snapshot of the rdf logs in the > > next months. At least to keep a weeks worth of rdf and rotate it > should > > be fine. > > I'm puzzled why you need to do that. The RDF/XML logs are live on > the web. If you want to copy them, take the one new file each day > and do something with it. In fact, I'd rather you didn't try to make > an alternative site. Ok, i just wanted to put them onto a spot i could get at them from work. but as I mailed this afternoon : someone already did this. > > to be honest, i think the best would be to push/mirror them onto a > > server that does not have this restriction of robots, i dont > understand > > why it works at home but not at work. (to be honest, i dont > understand > > this error message at all) > > Because the site was hit by agressive web robot client that tried to > download all the chatlogs every hour, several requests per second. > When it reached 70% of our entire site's requests and refused to > respect robots.txt, I banned the client and the IP addresses. We are > installing a new server, but I'm sure I don't want it sucked up with > these requests either. > > There is only one new file each day in RDF, text and HTML > respectively. > Mirroring would take a few seconds. Ok, thanks, mike ===== James Michael DuPont http://introspector.sourceforge.net/ __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com
Received on Wednesday, 15 January 2003 12:07:12 UTC