Re: HTTP 429

Hi Eric,

* Eric Walker <email@owlcroft.com> [2024-04-21 01:38-0700]
>Dear Folks:
>
>I have a small web site, circa 65 pages, for which I would like 
>to be able to validate its HTML (and CSS) in an automated 
>process, because it will ultimately be used by non-tech 
>personnel.
>
>I have PHP code that submits a properly formatted GET request.  
>When used once or twice, it generates the expected response.  I 
>had set it to sleep for 1 second between calls, as that is an 
>interval that I have often seen described as sufficient to not 
>be hammering a server.

I think that's a good guideline for general web requests but 
generating and returning validation results is much more 
expensive than serving static resources like web pages or images. 
I'm not sure how to quantify the difference, but it's certainly 
several orders of magnitude more expensive.

>Regrettably, after 4 or 5 submissions, the Validator starts 
>returning HTTP 429 responses.

I'm surprised that would happen with the current rate limits we 
have in place. If you're willing to contact me off list with the 
IP addresses from which your requests are being sent I can 
investigate further.

>What is frustrating is that, at least so far as I can see, 
>there is no information on what would be an acceptable delay 
>between submissions, a datum that, as I understand it, is often 
>supplied when a 429 response is being sent.
>
>I suggest that, at the least, you could provide that datum in 
>the response.  I would also be grateful if you could let me 
>know what an acceptable sleep time is: it is obviously  more 
>than 1 second, but by how much I don't know.  I have no desire 
>to hammer your servers, but surely there is some interval that 
>is acceptable.
>
>Thank you for your time reading this.

We have a number of rules in place that look for different 
request characteristics before deciding whether to block or 
challenge a request, so it would be hard to express that as a 
single figure like the number of seconds between requests.

Unfortunately these services receive millions of abusive requests 
per day so we have had to add these rate limits in order to keep 
the services available. We have tried not to impact legitimate 
users including reasonable use via automated systems like yours.

-- 
Gerald Oskoboiny <gerald@w3.org>
http://www.w3.org/People/Gerald/

Received on Tuesday, 23 April 2024 03:52:26 UTC