Re: W3ID Latency

Hi David,

Thanks for the explanation. I’m still getting pretty substantial latency today, maybe 5 seconds per request, although it’s better than yesterday.

I guess W3ID is running on Apache HTTP? Are there any technical solutions available, like tweaking the configuration according these suggestions<https://httpd.apache.org/docs/trunk/misc/security_tips.html#dos>? Is the issue purely one of volume, or are the individual requests taking an unusual amount of time? I had hoped that a pure proxy server would be near impossible to DDOS but I’m no expert.

Cheers,
Michael
From: David I. Lehn <dil@lehn.org>
Date: Tuesday, 22 July 2025 at 4:57 AM
To: Michael Milton <milton.m@wehi.edu.au>
Cc: public-perma-id@w3.org <public-perma-id@w3.org>
Subject: Re: W3ID Latency
On Sun, Jul 20, 2025 at 10:32 PM Michael Milton <milton.m@wehi.edu.au<mailto:milton.m@wehi.edu.au>> wrote:
W3IDs are failing to resolve or loading very slowly. A good example is https://w3id.org/ro/crate/1.1/context<https://urldefense.com/v3/__https:/w3id.org/ro/crate/1.1/context__;!!KKt0acMt!db1HYx-5fiOWMheRXo8KSe7vu9vjGLUDjk_LgxVzakrf7ljYiGlbwMAFeAa9xlyl6O7UzIgd74wZ$> and even https://w3id.org/WEHI-SODA-Hub/omerocrate/upload<https://urldefense.com/v3/__https:/w3id.org/WEHI-SODA-Hub/omerocrate/upload__;!!KKt0acMt!db1HYx-5fiOWMheRXo8KSe7vu9vjGLUDjk_LgxVzakrf7ljYiGlbwMAFeAa9xlyl6O7UzH4jbVq2$> (which doesn’t have anything it’s supposed to resolve to) takes a long time to 404.

Is there an issue with the proxy server?


It wasn't any URL in particular, it was bots getting out of control scraping many URLs.  I got *lots* of monitoring alerts for the issue around when this was opened.  I'm not sure what to do about it but throw more money/resources at the problem?  While the request rates are higher, they look like a totally reasonable rate for the system to handle.  We'll keep an eye on it and have some thoughts on mitigations, but the issue has been rare so far, so the "wait and it'll be fine" theory has worked.

-dave