[w3c/ServiceWorker] Unable to insert conditional-request headers in Service Worker? (Issue #1699)

I'm currently developing a browser game that operates as a single-page web application, distributed as a GitHub pages site. There is no build step, as everything is written in pure HTML/SVG, CSS, or JS, and so the GH pages source can simply be the HEAD of the repository. However, since this is a build-free repo, this means that all the subresources are fetched individually, and thus individual script files can end up being out-of-sync, if some of them are retrieved from cache and others from the network.

I've lost count of the number of bug reports I've gotten for weird one-off failures caused by out-of-sync resources or the number of times I've had to instruct my players on how to do a full ctrl-shift-R reload, so I decided that in order to fix the issue once and for all, I'd write a ServiceWorker that could monitor the requests and ensure that the player is using a _consistent_ snapshot of the codebase, and at the same time I could cache the game's resources so that players could continue to play even while offline.

I've run into trouble with the caching functionality, however. I'm using [RFC9111 4. Constructing Responses From Caches](https://httpwg.org/specs/rfc9111.html#constructing.responses.from.caches) as a behavior guide, but I'm running into unexpected trouble implementing [4.3.1. Sending a Validation Request](https://httpwg.org/specs/rfc9111.html#validation.sent), which begins:

> When generating a conditional request for validation, a cache either starts with a request it is attempting to satisfy or — if it is initiating the request independently — synthesizes a request using a stored response by copying the method, target URI, and request header fields identified by the Vary header field ([Section 4.1](https://httpwg.org/specs/rfc9111.html#caching.negotiated.responses)).

Unfortunately, I'm completely unable to do that. When my service worker receives a `FetchEvent`, the request headers are immutable, even if I've created a new Request object using `new Request(evt.request)`, and so I can't add `If-None-Match` or `If-Modified-Since`. On the other hand, if I try to create an empty Request object, I can't copy whatever headers might have been specified in the Vary of the cached response, because the incoming Request's headers are shielded and can't be retrieved for security/privacy reasons.

Even worse, the failure was entirely silent. The following _should_ have been a conservative but functional cache-verification function:

```js
async function verifyCacheResult(req, cacheResult) {
    const origReq = req;
    req = new Request(req);

    if (cacheResult.headers.has("etag")) {
        req.headers.append("If-None-Match", cacheResult.headers.get("etag"));
    }
    if (cacheResult.headers.has("last-modified")) {
        req.headers.append("If-Modified-Since", cacheResult.headers.get("last-modified"));
    }

    const response = await fetch(req.clone());

    if (response.status === 200) {
        // New response, update cache
        const cache = await caches.open("direct");
        cache.put(origReq, response);
    }

    return response;
}
```

And for a time, it seemed like it was working, until I happened to notice while I was tracking down a different issue that I was getting _way_ too many cache updates. It took me two days of research to discover that the problem was that `req.headers` has an entirely invisible attribute called "guard", which cannot be interrogated by JavaScript, and which causes the call to `req.headers.append()` to fail _silently_. I'll note that this is mentioned nowhere on the [MDN page for Headers.append](https://developer.mozilla.org/en-US/docs/Web/API/Headers/append), not even in passing, but that's obviously an issue the MDN team ought to be solving.

As a result, not only does my service worker implementation not improve on the browser's own HTTP cache, it makes things _significantly_ worse, as _every single subresource_ is subjected to an unconditional HTTP fetch on every single load of the page. If I hadn't noticed the issue - which, again, generated no errors, which is apparently allowable by spec (???) - I would have been responsible for a very small-scale DDoS of Github, as every player's browser would attempt to revalidate every subresource, and the server would report that every single subresource had been modified, which would trigger a transparent reload in the player's browser, at which point the service worker would attempt to revalidate the newly-cached responses (which were the same as the old cached responses in the first place)...

I've put the service worker implementation on hold for now, as there are other features I can work on that now seem more likely to bear fruit, but given that Service Workers are explicitly intended and touted as a programmable browser-cache system, why can't I set conditional-request headers on outgoing requests? How am I supposed to do this and still provide proper semantics for cache validation?

-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3c/ServiceWorker/issues/1699
You are receiving this because you are subscribed to this thread.

Message ID: <w3c/ServiceWorker/issues/1699@github.com>