[whatwg/fetch] Proposal: Easier request/response rewrites (#671)

Hi all,

For those that don't know, I'm the tech lead for [Cloudflare Workers](https://blog.cloudflare.com/introducing-cloudflare-workers/), which implements the Service Workers API (but runs code on Cloudflare's servers rather than in the browser).

In general the Service Workers and Fetch APIs have been very good for us (certainly better than what we would have ended up with had we invented our own). But, based on user feedback we are finding some pain points. This is the first of a few proposals I'll be making to solve these issues.

/cc @harrishancock who does most of the API implementation work on our team.

## Problem statement

Today, making minor "rewrites" to a Request or Response (e.g. starting from an existing object and modifying just one property or header) is not very ergonomic. I'm not sure if this is common in browser Service Workers, but it is probably the dominant use case for Cloudflare Workers, so we're very interested in making it better.

For example, consider the case where I want to change the hostname, change the redirect mode to "follow" (to resolve redirects server-side), and add a header. This might look like:

```javascript
addEventListener("fetch", event => {
  let request = event.request

  // Change URL.
  let url = new URL(event.request.url)
  url.hostname = "example.com"
  request = new Request(url, request)
      
  // Change the redirect mode.
  request = new Request(request, { redirect: "follow" })
      
  // Add a header.
  request.headers.add("X-Example", "foo")

  event.respondWith(fetch(request))
})
```

Notice how the best way to change each property is wildly different!

* The only way (AFAIK) to modify the URL while keeping everything else the same is `request = new Request(newUrl, request)`. The fact that this works is somewhat of a fluke: we're actually duck-typing the old request object as a `RequestInit`, which happens to work because all the member names line up. It's unclear to me if the spec actually intended for this to work, but without this trick, there's no way to construct the new request without enumerating every request property individually, which is error-prone and not future-proof.
* We modify `redirect` in the intended way, by passing the old request as the first parameter to the new request's constructor, and passing a `RequestInit` containing only `redirect`. This is fine.
* Modifying `headers` via `RequestInit` is inconvenient because it replaces all of the headers. Since we don't want to remove the existing headers, we'd need to make a copy `Headers` object first, modify the copy, then pass that in `RequestInit`. It turns out, though, that once we've made our own `Request` object, we can just modify its `headers` directly. This is convenient, but weirdly inconsistent: properties like `url` and `redirect` cannot be modified post-construction.

When it comes to the `Response` type, we have a bigger problem: you cannot pass an existing response object as the first parameter to `Response`'s constructor, the way you can do with requests. (If you try to do so, the response object will be stringified as `[object Response]` and that will become the new response's body.) So, if you want to modify the status code of a Response:

```javascript
addEventListener("fetch", event => {
  if (new URL(event.request.url).pathname.startsWith("/hidden/")) {
    // Mask our hidden directory by faking 404.
    event.respondWith(fetch("/404-page.html")
        .then(response => {
      // 404-page.html returns with status 200. Give it status 404.
      return new Response(response.body, {
        status: 404,
        statusText: "Not Found",
        headers: response.headers
      })
    }))
  }
})
```

This is bad, because if `Response` and `ResponseInit` are ever extended with a new field, that field will be inadvertently dropped during the rewrite. (We commonly see people doing Request rewrites this way, too, where it's an even bigger issue as `RequestInit` has quite a few fields that tend to be forgotten.)

## Proposal

Let's make `Request`'s constructor be the One True Way to rewrite requests. To that end:

* Define `RequestInit.url` as an alternative way to specify the URL. This field would only be used when the constructor's first parameter is an existing request object, in which case `RequestInit.url` overwrites the URL. (It's important that `RequestInit.url` is ignored when the first parameter is itself a string URL. Otherwise, existing code which rewrites URLs using the `request = new Request(url, request)` idiom would break.)
* Define `RequestInit.setHeaders` and `RequestInit.appendHeaders` as type `record<ByteString, ByteString>`. If specified, this is equivalent to calling `request.headers.set()` or `request.headers.append()` with each key/value pair after the request object is constructed.

Similarly, let's fix `Response` to use the same rewrite idiom:

* Allow `Response`'s constructor to take another `Response` object as the first parameter, in the same way `Request`'s constructor does today.
* Define `ResponseInit.body` as an alternative way to override the body, in the specific case where the constructor's first parameter is an existing `Response` object.
* Define `ResponseInit.setHeaders` and `ResponseInit.appendHeaders` to work the same as with requests.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/fetch/issues/671