Re: [whatwg/fetch] Request body streams should use chunked encoding (#966)

> @bradisbell [wrote](https://github.com/whatwg/fetch/issues/966#issuecomment-592535669):
> There are servers today (such as Icecast) that will probably never support HTTP/2, which work just fine by receiving a stream sent over an HTTP request. In my opinion, the complexity of going to HTTP/2 is massive for projects like these. There are 20 years worth of deployments for Icecast alone.

It does not seem "massive" to potentially place a forward proxy in place (e.g. a service like [Caddy](https://caddyserver.com/)) for such services. That may seem dismissive, but it's about respecting the [priority of constituencies](https://www.w3.org/TR/html-design-principles/#priority-of-constituencies) and placing users first. Using our Windows users over a 28 day window, approximately 6.7% of requests in Chrome go through a locally-configured TLS endpoint. However, by user, that's approximately 17.3% of Windows users going through such a service within that same time period, indicating a substantial portion of our users.

The most common cause of this, by far, is the use of [antivirus that performs TLS inspection](https://www.virusbulletin.com/blog/2017/02/security-products-and-https-lets-do-it-better/) of traffic, acting as a transparent proxy to inspect requests and responses, scanning and modifying them. We have [ample experience](https://www.youtube.com/watch?v=_mE_JmwFi1Y) in the challenges these services pose to making otherwise sensible ecosystem changes.

However, unlike TLS 1.3, which involved cryptographic negotiation, the failure scenario is not so easily tested. Historically, when considering features to roll out, such as the development of QUIC and TLS 1.3, the Chrome networking team constructed careful test scenarios to measure the success rate of various proposals. That is, using a well-known server endpoint which was configured correctly, carefully measuring the success rate of various client approaches. If @yutakahirano and @wenbozhu were proposing that approach first, to gather real world data without necessarily making it part of the Web Platform, I'd be much less concerned.

However, even when that approach was used, it's not perfect, because not all users opt-in to supporting Chrome experiments and metrics, and as a result, their environments are not considered in the design. The aforementioned link regarding TLS 1.3 shows what can happen there, while ultimately, QUIC had to decide to refuse to negotiate in such scenarios, because of middleboxes improperly implementing the protocol.

I can understand your appeal to other libraries, and the spec, as it seems like it should "just work". However, the Chrome team has ample experience that the real world [is anything but](https://tools.ietf.org/html/draft-iab-protocol-maintenance) that. The most depressing example of this I know was a popular DSL router (the most popular in the US at the time) [shutting down DNS](https://bugs.chromium.org/p/chromium/issues/detail?id=122566) if too many requests were received. The combination of DNS prefetch plus a change in our network resolver code, both fully spec conformant, would have knocked millions of users offline, had it not been caught by a Chrome VP who happened to use that particular router at home, and was able to have multiple Chrome engineers over to investigate and diagnose why Chrome Canary was knocking them offline.

> It's possible I don't know all of the complexities you're talking about, but it feels trivial to me for the JS developer:
> 
> ```
> fetch('https://example.com/some-stream', {
>   method: 'PUT',
>   body: someReadableStream
> });
> ```

Indeed, the API is the simple part. The concern is the intersection that chunked encoding is going to have for that 17.3% of Windows users (ignoring other platforms for sake of discussion). The 'best case' scenario is this feature simply does not work, and such fetch requests fail. The Web developer now has to try to figure out why it's failing, and the user experience of such failure is, more often than not, poor, because the developer has failed to account for the possibility, since it's "trivial" to do. The worst case, however, is that this can interfere with or disrupt such transparent proxies.

Is it a bug in the proxy if they do? Absolutely. Is it something someone could do with Curl, wget, ffmpeg, or any other native program running on the device? Yup, unfortunately. However, the prospect of exposing this to the Web, with the risk of disruption and incompatibility, is high, particularly given this is an area we know has extremely high likelihood of being implemented incorrectly.

The proposal to restrict to HTTP/2 attempts to mitigate this risk, on the client side, as generally, those proxies will not negotiate HTTP/2 or, if they do, are less likely to have issues in this particular area of code because it's a [well-oiled mechanism](https://tools.ietf.org/html/draft-ietf-tls-grease-02#section-1) of HTTP/2.

I can understand this seems overly conservative, but the Chrome Networking team has seen all manner of depressing things, and that's why it [makes us nervous](https://github.com/whatwg/fetch/issues/966#issuecomment-554985629).  Typically, before exposing something to the Web, let alone as an API, we've done wide-scale measurements as best we can to assess compatibility risks. For features that we expect may affect users who have not opted in to User Metrics (e.g. Enterprise users), we're even more cautious in the deployment, because we know it represents a *significant* investment to get working correctly.

The suggestions here are based on strategies to reduce the risk, while still hopefully making forward progress on addressing author's real needs, even if not completely. I'd much rather there be 10,000 sad Icecast authors than [172 million](https://venturebeat.com/2015/05/28/google-chrome-now-has-over-1-billion-users/) sad Chrome users. Even if we assume only 0.1% of users would encounter some breakage, that's still an order of magnitude more sadness. And I hate sad users 💔 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/fetch/issues/966#issuecomment-592557034