Re: Proposal: Adopt State Synchronization into HTTPbis

Server-sent events (SSE) demonstrates that there is interest in a simple
method for receiving events/updates from a server. It doesn't cover every
use case but offers a great alternative to WebSockets and WebTransport if
you just need uni-directional messaging of short text-based events.
Similarly, I don't think subscriptions need to be built upon WebSockets or
WebTransport unless there is need for the additional features offered by
them.

Regarding your example, Michael: Does the response body, which contains the
updates, adapt its syntax to the used HTTP protocol? Do you suggest that
subscriptions over HTTP/2 generate the updates as additional HTTP/2
responses? If so, this would require subscriptions to be implemented inside
the HTTP client itself instead of being a feature that a user can implement
based upon existing and available HTTP clients. In addition, this raises
questions about how to handle situations where the used protocol changes as
requests and responses are forwarded through proxies and gateways.

I also wonder if we could reuse multipart responses for delivering updates.
The response to a subscription request would be a streamed multipart
response where each "part" is one update. The update can include header
fields as well as content, similar to your example. Status codes in the
update would not be directly possible, but I'm not sure if that's a big
loss.

All in all, I enjoy the idea, but think that we can achieve this already
with the existing features we have.

Best regards,
Marius Kleidl

On Wed, Oct 9, 2024 at 1:18 AM Michael Toomim <toomim@gmail.com> wrote:

> Josh Cohen and I are considering publishing a new draft on Subscriptions
> in HTTP, and as we think through the big design decisions, I ran across
> this excellent question from Watson Ladd bringing up the most fundamental
> question of them all:
> On 11/6/23 4:47 PM, Watson Ladd wrote:
>
> On Tue, Oct 31, 2023 at 7:12 PM Michael Toomim <toomim@gmail.com> <toomim@gmail.com> wrote:
>
> At IETF 118 I will present a proposal to adopt State Synchronization work into HTTPbis:
>
> Braid-HTTP: https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http [1]
>
> <...snip...>
>
> The big sticking point for me is subscriptions. This is a deviation
> from the request/response paradigm that goes pretty deep into how
> clients and servers are coded and the libraries they use. It can of
> course be stuck on top of WebTransport, which might be the right way
> to do it, but then doesn't integrate with the other three parts.
>
> You might be better trying to layer this on top of HTTP and
> WebTransport, as ugly as that can be with regard to what
> intermediaries can do in order to get it into the hands of people
> faster, but if there's some strong reason not to do that I'm all ears.
>
> Watson raises a basic choice in designing Subscriptions:
>
>    1. Do we dare extend the basic request/response model to allow
>    long-lived subscriptions?
>    2. Or are subscriptions better layered on top— inside a WebSocket, or
>    WebTransport connection?
>
> I argue that (1) is actually a *much* better choice. Yes, it is
> fundamental. It extends the basic architecture of HTTP (hat tip to Roy
> Fielding) by extending REST into RESS (REpresentational State
> Synchronization). It adds a new basic feature to HTTP— the ability to
> subscribe to any resource, and get notified of its changes over time;
> throughout the entire web and HTTP ecosystem. Clients will stop guessing
> whether to reload cache, and will stop making redundant requests.
> Servers—which *authoritatively know* when resources change—will promise to
> tell clients, automatically, and optimally. Terabytes of bandwidth will be
> saved. Millions of lines of cache invalidation logic will be eliminated.
> Quadrillions of dirty-cache bugs will disappear. In time, web browser
> "reload" buttons will become obsolete, across the face of earth.
>
> The alternative (2) is to add subscriptions on top of a WebSocket or
> WebTransport API, separate from HTTP resource semantics. But then HTTP
> resources themselves will not be subscribable. Programmers will be annoyed,
> and limit their use of HTTP to bootstrapping the initial HTML, CSS and
> Javascript; migrating all the interesting state onto this separate
> WebSocket or WebTransport mechanism, which will then require more and more
> of HTTP's features being added back into it: like (a) being able to GET the
> state, and also PUT changes to the it, over (b) multiple content-types
> (e.g. text/json, text/html, and image/png), while (c) supporting various
> PATCH types, across (d) a full-featured network of Proxies, Caches, and
> CDNs to scale the network. In conclusion, choice (2) leads to reinventing
> HTTP, within a WebSocket/Transport... on top of HTTP.
>
> The clear way forward is subscribing directly to HTTP and REST state. An
> elegant way to see this is extending Request/Response into
> Request/Multiresponse. The Subscription can be a Request that receives
> multiple Responses, one for each update to the resource. There are many
> ways to format a Multiresponse; here's a straightforward and
> backwards-compatible option:
>
>       Request:
>
>          GET /chat
>          Subscribe: timeout=10s
>
>       Response:
>
>          HTTP/1.1 104 Multiresponse
>          Subscribe: timeout=10s
>          Current-Version: "3"
>
>          HTTP/1.1 200 OK
>          Version: "2"
>          Parents: "1a", "1b"
>          Content-Type: application/json
>          Content-Length: 64
>
>          [{"text": "Hi, everyone!",
>            "author": {"link": "/user/tommy"}}]
>
>          HTTP/1.1 200 OK
>          Version: "3"
>          Parents: "2"
>          Content-Type: application/json
>          Merge-Type: sync9
>          Content-Length: 117
>
>          [{"text": "Hi, everyone!",
>            "author": {"link": "/user/tommy"}}
>           {"text": "Yo!",
>            "author": {"link": "/user/yobot"}]
>
> This is backwards-compatible because it encodes multiple responses into a
> regular response body that naïve intermediaries will just pass along
> blindly, like SSE. But upgraded H2 and H3 implementations can have native
> headers & body frames that repeat. It's all quite elegant. It fits right
> into HTTP. It feels as if HTTP was designed to make it possible.
>
> We can add subscriptions to the basic fabric of HTTP, and free application
> programmers from having to write cache-invalidation logic. This will (a)
> eliminate bugs and code complexity; while simultaneously (b) improving
> performance across the internet, and (c) giving end-users the functionality
> of a realtime web by default. This is a fundamental change, but it is
> overwhelming beneficial. Then we can update Roy's dissertation. It's a good
> one, and deserves our care.
>
> Michael
>