Re: Comments on Extensible Prioritization Scheme for HTTP

On Sat, Apr 09, 2022 at 09:58:16AM -0400, Glenn Strauss wrote:
> On Sat, Apr 09, 2022 at 02:37:56PM +0200, Willy Tarreau wrote:
> > On Sat, Apr 09, 2022 at 07:25:57AM -0400, Glenn Strauss wrote:
> > > For widest client compatibility, lighttpd also plan to omit sending
> > > SETTINGS_NO_RFC7540_PRIORITIES=1
> > 
> > HAProxy doesn't implement priority either and currently sends nothing.
> > However once this draft becomes standard, I plan to send this setting
> > to inform compatible clients that they do not have to expect any
> > particular behavior from it on this point. It's not a matter of client
> > compatibility, H2 agents are required to ignore unkonwn settings, so
> > those unaware of this setting will not change anything, while those
> > aware of it will benefit from it.
> 
> It would be trivial for lighttpd to unconditionally send
> SETTINGS_NO_RFC7540_PRIORITIES=1 with server connection preface.
> 
> However, I am choosing not to do so (at least for now) due to:
> https://mikebishop.github.io/http-misc-extensions/draft-bishop-httpbis-grease.html
> "The need for such a technique was demonstrated recently by an HTTP/2
>  implementation which closed the connection upon receipt of an unknown
>  setting."
> 
> I do not know which client this was or how widely it is used, but it is
> mentioned in the grease draft as a good reason supporting greasing
> HTTP/2 SETTINGS.

It can happen from time to time on early implementations, which is why
greasing is good. But usually such implementations don't last long.
Several times we noticed trouble with websocket a long time ago and H2
more recently that were caused in fact by anti-malware programs plugged
into browsers and inspecting the stream. Having the user disable them
was enough to fix the trouble, which was likely sufficient to report
the issue to the vendor and have it fixed soon. It turns out that the
whole web mostly runs on very few implementations (both clients and
servers), and if you have just 2-3 of them that induce breakage with a
non-compliant one, that last one is urged to get fixed otherwise it will
become so much unusable that it will disappear.

> I have completed implementing PRIORITY_UPDATE on my public dev branch.
> I want to avoid a lighttpd upgrade breaking an existing client when
> PRIORITY_UPDATE is not yet standardized.

You shouldn't worry about this, really. Implementations that break with
lighttpd will have trouble visiting many sites, and would probably have
trouble with some sites already. It's more difficult when you deal with
embedded clients (software updates from cars, IoT devices, payment
terminals etc) that are not updated often, because you may have to add
an option to disable this. But quite frankly, you'd rather implement
what you need to improve your users' experience than live in the past by
fear of breaking already broken clients. I'm personally very careful about
not breaking stuff, and despite this we continue to move forward. A few
times we've met trouble with extremely popular clients that were very
quickly resolved, either on our side if we were faulty, or on the browser
side when it was their turn, and in the end users were very pleased by
the reactivity and already don't remember about this.

The real risk you're facing by implementing something that is not standardized
is to have to change some frame IDs or the behavior id these were to change
later. That's a risk I'm willing to take during development but not in LTS
versions, even though it's extremely low.

> > > Currently, SETTINGS_NO_RFC7540_PRIORITIES=1 is proposed for client to
> > > tell the server that the server need not keep RFC7540 PRIORITY state,
> > > but existing clients do not send that, and many also do not utilize
> > > PRIORITY.
> > 
> > The most important is that many clients *do* support priority, that
> > these ones are already deployed and working fine, and that you cannot
> > suddenly break them by telling them "ah, you didn't read the news, the
> > default value recently changed and while you used to comply with the
> > standard now you violate it".
> 
> Prioritization signals may be ignored by the server.  Clients will not
> "break".

They can notice a serious difference. I've seen this exactly once. One
hosting provider hosts hundreds of applications among which one has much
worse load time in H2 than H1 through haproxy, and they verified that when
joining the server directly, priorities would address this. Thus they
disabled H2 for this specific application and that was all. That's what
I mean by "breaking". If the application is designed to firmly rely on
priorities, it may be degraded. And in my opinion it's not acceptable
to cause that trouble with perfectly complying software.

>  SETTINGS_NO_RFC7540_PRIORITIES=1 is not standardized, so it is
> to be expected that most/all current clients do not send that setting,
> and it is unknown when they will be upgraded, and when upgraded,
> unknown if this will be a required setting.  For server implementations
> which implement PRIORITY, this may be a missing signal.

Sure but if the client sends this it also stops expecting the regular
priorities to be used. That's the point.

> Will a future revision of HTTP/2 include this signal, saying that a
> client SHOULD send SETTINGS_NO_RFC7540_PRIORITIES=1 in the client
> connection preface if the client does not intend to use RFC7540
> PRIORITY?  Or will a future revision of HTTP/2 deprecate PRIORITY?

We don't really care. Clients and servers *are* expected to support
PRIORITY. It's not an option in the protocol, it's not negotiated.
Without a specific extension like this draft, both sides are expected
to implement and support it. It's just that it's easy to silently ignore
and complex to implement with little return on investment and many of
us delayed that implementation to the point of no more intending to
implement it.

> > That's why one must never change default values once adopted, and why
> > the default set must be well thought before a standard is released!
> 
> I agree with you that this setting might be better as an advertisement
> of bitflags, supporting up to 32 different prioritization schemes,
> with 0 indicating none supported.  (bitflags in one SETTING would be an
> alternative to separate SETTINGs to enable PRIORITY, PRIORITY_UPDATE.)

Yes I also thought about a bit flag that would do it fine to let both
sides decide which one to agree on.

> That would suggest the server send something akin to
> SETTINGS_PRIORITIZATION_SCHEMES
>   0x0 none
>   0x1 PRIORITY_UPDATE
>   0x2 PRIORITY
>   0x4 ???
>   0x8 ...
> and the flags can be OR'd together.

I would personally swap 1 & 2 to keep them in chronological and/or
preference order. I.e. AND the two peers' flags and pick the topmost
bit. But that's a detail.

> If the client sent SETTINGS_PRIORITIZATION_SCHEMES, that would be a hint
> to the server, especially if the bitflag for PRIORITY were omitted so
> that the server knows affirmatively that the server need not keep
> PRIORITY state.

Definitely.

> Absense of the setting from from server connection preface would
> indicate RFC7540 defaults or could be affirmatively indicated with
>   SETTINGS_PRIORITIZATION_SCHEMES=2
> 
> In the above
>   SETTINGS_PRIORITIZATION_SCHEMES=1 == SETTINGS_NO_RFC7540_PRIORITIES=1
> but 
>   SETTINGS_PRIORITIZATION_SCHEMES=0 indicates no prioritization,
>     as opposed to SETTINGS_NO_RFC7540_PRIORITIES=0 which indicates
>     RFC7540 defaults, and could be equally achieved by omitting
>     SETTINGS_NO_RFC7540_PRIORITIES=0.
> 
> Willy, I like your proposal to make the setting a more bona fide
> advertisement and hope that the PRIORITY_UPDATE proposal authors will
> take another look at SETTINGS_NO_RFC7540_PRIORITIES

I agree that it would be more durable and a bit cleaner, but quite
frankly, I don't expect this to be that big a game changer, because I
wouldn't be surprised if in 10 years we'll only have H1 and H3 (or H4),
i.e. legacy vs low-latency, thus I suspect that this setting will not
evolve further.

> https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-priority-12#section-2.1
>    The value of SETTINGS_NO_RFC7540_PRIORITIES MUST be 0 or 1.  Any
>    value other than 0 or 1 MUST be treated as a connection error (see
>    Section 5.4.1 of [HTTP2]) of type PROTOCOL_ERROR.  The initial value
>    is 0.

Indeed this one could be changed to allow other values, in the spirit
of forward compatibility and greasing.

Willy

Received on Saturday, 9 April 2022 16:20:45 UTC