Re: Design: Ignored Unknown Frame Types and Intermediaries

Hi Roberto

The beauty of extensions as opposed to protocol versions, is that it's easier to run several of these experiments at once.

So it's great that you can have versions "HTTP/2.1NewFlowControlModel" and "HTTP/2.1DynamicPriorities" and "HTTP/2.1NewHeadersFormat", but then the server can only choose one of these experiments. And the safest route for middleboxes to take will be to force the protocol down to the HTTP/2.0 which they understand. That may be what we want firewalls to do, but other proxies may not be affected by some of these.

But I agree that we should limit what non-version-changing extensions are allowed to do. We should require that if the extension is either ignored by the recipient or removed by a middlebox, no harm would be done (except the new functionality not working)

Yoav

On May 14, 2013, at 2:15 AM, Roberto Peon <grmocg@gmail.com<mailto:grmocg@gmail.com>> wrote:

Everything in the middle would have had its chance to do something when the initial negotiation goes through its end-to-end route.

Using DATA frames with a format that the intermediary doesn't understand (many of them do examine content) doesn't guarantee that the session will work properly either.

A RST_STREAM is insufficient if any modification to shared state has occurred. It still requires that the client or server keep state about when it should not use these unknown opcodes when speaking to an endpoint from a specific network (since the client likely roams amongst different typologies these days).

Things that wouldn't cause session-level problems, nor stream-level problems if removed:

  *   An unknown opcode which provides a more detailed hint about prioritization
  *   An unknown opcode which provides data about the state of the browser cache
  *   An unknown opcode which provides data about connection-state, number of connections, etc.
  *   An unknown opcode which points out when flow control is blocked

The above are examples of things I'm saying should be allowed in a known version of the protocol, and which are fine to remove/ignore if not understood.


The section below contains examples of things which shouldn't be allowed in a known version of the protocol, but should be fine in some variant of it if all parties agree to it.
Things that would obviously cause session-level problems if removed:

  *   An unknown opcode which touches on flow control
  *   An unknown opcode which modifies compression state
  *   An unknown opcode which reserves/assigns/changes stream IDs
  *   An unknown opcode which closes multiple streams.

These I'm not sure about either way. I've already had the displeasure of dealing with things that cause stream-level problems, and I'm not excited about the prospect of having to figure this out (second hand, as it usually happens as some tiny segment of the populace is having problems and the percentage of them who actually *tell* you about it small, and then those who are willing to help you debug it is even smaller...).

Things that wouldn't cause session-level problems, but would cause stream-level problems if removed:

  *   An unknown opcode which indicates that byte ranges come out of order (and which specifies that order)
  *   An unknown opcode which indicates that the encoding type has changed.
  *   An unknown opcode which indicates that the rest of the payload is being sent uncompressed or compressed (when it was the opposite before).

As I've said before, I believe that this should be a living protocol, and not ossified. That does imply that both experimentation needs to be accommodated and that successful experimentation becomes interoperable.

ALPN and NPN work/worked because their addition to TLS caused no failure when ignored or not understood. Now that we have ALPN and NPN, and now that using a new version is so amazingly easy in comparison to what it was before, it makes sense to use that fact to reduce complexity instead of falling into the same trap that HTTP fell into w.r.t. extensions and unused features. I believe that this will allow for more experimentation, more quickly, and much faster standardized interop than will extensions that require significant work and complexity to deal with failure scenarios.


-=R


On Mon, May 13, 2013 at 3:35 PM, James M Snell <jasnell@gmail.com<mailto:jasnell@gmail.com>> wrote:
On Mon, May 13, 2013 at 3:08 PM, Roberto Peon <grmocg@gmail.com<mailto:grmocg@gmail.com>> wrote:
> I believe that, if one does remove/ignore such frames the session may not be
> as optimal as it might otherwise be, but the rule, by definition, precludes
> the use of anything which would hinder interop if emitted, but then ignored
> by the receiver.
> The rule is that one MUST NOT send a frame if its removal would cause
> session corruption/termination/problems.
> So, can you provide an example? :)
>

The challenge with this is that the sender may know what it and the
origin server supports, it will generally have absolutely no clue what
everything in the middle supports... this means the default position
would be: don't ever send anything other than the core frame types,
which kills the whole point of allowing extensibility in the first
place (and yes, I know your position on extensibility ;-) ...).

In the worse case, what you'll end up with are implementations that
use DATA frames to tunnel (hack) their extension stuff through..
(because they know those will make it thru). We see that anti-pattern
repeated time and time again with things like X-Post-Method-Override,
DNS TXT records, SOA POST tunneling, Deletes-using-GET or POST
methods, etc. I'd rather not encourage similar abuse here.

>
> I think the version string is the most robust way to ensure that an
> intermediary does the right thing-- If an intermediary sees a version string
> it doesn't recognize, it simply doesn't allow the negotiation of that
> variant.

Imo, Having a rule that says either forward everything or forward
nothing achieves the same result, without the need for any additional
"negotiation".

> This is more simple because the lack of an explicit negotiation for this
> will lead to either:
> 1) implementations which ignore the spec, since their mission is to provide
> security and consistency
> 2) Clients, servers, and proxies all maintaining a heuristic and
> table/history about when and why sessions were terminated when a not
> standardized opcode was
>  I don't want to have to have maintain a table of endpoints which have sent
> me some error-code or terminated my session because I included an unknown
> opcode type (that would be pretty complicated). This would have to be done
> for both sides, for that matter.
>

Such a table would not be necessary. If my user agent sends a
particular extension frame and a middlebox responds with a
RST_STREAM('UNSUPPORTED_FRAME') then I just move on and either try
something else or give up. Otherwise, if the middlebox forwards those
frames on, things just work. There's no reason to terminate the
session, just cancel the stream within the session. There's also no
reason to keep any kind of table for what frame types work or don't...
you either use it and it doesn't work, or you use it and it does.

- James

> I'd prefer to say that an intermediary MUST NOT touch things in the session
> which it does not recognize if it doesn't know the version, but allowed the
> version to be negotiated, and it MAY remove things it doesn't know from a
> session when the version is one which is fully specified.
>
> That allows the intermediary to simply not negotiate things it doesn't like
> on a per-connection basis, instead of having to have this complex cache of
> 'this feature doesn't work when X,Y,Z'.
>
> James has pointed out that a reverse proxy may still have some difficulties
> with this, assuming it wishes to support an unknown extention, but, I think
> that trade (no longer requiring clients and servers to have to worry about
> this, while having the proxy continue to worry about it) is reasonable.
>
> -=R
>
>
>
> On Mon, May 13, 2013 at 2:13 PM, Martin Thomson <martin.thomson@gmail.com<mailto:martin.thomson@gmail.com>>
> wrote:
>>
>> On 13 May 2013 11:37, Roberto Peon <grmocg@gmail.com<mailto:grmocg@gmail.com>> wrote:
>> > James:
>> > Can you construct a case where, if you follow the rule spelled out in my
>> > earlier email, you fail to achieve interop (because I can't)?
>>
>> It's trivially possible to construct a scenario where this happens,
>> but only if you don't write down a "MUST ignore" rule.  Once the rule
>> is in place, then you are constraining future extensibility.  A "MUST
>> ignore" rule is the easiest rule to get right, but there are other
>> models you can use.
>>
>> > The rule is, essentially:
>> > If a party to the communication ignores (or removes) something it don't
>> > understand, that must not screw up the session.
>>
>> The ignore/remove distinction is very important.  You can't
>> selectively remove; it's all or nothing.  Either remove everything you
>> don't know about or leave it all in.
>>
>> This consideration, along with James' hop-by-hop question does suggest
>> a relatively simple way out:
>>
>> All unsupported/unknown frames that have a non-zero stream identifier
>> MUST be ignored.  If a stream is forwarded by an intermediary, all
>> unsupported/unknown frames MUST either be forwarded or removed; an
>> intermediary MUST NOT selectively forward unsupported frame types.
>> Unsupported/unknown frames with a zero stream identifier MUST be
>> ignored and MUST NOT be forwarded.
>>
>> > That implies that, when you add anything that must be interpreted, it
>> > then
>> > must be declared in the version string (i.e. new version) and thus
>> > agreed
>> > upon by both parties up front, and if you don't negotiate that other
>> > version, you don't get to add frames whose removal would screw up the
>> > session.
>>
>> Yeah, we addressed that early on.  If you want to guarantee that the
>> other guy is going to support something, either work out how to agree
>> in-session (with those ignored frames) or negotiate a new protocol.
>
>



Email secured by Check Point