Re: Design: Ignored Unknown Frame Types and Intermediaries

Everything in the middle would have had its chance to do something when the
initial negotiation goes through its end-to-end route.

Using DATA frames with a format that the intermediary doesn't understand
(many of them do examine content) doesn't guarantee that the session will
work properly either.

A RST_STREAM is insufficient if any modification to shared state has
occurred. It still requires that the client or server keep state about when
it should not use these unknown opcodes when speaking to an endpoint from a
specific network (since the client likely roams amongst different
typologies these days).

Things that wouldn't cause session-level problems, nor stream-level
problems if removed:

   - An unknown opcode which provides a more detailed hint about
   prioritization
   - An unknown opcode which provides data about the state of the browser
   cache
   - An unknown opcode which provides data about connection-state, number
   of connections, etc.
   - An unknown opcode which points out when flow control is blocked

The above are examples of things I'm saying should be allowed in a known
version of the protocol, and which are fine to remove/ignore if not
understood.


The section below contains examples of things which shouldn't be allowed in
a known version of the protocol, but should be fine in some variant of it
if all parties agree to it.
Things that would obviously cause session-level problems if removed:

   - An unknown opcode which touches on flow control
   - An unknown opcode which modifies compression state
   - An unknown opcode which reserves/assigns/changes stream IDs
   - An unknown opcode which closes multiple streams.


These I'm not sure about either way. I've already had the displeasure of
dealing with things that cause stream-level problems, and I'm not excited
about the prospect of having to figure this out (second hand, as it usually
happens as some tiny segment of the populace is having problems and the
percentage of them who actually *tell* you about it small, and then those
who are willing to help you debug it is even smaller...).

Things that wouldn't cause session-level problems, but would cause
stream-level problems if removed:

   - An unknown opcode which indicates that byte ranges come out of order
   (and which specifies that order)
   - An unknown opcode which indicates that the encoding type has changed.
   - An unknown opcode which indicates that the rest of the payload is
   being sent uncompressed or compressed (when it was the opposite before).


As I've said before, I believe that this should be a living protocol, and
not ossified. That does imply that both experimentation needs to be
accommodated and that successful experimentation becomes interoperable.

ALPN and NPN work/worked because their addition to TLS caused no failure
when ignored or not understood. Now that we have ALPN and NPN, and now that
using a new version is so amazingly easy in comparison to what it was
before, it makes sense to use that fact to reduce complexity instead of
falling into the same trap that HTTP fell into w.r.t. extensions and unused
features. I believe that this will allow for more experimentation, more
quickly, and much faster standardized interop than will extensions that
require significant work and complexity to deal with failure scenarios.


-=R


On Mon, May 13, 2013 at 3:35 PM, James M Snell <jasnell@gmail.com> wrote:

> On Mon, May 13, 2013 at 3:08 PM, Roberto Peon <grmocg@gmail.com> wrote:
> > I believe that, if one does remove/ignore such frames the session may
> not be
> > as optimal as it might otherwise be, but the rule, by definition,
> precludes
> > the use of anything which would hinder interop if emitted, but then
> ignored
> > by the receiver.
> > The rule is that one MUST NOT send a frame if its removal would cause
> > session corruption/termination/problems.
> > So, can you provide an example? :)
> >
>
> The challenge with this is that the sender may know what it and the
> origin server supports, it will generally have absolutely no clue what
> everything in the middle supports... this means the default position
> would be: don't ever send anything other than the core frame types,
> which kills the whole point of allowing extensibility in the first
> place (and yes, I know your position on extensibility ;-) ...).
>
> In the worse case, what you'll end up with are implementations that
> use DATA frames to tunnel (hack) their extension stuff through..
> (because they know those will make it thru). We see that anti-pattern
> repeated time and time again with things like X-Post-Method-Override,
> DNS TXT records, SOA POST tunneling, Deletes-using-GET or POST
> methods, etc. I'd rather not encourage similar abuse here.
>
> >
> > I think the version string is the most robust way to ensure that an
> > intermediary does the right thing-- If an intermediary sees a version
> string
> > it doesn't recognize, it simply doesn't allow the negotiation of that
> > variant.
>
> Imo, Having a rule that says either forward everything or forward
> nothing achieves the same result, without the need for any additional
> "negotiation".
>
> > This is more simple because the lack of an explicit negotiation for this
> > will lead to either:
> > 1) implementations which ignore the spec, since their mission is to
> provide
> > security and consistency
> > 2) Clients, servers, and proxies all maintaining a heuristic and
> > table/history about when and why sessions were terminated when a not
> > standardized opcode was
> >  I don't want to have to have maintain a table of endpoints which have
> sent
> > me some error-code or terminated my session because I included an unknown
> > opcode type (that would be pretty complicated). This would have to be
> done
> > for both sides, for that matter.
> >
>
> Such a table would not be necessary. If my user agent sends a
> particular extension frame and a middlebox responds with a
> RST_STREAM('UNSUPPORTED_FRAME') then I just move on and either try
> something else or give up. Otherwise, if the middlebox forwards those
> frames on, things just work. There's no reason to terminate the
> session, just cancel the stream within the session. There's also no
> reason to keep any kind of table for what frame types work or don't...
> you either use it and it doesn't work, or you use it and it does.
>
> - James
>
> > I'd prefer to say that an intermediary MUST NOT touch things in the
> session
> > which it does not recognize if it doesn't know the version, but allowed
> the
> > version to be negotiated, and it MAY remove things it doesn't know from a
> > session when the version is one which is fully specified.
> >
> > That allows the intermediary to simply not negotiate things it doesn't
> like
> > on a per-connection basis, instead of having to have this complex cache
> of
> > 'this feature doesn't work when X,Y,Z'.
> >
> > James has pointed out that a reverse proxy may still have some
> difficulties
> > with this, assuming it wishes to support an unknown extention, but, I
> think
> > that trade (no longer requiring clients and servers to have to worry
> about
> > this, while having the proxy continue to worry about it) is reasonable.
> >
> > -=R
> >
> >
> >
> > On Mon, May 13, 2013 at 2:13 PM, Martin Thomson <
> martin.thomson@gmail.com>
> > wrote:
> >>
> >> On 13 May 2013 11:37, Roberto Peon <grmocg@gmail.com> wrote:
> >> > James:
> >> > Can you construct a case where, if you follow the rule spelled out in
> my
> >> > earlier email, you fail to achieve interop (because I can't)?
> >>
> >> It's trivially possible to construct a scenario where this happens,
> >> but only if you don't write down a "MUST ignore" rule.  Once the rule
> >> is in place, then you are constraining future extensibility.  A "MUST
> >> ignore" rule is the easiest rule to get right, but there are other
> >> models you can use.
> >>
> >> > The rule is, essentially:
> >> > If a party to the communication ignores (or removes) something it
> don't
> >> > understand, that must not screw up the session.
> >>
> >> The ignore/remove distinction is very important.  You can't
> >> selectively remove; it's all or nothing.  Either remove everything you
> >> don't know about or leave it all in.
> >>
> >> This consideration, along with James' hop-by-hop question does suggest
> >> a relatively simple way out:
> >>
> >> All unsupported/unknown frames that have a non-zero stream identifier
> >> MUST be ignored.  If a stream is forwarded by an intermediary, all
> >> unsupported/unknown frames MUST either be forwarded or removed; an
> >> intermediary MUST NOT selectively forward unsupported frame types.
> >> Unsupported/unknown frames with a zero stream identifier MUST be
> >> ignored and MUST NOT be forwarded.
> >>
> >> > That implies that, when you add anything that must be interpreted, it
> >> > then
> >> > must be declared in the version string (i.e. new version) and thus
> >> > agreed
> >> > upon by both parties up front, and if you don't negotiate that other
> >> > version, you don't get to add frames whose removal would screw up the
> >> > session.
> >>
> >> Yeah, we addressed that early on.  If you want to guarantee that the
> >> other guy is going to support something, either work out how to agree
> >> in-session (with those ignored frames) or negotiate a new protocol.
> >
> >
>