Re: HTTP/2 and Websockets

On 21 November 2014 13:13:02 GMT+08:00, Andy Green <andy@warmcat.com> wrote:
>
>
>On 21 November 2014 12:33:45 GMT+08:00, Yutaka Hirano
><yhirano@google.com> wrote:
>>>
>>>  - WSDATA's 8-bit flags field, is the ws flags [fin][rsv1-3][opcode]
>>
>>We may want END_STREAM as well.
>
>Good point... and to be friendly with http2's obfuscation goals,
>PADDING.
>
>It means we need an extra byte to hold all the flags... well it's not
>too bad per-frame.
>
>If so, we could also do everything the same reusing using DATA with a
>flag "WS", indicating the payload is [WS_FLAGS byte + ws payload]. 
>That has an advantage --->
>
>>Corresponding http2 frames can substitue ping / pong / close (note:
>>WebSocket ones have payload though h2 ones don't). Shoud we have
>>opcodes
>>for them?
>
>We should probably keep the ws ones too.  Things like ws echo services
>do use the payload.  Actually endpoints can prefer the http2 ones if
>the see they can use them and know how to.  But supporting both
>everyone's happy.
>
>Ws has his own close codes and payload, if we try mapping those on
>http2 codes etc it might be a step too far.  So keeping ws close frame
>will be good.
>
>Close is a bit tricky if we try to say that a WSDATA close is an
>implicit http2 stream close everybody has to understand that, even if
>they were intermediaries that didn't really understand WSDATA and were
>just passing it through.  That's probably too much to ask.
>
>If we re-use DATA with a WS flag, the extra advantage is all http2
>intermediaries and endpoints already know the meaning of DATA frame
>with END_STREAM, so even if they do not follow what WS flag on DATA
>means, they can still follow the stream lifecycle without extra work.
>
>How realistic is it to get a DATA flag bit assigned now?

Ah since the endpoints negotiated ws using HEADERS, actually we could do without a WS flag on DATA.

The fact that it's DATA for stream ID xxx and that ID went through a ws handshake means both sides and all intermediaries present during the ws handshake can know it is ws protocol in the DATA without an explicit flag.

So we could just use DATA for this without any modification.  Peers that understand websockets can know in some streams there's a WS_FLAGS byte at the start of the DATA payload.  Peers that don't understand websockets just move opaque DATA payloads around as usual and only see normal DATA semantics going on.

-Andy

>-Andy
>
>>
>>On Fri, Nov 21, 2014 at 1:13 PM, Andy Green <andy@warmcat.com> wrote:
>>
>>>
>>>
>>> On 21 November 2014 11:38:26 GMT+08:00, Andy Green
><andy@warmcat.com>
>>> wrote:
>>> >
>>> >
>>> >On 21 November 2014 11:30:46 GMT+08:00, Yutaka Hirano
>>> ><yhirano@google.com> wrote:
>>> >>On Fri, Nov 21, 2014 at 11:42 AM, Andy Green <andy@warmcat.com>
>>wrote:
>>> >>
>>> >>>
>>> >>>
>>> >>> On 21 November 2014 10:02:28 GMT+08:00, Yutaka Hirano
>>> >><yhirano@google.com>
>>> >>> wrote:
>>> >>> >>
>>> >>> >> I think you have to take the atomic large ws frame thing as a
>>> >>genuine
>>> >>> >> problem because http2 has the transmit credit concept.  So
>>even
>>> >if
>>> >>> >you
>>> >>> >> buffered one ws frame, you can't sit there spewing as much
>>http2
>>> >>DATA
>>> >>> >frame
>>> >>> >> as it needs to atomically encapsulate it, without sizing your
>>> >>http2
>>> >>> >frame
>>> >>> >> to suit the tx credit.
>>> >>> >> But it's OK because the implementation can transparently
>>fragment
>>> >>the
>>> >>> >ws
>>> >>> >> data using the ws message semantics... I think there's no
>>choice
>>> >>but
>>> >>> >to
>>> >>> >> take that approach.
>>> >>> >
>>> >>> >Sorry I don't understand what you are proposing. Can you
>>explain?
>>> >>>
>>> >>> I'm agreeing with what was already written by someone else on
>the
>>> >>thread.
>>> >>>
>>> >>> Talking about buffering huge ws frames until you have enough to
>>> >issue
>>> >>it
>>> >>> all in one big http2 DATA frame will not fly.
>>> >>>
>>> >>> If you're using this putative ws-over-http2 scheme, and you get
>>> >given
>>> >>a
>>> >>> huge ws frame to transmit, you should fragment it using RFC6455
>>> >>message
>>> >>> semantics to some implementation-defined limit that is friendly
>>for
>>> >>mux'd
>>> >>> http2 transport.
>>> >>>
>>> >>Thanks.
>>> >>
>>> >>Strictly speaking, RFC6455 allows an extension to give meaning to
>>> >>WebSocket
>>> >>frames, so merging / fragmenting frames breaks such extensions.
>>> >>We discussed this problem in HyBi and many of us said "don't
>care".
>>> >
>>> >Yeah extensions except for compression have just not come into
>>> >existence.
>>> >
>>> >So I also see it as don't care.  I'm not even sure it's true since
>>the
>>> >intention during ws discussion was an intermediary can fragment
>>frames
>>> >same as how tcp packets may be fragmented.
>>> >
>>> >>In any case, an http/2 frame cannot be bigger than 2^24 (or 2^14
>>> >>without an
>>> >>explicit permission), so I think we don't have to worry about DoS.
>>> >
>>> >... I don't see that.  If I keep spamming 16MB frames even on one
>>> >stream on a consumer link your latency goes to pieces and if
>>something
>>> >is doing multiple instances of it even a big pipe will feel pain.
>>> >
>>> >>> >I guess it can be done in parallel with http2 coming to an end
>>> >>rather
>>> >>> >than
>>> >>> >> trying to block it, just by defining some new optional frame
>>> >>types...
>>> >>> >
>>> >>> >I agree to define a ws-dedicated frame type and use it.
>>> >>>
>>> >>> Super... has anyone proposed how to map RFC6455 to http2 framing
>>in
>>> >>detail
>>> >>> yet?
>>> >>>
>>> >>I think not.
>>> >>I did list several ways at [1], but I deleted it from the next
>>> >
>>> >Alright I will go look at these.
>>>
>>> 5.2.1 is like pure tunneling, it's simple alright but it's bloated
>>for
>>> small frames.
>>>
>>> 5.2.2 has a needless flushing thing on top... in a real ws setup
>>pieces of
>>> data can arrive any old way they should be passed on as they come or
>>> coalsced according to intermediary policy, I don't see the point.
>>>
>>> 5.3.3 headers + data per websocket frame... super inefficient
>>>
>>> Has it already been discussed to just merge http2 frame with ws
>>frame?
>>>
>>>  - use the headers scheme you defined to negotiate the connection +
>>ws
>>> protocol
>>>
>>>  - There's an http2 frame type called like WSDATA
>>>
>>>  - WSDATA's 8-bit flags field, is the ws flags [fin][rsv1-3][opcode]
>>>
>>>  - WSDATA's payload, is the ws frame payload... the ws framing
>>disappears
>>> completely into the http2 framing.
>>>
>>> ... what problems does this create?
>>>
>>> -Andy
>>>
>>> >The discussion is moribund somebody should probably issue a
>>'stalking
>>> >horse' since it's easier for people to jump in and tell you you're
>>> >doing it wrong ^^
>>> >
>>> >-Andy
>>> >
>>> >>version[2]
>>> >>because HTTP/2 situation had changed.
>>> >>
>>> >>1:
>>>
>>>>http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-00
>>> >>2:
>>>
>>>>http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-01
>>> >>
>>> >>
>>> >>>
>>> >>> -Andy
>>> >>>
>>> >>> >
>>> >>> >On Fri, Nov 21, 2014 at 10:47 AM, Andy Green <andy@warmcat.com>
>>> >>wrote:
>>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> On 21 November 2014 04:11:53 GMT+08:00, Robert Collins <
>>> >>> >> robertc@robertcollins.net> wrote:
>>> >>> >> >On 15 October 2014 00:00, Amos Jeffries
>><squid3@treenet.co.nz>
>>> >>> >wrote:
>>> >>> >> >> -----BEGIN PGP SIGNED MESSAGE-----
>>> >>> >> >> Hash: SHA1
>>> >>> >> >>
>>> >>> >> >> On 14/10/2014 11:01 p.m., Robert Collins wrote:
>>> >>> >> >>> On 1 October 2014 23:37, Amos Jeffries wrote:
>>> >>> >> >>>
>>> >>> >> >>>>> All the implementor discussion I've seen during the
>>> >>> >> >>>>>> HTTP/2 discussions has focused on how intermediaries
>>want
>>> >>to
>>> >>> >> >>>>>> be scalable: and buffering is anti-scaling. So - is it
>>a
>>> >>> >> >>>>>> pragmatic concern, or do we expect DATA stream
>>buffering
>>> >to
>>> >>> >> >>>>>> take place [outside of protocol gateways converting to
>>> >>> >> >>>>>> HTTP/1.1 where non upload can require buffering - and
>>note
>>> >>> >> >>>>>> that such a gateway can't carry ws anyway unless its
>>aware
>>> >>of
>>> >>> >> >>>>>> it, and if its aware of it, it can make sure it does
>>not
>>> >>> >> >>>>>> buffer].
>>> >>> >> >>>>
>>> >>> >> >>>>
>>> >>> >> >>>> I think the problem is not buffering in HTTP/2 per-se
>but
>>> >the
>>> >>> >> >>>> DATA frame (de-)aggregation that can happen if the
>frames
>>> >are
>>> >>> >> >>>> buffered by general network conditions (ie in TCP
>>> >>bottlenecks).
>>> >>> >> >>>> This would not be good for a 1:1 relationship between
>>DATA
>>> >>and
>>> >>> >ws
>>> >>> >> >>>> frames.
>>> >>> >> >>>>
>>> >>> >> >>>> Amos
>>> >>> >> >>>
>>> >>> >> >>> So hang on a second here. If we say that ws frames can't
>>be
>>> >>split
>>> >>> >> >>> over multiple HTTP/2 frames that implies that we have to
>>> >>buffer
>>> >>> >> >>> them until there is enough in the window to transmit a
>>> >>> >potentially
>>> >>> >> >>> very large packet all at once. It also conflicts with
>>RFC6455
>>> >>-
>>> >>> >the
>>> >>> >> >>> specific intent there is to not be a stream based system.
>>> >>> >> >>
>>> >>> >> >> If a ws frame *has* to be that long, not doing so would
>>block
>>> >>the
>>> >>> >> >> entire HTTP/2 connection until all bytes of that frame
>were
>>> >>> >delivered
>>> >>> >> >> anyway. So you trade off buffering that single frame at
>the
>>> >>> >sender,
>>> >>> >> >> versus blocking all HTTP/2 traffic end-to-end.
>>> >>> >> >>
>>> >>> >> >> If the ws data is so critical to get transmitted fast why
>>is
>>> >>that
>>> >>> >> >> single ws frame so large to begin with? surely it would be
>>> >>> >> >transmitted
>>> >>> >> >> faster as a sequence of WS + *WSDATA frames emited as the
>>> >>payload
>>> >>> >was
>>> >>> >> >> available to send.
>>> >>> >> >
>>> >>> >> >I agree that its inconsistent which is why I don't think it
>>> >>matters
>>> >>> >>
>>> >>> >> I am the author of libwebsockets, we are adding http2 support
>>at
>>> >>the
>>> >>> >> moment.  The basic http2 serving is done and works for http,
>>but
>>> >>> >we're all
>>> >>> >> dressed up and nowhere to go in terms of treating websocket
>>> >>> >connections as
>>> >>> >> just another kind of http2, since the framing is "TBD".  I am
>>> >>sorry I
>>> >>> >am a
>>> >>> >> bit late to the party.
>>> >>> >>
>>> >>> >> I think you have to take the atomic large ws frame thing as a
>>> >>genuine
>>> >>> >> problem because http2 has the transmit credit concept.  So
>>even
>>> >if
>>> >>> >you
>>> >>> >> buffered one ws frame, you can't sit there spewing as much
>>http2
>>> >>DATA
>>> >>> >frame
>>> >>> >> as it needs to atomically encapsulate it, without sizing your
>>> >>http2
>>> >>> >frame
>>> >>> >> to suit the tx credit.
>>> >>> >>
>>> >>> >> But it's OK because the implementation can transparently
>>fragment
>>> >>the
>>> >>> >ws
>>> >>> >> data using the ws message semantics... I think there's no
>>choice
>>> >>but
>>> >>> >to
>>> >>> >> take that approach.
>>> >>> >>
>>> >>> >> Otherwise you get into being able to DoS even an http2 "big
>>pipe
>>> >>> >> aggregation" by just one mux element spewing an endless ws
>>frame
>>> >>and
>>> >>> >> blocking every other mux'd connection... it cannot be right.
>>> >>> >>
>>> >>> >> >and mapping down to h2 frames as a sequence of octets would
>>be
>>> >>fine.
>>> >>> >> >But you seem to both agree with my reasoning and disagree
>>with
>>> >my
>>> >>> >> >conclusion. This is confusing.
>>> >>> >> >
>>> >>> >> >>> I was suggesting that we just treat the HTTP/2 stream
>like
>>> >the
>>> >>> >TCP
>>> >>> >> >>> connection in RFC 6455 - the conversation from stream to
>>> >>message
>>> >>> >> >>> based semantics and so on can take place above that in
>the
>>ws
>>> >>> >> >>> implementation - and that we should still apply the
>>> >>transmission
>>> >>> >> >>> windows etc to ws streams.
>>> >>> >>
>>> >>> >> Yes ---^ this is how it has to be I think.
>>> >>> >>
>>> >>> >> >> If you do that you loose any and all benefits from HTTP/2
>>> >>frames.
>>> >>> >> >> Everything from ws frame headers to data content becomes
>>> >>> >semantically
>>> >>> >> >> identical to the opaque payload of a DATA frame on an
>>HTTP/2
>>> >>> >CONNECT
>>> >>> >> >> request. I believe Yutaka is seeking to get away from that
>>> >>> >situation
>>> >>> >> >> where DATA frames may be split, joined or buffered at any
>>> >>point.
>>> >>> >> >
>>> >>> >> >Sorry, I just don't follow that. We have a primitive which
>>> >>appears
>>> >>> >to
>>> >>> >> >fit ws entirely, with the only caveat being that we haven't
>>> >>defined
>>> >>> >> >the mapping from the high level frames to the h2 primitives.
>>If
>>> >>the
>>> >>> >>
>>> >>> >> Yeah.
>>> >>> >>
>>> >>> >> >spec identifies how ws is negotiated and framed within h2,
>>its
>>> >>not
>>> >>> >> >opaque at all. And ws implementations that support raw ws
>>(which
>>> >>> >> >they'll do for quite some time...) have to deal with tcp
>>which
>>> >>> >offers
>>> >>> >> >no better semantics than this.
>>> >>> >>
>>> >>> >> Right now if I understood it the ws connections can still
>>> >>negotiate
>>> >>> >> themselves transparently inside http2 mux connections, using
>>the
>>> >>> >RFC6455
>>> >>> >> upgrade on their individual session ID, do the extra RTT and
>>tx
>>> >>data
>>> >>> >> masking.
>>> >>> >>
>>> >>> >> Formalizing how to encapsulate the same thing in http2
>doesn't
>>> >buy
>>> >>> >much
>>> >>> >> above that... the benefit we can get is map the RFC6455
>>framing
>>> >on
>>> >>to
>>> >>> >http2
>>> >>> >> native framing and get rid of the duplication simple
>>> >encapsulation
>>> >>> >has (for
>>> >>> >> many small frames, it would be really painful overhead
>>actually).
>>> >
>>> >>So
>>> >>> >if we
>>> >>> >> will do anything, it should indeed be define how to map
>>RFC6455
>>> >>> >framing on
>>> >>> >> http2 framing.
>>> >>> >>
>>> >>> >> I guess it can be done in parallel with http2 coming to an
>end
>>> >>rather
>>> >>> >than
>>> >>> >> trying to block it, just by defining some new optional frame
>>> >>types...
>>> >>> >>
>>> >>> >> -Andy
>>> >>> >>
>>> >>> >> >-Rob
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>>
>>> >>>
>>>
>>>

Received on Friday, 21 November 2014 06:16:38 UTC