Re: HTTP/2 and Websockets

On 22 November 2014 00:54:47 GMT+08:00, Jason Greene <jason.greene@redhat.com> wrote:
>
>> On Nov 21, 2014, at 5:53 AM, Andy Green <andy@warmcat.com> wrote:
>> 
>> 
>> 
>> On 21 November 2014 18:57:13 GMT+08:00, Amos Jeffries
><squid3@treenet.co.nz> wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>> 
>>> On 21/11/2014 7:58 p.m., Andy Green wrote:
>>>> 
>>>> 
>>>> On 21 November 2014 14:41:19 GMT+08:00, Yutaka Hirano
>>>> <yhirano@google.com> wrote:
>>>>> On Fri, Nov 21, 2014 at 12:38 PM, Andy Green <andy@warmcat.com>
>>>>> wrote:
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 21 November 2014 11:30:46 GMT+08:00, Yutaka Hirano
>>>>> <yhirano@google.com>
>>>>>> wrote:
>>>>>>> On Fri, Nov 21, 2014 at 11:42 AM, Andy Green
>>>>>>> <andy@warmcat.com>
>>>>> wrote:
>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 21 November 2014 10:02:28 GMT+08:00, Yutaka Hirano
>>>>>>> <yhirano@google.com>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> I think you have to take the atomic large ws frame
>>>>>>>>>> thing as a
>>>>>>> genuine
>>>>>>>>>> problem because http2 has the transmit credit concept.
>>>>>>>>>> So even
>>>>> if
>>>>>>>>> you
>>>>>>>>>> buffered one ws frame, you can't sit there spewing as
>>>>>>>>>> much
>>>>> http2
>>>>>>> DATA
>>>>>>>>> frame
>>>>>>>>>> as it needs to atomically encapsulate it, without
>>>>>>>>>> sizing your
>>>>>>> http2
>>>>>>>>> frame
>>>>>>>>>> to suit the tx credit. But it's OK because the
>>>>>>>>>> implementation can transparently
>>>>> fragment
>>>>>>> the
>>>>>>>>> ws
>>>>>>>>>> data using the ws message semantics... I think there's
>>>>>>>>>> no
>>>>> choice
>>>>>>> but
>>>>>>>>> to
>>>>>>>>>> take that approach.
>>>>>>>>> 
>>>>>>>>> Sorry I don't understand what you are proposing. Can you
>>>>>>>>> explain?
>>>>>>>> 
>>>>>>>> I'm agreeing with what was already written by someone else
>>>>>>>> on the
>>>>>>> thread.
>>>>>>>> 
>>>>>>>> Talking about buffering huge ws frames until you have
>>>>>>>> enough to
>>>>> issue
>>>>>>> it
>>>>>>>> all in one big http2 DATA frame will not fly.
>>>>>>>> 
>>>>>>>> If you're using this putative ws-over-http2 scheme, and you
>>>>>>>> get
>>>>> given
>>>>>>> a
>>>>>>>> huge ws frame to transmit, you should fragment it using
>>>>>>>> RFC6455
>>>>>>> message
>>>>>>>> semantics to some implementation-defined limit that is
>>>>>>>> friendly
>>>>> for
>>>>>>> mux'd
>>>>>>>> http2 transport.
>>>>>>>> 
>>>>>>> Thanks.
>>>>>>> 
>>>>>>> Strictly speaking, RFC6455 allows an extension to give
>>>>>>> meaning to WebSocket frames, so merging / fragmenting frames
>>>>>>> breaks such extensions. We discussed this problem in HyBi and
>>>>>>> many of us said "don't care".
>>>>>> 
>>>>>> Yeah extensions except for compression have just not come into
>>>>> existence.
>>>>>> 
>>>>>> So I also see it as don't care.  I'm not even sure it's true
>>>>>> since
>>>>> the
>>>>>> intention during ws discussion was an intermediary can
>>>>>> fragment
>>>>> frames same
>>>>>> as how tcp packets may be fragmented.
>>>>>> 
>>>>>>> In any case, an http/2 frame cannot be bigger than 2^24 (or
>>>>>>> 2^14 without an explicit permission), so I think we don't
>>>>>>> have to worry about DoS.
>>>>>> 
>>>>>> ... I don't see that.  If I keep spamming 16MB frames even on
>>>>>> one
>>>>> stream
>>>>>> on a consumer link your latency goes to pieces and if something
>>>>>> is
>>>>> doing
>>>>>> multiple instances of it even a big pipe will feel pain.
>>>>>> 
>>>>>> 
>>>>> The spec says: Length: The length of the frame payload expressed
>>>>> as an unsigned 24-bit integer. Values greater than 2^14 (16,384)
>>>>> MUST NOT be sent unless the receiver has set a larger value for
>>>>> SETTINGS_MAX_FRAME_SIZE. So by default 2^14 is the upper limit.
>>>>> Larger frames will be used only if the receiver wants.
>>>> 
>>>> Okay.  But if buffering whole ws frames no matter how big was the
>>>> plan, and the one end wanted to send a 16MB ws frame, the other end
>>>> would say, "sure" you can send me >16K and he'd spam his 16MB
>>>> frames.  Default 16K limit doesn't help much because the server is
>>>> complicit in thinking sending huge ws frames is good.
>>>> 
>>> 
>>> You switched between "receiver" and "server" there in your concepts.
>> 
>> Well okay, it's not my day job.
>> 
>>> Overlooking the fact that intermediaries in HTTP can cut DATA up
>into
>>> smaller frames and stitch them together into larger ones as needed
>>> hop-by-hop. So what your sender gets told it can send on the
>>> client->proxy connection may not be what gets delivered to the
>server.
>>> 
>>> That was the major reason for WSDATA not being HTTP/2 DATA. That
>flags
>>> byte being mentioned at the beginning of DATA payload will get
>screwed
>>> over and moved around too much. Unless it is at fixed positions of
>the
>>> total WS stream inside the DATA payloads.
>> 
>> What does 'too much' mean... it should not attempt to preserve the
>original ws frame sizes, just send http2 frames of a size that suit it
>when an amount of ws payload data that suits it has appeared.  So the
>bit of tx logic that chops up the ws payload data, takes care of
>deferring FIN until the last chunk.  And the receiver doesn't have to
>do anything just pass up the frames it got with the ws flags it got.
>> 
>> What's the problem?
>> 
>>> But then, as has been mentioned since HEADERS initiated a WS stream,
>>> why is DATA needing to be re-flagged specially? it is opaque payload
>>> bytes of the stream already flagged as being a WS transaction.
>> 
>> I think it's alright being an implicit attribute of the stream.
>> 
>> But I understood the general objection from Hirano-san, it's that
>there might be dumb proxies who didn't follow the update activity on
>that stream and might parse what they see there as... http2 data.
>> 
>> 'Magic bad proxies' was a stick that extinguished a lot of discussion
>on hybi.... he's not going to get fired for shaking that stick at us. 
>It doesn't damage the argument to point out there's no evidence the bad
>proxy exists.  But if it's WSDATA and http2 wraps up without containing
>WSDATA this stuff will never get deployed, because if anybody doesn't
>understand WSDATA the connection won't work even if the handshake does,
>lol
>
>A bad proxy that corrupts DATA frames is going to break more than web
>sockets, it will also break common HTTP use cases. 

Yeah.

So what's the plan to get websockets working on HTTP2?  This is the place where we're supposed to be sorting that out.  But it's looking stalled.

Since there is no active plan right now, the default situation is a big nothing and perhaps a big crappy rollout of WSDATA frames that nobody supports leading to negotiated and then dead WS connections.

Is that an OK result for everybody?

As discussed - nobody disagreed with the logical workability of this - it can be done concisely over DATA today with zero change or delay to HTTP2.

The objection from Hirano-san is if we put [1 byte WS flags] + [ WS payload ] in HTTP2 DATA frames, "a bad proxy / intermediary" may think it's OK to mangle the content of DATA frames as if it was HTTP content.

Personally I don't think it's realistic, but I saw this argument block and destroy things on hybi, so we have to shrug and pretend it is a serious issue.  One of the reasons this argument is so damaging and unhelpful is when you ask, "how broken is that proxy exactly", there is never any limit to how deranged it is in theory, because it can be as deranged as you like, and the guy proposing it does not want to give up any ground by saying, "well, I guess it might not flip bits randomly then", or "yes it must understand http headers".  So we get into arguments about the characteristics of the nonexistant magic broken proxy rather than do engineering.

How about

1) Exchange fake HTTP headers after the ws upgrade completes, which is like GET "endless-crap" replied to by 200 mime-type: binary no content-length, the receiver of each fake header frame knows it's coming and swallows it.  So we help the magic broken proxy to get to a state where he's not going to look at the connection content.  Unless the proxy munges HTTP2 binary files... do we really believe that?

2) Add a CRC16/32 or so on the frame or message.... this is expensive to compute and check, though probably about on a par with the ws masking.

... any other ideas?

-Andy

>--
>Jason T. Greene
>WildFly Lead / JBoss EAP Platform Architect
>JBoss, a division of Red Hat

Received on Monday, 24 November 2014 23:35:10 UTC