Re: HTTP/2 and Websockets

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 25/11/2014 12:34 p.m., Andy Green wrote:
> 
> 
> On 22 November 2014 00:54:47 GMT+08:00, Jason Greene 
> <jason.greene@redhat.com> wrote:
>> 
>>> On Nov 21, 2014, at 5:53 AM, Andy Green <andy@warmcat.com> 
>>> wrote:
>>> 
>>> 
>>> 
>>> On 21 November 2014 18:57:13 GMT+08:00, Amos Jeffries
>> <squid3@treenet.co.nz> wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>> 
>>>> On 21/11/2014 7:58 p.m., Andy Green wrote:
>>>>> 
>>>>> 
>>>>> On 21 November 2014 14:41:19 GMT+08:00, Yutaka Hirano 
>>>>> <yhirano@google.com> wrote:
>>>>>> On Fri, Nov 21, 2014 at 12:38 PM, Andy Green 
>>>>>> <andy@warmcat.com> wrote:
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 21 November 2014 11:30:46 GMT+08:00, Yutaka Hirano
>>>>>> <yhirano@google.com>
>>>>>>> wrote:
>>>>>>>> On Fri, Nov 21, 2014 at 11:42 AM, Andy Green 
>>>>>>>> <andy@warmcat.com>
>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 21 November 2014 10:02:28 GMT+08:00, Yutaka 
>>>>>>>>> Hirano
>>>>>>>> <yhirano@google.com>
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> I think you have to take the atomic large ws 
>>>>>>>>>>> frame thing as a
>>>>>>>> genuine
>>>>>>>>>>> problem because http2 has the transmit credit 
>>>>>>>>>>> concept. So even
>>>>>> if
>>>>>>>>>> you
>>>>>>>>>>> buffered one ws frame, you can't sit there 
>>>>>>>>>>> spewing as much
>>>>>> http2
>>>>>>>> DATA
>>>>>>>>>> frame
>>>>>>>>>>> as it needs to atomically encapsulate it, 
>>>>>>>>>>> without sizing your
>>>>>>>> http2
>>>>>>>>>> frame
>>>>>>>>>>> to suit the tx credit. But it's OK because the
>>>>>>>>>>>  implementation can transparently
>>>>>> fragment
>>>>>>>> the
>>>>>>>>>> ws
>>>>>>>>>>> data using the ws message semantics... I think 
>>>>>>>>>>> there's no
>>>>>> choice
>>>>>>>> but
>>>>>>>>>> to
>>>>>>>>>>> take that approach.
>>>>>>>>>> 
>>>>>>>>>> Sorry I don't understand what you are proposing. 
>>>>>>>>>> Can you explain?
>>>>>>>>> 
>>>>>>>>> I'm agreeing with what was already written by
>>>>>>>>> someone else on the
>>>>>>>> thread.
>>>>>>>>> 
>>>>>>>>> Talking about buffering huge ws frames until you 
>>>>>>>>> have enough to
>>>>>> issue
>>>>>>>> it
>>>>>>>>> all in one big http2 DATA frame will not fly.
>>>>>>>>> 
>>>>>>>>> If you're using this putative ws-over-http2
>>>>>>>>> scheme, and you get
>>>>>> given
>>>>>>>> a
>>>>>>>>> huge ws frame to transmit, you should fragment it 
>>>>>>>>> using RFC6455
>>>>>>>> message
>>>>>>>>> semantics to some implementation-defined limit
>>>>>>>>> that is friendly
>>>>>> for
>>>>>>>> mux'd
>>>>>>>>> http2 transport.
>>>>>>>>> 
>>>>>>>> Thanks.
>>>>>>>> 
>>>>>>>> Strictly speaking, RFC6455 allows an extension to
>>>>>>>> give meaning to WebSocket frames, so merging /
>>>>>>>> fragmenting frames breaks such extensions. We
>>>>>>>> discussed this problem in HyBi and many of us said
>>>>>>>> "don't care".
>>>>>>> 
>>>>>>> Yeah extensions except for compression have just not
>>>>>>> come into
>>>>>> existence.
>>>>>>> 
>>>>>>> So I also see it as don't care.  I'm not even sure
>>>>>>> it's true since
>>>>>> the
>>>>>>> intention during ws discussion was an intermediary can
>>>>>>>  fragment
>>>>>> frames same
>>>>>>> as how tcp packets may be fragmented.
>>>>>>> 
>>>>>>>> In any case, an http/2 frame cannot be bigger than
>>>>>>>> 2^24 (or 2^14 without an explicit permission), so I
>>>>>>>> think we don't have to worry about DoS.
>>>>>>> 
>>>>>>> ... I don't see that.  If I keep spamming 16MB frames 
>>>>>>> even on one
>>>>>> stream
>>>>>>> on a consumer link your latency goes to pieces and if 
>>>>>>> something is
>>>>>> doing
>>>>>>> multiple instances of it even a big pipe will feel
>>>>>>> pain.
>>>>>>> 
>>>>>>> 
>>>>>> The spec says: Length: The length of the frame payload 
>>>>>> expressed as an unsigned 24-bit integer. Values greater 
>>>>>> than 2^14 (16,384) MUST NOT be sent unless the receiver
>>>>>> has set a larger value for SETTINGS_MAX_FRAME_SIZE. So
>>>>>> by default 2^14 is the upper limit. Larger frames will be
>>>>>> used only if the receiver wants.
>>>>> 
>>>>> Okay.  But if buffering whole ws frames no matter how big
>>>>> was the plan, and the one end wanted to send a 16MB ws
>>>>> frame, the other end would say, "sure" you can send me >16K
>>>>> and he'd spam his 16MB frames.  Default 16K limit doesn't
>>>>> help much because the server is complicit in thinking
>>>>> sending huge ws frames is good.
>>>>> 
>>>> 
>>>> You switched between "receiver" and "server" there in your 
>>>> concepts.
>>> 
>>> Well okay, it's not my day job.
>>> 
>>>> Overlooking the fact that intermediaries in HTTP can cut
>>>> DATA up
>> into
>>>> smaller frames and stitch them together into larger ones as 
>>>> needed hop-by-hop. So what your sender gets told it can send
>>>> on the client->proxy connection may not be what gets
>>>> delivered to the
>> server.
>>>> 
>>>> That was the major reason for WSDATA not being HTTP/2 DATA. 
>>>> That
>> flags
>>>> byte being mentioned at the beginning of DATA payload will
>>>> get
>> screwed
>>>> over and moved around too much. Unless it is at fixed
>>>> positions of
>> the
>>>> total WS stream inside the DATA payloads.
>>> 
>>> What does 'too much' mean... it should not attempt to preserve 
>>> the
>> original ws frame sizes, just send http2 frames of a size that
>> suit it when an amount of ws payload data that suits it has
>> appeared. So the bit of tx logic that chops up the ws payload
>> data, takes care of deferring FIN until the last chunk.  And the
>> receiver doesn't have to do anything just pass up the frames it
>> got with the ws flags it got.
>>> 
>>> What's the problem?
>>> 
>>>> But then, as has been mentioned since HEADERS initiated a WS 
>>>> stream, why is DATA needing to be re-flagged specially? it
>>>> is opaque payload bytes of the stream already flagged as
>>>> being a WS transaction.
>>> 
>>> I think it's alright being an implicit attribute of the
>>> stream.
>>> 
>>> But I understood the general objection from Hirano-san, it's 
>>> that
>> there might be dumb proxies who didn't follow the update
>> activity on that stream and might parse what they see there as...
>> http2 data.
>>> 
>>> 'Magic bad proxies' was a stick that extinguished a lot of 
>>> discussion
>> on hybi.... he's not going to get fired for shaking that stick
>> at us. It doesn't damage the argument to point out there's no
>> evidence the bad proxy exists.  But if it's WSDATA and http2
>> wraps up without containing WSDATA this stuff will never get
>> deployed, because if anybody doesn't understand WSDATA the
>> connection won't work even if the handshake does, lol
>> 
>> A bad proxy that corrupts DATA frames is going to break more
>> than web sockets, it will also break common HTTP use cases.
> 
> Yeah.
> 
> So what's the plan to get websockets working on HTTP2?  This is
> the place where we're supposed to be sorting that out.  But it's
> looking stalled.
> 
> Since there is no active plan right now, the default situation is
> a big nothing and perhaps a big crappy rollout of WSDATA frames
> that nobody supports leading to negotiated and then dead WS
> connections.
> 
> Is that an OK result for everybody?
> 
> As discussed - nobody disagreed with the logical workability of
> this - it can be done concisely over DATA today with zero change or
> delay to HTTP2.
> 
> The objection from Hirano-san is if we put [1 byte WS flags] + [
> WS payload ] in HTTP2 DATA frames, "a bad proxy / intermediary"
> may think it's OK to mangle the content of DATA frames as if it was
> HTTP content.
> 
> Personally I don't think it's realistic, but I saw this argument 
> block and destroy things on hybi, so we have to shrug and pretend
> it is a serious issue.  One of the reasons this argument is so
> damaging and unhelpful is when you ask, "how broken is that proxy
> exactly", there is never any limit to how deranged it is in theory,
> because it can be as deranged as you like, and the guy proposing it
> does not want to give up any ground by saying, "well, I guess it
> might not flip bits randomly then", or "yes it must understand http
> headers". So we get into arguments about the characteristics of the
> nonexistant magic broken proxy rather than do engineering.

I think Hirano-san's proxy case is a good one to work with. But limit
it to being a HTTP/2 compliant proxy. That limits it to
slicing/aggregating DATA frames any way it likes but not re-ordering
or changing payload bytes unless there are transformations which the
HTTP/2 headers permit (the TCP analogy).

A truely broken magic proxy does not exist yet in any HTTP/2
implementations and HTTP/2 itself is taking the "hard line" of
defining RST_STREAM or GOAWAY to be used when any kind of binary level
broken proxy is detected.

So the magic proxy can take your 1-byte flags + 6 byte WS payload DATA
frame and slice it. Delivering 1 byte DATA frame followed by 6 byte
DATA frame to the server. This is particularly likely to happen if
your WS client writes the flags and WS payload in separate operations
to the HTTP/2 library/backend.
 - all you need to resolve this is a length+flags+payload structure to
the WS data. You can do that at the HTTP//2 stream level, or inside
the DATA frame payload.
 - if you dont know the length in advance then stream level is best
for HTTP/2 compatibility.

Amos
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBAgAGBQJUc9jhAAoJELJo5wb/XPRjDsAH/21qGwDJVC2iOEYDeBEb5rBG
kgYAeMri9yLrRis/4Pv0QtEQgxxMNVkWxDP+NNKmwCkhz3dF3X/y0v8ryXyr40aE
Kw6+m5vsLEtQ1/d7Xxbkz8iqJBZsM/1fklRGJ2G0YKLUED4KDf2hKmnkQwbDi7la
LUV1t6ij50gT9VmfW6r0dUwnLOt2M9qYRbTbGmgQbuc/6Zsqqp1VNtSOTpG/HC3+
xl3iZCDTJYyi01lNXZNsqzlXrx8zLT0i7KdE+QM+Vg/v708rUAp8sF26jxzsB/M2
d9oL2jXBJQ4ydh16Od5lwJrKlCqK7QHg3QHxRe8wmP9dabzHUVef3QmqPtoXJwg=
=9I96
-----END PGP SIGNATURE-----

Received on Tuesday, 25 November 2014 01:19:14 UTC