Re: Design Issue: Separate HEADERS and PRIORITY Frames, Eliminate HEADERS+PRIORITY

On Sat, May 25, 2013 at 8:44 PM, Jeff Pinner <jpinner@twitter.com> wrote:

> I would say the server drops the HEADERS frame which I suppose means that
> PRIORITY opens the stream (just as PUSH_PROMISE does) since the client
> can't distinguish between:
>
> server receives PRIORITY
> server generates RST_STREAM
> server receives HEADERS
>
> and
>
> server receives PRIORITY
> server receives HEADERS
> server generates RST_STREAM
>
> so it would be equivalent to PRIORITY having created the stream, the
> RST_STREAM closing it and the client not bothering to send the DATA frames.
>
>
> It may be the case that we only want HEADERS frames to be able to create
> streams, in which case yes that would prohibit sending a PRIORITY frame
> before the HEADERS frame. But I don't know if that is being overly
> restrictive.
>

This is what I favor, and it is at most 8 bytes of overhead (send an empty
HEADERS frame before whatever else) in non-HTTP cases where metadata about
the stream opening is not needed (nothing that I can think of would do this
today).


>
> Another option is a flag on the HEADERS frame similar to the continuation
> bit that says a PRIORITY frame is following and must be sent after the
> continued headers are received.
>

This is basically what HEADERS+PRIORITY is, without the overhead of the
frame-header.

-=R


>
>
> On Sat, May 25, 2013 at 2:32 PM, Roberto Peon <grmocg@gmail.com> wrote:
>
>> 1) I'm not worried about prioritization after stream start-- that race is
>> unavoidable :)
>> 2) Agreed. Happy to handle this complexity-- it requires that I
>> allocation nothing new. At worst I remove the stream from the priority
>> structure and re-add, or simply report that the stream is already dead.
>> 3) This is what I'm unhappy about. PRIORITY before HEADERS means that I
>> have an additional state transition and more state to track.
>> I agree that it seems more simple on the surface, but I also think that a
>> decoupled PRIORITY frame leads to more code complexity given the need to
>> handle client-directed priority for responses at the server.
>>
>>
>> It is actually more complex to decouple these at stream initiation time,
>> since, at stream initiation time, I want to know both what I'm doing, and
>> how important it is. A lack of one or the other leads to additional state
>> tracking, additional lookups, races, ambiguity over when the proxy should
>> begin to make connections, etc.
>>
>> As an example, in a world where PRIORITY is decoupled from HEADERS, what
>> happens in the race where I've reset the stream after receiving the
>> PRIORITY frame, but before receiving the HEADERS frame?
>>
>> client sends Priority(x),stream(3)
>> client sends Headers(x),stream(3)
>>
>> server receives Priority(x), stream(3)
>> server sends RST stream(3)
>> server receives Headers(x), stream(3)
>> ... now what?
>>
>> I see a few obvious outcomes, none of which seem appealing to me:
>> 1) The server processes the request, but now without knowledge of the
>> priority
>> 2) The server keeps track of streams it has reset for some time period
>> (this would have to be a map or similar) and times them out after some time
>> period (e.g. sends a PING and awaits the PONG)
>> 3) The server is disallowed from sending the RST before it has received
>> HEADERS, and must maintain the additional data that is perhaps not ever
>> useful
>>
>> In the case where we have HEADERS+PRIORITY, and where stream creation is
>> accomplished ONLY via HEADERS or HEADERS+PRIORITY, this mess goes away.
>> Yes, we still have reprioritization to deal with, but with the guarantee
>> of the previous sentence, if I see a PRIORITY for a stream I don't know, I
>> just ignore it (and generate a RST stream with appropriate error code)
>>
>> In a world where we agree that streams are only started by some explicit
>> thing (HEADERS*), the code is also more simple, because I don't have to
>> maintain state to see what streams have finished vs never created, etc.
>> etc. and because there are fewer races and less ambiguity.
>>
>> -=R
>>
>>
>>
>>
>>
>>
>>
>> On Sat, May 25, 2013 at 1:09 PM, Jeff Pinner <jpinner@twitter.com> wrote:
>>
>>> You get the same race condition by introducing re-prioritization since
>>> that is equivalent to sending the priority after initiating a stream with
>>> default priority.
>>>
>>> I guess my argument is:
>>>
>>> 1. Once we introduce the PRIORITY frame and re-prioritization you have
>>> to handle all the additional complexity of receiving PRIORITY after HEADERS.
>>> 2. We already have to handle receiving PRIORITY before HEADERS (i.e.
>>> empty continued header blocks).
>>> 3. Clients have the option to always send PRIORITY first if they want to
>>> ensure that it can be considered by the server.
>>>
>>> So introducing PRIORITY as a frame means you now have two ways to
>>> accomplish the same thing and you have already taken the complexity hit, so
>>> lets try to make it slightly simpler by only having one way.
>>>
>>>
>>>
>>>
>>>
>>> On Sat, May 25, 2013 at 12:52 PM, Roberto Peon <grmocg@gmail.com> wrote:
>>>
>>>> Nah-- I've been proposing that when the continuation bit is set, you
>>>> may ONLY continue sending HEADERS frames for that stream until the set of
>>>> headers is done (at which point you can send whatever again), basically
>>>> because of both complexity and the DoS stuff inherent in incomplete state.
>>>> :)
>>>>
>>>> I agree that the best way to handle the PRIORITY DoS is to treat the
>>>> receipt of one on a stream which isn't known as a protocol error. This,
>>>> however re-invokes the race condition where a stream has been created but
>>>> we don't know the priority. We either need to know the priority before the
>>>> headers are completed.
>>>>
>>>> Honestly, I'm not sure what the spec allows right now, but what I said
>>>> in the previous email (you have to begin a stream with either HEADERS or
>>>> HEADERS+PRIORITY) is what I understand is the proposal to be.
>>>> It would be different from SYN_STREAM-- you could begin a stream with
>>>> the equivalent of SYN_REPLY (HEADERS)
>>>> -=R
>>>>
>>>>
>>>> On Sat, May 25, 2013 at 12:44 PM, Jeff Pinner <jpinner@twitter.com>wrote:
>>>>
>>>>> A rebuttal of the DoS argument for the server case. Consider the
>>>>> following:
>>>>>
>>>>> 1. We currently allow up to N concurrent streams on the server
>>>>> (configured via settings).
>>>>> 2. With reducing the frame size you have made the argument that
>>>>> multiple HEADERS frame may be required on an individual stream to comprise
>>>>> a HTTP request.
>>>>>
>>>>> This means that a malicious client could always send N
>>>>> HEADERS+PRIORITY frames, all with some "continuation" bit set (and possibly
>>>>> all without any headers), causing the server to allocate state for all N
>>>>> streams.
>>>>>
>>>>> This would be identical to issuing N PRIORITY frames for the next N
>>>>> open streams followed by N HEADERS frames.
>>>>>
>>>>> We can also protect the server against DoS by either requiring that
>>>>> the receipt of a PRIORITY frame for a Stream ID that is invalid result in a
>>>>> PROTOCOL_ERROR the same way that receipt of a HEADERS+PRIORITY frame would.
>>>>>
>>>>> As for the additional state transitions, the additional complexity in
>>>>> the session management logic is introduced as soon as you allow more than
>>>>> one HEADERS frame per request and unaffected by separating HEADERS and
>>>>> PRIORITY (again since a HEADERS+PRIORITY frame with no headers is
>>>>> equivalent).
>>>>>
>>>>>
>>>>> As for issuing HEADERS from the client, does the spec now allowing
>>>>> mapping HTTP requests to either HEADERS or HEADERS+PRIORITY frames? So this
>>>>> is different than what was done with SynStream?
>>>>>
>>>>>
>>>>> On Sat, May 25, 2013 at 12:24 PM, Roberto Peon <grmocg@gmail.com>wrote:
>>>>>
>>>>>> Right now, we've proposed the following frame types:
>>>>>>
>>>>>> HEADERS+PRIORITY
>>>>>> HEADERS
>>>>>> PRIORITY (to be added)
>>>>>>
>>>>>> While I'm strongly against removal for the HEADERS+PRIORITY frame, I
>>>>>> do agree that the PRIORITY frame is probably reasonable.
>>>>>>
>>>>>> Playing devil's advocate on the PRIORTY frame, though, I ask myself
>>>>>> the question:
>>>>>> Since a HEADERS+PRIORITY frame with no payload would be the same as a
>>>>>> PRIORTY frame (and we have to handle the empty-payload case for header
>>>>>> blocks anyway), are we buying anything?
>>>>>> I don't have a good answer except that the PRIORITY frame "smells"
>>>>>> right, and it has a low implementation cost in terms of complexity if we're
>>>>>> doing any reprioritization.
>>>>>>
>>>>>> So, onto why I hate the idea of *requiring* separate frames for
>>>>>> PRIORITY and HEADERS:
>>>>>> In the PUSH_PROMISE case, the client can have indicated that it will
>>>>>> not accept these streams by sending a SETTINGS frame, and much of the time
>>>>>> the client has orders of magnitude more memory to spend per connection that
>>>>>> does the server. PUSH_PROMISE is a totally different beast from PRIORITY.
>>>>>> PRIORITY, on the other hand is not optional and can't be disabled--
>>>>>> if we did remove/disallow it, then we'd have crippled multiplexing for the
>>>>>> browser use-case.
>>>>>> If we allow PRIORITY to perform allocations, then we've increased
>>>>>> complexity in the servers (more state transitions and state to store) and
>>>>>> thus increased the number of DoS vectors against the servers for
>>>>>> essentially zero gain.
>>>>>>
>>>>>> In your (non browser) use case, you'd just use HEADERS to initiate a
>>>>>> stream from the client when you don't care to set the priority. You don't
>>>>>> have to use HEADERS+PRIORITY. Your overall complexity should be unchanged.
>>>>>> In the browser use-case, HEADERS+PRIORITY would be used nearly all
>>>>>> the time, since the communication of priority is of very high importance
>>>>>> (else the browser must implement and play heuristic waiting games for
>>>>>> requests).
>>>>>>
>>>>>> In either case a PRIORITY frame could be used to reprioritize a
>>>>>> stream.
>>>>>>
>>>>>> -=R
>>>>>>
>>>>>>
>>>>>> On Sat, May 25, 2013 at 11:58 AM, Jeff Pinner <jpinner@twitter.com>wrote:
>>>>>>
>>>>>>> The color of my shed:
>>>>>>>
>>>>>>> I would like to see us remove HEADERS+PRIORITY entirely and add a
>>>>>>> separate PRIORITY frame.
>>>>>>>
>>>>>>> I don't agree that separating them will simply cause an additional 4
>>>>>>> bytes to be sent on every request. While it might be true that most
>>>>>>> browsers will set the priority of a request, I don't think that all clients
>>>>>>> necessarily will (I have a mobile client that uses HTTP for API requests
>>>>>>> and have not found the priority mechanism necessary -- at least not yet).
>>>>>>>
>>>>>>> I could imagine that it would be acceptable to send the PRIORITY
>>>>>>> frame before the HEADERS frame and still mandate that HEADERS frames (and
>>>>>>> now only HEADERS frames) initiate streams. While this does cause some extra
>>>>>>> state allocation, we already have to do something similar with PUSH_PROMISE
>>>>>>> where we must track that the "stream identifier MUST be a valid choice for
>>>>>>> the next stream sent" and then initiate the stream later.
>>>>>>>
>>>>>>> This would also give us a mechanism to send priority changes for
>>>>>>> outstanding requests at the framing layer, and could allow priorities to be
>>>>>>> set for pushed responses should it prove useful.
>>>>>>>
>>>>>>> - Jeff
>>>>>>>
>>>>>>>
>>>>>>> On Tue, May 21, 2013 at 2:55 PM, James M Snell <jasnell@gmail.com>wrote:
>>>>>>>
>>>>>>>> On Tue, May 21, 2013 at 2:43 PM, Roberto Peon <grmocg@gmail.com>
>>>>>>>> wrote:
>>>>>>>> > Sending the PRIORITY frame *MUST* cause state allocation at the
>>>>>>>> receiver,
>>>>>>>> > else it was useless to send before the HEADERS frame. As you
>>>>>>>> point out, at
>>>>>>>> > minimum it must allocate a stream ID and priority field, and for
>>>>>>>> most
>>>>>>>> > implementations it will also need to include so mechanism of
>>>>>>>> pointing out
>>>>>>>> > that the headers don't exist, so, probably between 16 to 24 bytes
>>>>>>>> worth of
>>>>>>>> > allocation on a 64 bit machine.
>>>>>>>> >
>>>>>>>>
>>>>>>>> Sorry, I wasn't clear in my initial response. Yes, there is some
>>>>>>>> state
>>>>>>>> that would need to be allocated but not the same as that when the
>>>>>>>> initial HEADERS frame is received, for instance.
>>>>>>>>
>>>>>>>> > If the PRIORITY frame was renamed to CHANGE_PRIORITY, would that
>>>>>>>> clarify
>>>>>>>> > anything? Priority changing is the current intent of that frame
>>>>>>>> type.
>>>>>>>> >
>>>>>>>>
>>>>>>>> Not particularly, because we'd still have the question of when to
>>>>>>>> use
>>>>>>>> HEADERS+PRIORITY vs. the combination of HEADERS and a
>>>>>>>> CHANGE_PRIORITY.
>>>>>>>> Can HEADERS+PRIORITY be used any time? For instance, could I send an
>>>>>>>> initial HEADERS frame on a stream then later send a HEADERS+PRIORITY
>>>>>>>> on the same stream? I honestly don't care how it ultimately ends up
>>>>>>>> so
>>>>>>>> long as (a) It's the simplest thing that could possibly work and (b)
>>>>>>>> Is easy to explain in the spec and easy for a developer to
>>>>>>>> implement.
>>>>>>>>
>>>>>>>> > btw, I am not particularly partial to the "any frame opening up a
>>>>>>>> stream"
>>>>>>>> > thing. I'm not completely against it though :)
>>>>>>>> > My reason for slightly preferring that streams must begin with
>>>>>>>> HEADERS or
>>>>>>>> > HEADERS+PRIORITY is that it is an explicit statement of intent,
>>>>>>>> and thus
>>>>>>>> > off-by-one, uninitialized var, etc. errors are more likely to be
>>>>>>>> detectable
>>>>>>>> > in a world where such is required.
>>>>>>>> >
>>>>>>>>
>>>>>>>> I would very much like to see us mandate that streams always
>>>>>>>> initiate
>>>>>>>> with a HEADERS / HEADERS+PRIORITY frame.
>>>>>>>>
>>>>>>>> - James
>>>>>>>>
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > -=R
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Tue, May 21, 2013 at 2:19 PM, James M Snell <jasnell@gmail.com>
>>>>>>>> wrote:
>>>>>>>> >>
>>>>>>>> >> On Tue, May 21, 2013 at 10:30 AM, William Chan (陈智昌)
>>>>>>>> >> <willchan@chromium.org> wrote:
>>>>>>>> >> > Thanks for describing these cases now. I had not thought of
>>>>>>>> them.
>>>>>>>> >> >
>>>>>>>> >> > If everyone's sold on reprioritization, then let's go ahead
>>>>>>>> and do this
>>>>>>>> >> > and
>>>>>>>> >> > have the debate about ditching HEADERS+PRIORITY or not. I want
>>>>>>>> to keep
>>>>>>>> >> > it. I
>>>>>>>> >> > don't like the idea of sending a PRIORITY frame first. Is
>>>>>>>> sending a
>>>>>>>> >> > PRIORITY
>>>>>>>> >> > frame going to trigger stream state allocation at the
>>>>>>>> receiver? What's
>>>>>>>> >> > the
>>>>>>>> >> > expectation? And if you don't have a priority for the HEADERS,
>>>>>>>> then you
>>>>>>>> >> > have
>>>>>>>> >> > the race that Roberto described.
>>>>>>>> >> >
>>>>>>>> >>
>>>>>>>> >> There is no reason to assume that sending a PRIORITY frame first
>>>>>>>> would
>>>>>>>> >> trigger stream state allocation at the receiver. At most, it
>>>>>>>> would
>>>>>>>> >> reserve the stream ID and store the priority value. The full
>>>>>>>> state
>>>>>>>> >> allocation would not occur until the HEADERS frame is received.
>>>>>>>> That
>>>>>>>> >> said, I'm not 100% dead set on removing HEADERS+PRIORITY, I
>>>>>>>> would just
>>>>>>>> >> like to simplify the protocol where it makes sense to, and even
>>>>>>>> then
>>>>>>>> >> only after it's been proven out in implementation. Having
>>>>>>>> separate
>>>>>>>> >> HEADERS, HEADERS+PRIORITY and PRIORITY frames is confusing, if
>>>>>>>> we can
>>>>>>>> >> do without separating them, we ought to do so.
>>>>>>>> >>
>>>>>>>> >> - James
>>>>>>>> >>
>>>>>>>> >> >
>>>>>>>> >> > On Tue, May 21, 2013 at 2:09 PM, Patrick McManus <
>>>>>>>> pmcmanus@mozilla.com>
>>>>>>>> >> > wrote:
>>>>>>>> >> >>
>>>>>>>> >> >>
>>>>>>>> >> >> On Tue, May 21, 2013 at 12:32 PM, William Chan (陈智昌)
>>>>>>>> >> >> <willchan@chromium.org> wrote:
>>>>>>>> >> >>>
>>>>>>>> >> >>>
>>>>>>>> >> >>> I support adding a new additional PRIORITY frame for stream
>>>>>>>> >> >>> reprioritization.
>>>>>>>> >> >>
>>>>>>>> >> >>
>>>>>>>> >> >> me too. Specifically I support this as a mechanism for the
>>>>>>>> client to be
>>>>>>>> >> >> able to explicitly prioritize an open pushed stream. I can
>>>>>>>> wait for
>>>>>>>> >> >> more
>>>>>>>> >> >> evidence about re-prioritizing, but in cases where the client
>>>>>>>> hasn't
>>>>>>>> >> >> ever
>>>>>>>> >> >> explicitly set the stream's priority I think we have evidence
>>>>>>>> that its
>>>>>>>> >> >> time
>>>>>>>> >> >> to do something.
>>>>>>>> >> >>>
>>>>>>>> >> >>>
>>>>>>>> >> >>> Unless there's a reason this needs to be in the current
>>>>>>>> http/2 draft
>>>>>>>> >> >>> sooner rather than later, I'd rather punt on this discussion
>>>>>>>> until we
>>>>>>>> >> >>> have
>>>>>>>> >> >>> implementation experience that can guide this debate.
>>>>>>>> >> >>
>>>>>>>> >> >>
>>>>>>>> >> >> I think there is experience here specifically related to push.
>>>>>>>> >> >>
>>>>>>>> >> >> e.g. You can easily configure mod_spdy to push images when
>>>>>>>> html is
>>>>>>>> >> >> pulled.
>>>>>>>> >> >> but you can't effectively dictate the relative priorities of
>>>>>>>> those two
>>>>>>>> >> >> things.
>>>>>>>> >> >>
>>>>>>>> >> >> Sure, you can define an explicit priority for those images
>>>>>>>> but priority
>>>>>>>> >> >> implementations are all about relative levels and the client
>>>>>>>> set the
>>>>>>>> >> >> priority of the html.
>>>>>>>> >> >>
>>>>>>>> >> >> You can argue that mod_spdy should have defined relative
>>>>>>>> priorities
>>>>>>>> >> >> (+/-
>>>>>>>> >> >> the associated stream) instead of constants.. that would be
>>>>>>>> better but
>>>>>>>> >> >> the
>>>>>>>> >> >> client still has no way to make sure those streams are at a
>>>>>>>> higher
>>>>>>>> >> >> priority
>>>>>>>> >> >> than a "save as" background stream (I've seen this one happen
>>>>>>>> as
>>>>>>>> >> >> mod_spdy
>>>>>>>> >> >> defaults to lowest priority when pushing), or a lower
>>>>>>>> priority than a
>>>>>>>> >> >> real-time video stream..
>>>>>>>> >> >>
>>>>>>>> >> >> plus there is no scale for the server to work with.. it might
>>>>>>>> set a +2
>>>>>>>> >> >> priority for pushed images but the client might be using +3
>>>>>>>> for pulled
>>>>>>>> >> >> images causing a mismatch in something that was intended to
>>>>>>>> be equally
>>>>>>>> >> >> weighted.
>>>>>>>> >> >>
>>>>>>>> >> >> at least with a priority frame the client can make those
>>>>>>>> adjustments in
>>>>>>>> >> >> a
>>>>>>>> >> >> RTT.
>>>>>>>> >> >>
>>>>>>>> >> >> Cheefully,
>>>>>>>> >> >> -Patrick
>>>>>>>> >> >>
>>>>>>>> >> >>
>>>>>>>> >> >
>>>>>>>> >> >
>>>>>>>> >>
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>