Re: Design Issue: Separate HEADERS and PRIORITY Frames, Eliminate HEADERS+PRIORITY

1) I'm not worried about prioritization after stream start-- that race is
unavoidable :)
2) Agreed. Happy to handle this complexity-- it requires that I allocation
nothing new. At worst I remove the stream from the priority structure and
re-add, or simply report that the stream is already dead.
3) This is what I'm unhappy about. PRIORITY before HEADERS means that I
have an additional state transition and more state to track.
I agree that it seems more simple on the surface, but I also think that a
decoupled PRIORITY frame leads to more code complexity given the need to
handle client-directed priority for responses at the server.


It is actually more complex to decouple these at stream initiation time,
since, at stream initiation time, I want to know both what I'm doing, and
how important it is. A lack of one or the other leads to additional state
tracking, additional lookups, races, ambiguity over when the proxy should
begin to make connections, etc.

As an example, in a world where PRIORITY is decoupled from HEADERS, what
happens in the race where I've reset the stream after receiving the
PRIORITY frame, but before receiving the HEADERS frame?

client sends Priority(x),stream(3)
client sends Headers(x),stream(3)

server receives Priority(x), stream(3)
server sends RST stream(3)
server receives Headers(x), stream(3)
... now what?

I see a few obvious outcomes, none of which seem appealing to me:
1) The server processes the request, but now without knowledge of the
priority
2) The server keeps track of streams it has reset for some time period
(this would have to be a map or similar) and times them out after some time
period (e.g. sends a PING and awaits the PONG)
3) The server is disallowed from sending the RST before it has received
HEADERS, and must maintain the additional data that is perhaps not ever
useful

In the case where we have HEADERS+PRIORITY, and where stream creation is
accomplished ONLY via HEADERS or HEADERS+PRIORITY, this mess goes away.
Yes, we still have reprioritization to deal with, but with the guarantee of
the previous sentence, if I see a PRIORITY for a stream I don't know, I
just ignore it (and generate a RST stream with appropriate error code)

In a world where we agree that streams are only started by some explicit
thing (HEADERS*), the code is also more simple, because I don't have to
maintain state to see what streams have finished vs never created, etc.
etc. and because there are fewer races and less ambiguity.

-=R







On Sat, May 25, 2013 at 1:09 PM, Jeff Pinner <jpinner@twitter.com> wrote:

> You get the same race condition by introducing re-prioritization since
> that is equivalent to sending the priority after initiating a stream with
> default priority.
>
> I guess my argument is:
>
> 1. Once we introduce the PRIORITY frame and re-prioritization you have to
> handle all the additional complexity of receiving PRIORITY after HEADERS.
> 2. We already have to handle receiving PRIORITY before HEADERS (i.e. empty
> continued header blocks).
> 3. Clients have the option to always send PRIORITY first if they want to
> ensure that it can be considered by the server.
>
> So introducing PRIORITY as a frame means you now have two ways to
> accomplish the same thing and you have already taken the complexity hit, so
> lets try to make it slightly simpler by only having one way.
>
>
>
>
>
> On Sat, May 25, 2013 at 12:52 PM, Roberto Peon <grmocg@gmail.com> wrote:
>
>> Nah-- I've been proposing that when the continuation bit is set, you may
>> ONLY continue sending HEADERS frames for that stream until the set of
>> headers is done (at which point you can send whatever again), basically
>> because of both complexity and the DoS stuff inherent in incomplete state.
>> :)
>>
>> I agree that the best way to handle the PRIORITY DoS is to treat the
>> receipt of one on a stream which isn't known as a protocol error. This,
>> however re-invokes the race condition where a stream has been created but
>> we don't know the priority. We either need to know the priority before the
>> headers are completed.
>>
>> Honestly, I'm not sure what the spec allows right now, but what I said in
>> the previous email (you have to begin a stream with either HEADERS or
>> HEADERS+PRIORITY) is what I understand is the proposal to be.
>> It would be different from SYN_STREAM-- you could begin a stream with the
>> equivalent of SYN_REPLY (HEADERS)
>> -=R
>>
>>
>> On Sat, May 25, 2013 at 12:44 PM, Jeff Pinner <jpinner@twitter.com>wrote:
>>
>>> A rebuttal of the DoS argument for the server case. Consider the
>>> following:
>>>
>>> 1. We currently allow up to N concurrent streams on the server
>>> (configured via settings).
>>> 2. With reducing the frame size you have made the argument that multiple
>>> HEADERS frame may be required on an individual stream to comprise a HTTP
>>> request.
>>>
>>> This means that a malicious client could always send N HEADERS+PRIORITY
>>> frames, all with some "continuation" bit set (and possibly all without any
>>> headers), causing the server to allocate state for all N streams.
>>>
>>> This would be identical to issuing N PRIORITY frames for the next N open
>>> streams followed by N HEADERS frames.
>>>
>>> We can also protect the server against DoS by either requiring that the
>>> receipt of a PRIORITY frame for a Stream ID that is invalid result in a
>>> PROTOCOL_ERROR the same way that receipt of a HEADERS+PRIORITY frame would.
>>>
>>> As for the additional state transitions, the additional complexity in
>>> the session management logic is introduced as soon as you allow more than
>>> one HEADERS frame per request and unaffected by separating HEADERS and
>>> PRIORITY (again since a HEADERS+PRIORITY frame with no headers is
>>> equivalent).
>>>
>>>
>>> As for issuing HEADERS from the client, does the spec now allowing
>>> mapping HTTP requests to either HEADERS or HEADERS+PRIORITY frames? So this
>>> is different than what was done with SynStream?
>>>
>>>
>>> On Sat, May 25, 2013 at 12:24 PM, Roberto Peon <grmocg@gmail.com> wrote:
>>>
>>>> Right now, we've proposed the following frame types:
>>>>
>>>> HEADERS+PRIORITY
>>>> HEADERS
>>>> PRIORITY (to be added)
>>>>
>>>> While I'm strongly against removal for the HEADERS+PRIORITY frame, I do
>>>> agree that the PRIORITY frame is probably reasonable.
>>>>
>>>> Playing devil's advocate on the PRIORTY frame, though, I ask myself the
>>>> question:
>>>> Since a HEADERS+PRIORITY frame with no payload would be the same as a
>>>> PRIORTY frame (and we have to handle the empty-payload case for header
>>>> blocks anyway), are we buying anything?
>>>> I don't have a good answer except that the PRIORITY frame "smells"
>>>> right, and it has a low implementation cost in terms of complexity if we're
>>>> doing any reprioritization.
>>>>
>>>> So, onto why I hate the idea of *requiring* separate frames for
>>>> PRIORITY and HEADERS:
>>>> In the PUSH_PROMISE case, the client can have indicated that it will
>>>> not accept these streams by sending a SETTINGS frame, and much of the time
>>>> the client has orders of magnitude more memory to spend per connection that
>>>> does the server. PUSH_PROMISE is a totally different beast from PRIORITY.
>>>> PRIORITY, on the other hand is not optional and can't be disabled-- if
>>>> we did remove/disallow it, then we'd have crippled multiplexing for the
>>>> browser use-case.
>>>> If we allow PRIORITY to perform allocations, then we've increased
>>>> complexity in the servers (more state transitions and state to store) and
>>>> thus increased the number of DoS vectors against the servers for
>>>> essentially zero gain.
>>>>
>>>> In your (non browser) use case, you'd just use HEADERS to initiate a
>>>> stream from the client when you don't care to set the priority. You don't
>>>> have to use HEADERS+PRIORITY. Your overall complexity should be unchanged.
>>>> In the browser use-case, HEADERS+PRIORITY would be used nearly all the
>>>> time, since the communication of priority is of very high importance (else
>>>> the browser must implement and play heuristic waiting games for requests).
>>>>
>>>> In either case a PRIORITY frame could be used to reprioritize a stream.
>>>>
>>>> -=R
>>>>
>>>>
>>>> On Sat, May 25, 2013 at 11:58 AM, Jeff Pinner <jpinner@twitter.com>wrote:
>>>>
>>>>> The color of my shed:
>>>>>
>>>>> I would like to see us remove HEADERS+PRIORITY entirely and add a
>>>>> separate PRIORITY frame.
>>>>>
>>>>> I don't agree that separating them will simply cause an additional 4
>>>>> bytes to be sent on every request. While it might be true that most
>>>>> browsers will set the priority of a request, I don't think that all clients
>>>>> necessarily will (I have a mobile client that uses HTTP for API requests
>>>>> and have not found the priority mechanism necessary -- at least not yet).
>>>>>
>>>>> I could imagine that it would be acceptable to send the PRIORITY frame
>>>>> before the HEADERS frame and still mandate that HEADERS frames (and now
>>>>> only HEADERS frames) initiate streams. While this does cause some extra
>>>>> state allocation, we already have to do something similar with PUSH_PROMISE
>>>>> where we must track that the "stream identifier MUST be a valid choice for
>>>>> the next stream sent" and then initiate the stream later.
>>>>>
>>>>> This would also give us a mechanism to send priority changes for
>>>>> outstanding requests at the framing layer, and could allow priorities to be
>>>>> set for pushed responses should it prove useful.
>>>>>
>>>>> - Jeff
>>>>>
>>>>>
>>>>> On Tue, May 21, 2013 at 2:55 PM, James M Snell <jasnell@gmail.com>wrote:
>>>>>
>>>>>> On Tue, May 21, 2013 at 2:43 PM, Roberto Peon <grmocg@gmail.com>
>>>>>> wrote:
>>>>>> > Sending the PRIORITY frame *MUST* cause state allocation at the
>>>>>> receiver,
>>>>>> > else it was useless to send before the HEADERS frame. As you point
>>>>>> out, at
>>>>>> > minimum it must allocate a stream ID and priority field, and for
>>>>>> most
>>>>>> > implementations it will also need to include so mechanism of
>>>>>> pointing out
>>>>>> > that the headers don't exist, so, probably between 16 to 24 bytes
>>>>>> worth of
>>>>>> > allocation on a 64 bit machine.
>>>>>> >
>>>>>>
>>>>>> Sorry, I wasn't clear in my initial response. Yes, there is some state
>>>>>> that would need to be allocated but not the same as that when the
>>>>>> initial HEADERS frame is received, for instance.
>>>>>>
>>>>>> > If the PRIORITY frame was renamed to CHANGE_PRIORITY, would that
>>>>>> clarify
>>>>>> > anything? Priority changing is the current intent of that frame
>>>>>> type.
>>>>>> >
>>>>>>
>>>>>> Not particularly, because we'd still have the question of when to use
>>>>>> HEADERS+PRIORITY vs. the combination of HEADERS and a CHANGE_PRIORITY.
>>>>>> Can HEADERS+PRIORITY be used any time? For instance, could I send an
>>>>>> initial HEADERS frame on a stream then later send a HEADERS+PRIORITY
>>>>>> on the same stream? I honestly don't care how it ultimately ends up so
>>>>>> long as (a) It's the simplest thing that could possibly work and (b)
>>>>>> Is easy to explain in the spec and easy for a developer to implement.
>>>>>>
>>>>>> > btw, I am not particularly partial to the "any frame opening up a
>>>>>> stream"
>>>>>> > thing. I'm not completely against it though :)
>>>>>> > My reason for slightly preferring that streams must begin with
>>>>>> HEADERS or
>>>>>> > HEADERS+PRIORITY is that it is an explicit statement of intent, and
>>>>>> thus
>>>>>> > off-by-one, uninitialized var, etc. errors are more likely to be
>>>>>> detectable
>>>>>> > in a world where such is required.
>>>>>> >
>>>>>>
>>>>>> I would very much like to see us mandate that streams always initiate
>>>>>> with a HEADERS / HEADERS+PRIORITY frame.
>>>>>>
>>>>>> - James
>>>>>>
>>>>>> >
>>>>>> >
>>>>>> > -=R
>>>>>> >
>>>>>> >
>>>>>> > On Tue, May 21, 2013 at 2:19 PM, James M Snell <jasnell@gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> On Tue, May 21, 2013 at 10:30 AM, William Chan (陈智昌)
>>>>>> >> <willchan@chromium.org> wrote:
>>>>>> >> > Thanks for describing these cases now. I had not thought of them.
>>>>>> >> >
>>>>>> >> > If everyone's sold on reprioritization, then let's go ahead and
>>>>>> do this
>>>>>> >> > and
>>>>>> >> > have the debate about ditching HEADERS+PRIORITY or not. I want
>>>>>> to keep
>>>>>> >> > it. I
>>>>>> >> > don't like the idea of sending a PRIORITY frame first. Is
>>>>>> sending a
>>>>>> >> > PRIORITY
>>>>>> >> > frame going to trigger stream state allocation at the receiver?
>>>>>> What's
>>>>>> >> > the
>>>>>> >> > expectation? And if you don't have a priority for the HEADERS,
>>>>>> then you
>>>>>> >> > have
>>>>>> >> > the race that Roberto described.
>>>>>> >> >
>>>>>> >>
>>>>>> >> There is no reason to assume that sending a PRIORITY frame first
>>>>>> would
>>>>>> >> trigger stream state allocation at the receiver. At most, it would
>>>>>> >> reserve the stream ID and store the priority value. The full state
>>>>>> >> allocation would not occur until the HEADERS frame is received.
>>>>>> That
>>>>>> >> said, I'm not 100% dead set on removing HEADERS+PRIORITY, I would
>>>>>> just
>>>>>> >> like to simplify the protocol where it makes sense to, and even
>>>>>> then
>>>>>> >> only after it's been proven out in implementation. Having separate
>>>>>> >> HEADERS, HEADERS+PRIORITY and PRIORITY frames is confusing, if we
>>>>>> can
>>>>>> >> do without separating them, we ought to do so.
>>>>>> >>
>>>>>> >> - James
>>>>>> >>
>>>>>> >> >
>>>>>> >> > On Tue, May 21, 2013 at 2:09 PM, Patrick McManus <
>>>>>> pmcmanus@mozilla.com>
>>>>>> >> > wrote:
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >> On Tue, May 21, 2013 at 12:32 PM, William Chan (陈智昌)
>>>>>> >> >> <willchan@chromium.org> wrote:
>>>>>> >> >>>
>>>>>> >> >>>
>>>>>> >> >>> I support adding a new additional PRIORITY frame for stream
>>>>>> >> >>> reprioritization.
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >> me too. Specifically I support this as a mechanism for the
>>>>>> client to be
>>>>>> >> >> able to explicitly prioritize an open pushed stream. I can wait
>>>>>> for
>>>>>> >> >> more
>>>>>> >> >> evidence about re-prioritizing, but in cases where the client
>>>>>> hasn't
>>>>>> >> >> ever
>>>>>> >> >> explicitly set the stream's priority I think we have evidence
>>>>>> that its
>>>>>> >> >> time
>>>>>> >> >> to do something.
>>>>>> >> >>>
>>>>>> >> >>>
>>>>>> >> >>> Unless there's a reason this needs to be in the current http/2
>>>>>> draft
>>>>>> >> >>> sooner rather than later, I'd rather punt on this discussion
>>>>>> until we
>>>>>> >> >>> have
>>>>>> >> >>> implementation experience that can guide this debate.
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >> I think there is experience here specifically related to push.
>>>>>> >> >>
>>>>>> >> >> e.g. You can easily configure mod_spdy to push images when html
>>>>>> is
>>>>>> >> >> pulled.
>>>>>> >> >> but you can't effectively dictate the relative priorities of
>>>>>> those two
>>>>>> >> >> things.
>>>>>> >> >>
>>>>>> >> >> Sure, you can define an explicit priority for those images but
>>>>>> priority
>>>>>> >> >> implementations are all about relative levels and the client
>>>>>> set the
>>>>>> >> >> priority of the html.
>>>>>> >> >>
>>>>>> >> >> You can argue that mod_spdy should have defined relative
>>>>>> priorities
>>>>>> >> >> (+/-
>>>>>> >> >> the associated stream) instead of constants.. that would be
>>>>>> better but
>>>>>> >> >> the
>>>>>> >> >> client still has no way to make sure those streams are at a
>>>>>> higher
>>>>>> >> >> priority
>>>>>> >> >> than a "save as" background stream (I've seen this one happen as
>>>>>> >> >> mod_spdy
>>>>>> >> >> defaults to lowest priority when pushing), or a lower priority
>>>>>> than a
>>>>>> >> >> real-time video stream..
>>>>>> >> >>
>>>>>> >> >> plus there is no scale for the server to work with.. it might
>>>>>> set a +2
>>>>>> >> >> priority for pushed images but the client might be using +3 for
>>>>>> pulled
>>>>>> >> >> images causing a mismatch in something that was intended to be
>>>>>> equally
>>>>>> >> >> weighted.
>>>>>> >> >>
>>>>>> >> >> at least with a priority frame the client can make those
>>>>>> adjustments in
>>>>>> >> >> a
>>>>>> >> >> RTT.
>>>>>> >> >>
>>>>>> >> >> Cheefully,
>>>>>> >> >> -Patrick
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >
>>>>>> >> >
>>>>>> >>
>>>>>> >
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>