Re: Stream State and PRIORITY Frames

On Thu, Jan 19, 2017 at 6:11 AM, Cory Benfield <cory@lukasa.co.uk> wrote:

>
> > On 19 Jan 2017, at 09:00, Amos Jeffries <squid3@treenet.co.nz> wrote:
> >
> > IIRC the intention behind the closing behaviour for idle streams was to
> > ensure that we could optimize away the need to maintain a list or array
> > of 2^31 state entries.
> >
> > Omitting PRIORITY from the MAX_STREAMS voids that benefit.
>
> No it doesn’t.
>
>

Agreed. There is no mechanism defined in the specification to communicate a
limit for the state retained due to prioritization/dependency information.
This is currently limited to implementation specific heuristics.



> Firstly, let’s note that the PRIORITY frame is defined not to change
> stream state. Receiving a PRIORITY frame on stream N leaves it in the idle
> state.

A server implementation can thus treat the receipt of a PRIORITY frame on
> any stream that has a higher ID than the last stream on which it received a
> HEADERS frame as having the *exact* same effect on state retention as
> receiving a PING frame. It doesn’t *require* the insertion of any stream
> state data.
>
> Specific implementations may require the insertion of stream state data if
> priority information is stored on the same structure as stream state
> information, but that is certainly not required. For example, receiving a
> bunch of PRIORITY frames does not cause the Python HTTP/2 implementation to
> allocate any more state information than receiving a bunch of PING frames.
>
> > It is causing state to be allocated on the server and on every
> > intermediary along the way which accepts it.
> >
> > Consider the effects of 2^31 PRIORITY frames being sent with different
> > IDs before the first HEADERS is used.
> > Notice how even this one case is markedly worse than sending just one
> > HEADERS with stream ID == 2^31 to waste server sockets.
>
> This problem is orthogonal to the one we’re discussing.
>
> So far I haven’t seen a proposal to say that PRIORITY frames should
> transition a stream out of the idle state. If we continue to leave RFC 7540
> saying that PRIORITY frames leave streams in the idle state, then
> definitionally they are excluded from the restrictions on
> MAX_CONCURRENT_STREAMS. So if you are allocating stream state for idle
> streams, then you are open to this DoS vector, but none of the proposals
> here are trying to address it.
>
> Additionally, I should note that the unbounded insertion of priority
> information into priority trees was discussed as an attack vector in this
> paper: https://www.imperva.com/docs/Imperva_HII_HTTP2.pdf. The paper was
> published in August of last year, and several implementations took steps to
> reduce their vulnerability to it (the Python priority implementation even
> filed a CVE, CVE-2016-6580).
>
> I absolutely support adding an erratum to indicate that implementations
> should resist unbounded insertion of PRIORITY information. But unless
> you’re willing to say that idle streams should be counted against
> MAX_CONCURRENT_STREAMS (which is a tricky thing to state), then the only
> way to resolve this problem at the spec level is to force PRIORITY frames
> to make state transitions, further complicating the state diagram for
> streams.
>
> > What exactly does it mean for them to set PRIORITY on a non-existent
> stream?
>
> It means exactly what RFC 7540 says it means. Quoting from Section 5.3.4
> of RFC 7540:
>
> > Similarly, streams that are in the "idle" state can be assigned priority
> or become a parent of other streams. This allows for the creation of a
> grouping node in the dependency tree, which enables more flexible
> expressions of priority. Idle streams begin with a default priority
> (Section 5.3.5).
>
> While we’re here, right below that section is:
>
> > The retention of priority information for streams that are not counted
> toward the limit set by SETTINGS_MAX_CONCURRENT_STREAMS could create a
> large state burden for an endpoint. Therefore, the amount of prioritization
> state that is retained MAY be limited.
>
> To this end, the Python priority implementation allows a user-configured
> maximum amount of data retention for priority information. This priority
> information is held separately from stream state information because they
> are entirely orthogonal concerns: in fact, the bits of code that manage
> priority information and those that manage stream state information are in
> entirely separate installable packages with no dependency relationship to
> each other.
>
> To my eye, there is a clear intent to allow setting of priority
> information on streams that are in the idle state, without that affecting
> the stream itself.
>
> Cory
>
>
>
>
>
>
>
>