Re: Question about HTTP 2.0 priority

On Thu, Oct 9, 2014 at 5:12 PM, Greg Wilkins <gregw@intalio.com> wrote:

>
> On 10 October 2014 05:35, Chad Austin <caustin@gmail.com> wrote:
>
>> And, of course, the most interesting thing about this entire discussion
>> is that managing a request priority queue is often very easy for both
>> clients and servers.
>
>
> Actually it can be very hard for a server and there are some tough issues
> to deal with.  Priority is going to be a very hard nut to crack on the
> Server.    From a clients point of view it is mostly about optimising round
> trips and network utilisation, but a server has to consider overall through
> put as well as keeping individual connections optimal.     It is probably
> not sensible for a server to throw all of it's cores at a single connection
> to optimally drive the network utilisation for 1 client when it may have
> 99,999+ other client connections to consider.
>
> Thus I believe that servers are going to have to limit the degree of
> parallelism they are prepared to allocated to a single connection,
> otherwise HTTP/2 will have very bad mechanical sympathy.
>

I did not mean that prioritization in general is easy all the time.  I was
just pointing that managing a priority queue is simpler than (or as simple
as) a dependency tree model.  I have yet to understand why HTTP 2 chose a
dependency tree for priority over a priority queue.  I understand, however,
that it's unlikely to change at this point in the process.

So lets say that we have a queue of 30 request in priority order.    A
> server may wish to limit the parallelism that it gives to this, so it may
> allocate threads only to the first few requests and it makes sense to pick
> them in priority order.    But what if those high priority requests are
> slow to generate?   Should the server still wait for them to complete even
> though the network connection for that client is idle at if it allocated a
> thread to a lower priority resource it might be able to progress?
>

In the past, to solve similar problems, I've broadly characterized request
handlers as CPU-bound, disk-bound, or whatever.  A-bound for some resource
A.  :)  Then each resource gets its own priority queue.  The HTTP 2
connection would push each request into the appropriate resource queue,
where it would be executed as resources are available.  Thus, if a bunch of
CPU-bound requests arrive, and ~all the cores are effectively busy, a
disk-bound request could still be processed and returned immediately.

This approach gets trickier when request handlers are bound on a mix of
multiple resources.

This will of course be further complicated by push, where 29 of those
> requests are generated from push_promises from information that the server
> knows about the base request, which could include time to generate
> dependencies and priorities assigned on previous pushes.  A server might
> know that it has some low priority quick resources that it can dispatch and
> server in the time it takes some high priority slow resources to be
> generated.  Static resources might be able to be asynchronously pushed
> while generated resources may need a thread, further complicating the
> resource allocation.
>
> cheers
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Greg Wilkins <gregw@intalio.com>  @  Webtide - *an Intalio subsidiary*
> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that
> scales
> http://www.webtide.com  advice and support for jetty and cometd.
>



-- 
Chad Austin
http://chadaustin.me