Re: Call for Consensus: Remove "reference set" from HPACK (to address #552)

Willy,

I see what you are saying.  However my experience with hpack is that
fiddling the bit encodings is going to give you +/- <1% difference.

I can make +/- >5% differences by picking different encoding strategies and
see similar differences over different stories over the test data.

So I think tweaking the encodings at this stage is really just operating
within the noise of the different headers.   So I don't think we should
change hpack in this way.  We just don't have the data to optimise the last
1% nor do we know if natural variation is such that it is pointless to try
to find a one-size-pefectly-fit-all solution.

Removing RefSet is definitely a good thing to do.

Removing the copy of static entries to the dynamic table make measurable
impact on compression, so I think we should remove it for simplicity and to
save CPU - it only being added because of the RefSet.

With those removed,  I'm 90% happy with hpack

Once you remove the static copy, I do think there is an argument to be made
to revert the static table to below the dynamic table. This is for
simplicity and because it will allow the common field indexes to be
precomputed as a single byte, saving CPU.      It does not appear to affect
the compression efficiency in any significant way.

With this I'd be 95% happy.

Encoding dates as integers does appear to give some additional
compression.  But apparently it has been proposed and rejected before by
the WG.     So while I think it would be good, I'm not going to advocate
that we revisit at this stage.


cheers













On 18 July 2014 01:45, Willy Tarreau <w@1wt.eu> wrote:

> Hi Greg,
>
> On Wed, Jul 16, 2014 at 04:00:33PM +1000, Greg Wilkins wrote:
> > On 16 July 2014 15:45, Willy Tarreau <w@1wt.eu> wrote:
> >
> > > The problem I'm seeing with this change is that you need one
> extra-byte to
> > > send litteral headers fields that are not present in the static table.
> > >
> >
> > The first 64 entries can still be sent with 1 byte.   So I think in many
> > common use-cases almost all the indexes will just be 1 byte.
> >
> > > Maybe
> > > on the other hand it rarely happens that they're not in the static
> table ?
> > >
> >
> > Well the static table has few name+value fields, but the ones it does
> have
> > are common.
>
> I'm noticing that we need only 4 bits to send any possible static
> header+value
> (assuming we swap accept-charset and accept-encoding), since all other
> values
> are empty and will never be used as-is but as litterals.
>
> So I think that based on your suggestion to split indexing of static and
> dynamic, I'm suspecting we could systematically split indexes between
> static
> and dynamic in opcodes. Indeed, some index fields are already small (6+ for
> litterals and 4+ for litterals not indexed/never indexed), so having the
> static first will force one extra byte for all dynamic headers. However,
> stealing one bit in unused value ranges could help make that situation
> better. Something along these lines seems possible (please forgive me for
> errors, I'm encoding as I write) :
>
>   11 XXXXXX => send 6+indexed header from *dynamic* table as-is  (0xC0
> reserved)
>   10 01XXXX => send 4+indexed header from *static* table as-is   (0x80
> reserved)
>
>   00 00XXXX => send 4+indexed *dynamic* header without indexing (unchanged)
>   10 00XXXX => send 4+indexed *static* header without indexing
>
>   10 1XXXXX => send litteral based on *dynamic* header table (0=new name
> or reserved)
>   01 XXXXXX => send litteral based on *static* header table (0=new name)
>
> The idea is that even when we have few bits to encode the index, we're not
> forced to add an extra byte to encode the values from the other table,
> whatever
> table we pick first as first.
>
> I don't know if you see what I mean of if that's even more confuse.
>
> Cheers,
> Willy
>
>


-- 
Greg Wilkins <gregw@intalio.com>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.