Re: #578: getting real-ish numbers for option 3

To clarify: we'd test with these synthetic headers to get an idea of how things that use custom headers would behave; we'd still test using the "normal" web corpus as we have before.

Cheers,


> On 24 Oct 2014, at 2:20 pm, Mark Nottingham <mnot@mnot.net> wrote:
> 
> I'm sensing a theme develop here, and it's entirely reasonable to say that we shouldn't block for 2+ months for an optimisation without any supporting data.
> 
> We already have an implementation of the proposal, so what we need now is a test corpus.
> 
> That presents a problem, since it's highly subjective (extension headers are very site-specific) and often fraught with privacy issues. 
> 
> What I'd suggest is that we agree upon an algorithm to synthesise some headers, do so, and use one corpus to test both approaches.
> 
> Straw-man:
> 
> - 10000 request messages, using the same gzip context
> - each request has five standard headers (:method :scheme :authority :path user-agent)
> - each standard header has a fixed value of an appropriate length
> - each request has five custom headers, selected from a pool of ten custom headers
> - each custom header has a field-name between 10 and 20 characters long
> - five of the custom headers has a field-value consisting of random data 40 characters long
> - five of the custom headers have a fixed value 40 characters long
> 
> Thoughts? 
> 
> 
>> On 24 Oct 2014, at 9:46 am, Michaela LaVan <mlavan@google.com> wrote:
>> 
>> Google is in favor of adopting Willy's proposal if there is data demonstrating improved performance as a result of the change. We may not be able to contribute data of our own in time for IETF 90, however.
>> 
>> On Thu, Oct 23, 2014 at 2:02 PM, Mike Bishop <Michael.Bishop@microsoft.com> wrote:
>> +1 to this.  Microsoft is not going to implement this in our code in the required time frame, but if there's data that can demonstrate substantial advantages across the board, we wouldn't object to pulling the change into the spec.
>> 
>> -----Original Message-----
>> From: Nicholas Hurley [mailto:hurley@todesschaf.org]
>> Sent: Thursday, October 23, 2014 10:39 AM
>> To: Mark Nottingham; HTTP Working Group
>> Subject: Re: Implementer intent -- option 3 for #578
>> 
>> I'm not convinced another interop is worth considering until this scheme has been shown to be a *significant* improvement for *all* use cases over the status quo. Mark has said the only four reasons we would accept any changes at this point are:
>> 
>>> a) editorial improvements
>>> b) substantial interop problems
>>> c) serious security issues
>>> d) changes that have broad consensus (i.e., we all agree it's worth
>>> it)
>> 
>> and furthermore went on to say that the only category this option falls under is (d), and that "if making these changes is controversial, we haven't met the bar".
>> 
>> So far, I haven't seen this be a totally non-controversial change (though I understand it's not my call to make). That said, there is (to my mind) a way the proponents of this change can remove any controversy
>> - by proving that this option is a wholesale improvement. We don't need another interop draft for that, just someone to run a decent number of test cases (that fall under both the "better with static table first"
>> and "better with dynamic table first" buckets) and show us the numbers.
>> Tatsuhiro has provided an implementation for anyone who doesn't want to update their own (or write their own), so let's wait to see what the numbers tell us before making what is currently at best a speculative change to the spec.
>> --
>> Peace,
>>  -Nick
>> 
>> 
>> 
> 
> --
> Mark Nottingham   https://www.mnot.net/
> 
> 

--
Mark Nottingham   https://www.mnot.net/

Received on Friday, 24 October 2014 03:26:15 UTC