Re: Browser detection for shepherd/css test runner

On Mar 31, 2015, at 2:43 PM, Christian Biesinger <cbiesinger@google.com> wrote:

> On Fri, Mar 27, 2015 at 9:45 PM, Peter Linss <peter.linss@hp.com> wrote:
>> The problem is, while there’s divergence in some areas, there’s still shared code in many (most?), and it currently requires human judgement to determine if passes form both count as two independent implementations or not.
> 
> Yes of course. I guess what you are implying is that the intent of
> these boxes is to see at a glance whether there's enough
> implementations to go from CR to REC...?

Correct, that was the primary purpose of the test harness. You'll notice after the table counts of how many passes are required to reach CR exit criteria.

> 
> To me, it was more useful to see how various browsers do, and in some
> cases (Flexbox, Grid, probably other more recent CSS specs) the
> implementations do diverge and may easily pass different tests;
> whether or not they should count as independent implementations is a
> different issue.

Understood, one a spec exit's CR, the focus of the test suite does switch to conformance testing. At some point I can take a look at augmenting the system to display WebKit and Blink in different columns, but count them as one implementation. Ideally I'd like a mechanism to know where they should be considered the same vs independent...

> 
>> If someone wants to generate a map (keyed by spec section) where there’s divergence (or the converse would be better, since that set isn’t getting bigger), I’d be happy to have the test harness automatically differentiate the results.
>> 
>> For the record, each result is keyed to the full UA string that generated it, so we can always go back and re-assign results to different products.
>> 
>>> 
>>> As for Presto, it is certainly of quickly declining relevance, but there are still a few areas where it supports features not supported in other browsers, or conforms better than other browsers. I think we should keep it for a while longer. As it provides useful information to the people working on the spec or the tests.
>>> 
>>> Besides, there's nothing wrong with a little tongue-in-cheek challenge to other browsers ("Even old Presto gets this right!”).
>> 
>> No reason to remove old passing implementations… I’ll take passes from Lynx if it helps get a spec to CR.
> 
> Fair enough. Is the UA detection good enough to not treat current
> Opera as Presto?

Yes, it goes by UA string. Opera's current UA string only mentions WebKit so that's what it's counted as. When we treat WebKit and Blink differently I can add some special case code to consider that Blink (though it would be best if the Opera US string actually listed Blink, for matter if Chrome would too...)