Re: Writing Modes PR & Testsuite

Thank you Geoffrey for running the tests and filing a test bug. Anti-alias
issues are not easy to deal with, as far as I understand, Gérard and Xidorn
are working on it in https://bugzilla.mozilla.org/show_bug.cgi?id=1258916.
In general, Gérard is following the CSS test guidelines to avoid anti-alias
issues, but the guidelines don't seem to be perfect. I agree that we should
continue working on improving tests, and multiple vendors working together
is important for issues like this, so your inputs are highly appreciated.

I don't think, however, anti-alias issues can affect the spec status, tests
can and should be improved regardless of the spec status. Hope W3C process
experts can commet if I'm wrong.

/koji

2017-01-12 14:48 GMT-08:00 Geoffrey Sneddon <me@gsnedders.com>:

> Yesterday, we resolved to publish either a CR or, preferably if the
> Process allows it, a PR of Writing Modes Level 3.
>
> However, I'm unconvinced we have a testsuite sufficient to leave CR
> (and proceed to PR) with; we seem to have hundreds of reftests which
> fail due to anti-aliasing differences.
>
> According to the test harness, Firefox passes 89.02% of the testsuite
> (which it makes out to be 1120 tests); running locally all automated
> tests, I get 1045 tests (910 parents, 135 subtests; as far as I'm
> aware the harness has no notion of harnesses), of which 457 pass and
> 588 fail: this implies that 89.02% of the entire testsuite cannot
> pass.
>
> Looking into many of the failures, it quickly became apparent that
> hundreds of these failures is down to anti-aliasing differences in
> reftests; I've filed a bug for this at
> <https://github.com/w3c/csswg-test/issues/1174>.
>
> As such, I at least don't view the testsuite as ready to publish a PR
> and would raise a formal objection if we resolved to do so.
>
> /Geoffrey.
>
>