RE: Conversion of MS CSS 2.1 tests to reftests

> From: public-css-testsuite-request@w3.org [mailto:public-css-testsuite-
> request@w3.org] On Behalf Of L. David Baron


> Given the way the spec is written, I don't think that's realistic.
> If we want that level of stability, we need a spec written much more
> formally.  In the current spec, people will keep finding issues, and
> we'll need to keep discussing them, no matter what the maturity
> level of the spec.

How we revision the CSS2.1 spec beyond REC certainly is an important issue. 
I would like to stick to issues that actually *prevent* a browser vendor 
from submitting an implementation report by the date that was agreed to in 
Oslo. If we can't even agree on when, how or even whether to get to REC,
what should happen beyond that doesn't seem any more likely to be resolved.

> > We've learned a lot after working on this suite and I think that
> > learning informs the 3.0 suite nicely, but I am loathe to spend
> > more time on 2.1 now that we are in the very end game.
> 
> We've clearly agreed in the past that while the snapshot of the test
> suite to be used for entering PR may be frozen at some point, we
> would continue to develop the test suite to improve
> interoperability.

True. John is talking about running the current snapshot and publishing
the results to get to PR.

> > I believe the benefit to changing some percentage of them to ref
> > tests is only incremental and will cause an overall delay.
> 
> I disagree; it allows browser vendors to share work rather than all
> do it separately.

This may be beneficial with respect to achieving the WG's deadline if the 
amount of time you have to contribute to complete this system, test it
 + the time spent running it and collecting results is equal or inferior 
to what it would take you to run the testcases on your own. That may be the 
case but I really don't know. It may also be that you deem the other benefits 
of having an automated system to be far more worthy of your time than running 
thousands of tests by hand. That is totally fine too, but this choice and its 
impact on the schedule are your responsibility. It's reasonable to ask whether 
the goals you are pursuing do allow you and others to make 10/15 or not.

John is assuming the answer to be 'No'. If you know him to be wrong, that's great
and we'd love to hear more about it. If he is correct though, you and others would 
have implicitly decided automation takes priority over submitting IRs on time and 
thus moved the deadline. Or at least made it more likely to be missed.

If that were the case, it would be helpful for you to at least say so clearly and
unambiguously.

Received on Monday, 20 September 2010 19:32:13 UTC