Re: Conversion of MS CSS 2.1 tests to reftests

On Monday 2010-09-20 16:28 +0000, John Jansen wrote:
> My entire concern is over the current time-frame. I realize I'm
> new to the group, but I am really just concerned with being done
> with 2.1 and moving on. For me, if we need to discuss any 2.1
> issues at TPAC, that is a failure on our part. 

Given the way the spec is written, I don't think that's realistic.
If we want that level of stability, we need a spec written much more
formally.  In the current spec, people will keep finding issues, and
we'll need to keep discussing them, no matter what the maturity
level of the spec.

> We've learned a lot after working on this suite and I think that
> learning informs the 3.0 suite nicely, but I am loathe to spend
> more time on 2.1 now that we are in the very end game.

We've clearly agreed in the past that while the snapshot of the test
suite to be used for entering PR may be frozen at some point, we
would continue to develop the test suite to improve
interoperability.

> We planned and reached consensus on a test process as a working
> group.  Changing or adding any processes to the working group's
> test suite at this point will only cause risk and delay.  The test
> cases are done and the suite is nearly complete.  Why would we
> re-engineer anything this is already done and working?  I oppose
> any change to the tests (other than the ongoing agreed upon review
> process) or the agreed-upon test case process at this point
> because I believe the end result will be a delay to getting the
> implementation reports.  

On which points in particular did we reach consensus?

> As we've seen already with Mozilla's feedback over the weekend,
> the primary cost for the implementation reports is going to be
> reviewing the tests for accuracy, not actually running them.

I don't think that's true; writing the messages I sent took a few
minutes out of the few hours that I spent.

> I believe the benefit to changing some percentage of them to ref
> tests is only incremental and will cause an overall delay.

I disagree; it allows browser vendors to share work rather than all
do it separately.

> Please note too: the test suite has been complete since January,
> so I was surprised (again, I'm new to the group, so forgive my
> surprise) that the group would want to start this work now, when
> we are literally just weeks from being done.

Complete by what definition?  I think most of the tests Mozilla
contributed weren't incorporated until the releases of the test
suite this summer.

-David

-- 
L. David Baron                                 http://dbaron.org/
Mozilla Corporation                       http://www.mozilla.com/

Received on Monday, 20 September 2010 18:06:53 UTC