RE: Conversion of MS CSS 2.1 tests to reftests

> 
> On 09/09/2010 12:46 PM, Geoffrey Sneddon wrote:
> > Hey,
> >
> > Attached is a diff to convert 830 tests to reftests (with a mere four
> > references!). The list of tests for each reference is based upon tests
> > that pass in Opera with the same screenshot, so anything that we fail
> > won't be included, but still, 830 tests is a nice start.
> >
> > It'd be nice if you could apply the diff (just adding five new files),
> > and adding it to the build system, sometime in the near future.
> 
> Arron, if this looks okay to you, I'm happy to check it in for you.
> 
> ~fantasai
> 
I would like to get some clarification on this ask of Arron before he commits to this work. 

Based on my understanding of how test ownership works, you are asking Microsoft to take ownership of these references moving forward, and so any changes to the tests referenced here will require that we update the references themselves. Correct?

Is your expectation that the 830 tests here are just the beginning? The test suite should be locked down in 5 days and then Implementation Reports should be complete within 30 days. If there are more changes coming, that seems to go against 'locking down'.

If there are no additional changes coming, it seems like reftesting ~8% of the suite isn't worth the cost/benefit analysis. Maybe I'm missing something, but I also think that each Vendor will want to run these tests manually anyway in order to verify your references. That feels like not only a 0 sum gain, but in reality extra work for everyone as we are approaching the end to CSS2.1 (which I what I believe we all want - to be done and moving on with CSS3).

Please verify, I may just be misunderstanding the expectations here.

-John Jansen

Received on Friday, 10 September 2010 22:19:01 UTC