Re: CR exit criteria and features at risk for HTML5

On 8/16/12 4:32 AM, James Graham wrote:
> Well the actual "contribute" part is basically "commit to a hg repo".
> The full instructions are at [1].
...
> [1] http://www.w3.org/html/wg/wiki/Testing/Submission/

Ah, excellent.  I hadn't realized things were that far along.  That's 
great to hear!

> Well we are using reftests for rendering tests and testharness.js tests
> for javascript-driven tests. Thanks to Ms2ger Mozilla has a harness for
> running testharness.js tests inside mochitest. I know this isn't quite
> as easy for you as "just write a mochitest"

It's close enough.  Shouldn't be a problem to write tests in this 
format.  I'll give it a shot.

> I'm not sure what you need us to do to help you mirror tests, but I am
> happy to help with anything that will make it easier for you.

It's worth checking with fantasai how she set up the CSSWG reftest 
mirroring and seeing if that can just be reused directly for the HTML 
tests on the W3C end.  Obviously at that point we can simply reuse our 
own end of it too.

> I don't see any reason that whatever infrastructure on your end works
> for contributing to CSS can't be trivially ported to work with HTML.

Agreed.

> Well I guess it depends what you think the value of a Rec. is.

Right, that was my original question.  ;)

> If you
> think that it's basically IPR related then there is value in getting
> there even if the details of the spec aren't all correct.

Agreed.

> It seems to me that people who take that view are also the most likely
> to stop contributing tests at all if there is no Rec. incentive; people
> who think that the value of Rec. is in reaching a level of
> interoperability can still aim to improve interoperability by writing
> and publishing tests even when it isn't coupled to a Process transition.

Yes.  My concerns are about the legal status of a REC and about other 
groups building on the REC and creating things that contradict what 
needs to actually happen once the bugs in the REC are fixed....

I guess as long as we make it clear that the REC is NOT a final word in 
any sense, this should hopefully not be too much of a problem.

>>> So there will be strong pressure to just drop those tests from the
>>> testsuite "for this
>>> version".
>>
>> In which case we've failed and might as well go home.  Which _is_ being
>> pretty tempting, for sure.
>
> If your claim is that the W3C Process doesn't work, then I can't say I
> disagree. I would much prefer a process where the spec — much like
> browsers — underwent continual development and there were time based
> snapshots for lawyers.

Amen.

> Figuring out a motivation beyond "improving the
> web as a platform" for people to write and contribute tests in this
> scenario is of course more difficult.

Even that might be sufficient motivation....

>>> So I don't really forsee a testsuite written for the purpose
>>> of proving interoperability actually helping interoperability as much as
>>> a testsuite written for the purpose or finding bugs.
>>
>> There seems to be 0 incentive to write the latter, unfortunately.  :(
>
> You don't think that you benefit as browser interoperability improves?

Well, ok.  I think there _should_ be incentive, but people aren't acting 
as if it's there.  I'll see what we (Mozilla) can do to lead by example 
here.

-Boris

Received on Thursday, 16 August 2012 15:58:16 UTC