Re: CR exit criteria and features at risk for HTML5

On 8/16/12 3:04 AM, James Graham wrote:
> In this case I'm not that worried because the W3C has the WHATWG to keep
> it in line. If the W3C spec can't be trusted on
> interopeability-affecting matters then the WHATWG version will become
> the de-facto reference for implementors. So as long as bugs in the spec
> are found at all there will be strong pressure on W3C to fix their
> branch or risk becoming irrelevant.

Even if we posit this, it seems like a failure mode best avoided by 
planning, not by UA implementors basically abandoning the W3C HTML spec 
as a useful reference...

> I would happily give up any mention of testsuites in the
> Process in exchange for solving *that* problem.

Well, a good start would be having a simple, advertized way to 
contribute tests.  I don't even know how I'd go about contributing a 
test for the HTML5 test suite, and my experience with contributing tests 
for other W3C working groups has been less than pleasant.  :(

Of course contributing test suites to browsers that are not Mozilla is 
not that easy for me either.  But for Mozilla, it takes me very little 
time to add a test to the Mozilla test suite.  If we can get 
contributing tests for the official test suite down to that level, I bet 
developers will be more likely to contribute.

Another alternative is what the CSS working group is doing: making it 
easier to sync tests back and forth between their test suite and the 
browser-developer test suites (e.g. by adopting a test format that the 
browser developers can use easily inside their own test suites, setting 
up procedures for mirroring tests, etc).  Then I could just check in my 
tests to the Mozilla test suite as I normally do and they'd 
automatically end up in the spec test suite.  ;)

> I'm not sure what you consider "the most risky" parts of the spec to be

The navigation and event queue bits.  I did say that...

> but in my experience the parts least likely to match reality in their
> details are the parts documenting the most fundamental behaviour of the
> web.

Indeed.  Like navigation.

> For example the specification of session history is -- I think --
> considerably more buggy than the specification of the 2D context or many
> other new features.

Yes.  Session history is definitely part of the whole navigation mess.

> nor are people likely to be happy stalling waiting for
> browsers to rewrite such fundamental code to get to Rec.

Why not?  Why bother going to Rec with a spec we _know_ is bogus?

> So there will be strong pressure to just drop those tests from the testsuite "for this
> version".

In which case we've failed and might as well go home.  Which _is_ being 
pretty tempting, for sure.

> So I don't really forsee a testsuite written for the purpose
> of proving interoperability actually helping interoperability as much as
> a testsuite written for the purpose or finding bugs.

There seems to be 0 incentive to write the latter, unfortunately.  :(

-Boris

Received on Thursday, 16 August 2012 07:21:52 UTC