Re: CR exit criteria and features at risk for HTML5

On Sat, 18 Aug 2012 05:36:36 +0200, Glenn Adams <glenn@skynav.com> wrote:

> On Fri, Aug 17, 2012 at 12:30 AM, Benjamin Hawkes-Lewis <
> bhawkeslewis@googlemail.com> wrote:
>
>> On Thu, Aug 16, 2012 at 4:35 PM, Glenn Adams <glenn@skynav.com> wrote:
>> > There are a number of formal standards organizations waiting on HTML5  
>> to
>> > reach REC (in any form whatsoever) so that they may publish standards
>> > documents that formally reference HTML5.
>>
>> Formally reference it for what purpose?
>>
>
> Does it matter? Basically, formally reference it in the normative  
> reference
> sections of their standards/specifications.
>

I agree about the necessity to get to Rec ASAP and issuing new Recs more  
frequently (html 5.1, 5.2 and so on), with the clear disclaimer (in case  
is not obvious) that specs cannot be bug free and new Rec may change (even  
in non backward compatible ways) if they were terribly broken and do not  
match reality (of course care should be taken to delay things to the next  
Rec where there is no good reason to have them in)

Note that in practice, from a technical point of view, this will be the  
same as having a long living standard  or a spec that keeps being in CR  
forever. So should not be an issue for implementers. But this way of  
working suites best other organizations referencing HTML5 (for the reason  
already described in this thread).

Of course you could try to convince all organizations on the planet  
referencing HTML5 to change their way of doing references, but it seems  
easier to do a procedural change in W3C if the end result doesn't affect  
the technical spec too much (since we are all aware that new version will  
come and people will have to be prepared to update their references)


>
> Some organizations writing downstream specs that reference HTML5 will
> write|publish their own compliance test regimes that effectively (i.e.,
> operationally) define what is correct as far as they are concerned. Such
> test regimes will also change over time.
>

This is a critical point IMO. Who will write/maintain test cases to be  
used (referenced) by these compliance test regimes?
If this is done by each single organization, based (maybe) on different  
versions of spec, I see a potential big mess where people may not be able  
to implement a UA that passes all test suites (or alternatively long  
battles on test correctness and which version of the spec to follow).  
These discussion may well happen in W3C as well, but at least would be  
limited to one, rather than several, group.

So to avoid this, I believe that test cases and spec should be tightly  
coupled and handled inside the same organization possibly by the same  
individuals.

But this doesn't necessary need to happen at the expense of speed, i.e. it  
doesn't necessary need to be tied to the Spec going to Rec.
Decoupling testing progress from Spec progress by making testing a first  
class citizen in W3C may be the only solution to get both a test suite and  
a quick path to Rec.
Having dedicated staff to help writing and maintaining test harnesses and  
other tools that would make easier to write and contribute test cases, to  
check which part of a spec is tested and which one is not, which test  
cases are passing on different browser and which ones are not, and so on,  
would be of great help.

And even having someone (staff or 3rd parties) writing tests if there is a  
lack of members contributed tests could be an option, to speed things up.

Testing must be as important as the spec, or even more important, so there  
should be a reasonable effort put into creating and maintaining a test  
suite and all tools and infrastructure needed around it.


/g


-- 
Giuseppe Pascale
TV & Connected Devices
Opera Software

Received on Friday, 31 August 2012 09:47:52 UTC