RE: [HTML testing] CfC: Adopt "Plan 2014" and make some specific related decisions

Yes we talked a bit about this today at our 'Meeting' today.  I think that it seems to be part of the charter for the TTF to start to build out this data starting with the table of contents.   It's also think that we do know of a number of features that are interoperable today (parser).

For example (don't read too deep or use this a pure statement)

4.8.2 The iframe element (http://dev.w3.org/html5/spec/single-page.html#the-iframe-element)

    Mostly interoperable and implemented across multiple browsers.
    Expectation is the 'seamless' attribute, which seems to currently only be implemented in chrome.
    Though web developers indicate this is great feature to have in the web platform.
    Sandbox is implemented in FireFox 17, IE10, Chrome at this point.
    
http://www.w3.org/2012/10/09-htmlt-irc

[08:14] <plh> I'm trying to generate a preliminary implementation report for tpac
[08:15] <krisk> I have some data as well that I was thinking about presenting
[08:15] <krisk> In an effort to help allow the spec to enter CR
[08:15] <jgraham> plh: Based on what data?
[08:16] <krisk> I was just going to have a chart graph based on how many tests have at least two browsers passing
[08:16] <plh> jgraham, based on the approved tests
[08:16] <krisk> ...and not document who specifically passed/failed
[08:16] <jgraham> That's not going to cover much
[08:16] <jgraham> Seems it would be really useful to have a measure of which parts of the spec actually have coverage
[08:17] <jgraham> As well as which parts have implementations passing the tests we do have
[08:17] <jgraham> And some qualitative data where coveriage isn't great
[08:17] <jgraham> e.g. Section X: no tests, but caniuse.com says 3 browsers implement this already
[08:17] <krisk> I was also going to go back a few years and show simalar data
[08:18] <krisk> I think it's important to focus on all the features that browsers have all added/imnplemented in the past 3->4 years
[08:19] <jgraham> I think I diagree with that
[08:19] <krisk> Stuff like History API, Canvas, HTML Audio/Video are all good examples of features that have been added and that are highly interoperable in current browsers
[08:19] <jgraham> *disagree
[08:20] <jgraham> At least, if the goal is to find non-interoperability (which it should be), then older stuff is at least as valuable
[08:20] <plh> in terms of determining the coverage, what data do we have?
[08:20] <krisk> Yes and I recall that the 2014 Plan is to remove stuff from the HTML5 spec that is not interoperable
[08:21] <jgraham> krisk: That sounds like a very dubious plan to me
[08:21] <krisk> Well another way to view the problem is that the spec is huge...
[08:22] <jgraham> Another way to view the problem is that the platform is large
[08:22] <jgraham> And interconnected
[08:22] <jgraham> small specs leave interoperability gaps at the edges
[08:22] <krisk> and we do have tests that cover big chunks of the spec and the test we do have don't show alot of interop issues
[08:22] <krisk> E.g. canvas, html5 video/audio,  parser
[08:22] <jgraham> Parser is the great success story :)
[08:23] <krisk> So it would seem to be important to have the WG agree that these parts of the spec are stable and enough to enter CR
[08:23] <krisk> Then we can start to look at other areas of the spec that don't have tests and are viewed as not interoperable
[08:23] <krisk> Make sense?
[08:26] <jgraham> More or less, I think
[08:26] <krisk> Sorry if I am typing to fast...
[08:26] <krisk> I suspect the co-chairs (plh) can correct me do want to see some more data from the testing task force
[08:27] <krisk> I think it would not be unreasonable to take the table of contents
[08:27] <krisk> ..e.g. http://dev.w3.org/html5/spec/single-page.html
[08:27] <krisk> and for each section have data...
[08:28] <krisk> For example
[08:28] <krisk> 4.8.2 The iframe element (http://dev.w3.org/html5/spec/single-page.html#the-iframe-element)
[08:29] <jgraham> I think that would be excellent
[08:29] <jgraham> If we understand what the limitations of the data are
[08:29] <krisk> Then list out what we know today...Implemented in browsers - sandbox is only implemented by webkit, ie and FF17
[08:31] <krisk> I would expect this to generate alot of discussion

-----Original Message-----
From: Sam Ruby [mailto:rubys@intertwingly.net] 
Sent: Tuesday, October 9, 2012 10:12 AM
To: public-html-testsuite@w3.org
Subject: [HTML testing] CfC: Adopt "Plan 2014" and make some specific related decisions

On 10/08/2012 05:02 PM, Maciej Stachowiak wrote:
>
> Recently, the Chairs of the HTML Working Group proposed a plan to 
> bring HTML5 to Recommendation by 2014. This plan was created in 
> consultation with W3C Management and key leaders of the W3C 
> Accessibility community. The plan has been discussed and the Chairs 
> have revised it based on feedback. At this time, the Chairs propose to 
> adopt this plan by consensus.

I'd like to draw this group's attention to a specific portion of the
plan[1]:

> We also begin work on a systematic HTML5.0 Testing Plan, with the goals being:
>
>   * identifying areas that are known to be interoperable and don't
>     need further tests.
>   * identify areas that are known not to be interoperable, and to be
>     removed without the need for investing time in the creation of
>     tests.
>   * for the remaining areas:
>       * systematically determine which features we currently have
>         test cases for
>       * systematically determine which features we still need
>          test cases for

The following may also be helpful: at risk features[2] and exit criteria [3].

Is this something the Test Suite Task Force can help out with?

- Sam Ruby

[1]
http://dev.w3.org/html5/decision-policy/html5-2014-plan.html#html5.0-milestones
[2] http://www.w3.org/html/wg/wiki/index.php?title=HTML5.0AtRiskFeatures
[3]
http://dev.w3.org/html5/decision-policy/public-permissive-exit-criteria.html

Received on Tuesday, 9 October 2012 21:59:51 UTC