Re: Ringmark, Core Mobile, and the Financial Times

On 6/11/12 4:03 PM, "Glenn Adams" <glenn@skynav.com> wrote:

>On Mon, Jun 11, 2012 at 4:38 AM, Robin Berjon <robin@berjon.com> wrote:
>
>On Jun 11, 2012, at 10:39 , James Graham wrote:
>> The best solution for bugginess is to have high quality public
>>testsuites for various features. The various W3C working groups already
>>curate tests for their various specs and have guidelines / frameworks
>>for writing tests. The main problem is sourcing the tests themselves; so
>>far the dominant contributors by far are browser vendors, and even they
>>do not yet contribute as much as you might hope due to a history of
>>writing tests in ways that that are specific to their particular browser
>>or testing setup. This has been improving recently, however.
>
>
>That's certainly part of the problem, but not the whole story either. For
>most of the issues that Robert brings up the features are actually
>supported and would be reported as such in traditional conformance test
>suites. The fact that they're there but simply unusable for developers
>would not be noted in the implementation report.
>
> 
>I brought this up at the recent WebApps F2F, namely, that the traditional
>purpose for test suites in the W3C has been to demonstrate
>implementability of a spec, i.e., as a tool to help persuade the director
>that a spec can move from CR to PR. Other purposes for testing, e.g.,
>interoperability and quality, have not been an explicit goal in the W3C
>test process, at least not the way it has been practiced in general.
>Certainly, some groups and some specs have had test suites created with
>this in mind, but the W3C process does not (presently) encode this as a
>goal in test creation or use.
>
>In addition, because the W3C does not perform any type of compliance
>testing (and it is unlikely to do so any time soon if ever), these other
>goals are typically given low priority or dropped entirely.
>
>I would suggest that we need to find ways to encode the goals of
>interoperability and quality (of the technical products, not the process
>itself) in the W3C Process in a manner that translates to specific
>actions in the way W3C test suites are developed and used.

Ties back another thing I brought up on the WebApps mailing list recently:
documented use cases and requirements. Quality of implementation tests
should assert the implementations fulfill the use cases a spec was
designed to meet. Effectively, this would close the circle and seed a more
iterative kind of development: use case -> requirements -> spec ->
implementation -> confirm implementation is spec compliant -> confirm
implementation meets use cases and requirements -> gather unmet or new use
cases. Repeat.

--tobie

Received on Tuesday, 12 June 2012 09:33:53 UTC