RE: Versioning and HTML -- CR exit criteria

I agree with Ashok.  If the specification is written with explicit MUST's (or mandatory rules) in it and the test cases are tied to these statements (or rules) then the testing and CR experience can be directly related back to the specification.

/paulc

Paul Cotton, Microsoft Canada
17 Eleanor Drive, Ottawa, Ontario K2E 6A3
Tel: (425) 705-9596 Fax: (425) 936-7329


-----Original Message-----
From: www-tag-request@w3.org [mailto:www-tag-request@w3.org] On Behalf Of ashok malhotra
Sent: Tuesday, April 28, 2009 9:45 AM
To: Larry Masinter
Cc: www-tag@w3.org WG
Subject: Re: Versioning and HTML -- CR exit criteria

 > Test cases, even if results are reported honestly, only verify 
implementation of the test cases and not of the specification.

This is tautologically correct but a bit strong.  A set of well written 
test cases *can* test compliance with the spec.

All the best, Ashok


Larry Masinter wrote:
> This is a little off-topic from "versioning and HTML" except
> for the assertion that once HTML exits CR, no incompatible
> changes will ever be necessary.
>
>
> IF there are two implementations that are actually built
> from reading the spec itself, and the implementations interoperate, then
> you have some confidence that the spec isn't incomprehensible and that
> it is actually possible to build SOMETHING interoperable based on it.
>
> The process assumes that the assertions that the implementations in 
> fact match the specification are made in good faith. Unfortunately,
> this isn't always the case. 
>
> Many specifications unfortunately are completely incomprehensible,
> and the CR exit criteria doesn't explicitly require that the 
> implementations weren't built using inside knowledge and the
> spec written after the fact.
>
> Even if the implementations are written based on the specification
> rather than the other way around, there is no process for verifying
> that they match. Test cases, even if results are reported honestly,
> only verify implementation of the test cases and not of the
> specification.
>
> Having only two implementations is hardly a guarantee of the
> utility of the specification for wide applicability.  Surely
> only two implementations aren't a guarantee that the considerations
> of the wide variety of devices, operating systems, usability
> concerns, international contexts, networking situations have
> really been considered, even for the simplest of specifications.
>
> As noted earlier, even if there are many implementations, all
> built based on the specifications, over time requirements change,
> and changing requirements might require incompatible changes.
>
> It is never possible to " ensure that problems with defining 
> behavior incorrectly for the long term are all caught."
>
> Larry
>

Received on Tuesday, 28 April 2009 13:55:38 UTC