Re[2]: CR exit criteria

Hi Lofton,

Friday, July 28, 2006, 6:44:46 PM, Lofton Henderson wrote:

> At 05:04 PM 7/28/2006 -0400, Benoit Bezaire wrote:

>>Hi,
>>
>>   I've been thinking about the CR exit criteria discussion. Here's my
>>   opinion on it.
>>
>>   First, I think Chris' request is reasonable (that is two
>>   successful passes for each 1.0 and 2.0 tests).

> I won't dispute "reasonable". But as I expressed in telecon, there
> are equally reasonable alternative views of it. Effectively, the
> functionality of the 1.0 subset of 2.0 is being subject to higher
> quality criteria to be reaffirmed in 2006, than were applied to
> accept it as Recommendation in 1999 and 2001.
Right, over the years the W3C has raised the quality criteria. You can
see the difference between SVG 1.0, 1.1 and soon of 1.2. It was my
understanding that all new specifications are subject to these
stricter rules.

>  The appropriateness of that can be debated (W3C Process is silent
> about such specifics). 

>>There is however a
>>   down side to it, and that is it could slow us down in our progress
>>   to Rec.

> And it is this potential that worries me. If the suggested quality
> criteria for the 1.0 subset had occurred to us earlier, we could
> have been working on this for the last year or so, in parallel with
> developing the new 2.0 tests. (In which case it wouldn't be an
> issue.)
As I said, I think the main thing that will slow us down is get some
of the vendors to provide results.

>>   Is there middle ground that can be reached? Probably.
>>
>>   I think we would have to agree that no new 1.0 tests can be created.
>>   Dealing with the existing one is plenty for now. If the CGM Open TC
>>   wants to create more tests later; that's up to them, but from a W3C
>>   perspective, we are only dealing with existing 1.0 tests. Ok?
>>
>>   Also, I don't think the working group should be trying to regroup
>>   the two test suites into a single one. That would be wasted cycles
>>   (in my opinion).

> I don't understand this comment, "regroup the two suites into a single
> one".  Explain?
Just ignore it please. It a bundling thing.

>>   Creating the matrix itself wouldn't take much time, the name of each
>>   test is available in the ICS pro-forma. This is mostly copy/paste
>>   work.

> Trivial.  (Actually, I can generate the *empty* new matrix easily from a
> list of test names, which I have.)

> However, once we go from 40 tests to 250+ tests, it is questionable whether
> manual construction and maintenance of the new matrix (editing an HTML
> table) is practical or reliable.  Some automation is likely 
> necessary.  E.g., vendors submit their results in a rigorous format -- even
> a careful XHTML table would suffice -- that can be merged automatically by
> XSLT.

> I can write such XSLT, but haven't done so yet because ... too lazy, and
> too hard to get people to adopt rigor in submitting results, so that 
> hand-editing 40 row HTML table has been easier, especially with few and
> infrequent changes.
I propose we first get the results, and deal with maintenance later.

>>   What is time consuming is if vendors provide inaccurate results;
>>   this can't happen. Also each vendor would have to be able to provide
>>   beta versions of their product for someone like Chris to verify the
>>   results.

> I object to that suggestion, unless you mean "spot check". I don't
> have a problem with spot-check. But we should NOT require a full
> verification of results, if that's what you mean. That is
> inappropriate (IMO), and without precedent (AFAIK). (Not to mention
> more work and delay.)
I'm not suggesting to make it a requirement. Chris can comment about
W3C policies here. I expect Itedo, as a courtesy, to provide a beta
version to W3C staff for the CR phase.

-- 
Regards,
 Benoit   mailto:benoit@itedo.com

This e-mail and any attachments are confidential and may be protected
by legal privilege. If you are not the intended recipient, be aware
that any disclosure, copying, distribution or use of this e-mail or
any attachment is prohibited. If you have received this e-mail in
error, please notify us immediately by returning it to the sender and
delete this copy from your system. Thank you for your cooperation. 


>>If we get a commitment from all the vendors to provide
>>   prompt and accurate results for each tests, it may be doable in
>>   relatively little time; if that's not the case, I'm afraid we'd be
>>   stuck in CR for a long time.
>>
>>   Thoughts on this?

> No strong opinion. I'm slightly pessimistic about anything involving
> 5 vendors/implementations and 250+ (1.0) tests being done quickly.

> For now, I'm preceding with the "lumpy" composite table, i.e., the
> one at the granularity of the ICS. I'll have something soon. (And
> possibly the issue will become moot, except for the labor to do the
> new 1.0-subset table.)

> -Lofton.

Received on Monday, 31 July 2006 14:37:27 UTC