RE: Re[2]: CR exit criteria

A side comment:

With the introduction of the ICS pages, all vendors committed to
do the complete testing and to keep those sheets up to date.
Time has shown that this did not happen (including our own sheets).
So I support the idea to have a full matrix here with clear results,
even if we don't pass every test.
This was the intention at the time we started with the ICS, and
I don't think there are features at risk because of lack of support.

Dieter 

> -----Original Message-----
> From: public-webcgm-wg-request@w3.org 
> [mailto:public-webcgm-wg-request@w3.org] On Behalf Of Benoit Bezaire
> Sent: Monday, July 31, 2006 4:55 PM
> To: WebCGM WG
> Subject: Re[2]: CR exit criteria
> 
> 
> Monday, July 31, 2006, 10:20:28 AM, Lofton Henderson wrote:
> 
> > See embedded...
> 
> > At 11:55 AM 7/31/2006 +0200, Chris Lilley wrote:
> 
> >>On Friday, July 28, 2006, 11:04:28 PM, Benoit wrote:
> >>
> >>BB> Hi,
> >>
> >>BB>   I've been thinking about the CR exit criteria 
> discussion. Here's my
> >>BB>   opinion on it.
> >>
> >>BB>   First, I think Chris' request is reasonable (that is two
> >>BB>   successful passes for each 1.0 and 2.0 tests).
> >>
> >>To be clear, I'm asking for testing 2.0. Some of the tests 
> to do that 
> >>are currently 1.0 tests. I assume that they would need to be 
> >>identified as WebCGM 2.0 files,though.
> 
> > I have been thinking about this question.
> 
> > One way to identify them as 1.0 files is to put them in a 
> place that 
> > says, "all of these 1.0 tests are valid 2.0 tests".  The 
> other way is 
> > to change the "1.0" to "2.0" in the MetDesc element in the CGM, to 
> > also make that change in the graphical text that labels and 
> appears in 
> > the legend block of the CGM, and regenerate all of the PNG files.
> 
> > I have done something like the latter before, when adapting 
> tests from 
> > ATA to WebCGM.  It's a bit of work.
> 
> > While the latter is more appealing from the perspective of 
> "goodness" 
> > of the test suite, and ought to be done eventually, on the 
> other hand 
> > it is somewhat "cosmetic".  I would like to think that, for now, we 
> > can live with the former for CR interoperability purposes 
> ("two pass").
> 
> > Thoughts?
> I'm ok with the former.
> 
> > One more...
> 
> >>BB>  There is however a
> >>BB>   down side to it, and that is it could slow us down in 
> our progress
> >>BB>   to Rec.
> >>
> >>BB>   Is there middle ground that can be reached? Probably.
> >>
> >>BB>   I think we would have to agree that no new 1.0 tests 
> can be created.
> >>BB>   Dealing with the existing one is plenty for now. If 
> the CGM Open TC
> >>BB>   wants to create more tests later; that's up to them, 
> but from a W3C
> >>BB>   perspective, we are only dealing with existing 1.0 tests. Ok?
> >>
> >>That was my proposal, yes.
> >>
> >>
> >>BB>   Also, I don't think the working group should be 
> trying to regroup
> >>BB>   the two test suites into a single one. That would be 
> wasted cycles
> >>BB>   (in my opinion).
> >>BB>
> >>BB>   Creating the matrix itself wouldn't take much time, 
> the name of each
> >>BB>   test is available in the ICS pro-forma. This is 
> mostly copy/paste
> >>BB>   work.
> >>
> >>BB>   What is time consuming is if vendors provide 
> inaccurate results;
> >>
> >>Right.
> >>
> >>BB>   this can't happen. Also each vendor would have to be 
> able to provide
> >>BB>   beta versions of their product for someone like Chris 
> to verify the
> >>BB>   results.
> >>
> >>Perhaps we could do some of that at the f2f meeting?
> 
> > That's a thought.
> 
> > Unfortunately, two of the implementors won't be there (unless we 
> > invited them, which from QAWG experience is apparently 
> legitimate for 
> > a "public" group; or maybe we could get a block of Zakim 
> time and join 
> > them in if there is need for discussion with them.)
> Would they be willing to send a beta version to you and/or Chris?
> 
> > Btw, the reason I objected earlier to a test by test, 
> vender by vender 
> > verification of all results, as opposed to spot-check or focus on 
> > reported problematic tests... Does it make a statement 
> about our view 
> > of the self-reporting by venders? Does anyone else in W3C 
> do this? Do 
> > we thereby start to establish new criteria for the two-pass 
> > convention, "independently verified interop data"? (Plus 
> ... it would 
> > dump on Chris about 1000 individual test claim
> > verifications: 4*250. I'm sure he wouldn't mind that in his 
> spare time 
> > :-) )
> I can only comment on what I've seen in the SVG Working Group.
> Basically, it's not in the vendors best interest to claim a 
> PASS if the actual result is a FAIL. To have an independent 
> reviewer is useful for those tests that 'unfortunately' 
> require some interpretation.
> 
> > -Lofton.
> 
> >>BB>  If we get a commitment from all the vendors to provide
> >>BB>   prompt and accurate results for each tests, it may be 
> doable in
> >>BB>   relatively little time; if that's not the case, I'm 
> afraid we'd be
> >>BB>   stuck in CR for a long time.
> >>
> >>BB>   Thoughts on this?
> >>
> >>--
> >>  Chris Lilley                    mailto:chris@w3.org
> >>  Interaction Domain Leader
> >>  Co-Chair, W3C SVG Working Group
> >>  W3C Graphics Activity Lead
> >>  Co-Chair, W3C Hypertext CG
> 
> --
> Regards,
>  Benoit   mailto:benoit@itedo.com
> 
> This e-mail and any attachments are confidential and may be 
> protected by legal privilege. If you are not the intended 
> recipient, be aware that any disclosure, copying, 
> distribution or use of this e-mail or any attachment is 
> prohibited. If you have received this e-mail in error, please 
> notify us immediately by returning it to the sender and 
> delete this copy from your system. Thank you for your cooperation. 
> 
> 
>

Received on Monday, 31 July 2006 16:32:15 UTC