RE: Re[2]: CR exit criteria

I agree, and am moving forward to set up the big matrix.

If I'm questioning to clarify about the context and W3C process, that is on 
principle.  But I don't propose to let that interfere with the most 
expedient path to finish W3C processing.  (Tho' I may whinge and moan a bit.)

I agree that it's useful, and doubt that features are at risk.

-Lofton.

At 06:32 PM 7/31/2006 +0200, Dieter  Weidenbrück wrote:

>A side comment:
>
>With the introduction of the ICS pages, all vendors committed to
>do the complete testing and to keep those sheets up to date.
>Time has shown that this did not happen (including our own sheets).
>So I support the idea to have a full matrix here with clear results,
>even if we don't pass every test.
>This was the intention at the time we started with the ICS, and
>I don't think there are features at risk because of lack of support.
>
>Dieter
>
> > -----Original Message-----
> > From: public-webcgm-wg-request@w3.org
> > [mailto:public-webcgm-wg-request@w3.org] On Behalf Of Benoit Bezaire
> > Sent: Monday, July 31, 2006 4:55 PM
> > To: WebCGM WG
> > Subject: Re[2]: CR exit criteria
> >
> >
> > Monday, July 31, 2006, 10:20:28 AM, Lofton Henderson wrote:
> >
> > > See embedded...
> >
> > > At 11:55 AM 7/31/2006 +0200, Chris Lilley wrote:
> >
> > >>On Friday, July 28, 2006, 11:04:28 PM, Benoit wrote:
> > >>
> > >>BB> Hi,
> > >>
> > >>BB>   I've been thinking about the CR exit criteria
> > discussion. Here's my
> > >>BB>   opinion on it.
> > >>
> > >>BB>   First, I think Chris' request is reasonable (that is two
> > >>BB>   successful passes for each 1.0 and 2.0 tests).
> > >>
> > >>To be clear, I'm asking for testing 2.0. Some of the tests
> > to do that
> > >>are currently 1.0 tests. I assume that they would need to be
> > >>identified as WebCGM 2.0 files,though.
> >
> > > I have been thinking about this question.
> >
> > > One way to identify them as 1.0 files is to put them in a
> > place that
> > > says, "all of these 1.0 tests are valid 2.0 tests".  The
> > other way is
> > > to change the "1.0" to "2.0" in the MetDesc element in the CGM, to
> > > also make that change in the graphical text that labels and
> > appears in
> > > the legend block of the CGM, and regenerate all of the PNG files.
> >
> > > I have done something like the latter before, when adapting
> > tests from
> > > ATA to WebCGM.  It's a bit of work.
> >
> > > While the latter is more appealing from the perspective of
> > "goodness"
> > > of the test suite, and ought to be done eventually, on the
> > other hand
> > > it is somewhat "cosmetic".  I would like to think that, for now, we
> > > can live with the former for CR interoperability purposes
> > ("two pass").
> >
> > > Thoughts?
> > I'm ok with the former.
> >
> > > One more...
> >
> > >>BB>  There is however a
> > >>BB>   down side to it, and that is it could slow us down in
> > our progress
> > >>BB>   to Rec.
> > >>
> > >>BB>   Is there middle ground that can be reached? Probably.
> > >>
> > >>BB>   I think we would have to agree that no new 1.0 tests
> > can be created.
> > >>BB>   Dealing with the existing one is plenty for now. If
> > the CGM Open TC
> > >>BB>   wants to create more tests later; that's up to them,
> > but from a W3C
> > >>BB>   perspective, we are only dealing with existing 1.0 tests. Ok?
> > >>
> > >>That was my proposal, yes.
> > >>
> > >>
> > >>BB>   Also, I don't think the working group should be
> > trying to regroup
> > >>BB>   the two test suites into a single one. That would be
> > wasted cycles
> > >>BB>   (in my opinion).
> > >>BB>
> > >>BB>   Creating the matrix itself wouldn't take much time,
> > the name of each
> > >>BB>   test is available in the ICS pro-forma. This is
> > mostly copy/paste
> > >>BB>   work.
> > >>
> > >>BB>   What is time consuming is if vendors provide
> > inaccurate results;
> > >>
> > >>Right.
> > >>
> > >>BB>   this can't happen. Also each vendor would have to be
> > able to provide
> > >>BB>   beta versions of their product for someone like Chris
> > to verify the
> > >>BB>   results.
> > >>
> > >>Perhaps we could do some of that at the f2f meeting?
> >
> > > That's a thought.
> >
> > > Unfortunately, two of the implementors won't be there (unless we
> > > invited them, which from QAWG experience is apparently
> > legitimate for
> > > a "public" group; or maybe we could get a block of Zakim
> > time and join
> > > them in if there is need for discussion with them.)
> > Would they be willing to send a beta version to you and/or Chris?
> >
> > > Btw, the reason I objected earlier to a test by test,
> > vender by vender
> > > verification of all results, as opposed to spot-check or focus on
> > > reported problematic tests... Does it make a statement
> > about our view
> > > of the self-reporting by venders? Does anyone else in W3C
> > do this? Do
> > > we thereby start to establish new criteria for the two-pass
> > > convention, "independently verified interop data"? (Plus
> > ... it would
> > > dump on Chris about 1000 individual test claim
> > > verifications: 4*250. I'm sure he wouldn't mind that in his
> > spare time
> > > :-) )
> > I can only comment on what I've seen in the SVG Working Group.
> > Basically, it's not in the vendors best interest to claim a
> > PASS if the actual result is a FAIL. To have an independent
> > reviewer is useful for those tests that 'unfortunately'
> > require some interpretation.
> >
> > > -Lofton.
> >
> > >>BB>  If we get a commitment from all the vendors to provide
> > >>BB>   prompt and accurate results for each tests, it may be
> > doable in
> > >>BB>   relatively little time; if that's not the case, I'm
> > afraid we'd be
> > >>BB>   stuck in CR for a long time.
> > >>
> > >>BB>   Thoughts on this?
> > >>
> > >>--
> > >>  Chris Lilley                    mailto:chris@w3.org
> > >>  Interaction Domain Leader
> > >>  Co-Chair, W3C SVG Working Group
> > >>  W3C Graphics Activity Lead
> > >>  Co-Chair, W3C Hypertext CG
> >
> > --
> > Regards,
> >  Benoit   mailto:benoit@itedo.com
> >
> > This e-mail and any attachments are confidential and may be
> > protected by legal privilege. If you are not the intended
> > recipient, be aware that any disclosure, copying,
> > distribution or use of this e-mail or any attachment is
> > prohibited. If you have received this e-mail in error, please
> > notify us immediately by returning it to the sender and
> > delete this copy from your system. Thank you for your cooperation.
> >
> >
> >

Received on Monday, 31 July 2006 16:50:10 UTC