Re: test suite distinctions [was: Re: Feedback on "The Matrix"]

It is the plan of the QAWG to produce several guidelines/checkpoints 
documents in the Framework document family.  These will cover the areas of:

** QA process and operational setup (first public WD, 1-feb)
** Specifications (recommendations)
** Technical Materials

Associated WAI-like checklists will allow scoring of processes, specs, 
materials according to checklists, and rating of the target with WAI-like 
conformance levels (A, AA, AAA).  I'm not sure that I'm understanding the 
suggestions in this thread.  Is it suggested that W3C should:

1.) not produce such goodness-rating specs/tools?
2.) produce them but don't, ourselves (W3C), apply them and publish results?
3.) something else?

See also embedded questions below...

At 01:44 PM 2/27/2002 -0500, Al Gilman wrote:
>At 10:13 AM 2002-02-27 , Alex Rousskov wrote:
> >Overall, the current solution may be sufficient. It is definitely the
> >simplest and least controversial one.
> >
>
>Two pieces of evidence in support of Alex's approach:
>
>1.  The analogy with UDDI+WSDL.
>
>The directory really just tells you that a service exists; everything else 
>is addressed in the service prospectus in a rich language.
>
>2.  An analogy with accessibility enforcement.
>
>The coarsely-quantized rating system of three conformance plateaux for web 
>accessibility as promulgated in WCAG 1.0 has very little "consensus 
>stability margin" behind it.

I don't follow the point here.  Could you please elaborate?

>The topic of conformance representations engenders a lot of ongoing 
>controversy in the accessibility domain.

"Representations" means "claims"?

Regards,
-Lofton.


>So best not to employ distinguished icons that can be interpreted as 
>connoting degrees of authority without prior careful review of how 
>different people will interpret, apply, and populate them.
>
>The distinctions suggested fall in "potentially invidious" territory, as I 
>see it.
>
>Al
>
> >On Wed, 27 Feb 2002, Tantek Celik wrote:
> >
> >> The "Test Suites" column is currently just a boolean
> >> (hyperlinked!)  indicator of whether or not there is anything even
> >> remotely resembling a test suite available for a particular
> >> technology.
> >>
> >> While this is useful, it would help significantly if the test
> >> suites which were actually hosted at w3.org used a "W3C" icon
> >> instead of the "hammer and wrench" icon.
> >
> >IMHO, "being hosted at w3.org" adds little information about the
> >quality or even availability of the test suite. Reflecting the state
> >of the suite (under construction, available, production quality, with
> >public results database, etc.) may be a good idea. In some cases,
> >however, assigning a state may be a controversial action. Rating the
> >quality of a suite would be even more controversial, of course.
> >
> >> This will help quickly call out at a glance which specs actually
> >> have official W3C test suites, vs. which have some sort of test
> >> suite or plan for a test suite, and which have no form of test
> >> suite at all.
> >
> >"Official" has little utility in this context, IMO. Whether I can use
> >the test suite now is far more important (to me, anyway).
> >
> >If W3C branding is important, perhaps there should be two columns:
> >"W3C endorsement" and "state/availability".
> >
> >The situation become even more complex when several test suites are
> >available and described on a separate page. In that case, one could
> >use the state of the "best" (e.g., already available) suite to assign
> >an icon since people are more likely to use the best tool if given a
> >choice.
> >
> >Overall, the current solution may be sufficient. It is definitely the
> >simplest and least controversial one.
> >
> >Alex.
> >

Received on Thursday, 28 February 2002 11:30:04 UTC