Re: The Evaluation Techniques Strike Back

>Wendy, today there are WCAG 1.0 techniques by mark-up; Core, HTML, CSS,
>SVG?  -  are there going to be in WCAG 2.0 testable *by mark-up format*
>that ATAG can reference?

Yes, see below. That's what I was pointing to in my [2] reference.

>  If true, then ATAG evaluation techniques would be
>"something like AERT but with different tool types" that reference the WCAG
>2.0 testable techniques for *mark-up formats* such as HTML, SVG, etc. Today
>AUWG will have to reference WCAG 1.0 techniques, even though they may not
>be as testable as the WCAG 2.0 ones will be.

I imagine ATAG will reference both levels (success criteria and 
technology-specific "tests" or whatever they end up being called).  It's 
hard to say what it will look like since it doesn't all exist yet.

One issue to keep in mind is that the WCAG 2.0 Techniques documents will 
not likely be normative (i.e. they will continue to be published as Notes, 
like they are now).  I doubt that has an effect on ATAG Techniques (since 
they are also informative), but just FYI.

> > The Techniques documents will have more testable criteria that look more
> > like the AERT. [2]
> >
> > Therefore, if WCAG provides what should be produced, ATAG should provide
> > how to help the author get it there....not a new idea....just restating
>it
> > FYI.  This follows along what we discussed last June in Amsterdam [3].
>
>great.
>
>Is there any discussion of a mock web site or set of files that has correct
>and incorrect mark-up so that both the "testable techniques" of WCAG 2.0
>and ATAG evaluation techniques could reference them?

Yes.  Matt May began work on a mock news site and Web AIM has a mock 
distance education site. If you  know of other mock sites that we can refer 
to, please let me know (seems there was at least one more that I am 
forgetting right now...).

We also plan to make all of the example code in the techniques documents 
"live" so that people can play with it.  Also, the W3C has a new Quality 
Assurance activity that has been helping with test suites for specs as they 
go through TR.  Therefore, I expect WCAG 2.0 to have a test suite.  ATRC 
has several test files which we can use as a basis for this (plus the live 
code from the WCAG Techniques docs, plus examples from the 
curriculum)...and lots of people have test files. It would be great to 
collect them.  It would be great to have AUWG's help in collecting them!

--wendy

>Jan Richards <jan.richards@utoronto.ca>@w3.org on 05/29/2002 09:48:32 AM
>
>Sent by:    w3c-wai-au-request@w3.org
>
>
>To:    "w3c-wai-au@w3.org" <w3c-wai-au@w3.org>
>cc:
>Subject:    Re: The Evaluation Techniques Strike Back
>
>
>
>
>Wendy A Chisholm wrote:
> >
> > I don't have any answers, but glad you are asking these questions.
>Another
> > thing to take into consideration is the work on WCAG 2.0. [1]  WCAG 2.0
>has
> > "success criteria" for each checkpoint - these are supposed to be
>testable.
>
>At this point the evaluation techniques are for ATAG 1.0, which
>references only WCAG 1.0. Once WCAG 2.0 is released, a new version of
>ATAG can be published that refers to the new version of WCAG. Only then
>we will be able to update the ATAG implementation techniques and ATAG
>evaluation techniques documents to take advantage of all the advances in
>WCAG 2.0, including the "success criteria".
>
> > The Techniques documents will have more testable criteria that look more
> > like the AERT. [2]
> >
> > Therefore, if WCAG provides what should be produced, ATAG should provide
> > how to help the author get it there....not a new idea....just restating
>it
> > FYI.  This follows along what we discussed last June in Amsterdam [3].
>
>Absolutely.
>
>Cheers,
>Jan
>
>
> >
> > At 05:20 PM 5/28/02, Jan Richards wrote:
> > >Hi all,
> > >
> > >It seems that the ATAG evaluation techniques are always on the agenda,
> > >but for some reason, we never quite get to them. As we put together an
> > >agenda for the Austria F2F, perhaps we should return to the subject and
> > >survey the numerous outstanding issues (which I will be placing in an
> > >issues page - linked from a new Evaluation Techniques sub-section on the
> > >AU homepage).
> > >
> > >As I see it, we need to come to a consensus on the following:
> > >
> > >1. Do we want the evaluation techniques to be a step-by-step procedure
> > >for people who are not familiar with ATAG? (i.e. "Open the file supplied
> > >and then perform X. If you see Y happen then the tool passes, if Z then
> > >it fails.")  Or will the evaluation techniques be intended to support
> > >evaluations by people familiar with ATAG (i.e. "Here are some things to
> > >keep in mind when assessing X in the tool")? - Either way, how can we
> > >avoid specifying things at the level of markup (which is best left to
> > >WCAG's whenever possible)? In other words, will we have to have a
> > >different set of tests for HTML, SVG?
> > >
> > >2. How will we take into account all the different kinds of tools? Will
> > >we break the evaluation techniques into groups by ATAG checkpoint? Will
> > >we end up with something like the AERT but with different tool types
> > >rather than different markup languages.?
> > >
> > >3. What will be the relationship be between the evaluation techniques
> > >and the implementation techniques? If we include implementation
> > >specifics in the tests (i.e. "To assess whether highlighting has been
> > >used in the dialog check whether any options are highlighted by ordering
> > >or color.") how will we avoid this being seen as limiting the creative
> > >flexibility of developers and becoming the de facto prescriptive
> > >requirements?
> > >
> > >4. How can we support evaluation of checkpoints dealing with accessible
> > >output (for WCAG P1, P2 and P3) when checking tools are not up to the
> > >task yet? How much testing of output is sufficient?
> > >
> > >5. How can we support checking of the accessibility interface
> > >checkpoints in guideline 7? Will we provide pointers to platform
> > >specific standards, "rules of thumb" for checking interfaces, etc.
> > >
> > >6. How should we use the QA work
> > >(http://www.w3.org/TR/2002/WD-qaframe-spec-20020515/)
> > >
> > >
> > >Answers? More questions? Comments?
> > >
> > >--
> > >Cheers,
> > >Jan
> > >

-- 
wendy a chisholm
world wide web consortium
web accessibility initiative
seattle, wa usa
/--

Received on Monday, 3 June 2002 13:50:28 UTC