W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > February 2012

Evaluation scheme with three options - proposal

From: Kerstin Probiesch <k.probiesch@googlemail.com>
Date: Sun, 19 Feb 2012 10:36:50 +0100
To: "'EVAL TF'" <public-wai-evaltf@w3.org>
Message-ID: <4f40c287.54310e0a.14f5.ffffe511@mx.google.com>
Hi all,

in our last teleconference we discussed a evaluation scheme with three
options based upon 100% Conformance. I appreciate these proposals and see
them as chance to integrate or point to the three documents of WCAG2:
Guidelines and SCs, Understanding and How to meet.

One proposal for handling the documents in an evaluation scheme, based upon
the normative guidelines and SCs as core:

=====
Option 1: WCAG 2.0 – Core Test ("light version" or whatever the wording
later will be)

# Guideline X (Heading)

## Checkpoint: SC XX (Subheading)

Result: pass/fail

Character: global/regional (or another wording) - – if regional: a list of
pages where the problem exists

## Checkpoint: SC XX (Subheading)

Result: pass/fail

Character: global/regional (or another wording) - – if regional: a list of
pages where the problem exists

(...)

=====

Use cases for Option1:
 
- experienced developers and clients who know WCAG2 and need just the
results, 
- comparative evaluations (20 hotel websites, city websites…)
- or for example just with the SCs of level a and a smaller scope as
pre-test to decide together with the client what the best next steps might
be (evaluation, consulting, probably workshops for editors)

=====

Option 2: WCAG 2.0 – Core incl. understanding (name?)

# Guideline X (Heading)

## Checkpoint: SC XX (Subheading)

Result: pass/fail

Character: global/regional (or another wording) – if regional: a list of
pages where the problem exists

Problem (Subheading): Description of existing problems and barriers for
users (here know how out of the understanding document could be part of the
description). 

## Checkpoint: SC XX (Subheading)

Result: pass/fail

Character: global/regional (or another wording) – if regional: a list of
pages where the problem exists

Problem (Subheading): Description of existing problems and barriers for
users (here know how out of the understanding document could be part of the
description).

(...)

======

Use cases: 

- comparative evaluations (depending on the specific time and costs)

- if a client just want descriptions

- regular tests like "evaluation of the week"

=====

Option 3: WCAG 2.0 – Core, understanding, how to meet (name?)

# Guideline X (Heading)

## Checkpoint: SC XX (Subheading)

Result: pass/fail

Character: global/regional (or another wording) – if regional: a list of
pages where the problem exists

Problem (Subheading): description/explanation of existing problems and
barriers for users (here know how out of the Understanding Document could be
part of the description).

Action (Subheading): Description of techniques for meeting the SC (could be
techniques which are already in the techniques document or new techniques
which are not in the document, but with which the SC can be met). Here even
usability aspects can play a role, like: you can do a, b, c or d – I/we
propose/recommend c.

## Checkpoint: SC XX (Subheading)

Result: pass/fail

Character: global/regional (or another wording) – if regional: a list of
pages where the problem exists

Problem (Subheading): description/explanation of existing problems and
barriers for users (here know how out of the Understanding Document could be
part of the description).

Action (Subheading): Description of techniques for meeting the SC (could be
techniques which are already in the techniques document or new techniques
which are not in the document, but with which the SC can be met). Here even
usability aspects can play a role, like: you can do a, b, c or d – I/we
propose/recommend c.

(...)

======

Use cases:

- test incl. consulting

- for clients who are not very familiar with accessibility and WCAG2

============

For a seal/badge or any formal confirmation Option 1 is the minimum.

A report might also / should? also have intro parts like:

- Short description of the Option 1, 2 or 3

- Something like a disclaimer ("results might not be complete, therefore it
is important to go through the page, view all similar elements and solve the
corresponding problems)

- Glossary (for specific terms we used in our methodology -like
regional/global – if we decide to use them)

- Documentation of the used OS, Browsers and Versions, probably used
assistive technologies incl. versions

- Tested Conformance Level (A, AA, AA)

- Results

- Summary, probably written as an overall impression - we discussed in this
list the 'motivation factor'. I think the aim of an evaluation is not to
motivate. Nevertheless, writing a nice overall impression in a report, may
have this function. Ok, except when there is nothing nice to say.

This scheme could probably also be used for processes, pdf, flash and so on
and I think it would be flexible enough (time, costs, ...) and in the same
time valid against the Conformance Requirements, because the core
(evaluation itself) is the same in every option.

Important, as I see it, is that the evaluator has the three different
aspects in mind and in the report, which I believe shouldn't be mixed:
evaluation (Core, testing SCs), explanation (description of the
problem/violation, understanding) and consulting (how to meet, usability,..)


The evaluator could document the "progress toward meeting success criteria
from all levels beyond the achieved level of conformance": If for example
the evaluation is for Level A with Option 3 the SCs of AA could also be
checked (pass/fail) without any further description or with further
description, depending on the contract.

Advantage: every evaluator or testing organization uses the methodology and
a standardized 'template' for the core and the evaluation itself. The
descriptions of existing barriers (explanatory part/understanding in Option
2 and 3) and the consulting part (How to meet, in Option 3) would be the
specific added value for the clients/the evaluator/the testing organization.


Thoughts?

Best

--Kerstin


-------------------------------------
Kerstin Probiesch - Freie Beraterin
Barrierefreiheit, Social Media, Webkompetenz
Kantstraße 10/19 | 35039 Marburg
Tel.: 06421 167002
E-Mail: mail@barrierefreie-informationskultur.de
Web: http://www.barrierefreie-informationskultur.de

XING: http://www.xing.com/profile/Kerstin_Probiesch
Twitter: http://twitter.com/kprobiesch
------------------------------------
Received on Sunday, 19 February 2012 09:36:36 GMT

This archive was generated by hypermail 2.3.1 : Friday, 8 March 2013 15:52:13 GMT