RE: Comments on WCAG-EM

Hello Raph,

Many thanks for the feedback that you have provided on WCAG-EM. And thank you for the compliment. 
In the coming period we will discuss all comments in the Evaluation Taskforce. The outcome of the discussions will be visible in a Disposition of Comments where we will address the comments and propose solutions supported by a rationale. Thanks for the possibility to contact you in case we have additional questions. We plan to start working on the comments in our next Telco. 

We also plan to do a test-run of WCAG-EM soon. For this we ask to evaluate a website provided by Eval Taskforce using WCAG-EM. Would you be interested to participate in such a test-run and do a full evaluation of a website using WCAG-EM? In that case, please contact Shadi or me.

Kindest regards,

Eric

________________________________________
Van: Rooij, R.P.L.A. de (Raph) - Logius [raph.de.rooij@logius.nl]
Verzonden: vrijdag 28 februari 2014 15:22
Aan: 'public-wcag-em-comments@w3.org'
CC: Velleman, Eric; 'Will Coffie'; Rooij, R.P.L.A. de (Raph) - Logius
Onderwerp: Comments on WCAG-EM

Dear WCAG-EM working group,

Thank you for making the draft of the Website Accessibility Conformance Evaluation Methodology available and your invitation to review the document.
Having worked on evaluation methodologies in the past, I must say that it most certainly is an impressive piece of work!

My remarks are enclosed in the e-mail; see below. Headings and text strings are used as references to the text in the original document. Each remark is numbered.

If there are questions, or a need to clarify things; please don't hesitate to contact me.


With kind regards,

Raph de Rooij
product manager open standards
........................................................
Logius
Ministry of the Interior and Kingdom Relations
Wilhelmina van Pruisenweg 52 | 2595 AN | The Hague | NL
P.O. Box 96810 | 2509 JE | Den Haag | The Hague | NL
........................................................
M +31-6-21160652
raph.de.rooij@logius.nl
www.logius.nl/english | www.webrichtlijnen.nl (in Dutch)
........................................................
Digital government service



======================================================

REVIEW

Prepared by:
Raph de Rooij, Logius, the Netherlands, raph.de.rooij@logius.nl, +31-621160652

Reference:
http://www.w3.org/TR/2014/WD-WCAG-EM-20140130/

======================================================

[H]Introduction[/H]


[01] String:
This methodology, describes the steps

[01] Remark:
The comma has no function and should be removed


[02] String:
The methodology can be used in conjunction with techniques or meeting WCAG 2.0 success criteria, such as the Techniques for WCAG 2.0 documented by W3C/WAI, but does not require this or any other specific set of techniques.

[02] Remark:
I agree with this sentence, but it has led to a bit of discussion here about the value and the status of the techniques for WCAG 2.0, which in practice is a very useful resource.
Such discussion can easily be avoided by slightly reformulation the sentence.
Proposal: "It is recommendable to use the methodology in conjunction with techniques for meeting WCAG 2.0 success criteria, such as the Techniques for WCAG 2.0 documented by W3C/WAI. However, use of this or any other specific set of techniques is not mandatory."


[H]Purposes for this Methodology[/H]


[03] String:
for example before releasing, acquiring, or redesigning the website.

[03] Remark:
Should be: "for example before releasing or acquiring, or after redesigning the website."
Rationale: why would you want to evaluate the accessibility of a website prior to redesigning it?  Unless resources are unlimited, it makes more sense to do it afterwards.


[04] String:
This methodology is designed for anyone who wants to follow a common procedure for evaluating the conformance of websites to WCAG 2.0. This includes:

[04] Remark:
Suggestion for one more group that this methodology is designed for:
"Compliance and quality assurance managers who are responsible for meeting legal requirements and assurance aspects of information systems."


[H]Terms and Definitions[/H}


[05] String:
(not apliccable)

[05] Remark:
Term to be added: Audit.
See also remark [14].


[06] String:
content authors, designers, programmers, quality assurance testers, and project managers

[06] Remark:
An important type of developer is not explicitly mentioned here. This developer glues together the work on the web interface that other actors are responsible for.
The front-end developer plays a key role when it comes to web accessibility. It now is enclosed in the word programmers, which doesn't do it justice.
Suggestion: replace "[...] programmers [..]" by "[...] front-end developers, back-end programmers [...]".


[07] String:
Website Using Responsive Design

[07] Remark:
This type of website doesn't belong here: all other types of websites described here can use Responsive Design.
Suggestion: add a *note* on Responsive Design here, not a new type.


[H]Particular Evaluation Contexts[/H]


[08] String:
Particular Evaluation Contexts

[08] Remark:
Suggestion to add one more evaluation context that is related to Evaluating During Development: "Evaluating During Authoring"

[1] This context is linked to ATAG 2.0 (draft) principle B.3: Authors are supported in improving the accessibility of existing content (ref: http://www.w3.org/TR/ATAG20/#principle_b3 )

[2] http://www.w3.org/TR/2013/WD-IMPLEMENTING-ATAG20-20131107/#sc_b315 describes how this particular context can contribute to the evaluation process and be integrated in the methodology.


[H]Step 1.b: Define the Conformance Target[/H]


[09] String:
Part of initiating the evaluation process is to define the target WCAG 2.0 conformance level ("A", "AA", or "AAA") to evaluate for

[09] Remark:
In my opinion, a (reference to a) note should be added here that requiring Level AAA conformance as a general policy for an entire site is not recommended.
(Reference: http://www.w3.org/TR/WCAG20/#conformance-reqs, 1. Conformance Level)


[H]Step 1.d: Define Evaluation Methods to be Used (Optional)[/H]


[10] String:
However, it is good practice to specify the initial evaluation methods to be used during the evaluation

[10] Remark:
I assume that the evaluator is required to adequately document the methods used, in cases where they are not yet documented elsewhere (for example: in the Techniques for WCAG 2.0).
Is this assumption true?
If yes, where is it described?


[H]Step 3: Select a Representative Sample[/H]


[11] String:
The actual size of sample web pages and web page states needed to evaluate a website depends on many factors including:

[11] Remark:
"Availability of evaluation findings and test results" is missing here.
If results of accessibility testing are available, it may very well be useful to assess whether such results can be reliably (re-)used in the evaluation. The usefulness depends on factors such as timeliness, completeness and correctness of the information provided, the format in which the results are available (example: EARL), the testing method used, persons that carried out the tests, etc.


[12] String:
Required level of confidence

[12] Remark:
No (reference to) information is available on this subject. Levels of confidence are not defined. Clarification is needed.


[H}Step 3.b: Include Other Relevant Web Pages[/H]


[13] String:
other web pages and web page states that are relevant to people with disabilities

[13] Remark:
It may be helpful here to briefly explain and/or give examples what is meant with "relevant to people with disabilities".


[H]Step 4: Audit the Selected Sample[/H]


[14] String:
audits (detailed evaluation)

[14] Remark:
Is 'detailed evaluation' the definition of the term 'audit'?
If this is the case, the term should be added to the section Terms and Definitions. A common definition of the term is available at http://www.projectauditors.com/Dictionary2/1.8/index.php/term/,62555c9cae53606f68555ca75a.xhtml : "A planned and documented activity performed by qualified personnel to determine by investigation, examination, or evaluation of objective evidence, the adequacy and compliance with established procedures, or applicable documents, and the effectiveness of implementation."


[H]Step 4.a: Check for Each Success Criterion[/H]


[15] String:
evaluators typically use additional methods to evaluate conformance to WCAG 2.0

[15] Remark:
See remark [10]


[H]Step 5: Record the Evaluation Findings[/H]


[16] String:
to ensure reliable outcomes

[16] Remark:
Suggestion: to ensure reliable and verifiable outcomes
For insight in reliability of outcomes, verifiability is a prerequisite.


[H]Step 5.c: Provide an Evaluation Statement (Optional)[/H]


[17] String:
Reason for not conforming to WCAG 2.0

[17] Remark:
In my opinion, this should be: Reason for not fully conforming to WCAG 2.0


________________________________

Dit bericht kan informatie bevatten die niet voor u is bestemd. Indien u niet de geadresseerde bent of dit bericht abusievelijk aan u is toegezonden, wordt u verzocht dat aan de afzender te melden en het bericht te verwijderen. De Staat aanvaardt geen aansprakelijkheid voor schade, van welke aard ook, die verband houdt met risico's verbonden aan het elektronisch verzenden van berichten.
This message may contain information that is not intended for you. If you are not the addressee or if this message was sent to you by mistake, you are requested to inform the sender and delete the message. The State accepts no liability for damage of any kind resulting from the risks inherent in the electronic transmission of messages. .

Received on Friday, 28 February 2014 14:47:02 UTC