[wbs] response to 'Approval for draft publication of WCAG-EM'

The following answers have been successfully submitted to 'Approval for
draft publication of WCAG-EM' (public) for Kerstin Probiesch.


---------------------------------
Abstract
----



 * ( ) accept this section as draft
 * (x) accept this section as draft with the following suggestions
 * ( ) I do not accept this section as draft
 * ( ) I abstain (not vote)
 Priority: strong

I think the term "optional" could be misleading and there is no definition
of this in the "Terms and Definitions" section. 

Suggestion: change "optional" to "not required" or "not mandatory" or "not
facultativ" or add a definition of "optional" in the "Terms and
Definitions" section, which clearly says that all sections which are marked
as optional are not facultativ. 




---------------------------------
Introduction
----



 * ( ) accept this section as draft
 * ( ) accept this section as draft with the following suggestions
 * (x) I do not accept this section as draft
 * ( ) I abstain (not vote)
 # Priority: strong

Location: "The methodology relies on WCAG 2.0 techniques such as the
Techniques for WCAG 2.0 documented by W3C/WAI, but is not limited to this
set of techniques."

Suggestion: Delete this sentence.

This is in contradiction with the idea and the concept of the Techniques
Document. Not if a technique is used is important but wether an SC is met
or not. Therefore a methodology for evaluating can't rely on techniques.
What the WG has published here:
http://www.w3.org/WAI/WCAG20/wcag2faq.html#techsnot is also relevant for
evaluators. Even with "but is not limited to this set of techniques." the
paragraph places too much emphasis on the techniques.


---------------------------------
Using This Methodology
----



 * ( ) accept this section as draft
 * ( ) accept this section as draft with the following suggestions
 * (x) I do not accept this section as draft
 * ( ) I abstain (not vote)
 # Priority: strong suggestion

Location: Required expertise

Current wording: "Users of this methodology are assumed to be knowledgeable
of WCAG 2.0"

Suggested revision: "Users of this methodology are assumed to be
knowledgeable of WCAG 2.0 which includes the Success Criteria, the  role of
techniques http://www.w3.org/TR/WCAG20-TECHS/intro.html) in general and the
overall concept of WCAG 2.0".

# Priority: strong

Location: Review Teams (Optional)

Current Wording: Whole section

Suggested revision: Delete the section "Review Teams"

Rationale: 

That Review Teams are helping to identify barriers "more effectively" is at
a first glance a logical statement, but because there are no systematic
data which provide evidence it is just a thesis or an opinion. Two
examples: a. is the evaluation of a Review Team with some months experience
"more effectively" than the evaluation carried out by an individual
evaluator with 5, 6, 7 years or more experience? b. what if a Review Team
are testing their own interpretation: are the results of this Review Team
better than the results of an individual evaluator who follows for examples
decisions of the WCAG Working Group? As this is an opinion which is not
substantiated by systematical data it can't be "highly recommended".

When we speak about evaluation we of course speak about accessibility, but
we also speak about business models and about a lot of money. Evaluators
are in competition with one another Individual Evalators are in competition
with Testing Organizations. WCAG-EM is for example explicitly mentioned in
the "European Accessibility Requirements for Public Procurement of Products
and Services in the ICT Domain" especially in this document:
http://www.etsi.org/deliver/etsi_en/301500_301599/301549/01.00.00_20/en_301549v010000c.pdf
(page 90, note 2). It is likely that the preference of Review Teams over
Individual Evaluators will have a negative impact on the free competition.

We should give guidance on how to evaluate web pages and should obstain
from all (as long as they are not facts) what is likely to prefer one group
of evaluators - even in a "mild" form.


---------------------------------
Scope of Applicability
----



 * ( ) accept this section as draft
 * (x) accept this section as draft with the following suggestions
 * ( ) I do not accept this section as draft
 * ( ) I abstain (not vote)
 # Medium

Location: Principle of Website Enclosure.

Current Wording: "When a target website is defined for evaluation, it is
essential that all web pages, web page states, and functionality within the
scope of this definition are considered for evaluation. Excluding such
aspects of a website from the scope of evaluation would likely conflict
with the WCAG 2.0 conformance requirements".

Suggestion and Rationale: For clarification it should be made clear that
also PDF and other formats have to be considered for evaluation and that
not only HTML is meant.


---------------------------------
Step 1: Define the Evaluation Scope
----



 * ( ) accept this section as draft
 * (x) accept this section as draft with the following suggestions
 * ( ) I do not accept this section as draft
 * ( ) I abstain (not vote)
 Priority: strong

Step 1d. I strongly support Gregg's suggestion for changing this step from

"W3C/WAI provides a set of publicly documented (non-normative) Techniques
for WCAG 2.0 that help evaluate conformance to WCAG 2.0 Success Criteria.
However, it is not necessary to use these particular techniques (see
Understanding Techniques for WCAG Success Criteria). Some evaluators might
use other methods (inline with the requirements for custom techniques) to
evaluate conformance to WCAG 2.0.W3C/WAI provides a set of publicly
documented (non-normative) Techniques for WCAG 2.0 that help evaluate
conformance to WCAG 2.0 Success Criteria."

to what Gregg suggested:

"W3C/WAI provides a set of publicly documented (non-normative) Techniques
for WCAG 2.0 that provide one way to meet the WCAG 2.0 Success Criteria.
W3C/WAI provides a set of publicly documented (non-normative) Techniques
for WCAG 2.0 that ONE WAY TO MEET THE WCAG 2.0 Success Criteria. However,
it is not necessary to use these particular techniques (see Understanding
Techniques for WCAG Success Criteria). Some AUTHORS might use other methods
(IN LINE with the requirements for custom techniques) to CREATE conformance
to WCAG 2.0 AND EVALUATORS SHOULD ACCEPT VIABLE ALTERNATIVE TECHNIQUES AS
WELL."


---------------------------------
Step 2: Explore the Target Website
----



 * ( ) accept this section as draft
 * (x) accept this section as draft with the following suggestions
 * ( ) I do not accept this section as draft
 * ( ) I abstain (not vote)
 Priority: strong

Location: Step 2c

Current Wording: "Web pages using varying technologies such as HTML, CSS,
JavaScript and WAI-ARIA;"

Suggestion: "Web pages using varying technologies and formats such as HTML,
CSS, JavaScript, WAI-ARIA, PDF etc."


---------------------------------
Step 3: Select a Representative Sample
----



 * ( ) accept this section as draft
 * (x) accept this section as draft with the following suggestions
 * ( ) I do not accept this section as draft
 * ( ) I abstain (not vote)
 I agree with David especially the following

"1) use the “size of website” criteria" 

and the concrete examples for sample sizes.

This will reduce possible variations and I believe will increase
consistency, which is necessary for reliable and valide results.


---------------------------------
Step 4: Audit the Selected Sample
----



 * ( ) accept this section as draft
 * ( ) accept this section as draft with the following suggestions
 * (x) I do not accept this section as draft
 * ( ) I abstain (not vote)
 I agree with David's comments.


---------------------------------
Step 5: Record the Evaluation Findings
----



 * ( ) accept this section as draft
 * ( ) accept this section as draft with the following suggestions
 * (x) I do not accept this section as draft
 * ( ) I abstain (not vote)
 Also on this David's statement reflects my own opinion.

One additional point:

# Priority: strong

Location and current wording: "About the Evaluation" point 1: Name of the
evaluator (responsible person, organization, or other entity)"

Suggestion: Names of the evaluators (responsible person, members of the
review teams, organization, or other entity)

Reason: transparency (especially because WCAG-EM is part of "European
Accessibility Requirements for Public Procurement of Products and Services
in the ICT Domain")

I also agree with Gregg's statement on 5d especially:

"(...) unless and until you have data showing that it actually works based
on evaluation of diverse sites -- we should not have this in the document."
(Gregg)

Unless the reliability of a scoring system is not proven it should not be
part of this document. Especially because (and as written before) WCAG-EM
is explicitly mentioned in the documents of "European Accessibility
Requirements for Public Procurement of Products and Services in the ICT
Domain" (M376).






These answers were last modified on 15 December 2013 at 09:29:18 U.T.C.
by Kerstin Probiesch

Answers to this questionnaire can be set and changed at
https://www.w3.org/2002/09/wbs/1/WCAG-EM-20131129/ until 2013-12-17.

 Regards,

 The Automatic WBS Mailer

Received on Sunday, 15 December 2013 09:30:02 UTC