Re: What happens when an evaluation statement already exists?

You understood my proposal in your final paragraph, and your concern seemingly comes from an unwillingness to trust a claim made by a web developer / owner because (in your words) "it is possible that the owner is using a restrictive set of techniques just to bypass real conformance to WCAG".  

I have before read in this groups responses to my emails a note of seeming disregard for the skills and honesty of web developers / website owners, and I sadly read it again here.  I personally think we need to trust people when they make a public statement, which is why I made my proposal for an addition to step 1.d:

"If documented evaluation procedures are provided along with an evaluation statement or WCAG 2.0 conformance claim, the documented evaluation procedures provided should take precedence over any other evaluation procedure when undertaking an evaluation."

I should like to state clearly that the WCAG 2.0 guidelines are developed for web developers, rather than evaluators.  I should also like to state that the evaluation methodology we have created is for use by first and third parties i.e. web developers and evaluators.

It is also important to underline that expert evaluators are very clearly not the only people to be able to say whether or not something is accessible, especially now that WCAG 2.0's sufficient techniques exist. 

Alistair 

On 20 May 2014, at 11:49, Ramón Corominas wrote:

> Alistair wrote:
> 
> 
>> My core question is "Have you ever created a web page and tested
> > it with different tools?  Did you get the same results?"
> 
> Yes, of course, but I doubt that a tool that claims itself to be WCAG compliant can lead to false positives or, if you prefer, if the tool reports that a conforming website is not meeting WCAG, then the tool should be discarded. If a tool is WCAG compliant, it should not detect any failure in a conforming website. Maybe the tool includes warnings for manual checks, but it is unlikely that a tool will say "the table is wrong because you didn't use @scope", it will probably say "I cannot find @scope, so check if the data cells are associated somehow to the appropriate header cells".
> 
> Yes, maybe today a tool says "the table is wrong because there are no <th> elements", but you used roles and other ARIA attributes to emulate the table structure, but then it is the tool what needs an update, not the website, and your claim is perfectly valid. If a tool states that a particular content is wrong, but this particular content is using a documented and accessibility supported technique, it is easy to claim that the tool has a flaw, and not your website
> 
> Therefore, I don't see the point of imposing a precedence of evaluation techniques (again, not developing techniques). As evaluator, I think that it is more important to ensure that the assessment does not depend on the *evaluation* techniques used by the owner, because it is possible that the owner is using a restrictive set of techniques just to bypass real conformance to WCAG (for example, using techniques that are not accessibility supported).
> 
> 
> > All I am saying is that respect should be paid to the
> > WCAG techniques, and their directly associated checks,
> > a developer implements in order to make their web pages
> > accessible.
> 
> Techniques are not normative. I think this sentence suggests that they can be used as normative rules for evaluating conformance, with precedence over the Conformance Requirements, that could be tested using different techniques.
> 
> 
> > If not, you will get the situation where a single wide-scale
> > monitoring tool is developed which tests certain specific things,
> > and in order to comply all web developers naturally start to make
> > their web pages accessible in exactly the same way.  Which is
> > not what WCAG is about...
> 
> I guess that risk is even greater if website oweners can force the techniques that must be used to an independent evaluator. Website owners might indeed focus their efforts to pass the tests from a specific tool, and then claim Conformance based only on the results of that tool. But maybe the tool use only a restrictive set of tests that do not cover every aspect of WCAG, or that do not take into account accessibility support, for example, or that do not recognise processes, etc.
> 
> So my vote is still for allowing evaluators to use any methods that they consider, but obviously they must use methods that ensure WCAG conformance, not "MyTool-Conformance". Maybe I am not understading your point, but I cannot imagine a situation where an independent analysis is not desirable.
> 
> My mainm concern is that your proposal...
> 
> "If documented evaluation procedures are provided along with an evaluation statement or WCAG 2.0 conformance claim, the documented evaluation procedures provided should take precedence over any other evaluation procedure when undertaking an evaluation."
> 
> 
> ... may also be read this way: "I am a website owner that used "MyTool", that is supposed to be a complete tool to automatically assess WCAG 2.0 conformance. "MyTool" told me that my website is conforming to Double-A level, and therefore I made a claim stating that my website meets AA. I don't mind if you used your own procedures and discovered that my website is not keyboard accessible or that the descriptions of my images are meaningless, because my documented evaluation procedure ("I used MyTool") must take precedence over yours."
> 
> Regards,
> Ramón.
> 
> 
>>> Ramón said: 
> >>
>>> From the "Techniques for WCAG 2.0" document:
>>> 
>>> "Please note that the contents of this document are informative (they provide guidance), and not normative (they do not set requirements for conforming to WCAG 2.0)."
>>> 
>>> The only normative document is the WCAG 2.0 Recommendation, so you cannot tell anyone that you have to use a particular technique to check the accessibility of a website. Techniques may -and will- change in the future due to new technologies or methods, varying accessibility support, wrong concepts or any other circumstance.
>>> 
>>> Of course, any WCAG-related evaluation procedure must ensure that WCAG 2.0 itself is met, taking into account that conformance can be reached through many different ways, provided that they are accessibility supported; indeed, WCAG Conformance comes from meeting the 5 Conformance Requirements, and the Success Criteria are indirectly pointed from them.
>>> 
>>> If a website owner claims conformance and the website is in fact WAG 2.0 conformant, the method used to assess this should be irrelevant. If a particular method to assess WCAG 2.0 conformance states that a conformant website is non-conformant, there is a flaw in the method, and not in the website or in WCAG.
>>> 
>>> That said, it is true that sometimes we may disagree about the interpretation of certain important things, such as the required degree of accessibility support, the applicability of conforming alternate versions or other things that can lead to different results.
>>> 
>>> Of course, these differences can be very important from a legal perspective, but I assume that WCAG 2.0 is a technical document, not a legal one, and if someone cannot access and a discrimination exists, it doesn't matter if you claim conformance, and you cannot excuse the discrimination because "technically" you meet the guidelines.
>>> 
>>> In any case, overimposing a precedence on the techniques used to evaluate (not the techniques used to develop) could lead to owners claiming conformance but not conforming to WCAG 2.0, and then pretending that you must use *their* techniques to evaluate, not yours. For example, they could say that they used the keyboard (without a screen reader) to check keyboard accessibility, but if I check with the screen reader running maybe the website is completely unusable. Should I agree with the claim just because they did not test a real world case? Must I consider this website accessible?
>>> 
>>> In conclusion, I am totally against any imposition that gives more precedence to a particular evaluation procedure, of course provided that the evaluation methods follow the WCAG Recommendation.
>>> 
>>> Regards,
>>> Ramón.
>>> 
>>> 
>>> Alistair said:
>>> 
>>>> If I say I have used this set of techniques, and I have undertaken all associated checks, I would not want to be told that I did not conform because I did not use another set of techniques - possibly bound together in some monitoring tool.  Or, think about two EU states using slightly different monitoring tools - which conflict.   Having chaired the Evaluation Methods Task Force for the EuroAccessibility Consortium, and a similar European task force in the initial WABCluster I can say that such conflicts are the rule, rather than the exception. So, a concrete proposal for an addition to step 1.d would be:
>>>> "If documented evaluation procedures are provided along with an evaluation statement or WCAG 2.0 conformance claim, the documented evaluation procedures provided should take precedence over any other evaluation procedure when undertaking an evaluation."
>>> 
>

Received on Tuesday, 20 May 2014 11:34:21 UTC