Questions & Review -- Website Accessibility Evaluation Methodology for WCAG 2.0 Requirements

Hi everyone,



I'm looking at this version:

http://www.w3.org/WAI/ER/conformance/ED-methodology-20110928.html



First some comments on a few requirements:

R02: Tool and browser independent

Some WCAG 2 techniques depend on specific target user agent scenarios.

-          Techniques provided to support older technologies and therefore won't apply to content not targeted to older software

-          ARIA/HTML5 techniques which depend on support in the specific tools and browsers used by the consumers of the content, but may not be sufficient if we assume all content must support older browsers.



R02 seems to imply that the evaluation criteria must be independent of the actual technologies end users employ to consume the content. Shouldn't evaluation criteria vary slightly depending on the likely usage scenarios?

R08. Address the needs of the target audience


Was the idea of including end user consumers as an indirect target audience discussed?  That might go a long way in ensuring the evaluation methodologies optimize the end user benefit. There doesn't seem to be much sense in requiring lowest common denominator techniques in modern web apps targeted only to modern software, where ARIA/HTML5 techniques are beneficial to users.


How will the needs of each audience be identified? I think listing them in this document would be a good idea because it could affect the specific requirements.


For example, "Website developers, suppliers, procurers, and owners wishing to evaluate during the development process"  -- this audience has a need to lower cost or at least maximize the real user benefit for the effort spent. That could be at odds with R02 because specific browsers/tools cannot be considered in evaluation.

R14: Include tolerance metrics

- What potentially informs the tolerance metrics, other than the amount of content & number of violations?

- Are the tolerance metrics intended to allow for a certain # of violations for a given amount of content?

- Can tolerance metrics be based on the WCAG requirement/failure as some are more crucial than others? That would seem to create a dependency on importance weights for WCAG 2 techniques & failures. Importance weights would seem to depend on the type of content and the usage scenarios.



Recommendations



*         A stated goal should be to optimize end-user benefits while making the most of development resources.

*         A stated assumption should be that technology and accessibility techniques change over time.

*         A stated requirement could be to consider the breadth of browser support and depth of technology used for a given site/project. I'm provided examples for reference, although it's too detailed for the requirements doc:

1.       Broad content maximizes support for older browsers, using only the very safest code:
Appropriate WCAG techniques - safe techniques supporting a broad list of ATs including legacy versions

2.       Middle ground content is designed or optimized for up-to-date browser versions
Appropriate WCAG techniques should not include items only for older ATs/browsers

3.       Deep, leading edge web content targets only very recent browsers
Appropriate WCAG techniques include WAI-ARIA and HTML5. In this case, fallback WCAG techniques are only necessary when not all accessible target browsers support the necessary markup.

4.       Device-specific content is the extreme end  -- takes advantage of device-specific features (deep) but has very shallow UA support:
Appropriate techniques should be restricted to those with potential impact. It should not be necessary to use techniques to support features which do not exist and are not planned.

*         A stated requirement should be to consider common content development scenarios, such as:

1.       Static documentation pages

2.       Content that behaves like a native application

3.       Content designed for mobile devices only

*         A stated requirement should be to define tolerance for using new techniques as a replacement for traditional techniques (for now, newer techniques include HTML5, ARIA and CSS3 -- but in the future, something could replace them as well).

*         A stated requirement should be to define tolerance for ignoring legacy techniques

*         The tolerance requirement should be more specific about granularity (by WCAG failure?) or other inputs to tolerance

*         The tool & browser independence requirement should be more specific

*         The audience section should either define some of needs or provide more guidance so that we're sure the requirements  make sense for the audience



Hope these suggestions are useful and not too repetitive .. in summary, both developers & end users benefit when content is made accessible for the environments it was designed for.



Thanks!

Aaron

Aaron Leventhal
Senior Product Manager, Accessibility
Research In Motion (RIM)
www.blackberry.com/accessibility<http://www.blackberry.com/accessibility>


---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.