Re: International Web Access Guidelines “Ineffective”, PhD thesis Claims

I think I would have to concur with Steve's conclusion. The thesis quite clearly states that they avoided including a control group. This would have allowed them to eliminate the usability findings which I think simply pollute the findings. They are important and they are key to developing good websites but the existence of these issues do not prove that WCAG doesn't work.

Definitely keen to seem more of this research but there seems to be a serious need to ensure that any poking and prodding around the current guidelines needs to be done with a measure of cognisance regarding how they are used in the wild.

Kevin

--
Kevin White
Principal Experience Designer
Nile | Experience & Service Design
 
tel:             0131 220 5671
mobile:     07791 895 472
web:          www.nilehq.com
address:  13-15 Circus Lane, Edinburgh, EH3 6SU



On 1 Jun 2013, at 19:52, Steve Green wrote:

> It’s good that this sort of research is being done, but I seriously question the findings. I have skim-read the 300-page thesis and a couple of things jump out.
>  
> 1.       Several of the common problems that the participants encountered are usability issues that affect everyone, so they will not (and should not) be covered by the WCAG. Two such examples are information architecture and incomplete site search indexing.
> 
> 2.       The author claims there was “no significant relationship between user-based measures of accessibility and most measures based on technical guidelines�, yet after eliminating the usability issues virtually every accessibility issue described in the report is covered by a WCAG requirement. This raises a question about the quality of the technical testing and the author’s level of understanding of the WCAG success criteria.
>  
> I disagree with the recommendation of “moving away from the technical conformance approach of web accessibility�. Technical conformance is a prerequisite for good accessibility, albeit I agree it is not sufficient. I have always advocated user testing but it is pointless unless the website is already close to conformance with WCAG level AA (12 of the 16 websites in the study did not even meet level A).
>  
> The 100+ hours of video recordings would be a wonderful resource if it was publicly available. I wish I had time to reassess them and re-do the technical testing because I suspect I would come to a rather different conclusion.
>  
> Steve Green
> Managing Director
> Test Partners Ltd
>  
> From: Jorge Fernandes [mailto:jorge.f@netcabo.pt] 
> Sent: 01 June 2013 18:11
> To: w3c-wai-ig@w3.org
> Subject: International Web Access Guidelines “Ineffective�, PhD thesis Claims
>  
> A recent published PhD thesis of York University [1] wrote the follow statement in the abstract:
>  
> "Comparisons between user problems and WCAG 1.0 and WCAG 2.0 did not show any significant relationship between user-based measures of accessibility and most measures based on technical guidelines."
>  
> I have difficulty mostly to understand what this kind of statements means.
>  
> - it's means that there aren't significative relationship between WCAG and user problems? or;
> - it's means that there aren't significative relationship between WCAG and the way we are trying to measure the WCAG? 
>  
> [1]
> Freire, André Pimenta. (2012). Disabled people and the Web: User-based measurement of accessibility. PhD thesis, University of York.
> http://etheses.whiterose.ac.uk/3873/
>  
> Jorge Fernandes
>  
> ⠨⠚⠕⠗⠛⠑ ⠨⠋⠑⠗���⠙⠑⠎
>  universalaccess.blogspot.com
>