Re: Accessible does not imply usable (was International Web Access Guidelines "Ineffective", PhD thesis Claims)

Hi Ian, Gregg and Jorge.

I think we are talking of 3 qualities: conformance to wcag,
accessibility, and usability.

Conformance is based on the degree with which pages/sites satisfy wcag
criteria; it is affected by subjectivity, depending on who and how and
under which circumstances does the audit. [1]

Usability could be considered as quality in use (the degree of
effectiveness, efficiency and satisfaction with which certain people in
certain conditions achieve certain goals). Notice that it is highly
contextualized, wrt people+situation+goals. Contextualization and
reference to parameters like effectiveness, efficiency and satisfaction
make it hard to evaluate, justify the several methods that one could
employ, and explain variability of results.

Accessibility is something in between, IMO. One could define it as the
level of effectiveness that can be achieved by particular groups of
people using a particular device (eg blind people using a screen reader
on a smartphone). But this is a definition out of many possible ones.
Given that there is no single definition that we agree on [2], it is
difficult to discuss it. Another source of ambiguity is the notion of
"people with disabilities": what kind of impairment? what degree of
impairment? what about situational impairments? or social ones? and if
we start talking about cognitive impairments, do we include also
language barriers? or cultural barriers?

Having said that, the statement made by the thesis perhaps refers to the
weak relationship between conformance and usability, when the latter is
concerned with people with certain disabilities, that use certain tools,
that are asked to perform certain things. I would expect such a weak
relationship, in fact, given the differences in these two qualities.
Although, as Greg said, one would need to read the entire work before
drawing conclusions (which I haven't done, yet). Most experiments, by
their nature, have to restrict the scope of the investigation if they
are expected to find something that can be generalized. This one is
likely not to be an exception, and one needs to understand what
assumptions and restrictions were made when it was carried out.

BTW, I don't think either of these 3 qualities can be assessed solely
using automated ways, not even conformance. See [3] for some recent
assessment.

[1] G. Brajnik, Y. Yesilada, S. Harper. Is accessibility conformance an
elusive property? A study of validity and reliability of WCAG 2.0 , ACM
Transactions on Accessible Computing, 2(4), March 2012; doi:
10.1145/2141943.2141946

[2] Y. Yesilada, G. Brajnik, M. Vigo, S. Harper. Understanding web
accessibility and its drivers , W4A '12: Proceedings of the
International Cross-Disciplinary Conference on Web Accessibility, April
2012; doi: 10.1145/2207016.2207027
http://dl.acm.org/citation.cfm?id=2207016.2207027&coll=DL&dl=ACM&CFID=83883151&CFTOKEN=24765113

[3] Markel Vigo, Justin Brown, and Vivienne Conway. 2013. Benchmarking
web accessibility evaluation tools: measuring the harm of sole reliance
on automated tests. In Proceedings of the 10th International
Cross-Disciplinary Conference on Web Accessibility (W4A '13). ACM, New
York, NY, USA, , Article 1 , 10 pages.
DOI=http://dx.doi.org/10.1145/2461121.2461124
http://dl.acm.org/citation.cfm?id=2461124&CFID=221360079&CFTOKEN=45047729


Best regards


  Giorgio Brajnik
______________________________________________________________________
Dip. di Matematica e Informatica | voice: +39 (0432) 55.8445
Università di Udine | fax: +39 (0432) 55.8499
Via delle Scienze, 206 | email: brajnik@uniud.it
Loc. Rizzi -- 33100 Udine -- ITALY | http://www.dimi.uniud.it/giorgio


On 06/02/13 02:11, Ian Sharpe wrote:
> Hi Greg
>  
> I really like how you have explained the difference between usability
> and accessibility in your first two points below as I feel these terms
> are often incorrectly considered synonymous. Indeed, I would emphasise
> this distinction by adding that accessibility does not imply usability
> and that when we talk about inclusive design or universal design we
> should be aiming to achieve usability, not just accessibility.
>  
> Ironically, I believe it is an unfortunate consequence of the
> communities excellent work and success promoting the wide-spread
> acceptance, recognition and adoption of WCAG   which has led to this
> confusion as I feel organisations now think that in order to make their
> sites and services satisfy any legislative obligations, all they need to
> do is ensure their sites and services are accessible as defined by WCAG:
> Despite the fact that the guidelines say that WCAG should be thought of
> not as making things accessible,  but as a minimum accessibility
> standard as you point out.
>  
> While for the most part it is true that compliance to WCAG is likely to
> mitigate the potential risk of litigation under antidiscrimination
> legislation and result in a more usable site or service than a site or
> service which does not conform to WCAG, it is absolutely not the case
> that the site or service will be "usable".
>  
> The problem is that it is harder to test usability and as far as I'm
> aware, the process cannot be automated.
>  
> This is why I feel that it is important to make this distinction between
> usability and accessibility and why testing with real users should form
> part of level AA conformance.  
>  
> Cheers
> Ian
>  
>  
>  
>  
>  *From:* Gregg Vanderheiden [mailto:gv@trace.wisc.edu]
> *Sent:* 01 June 2013 19:53
> *To:* Jorge Fernandes
> *Cc:* w3c-wai-ig@w3.org
> *Subject:* Re: International Web Access Guidelines “Ineffective”, PhD
> thesis Claims
> 
> On Jun 1, 2013, at 12:11 PM, Jorge Fernandes <jorge.f@netcabo.pt
> <mailto:jorge.f@netcabo.pt>> wrote:
>> A recent published PhD thesis of York University [1] wrote the follow
>> statement in the abstract:
>> "Comparisons between user problems and WCAG 1.0 and WCAG 2.0 did not
>> show any significant relationship between user-based measures of
>> accessibility and most measures based on technical guidelines."
>> I have difficulty mostly to understand what this kind of statements means.
>> - it's means that there aren't significative relationship between WCAG
>> and user problems? or;
>> - it's means that there aren't significative relationship between WCAG
>> and the way we are trying to measure the WCAG? 
>> [1]  Freire, André Pimenta. (2012). /Disabled people and the Web:
>> User-based measurement of accessibility./ PhD thesis, University of York.
>> http://etheses.whiterose.ac.uk/3873/
>> Jorge Fernandes
> 
> Hi Jorge, 
> 
> Not sure - but It sounds like it says that there is not significant
> relationship between what users think is accessible and what WCAG (a
> technical guideline) says is accessible.
> 
> Of course unless you look at the study - you don't know what the data
> really say -- or if they were collected properly etc.  So I would read
> the whole thesis before drawing  any conclusions.
> 
> I find it hard to believe that there is no relationship between WCAG
> conformance and usability by people who are blind for example. 
> 
> ---
> On a related note...
> 
> I also read a study that showed that conforming to WCAG didn’t make a
> page usable by PWD (or to anyone).   What they were looking at was pages
> that were basically badly designed and unusable (or very poor usability)
> by everyone -- and WCAG didn’t fix the pages.      This is no surprise
> since one of the precepts in WCAG was that it only focus on things that
> were a problem ONLY for people with disabilities - or that affect them
> in greatly disproportionate wasy.    That is WCAG was restrictued to
> focusing on "accessibility" (discriminating against people with
> disabilities)  vs general usability (bad design in general for all
> users)  problems.        The moral here is that if pages aren't usable
> in general -- they will never be very usable by people with disabilities
> -- since they must navigate the non-usability through reduced access.  
>   So to make pages usable by people with disabilities,  make them both
> usable by all and WCAG conformant. 
> 
> ---
> 
> One that had me thinking was another study that showed that certain
> usability problems slowed users WITHOUT disabilities down by X seconds
>  but it slowed people who were blind down by (I think it was)  5x
> seconds.      This sounded like disproportionate - and an accessibility
> problem.   Then elsewhere in the paper it reported that these same
> subject who were blind took  5 times as long to complete all of their
> tasks.     So this made me think.    If it slowed regular uses down by
>  Y%  and it slowed blind users down by the same Y%    (the blind users
> taking 5 times longer in general)  then maybe it wasn’t disproportionate
> because is slowed everyone down by the same percent.   Now I had two
> different conclusions from the same data.   
> 
> ---
> 
> At any rate - a few things seem clear to me.
> 
>  1. *Usability is critical to accessibility.  * In fact Accessibility is
>     just a dimension of Usability.   It is Usability to certain groups.
>       Is the page easily used, hard to use, or unusable.   Pages can be
>     all three for people with and without disabilities.   An advanced
>     Genetics site for example may be completely unusable by anyone
>     without advance genetics background.   And in a way it does
>     discriminate against all of us who do not understand advanced
>     genetics.  But this is legal.  It is only illegal to discriminate
>     (make pages that cannot be used - or used effectively) by people on
>     the basis of race, creed, color, religion, and with the ADA,
>     disability  (If you are among those that think the ADA was intended
>     to include all public places including the Internet - even though
>     the internet was not such a common public place when the ADA was
>     written so was not mentioned specifically).   But whether you
>     believe this or the opposite,  *a page that is unusable can't be
>     made usable by just fixing the parts that are a problem ONLY for
>     people with disabilities (WCAG).  The rest of the unusable still
>     remains. 
> 
>     *
>  2. *WCAG conformance does not make things accessible to all. * In fact
>     is says this right in the WCAG intro.   WCAG should be thought of
>     not as making things accessible,  but as*a minimum accessibility
>     standard.  *Not the end of what one should do but the beginning.    
>     Even doing all three levels (which can be very hard on some specific
>     types of pages) will not make pages that everyone with every type,
>     degree and combination of disabilities can use.   *The advisory
>     techniques are a good place to start to look for more things that
>     can be done.  Then read books and papers on still more.  Finally,
>     work side by side with people who have disabilities (a wide variety)
>     and come to understand the disabilities and the impact of web design
>     on them.   There are so many things that can be done in the design
>     of a specific web site -- to make it more USABLE by people with
>     disabilities;  things that cannot be boiled down to rules and are
>     different from site to site or type of page to different type of page. 
> 
>     *
>  3. *We need to READ papers and understand them.   We can't just pull
>     conclusions from them. * People who do research know that in all
>     fields there is an amazing number (and percent) of papers published
>     that state conclusions that are not supported by their data and
>     design.  This is not done out of malice by authors -- just oversight
>     or weak design.   Good research is very hard to do.  And doing it in
>     areas that have direct implications on daily life I even harder
>     (extremely difficult)  to do in a controlled and valid way.  
>      Really good papers are actually relatively rare overall and rarer
>     in areas of applied research.   
>      1. By the way this is no comment on this particular thesis !   I
>         have not had the time to read it -- so make no comment on it
>         specifically !    (though I am going to pull a copy and try to
>         read it this summer to see what is to be learned from his work). 
> 
> 
> 
> ciao 
> 
> /Gregg/
> --------------------------------------------------------
>

Received on Sunday, 2 June 2013 09:48:14 UTC