Re: evaluating web applications (was Re: Canadian Treasury Board accessibility assessment methodology)

Hi all,

Is there a suggested approach/procedure for sampling functionality 
within an application, as we have for web pages within a website? Is 
this usual practice that web accessibility evaluators take?

Best,
   Shadi


On 22.5.2012 23:24, Peter Korn wrote:
> Shadi,
>
> I don't believe one can make an effective, useful, meaningful conformance claim
> <http://www.w3.org/TR/WCAG/#conformance-claims>  about many classes of web
> applications today. That class includes things like web mail, and many kinds of
> portal applications (particularly where they only employ a single URI).
>
> I do believe it will be possible to evaluate web applications for accessibility
> - similar to evaluating non-web applications for accessibility - but I expect we
> will need to do something that is different from the binary
> "perfection"/"imperfection" of the current conformance claim
> <http://www.w3.org/TR/WCAG/#conformance-claims>  rubric. The Canadian Treasury
> Board example takes a step along that path in shifting from one binary
> "perfection"/"imperfection" statement to a two tiered, percentage collection of
> 38 binary "perfection"/"imperfection" statements. But we need to go further than
> that.
>
> I think the components of such a successful evaluation will need to:
>
>    * Recognize (as EvalTF is already doing) that only a sampling/subset of
>      everything that a user can encounter can be effectively evaluated in a
>      finite and reasonable amount of time
>    * Provide greater granularity in the evaluation reporting - one that is
>      designed to accommodate the reality of imperfect software while nonetheless
>      providing useful information to those consuming the evaluation report such
>      that they can make informed decisions based on it
>    * Incorporate the concepts (as EvalTF is starting to do) of uses (or use
>      cases) of the application so that the evaluation is meaningful in the
>      context of how the web application will be used
>
>
> I am eager to get further into these discussions in EvalTF, some of which may be
> logical things to discuss as we review feedback from the public draft (including
> some of the Oracle feedback... :-). And as I mentioned, we've already started
> exploring some of this already.
>
>
> Peter
>
>
> On 5/22/2012 2:09 PM, Shadi Abou-Zahra wrote:
>>  Hi Peter,
>>
>>  Does that mean that web applications cannot be evaluated?
>>
>>  Best,
>>  Shadi
>>
>>
>>  On 22.5.2012 20:40, Peter Korn wrote:
>>>  Shadi,
>>>
>>>  As is clear from the Notes&  Examples under their definition of "Web page" at
>>>  the bottom of the URL you circulated (below), it is clear they are looking to
>>>  assess on a Pass/Fail basis the full complexity of web applications. As we've
>>>  explored in recent EvalTF meetings, that is a very challenging thing to do,
>>>  given how dynamic web applications can be (cf. their examples of a "Web mail
>>>  program" and a "customizable portal site"). It is challenging in normal software
>>>  testing to determine whether you have reached every possible code path&  every
>>>  possible configuration of the structure behind a single URI, let alone answer
>>>  Pass/Fail for each and every WCAG A/AA SC for those.
>>>
>>>
>>>  Regards,
>>>
>>>  Peter
>>>
>>>  On 5/22/2012 6:10 AM, Shadi Abou-Zahra wrote:
>>>>  Dear Group,
>>>>
>>>>  Ref:<http://www.tbs-sct.gc.ca/ws-nw/wa-aw/wa-aw-assess-methd-eng.asp>
>>>>
>>>>  David MacDonald pointed out the accessibility assessment methodology of the
>>>>  Canadian Treasury Board, in particular the scoring they use.
>>>>
>>>>  Best,
>>>>  Shadi
>>>>
>>>
>>>  --
>>>  Oracle<http://www.oracle.com>
>>>  Peter Korn | Accessibility Principal
>>>  Phone: +1 650 506 9522<tel:+1%20650%20506%209522>
>>>  Oracle Corporate Architecture Group
>>>  500 Oracle Parkway | Redwood City, CA 94065
>>>  --------------------------------------------------------------------------------
>>>  Note: @sun.com e-mail addresses will shortly no longer function; be sure to use:
>>>  peter.korn@oracle.com to reach me
>>>  --------------------------------------------------------------------------------
>>>  Green Oracle<http://www.oracle.com/commitment>  Oracle is committed to
>>>  developing practices and products that help protect the environment
>>
>
> --
> Oracle<http://www.oracle.com>
> Peter Korn | Accessibility Principal
> Phone: +1 650 506 9522<tel:+1%20650%20506%209522>
> Oracle Corporate Architecture Group
> 500 Oracle Parkway | Redwood City, CA 94065
> --------------------------------------------------------------------------------
> Note: @sun.com e-mail addresses will shortly no longer function; be sure to use:
> peter.korn@oracle.com to reach me
> --------------------------------------------------------------------------------
> Green Oracle<http://www.oracle.com/commitment>  Oracle is committed to
> developing practices and products that help protect the environment

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Wednesday, 23 May 2012 07:31:34 UTC