RE: evaluating web applications (was Re: Canadian Treasury Board accessibility assessment methodology)

Hi all
I was recently doing some application testing against WCAG 2.0 for a government agency here in Western Australia.  They asked me to test every page of their applications against the WCAG 2.0 criteria. The trickiest part was identifying the pages when they used the same URI.  I had to include a table with the paths or user tasks (e.g. if the user choose 'service a', then they would then be presented with 'scenario a' etc.)  Other than that, it didn't pose any different problems then assessing a normal website.  It just required lots of annotations of paths and actions.


Regards

Vivienne L. Conway, B.IT(Hons), MACS CT, AALIA(cs)
PhD Candidate & Sessional Lecturer, Edith Cowan University, Perth, W.A.
Director, Web Key IT Pty Ltd.
v.conway@ecu.edu.au
v.conway@webkeyit.com
Mob: 0415 383 673

This email is confidential and intended only for the use of the individual or entity named above. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify me immediately by return email or telephone and destroy the original message.
________________________________________
From: Shadi Abou-Zahra [shadi@w3.org]
Sent: Wednesday, 23 May 2012 3:30 PM
To: Peter Korn
Cc: Eval TF
Subject: Re: evaluating web applications (was Re: Canadian Treasury Board   accessibility assessment methodology)

Hi all,

Is there a suggested approach/procedure for sampling functionality
within an application, as we have for web pages within a website? Is
this usual practice that web accessibility evaluators take?

Best,
   Shadi


On 22.5.2012 23:24, Peter Korn wrote:
> Shadi,
>
> I don't believe one can make an effective, useful, meaningful conformance claim
> <http://www.w3.org/TR/WCAG/#conformance-claims>  about many classes of web
> applications today. That class includes things like web mail, and many kinds of
> portal applications (particularly where they only employ a single URI).
>
> I do believe it will be possible to evaluate web applications for accessibility
> - similar to evaluating non-web applications for accessibility - but I expect we
> will need to do something that is different from the binary
> "perfection"/"imperfection" of the current conformance claim
> <http://www.w3.org/TR/WCAG/#conformance-claims>  rubric. The Canadian Treasury
> Board example takes a step along that path in shifting from one binary
> "perfection"/"imperfection" statement to a two tiered, percentage collection of
> 38 binary "perfection"/"imperfection" statements. But we need to go further than
> that.
>
> I think the components of such a successful evaluation will need to:
>
>    * Recognize (as EvalTF is already doing) that only a sampling/subset of
>      everything that a user can encounter can be effectively evaluated in a
>      finite and reasonable amount of time
>    * Provide greater granularity in the evaluation reporting - one that is
>      designed to accommodate the reality of imperfect software while nonetheless
>      providing useful information to those consuming the evaluation report such
>      that they can make informed decisions based on it
>    * Incorporate the concepts (as EvalTF is starting to do) of uses (or use
>      cases) of the application so that the evaluation is meaningful in the
>      context of how the web application will be used
>
>
> I am eager to get further into these discussions in EvalTF, some of which may be
> logical things to discuss as we review feedback from the public draft (including
> some of the Oracle feedback... :-). And as I mentioned, we've already started
> exploring some of this already.
>
>
> Peter
>
>
> On 5/22/2012 2:09 PM, Shadi Abou-Zahra wrote:
>>  Hi Peter,
>>
>>  Does that mean that web applications cannot be evaluated?
>>
>>  Best,
>>  Shadi
>>
>>
>>  On 22.5.2012 20:40, Peter Korn wrote:
>>>  Shadi,
>>>
>>>  As is clear from the Notes&  Examples under their definition of "Web page" at
>>>  the bottom of the URL you circulated (below), it is clear they are looking to
>>>  assess on a Pass/Fail basis the full complexity of web applications. As we've
>>>  explored in recent EvalTF meetings, that is a very challenging thing to do,
>>>  given how dynamic web applications can be (cf. their examples of a "Web mail
>>>  program" and a "customizable portal site"). It is challenging in normal software
>>>  testing to determine whether you have reached every possible code path&  every
>>>  possible configuration of the structure behind a single URI, let alone answer
>>>  Pass/Fail for each and every WCAG A/AA SC for those.
>>>
>>>
>>>  Regards,
>>>
>>>  Peter
>>>
>>>  On 5/22/2012 6:10 AM, Shadi Abou-Zahra wrote:
>>>>  Dear Group,
>>>>
>>>>  Ref:<http://www.tbs-sct.gc.ca/ws-nw/wa-aw/wa-aw-assess-methd-eng.asp>
>>>>
>>>>  David MacDonald pointed out the accessibility assessment methodology of the
>>>>  Canadian Treasury Board, in particular the scoring they use.
>>>>
>>>>  Best,
>>>>  Shadi
>>>>
>>>
>>>  --
>>>  Oracle<http://www.oracle.com>
>>>  Peter Korn | Accessibility Principal
>>>  Phone: +1 650 506 9522<tel:+1%20650%20506%209522>
>>>  Oracle Corporate Architecture Group
>>>  500 Oracle Parkway | Redwood City, CA 94065
>>>  --------------------------------------------------------------------------------
>>>  Note: @sun.com e-mail addresses will shortly no longer function; be sure to use:
>>>  peter.korn@oracle.com to reach me
>>>  --------------------------------------------------------------------------------
>>>  Green Oracle<http://www.oracle.com/commitment>  Oracle is committed to
>>>  developing practices and products that help protect the environment
>>
>
> --
> Oracle<http://www.oracle.com>
> Peter Korn | Accessibility Principal
> Phone: +1 650 506 9522<tel:+1%20650%20506%209522>
> Oracle Corporate Architecture Group
> 500 Oracle Parkway | Redwood City, CA 94065
> --------------------------------------------------------------------------------
> Note: @sun.com e-mail addresses will shortly no longer function; be sure to use:
> peter.korn@oracle.com to reach me
> --------------------------------------------------------------------------------
> Green Oracle<http://www.oracle.com/commitment>  Oracle is committed to
> developing practices and products that help protect the environment

--
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

This e-mail is confidential. If you are not the intended recipient you must not disclose or use the information contained within. If you have received it in error please return it to the sender via reply e-mail and delete any record of it from your system. The information contained within is not the opinion of Edith Cowan University in general and the University accepts no liability for the accuracy of the information provided.

CRICOS IPC 00279B

Received on Thursday, 24 May 2012 13:22:48 UTC