AW: evaluating web applications (was Re: Canadian Treasury Board accessibility assessment methodology)

Hi Peter, Shadi,

if we would work out "something that is different" from the pass/fail which
obviously is not compliant with the conformance requirements it wouldn't be
an evaluation methodology for WCAG 2.0 anymore. Of course: part of reality
is imperfect software. Part of reality are also "imperfect" developers and
"imperfect" online editors. The question for me is: if we consider these
aspects why then promote for example ATAG? Another problem for me is: the
more granular evaluations are the less reliable they will be.

Regards

Kerstin



Von: Peter Korn [mailto:peter.korn@oracle.com] 
Gesendet: Dienstag, 22. Mai 2012 23:24
An: Shadi Abou-Zahra
Cc: Eval TF
Betreff: Re: evaluating web applications (was Re: Canadian Treasury Board
accessibility assessment methodology)

Shadi,

I don't believe one can make an effective, useful, meaningful conformance
claim about many classes of web applications today.  That class includes
things like web mail, and many kinds of portal applications (particularly
where they only employ a single URI).

I do believe it will be possible to evaluate web applications for
accessibility - similar to evaluating non-web applications for accessibility
- but I expect we will need to do something that is different from the
binary "perfection"/"imperfection" of the current conformance claim rubric. 
The Canadian Treasury Board example takes a step along that path in shifting
from one binary "perfection"/"imperfection" statement to a two tiered,
percentage collection of 38 binary "perfection"/"imperfection" statements. 
But we need to go further than that.

I think the components of such a successful evaluation will need to:
• Recognize (as EvalTF is already doing) that only a sampling/subset of
everything that a user can encounter can be effectively evaluated in a
finite and reasonable amount of time
• Provide greater granularity in the evaluation reporting - one that is
designed to accommodate the reality of imperfect software while nonetheless
providing useful information to those consuming the evaluation report such
that they can make informed decisions based on it
• Incorporate the concepts (as EvalTF is starting to do) of uses (or use
cases) of the application so that the evaluation is meaningful in the
context of how the web application will be used

I am eager to get further into these discussions in EvalTF, some of which
may be logical things to discuss as we review feedback from the public draft
(including some of the Oracle feedback... :-).  And as I mentioned, we've
already started exploring some of this already.


Peter


On 5/22/2012 2:09 PM, Shadi Abou-Zahra wrote: 
Hi Peter, 

Does that mean that web applications cannot be evaluated? 

Best, 
  Shadi 


On 22.5.2012 20:40, Peter Korn wrote: 

Shadi, 

As is clear from the Notes&  Examples under their definition of "Web page"
at 
the bottom of the URL you circulated (below), it is clear they are looking
to 
assess on a Pass/Fail basis the full complexity of web applications. As
we've 
explored in recent EvalTF meetings, that is a very challenging thing to do, 
given how dynamic web applications can be (cf. their examples of a "Web mail

program" and a "customizable portal site"). It is challenging in normal
software 
testing to determine whether you have reached every possible code path& 
every 
possible configuration of the structure behind a single URI, let alone
answer 
Pass/Fail for each and every WCAG A/AA SC for those. 


Regards, 

Peter 

On 5/22/2012 6:10 AM, Shadi Abou-Zahra wrote: 

 Dear Group, 

 Ref:<http://www.tbs-sct.gc.ca/ws-nw/wa-aw/wa-aw-assess-methd-eng.asp> 

 David MacDonald pointed out the accessibility assessment methodology of the

 Canadian Treasury Board, in particular the scoring they use. 

 Best, 
 Shadi 

-- 
Oracle<http://www.oracle.com> 
Peter Korn | Accessibility Principal 
Phone: +1 650 506 9522<tel:+1%20650%20506%209522> 
Oracle Corporate Architecture Group 
500 Oracle Parkway | Redwood City, CA 94065 
----------------------------------------------------------------------------
---- 
Note: @sun.com e-mail addresses will shortly no longer function; be sure to
use: 
peter.korn@oracle.com to reach me 
----------------------------------------------------------------------------
---- 
Green Oracle<http://www.oracle.com/commitment>  Oracle is committed to 
developing practices and products that help protect the environment 


-- 

Peter Korn | Accessibility Principal
Phone: +1 650 506 9522 
Oracle Corporate Architecture Group
500 Oracle Parkway | Redwood City, CA 94065 
________________________________________
Note: @sun.com e-mail addresses will shortly no longer function; be sure to
use: peter.korn@oracle.com to reach me 
________________________________________
Oracle is committed to developing practices and products that help protect
the environment

Received on Wednesday, 23 May 2012 05:23:06 UTC