W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > January 2013

RE: Combining evaluations

From: David MacDonald <david100@sympatico.ca>
Date: Thu, 31 Jan 2013 09:18:17 -0500
Message-ID: <BLU0-SMTP55A03FDFB6EA063139304AFE1D0@phx.gbl>
To: "'Vivienne CONWAY'" <v.conway@ecu.edu.au>, "'Roberto Scano \(IWA/HWG\)'" <r.scano@webprofession.com>, "'Velleman, Eric'" <evelleman@bartimeus.nl>, <public-wai-evaltf@w3.org>
Although I really love the work James and Michael have done on NVDA, and
their progress is incredible, I do not think a freeware product should be
considered the baseline for accessibility support... I realize that many
evaluators use it to check their assumptions, because it does not require an
investment, but in complex environments, NDVA is still progressing, not
something that should be depended upon... For instance, NDVA does not read
table headers, nor recognize properly coded complex tables...  

I don't think we should require developers to build to the most inexpensive
AT software. I would love a world where all accommodations are very
inexpensive... but an electric wheelchair costs money, and it is a
legitimate accommodation. I think the same is true for software ... (of
course we know all AT has bugs, just like every electric wheelchair has its
weaknesses and strengths)

Anyway, this is a philosophical issue we perhaps also need to address. But
forcing everyone on an evaluation team to test to NVDA because some don't
use JAWS, is perhaps not the optimal, nor a realistic picture of what people
with disabilities are doing ... my experience is that most people with
disabilities are still using the best AT products they can find... even in
Guatemala when I was there, they were using just 2 versions back and that
was only because the Spanish version of JAWS was there.

The WebAim survey shows that most are using several screen readers.

David MacDonald

CanAdapt Solutions Inc.
  Adapting the web to all users
            Including those with disabilities

-----Original Message-----
From: Vivienne CONWAY [mailto:v.conway@ecu.edu.au] 
Sent: January-31-13 8:42 AM
To: Roberto Scano (IWA/HWG); 'Velleman, Eric'; public-wai-evaltf@w3.org;
Subject: RE: Combining evaluations

Combining evaluations using different tools is a difficult one to address.
We use up to 3 evaluators and I like them to use the same AT for
consistency.  However I can envision a testing environment where testers
each use the AT they are most familiar with.  This would add more coverage
to the testing - eg. could navigate with JAWS but not with NVDA or similar.
We all find differences in how AT handled different aspects of the website.
People might have a problem combining them when 1 page is tested with JAWS
and one with NVDA obtaining different results - perhaps 1 passing and 1 not.
I think in that situation I'd prefer the pages be tested with the same AT.
It would be okay to use different AT if it was a situation of cross-checking
results as above with JAWS and NVDA.


Vivienne L. Conway, B.IT(Hons), MACS CT, AALIA(cs) PhD Candidate & Sessional
Lecturer, Edith Cowan University, Perth, W.A.
Director, Web Key IT Pty Ltd.
Mob: 0415 383 673

This email is confidential and intended only for the use of the individual
or entity named above. If you are not the intended recipient, you are
notified that any dissemination, distribution or copying of this email is
strictly prohibited. If you have received this email in error, please notify
me immediately by return email or telephone and destroy the original
From: Roberto Scano (IWA/HWG) [r.scano@webprofession.com]
Sent: Sunday, 27 January 2013 4:21 PM
To: 'Velleman, Eric'; public-wai-evaltf@w3.org; david100@sympatico.ca
Subject: R: Combining evaluations

This is one of the problem of WCAG 2.0 that we have also noticed in italian
government working group for accessibility. The definition of accessibility
supported regarding different assistive technologies in a real big problem,
especially if the conformance declaration isn't a voluntary declaration but
a law requirement.
I think that evalutation could be do by different people but with the same
"baseline" (same technologies for testing), otherwise I think is required
that any evalutator must evalutate the same pages for conformance.

Roberto Scano
International Webmasters Association / The HTML Writers Guild

-----Messaggio originale-----
Da: Velleman, Eric [mailto:evelleman@bartimeus.nl]
Inviato: sabato 26 gennaio 2013 23:23
A: public-wai-evaltf@w3.org; david100@sympatico.ca
Oggetto: Combining evaluations

Dear EvalTF,

Although we mostly agree that "Accessibility support must be uniform
throughout a single website", there are a few comments in the latest survey
[1]. I would like to discuss this a bit more on the list, specifically a bit
more about combining evaluations.

In WCAG it is not a problem if you are a person with a disability and you
have to use multiple user-agents, assistive technology and other tools and
configurations to use a single web page. We do not change this in WCAG-EM:
If you want to evaluate using multiple  versions of user agents, tools and
screenreaders, that is ok. We only say that this should then be uniform
throughout the single website evaluation. For the next single website, you
can use another configuration, other tools, assistive technology etc.. This
may be depending on the evaluation commissioner, the budget, time etc.

The question is: Can we combine two evaluations by two different evaluators
(using different configurations for evaluation) of two different parts of a
single website into one accessibility evaluation statement for the single

Both results of the evaluators are of course valid. but can they be combined
if they use different configurations and are about different parts of a
single website?

Kindest regards,


[1] <https://www.w3.org/2002/09/wbs/48225/evaltfq7/results#x2586>

This e-mail is confidential. If you are not the intended recipient you must
not disclose or use the information contained within. If you have received
it in error please return it to the sender via reply e-mail and delete any
record of it from your system. The information contained within is not the
opinion of Edith Cowan University in general and the University accepts no
liability for the accuracy of the information provided.

Received on Thursday, 31 January 2013 14:18:53 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:40:23 UTC