Re: Using AT for evaluation

All:

I can definitely see Detlev's point, and I concur about "highly
recommended" as an addition.  However, I still request that we not limit
ourselves or our audience to simply a screen reader.



Thanks,
Elle



On Wed, Apr 4, 2012 at 12:46 PM, RichardWarren
<richard.warren@userite.com>wrote:

> Aah,
>
> Detlev has a point. Our methodology is to check for compliance with the
> W3C guidelines, it is not, in and of itself, to test for universal
> accessibility. The guidelines do not require, as an essential component,
> that the site be tested with a screen reader. On that basis we do not have
> the right to insist on using a screen reader (of whatever flavour). As
> Detlev says it is possible to check for conformance without using a screen
> reader, and not all Screen readers are compliant themselves.
>
> HOWEVER I feel that making the use of a screen reader "optional" is not
> sufficient. I sense the feeling of the group is that screen reader testing
> is important - so I wonder if we can call it's use  "recommended", or even
> "highly recommended" rather than "optional".
>
> Richard
>
> -----Original Message----- From: Detlev Fischer
> Sent: Wednesday, April 04, 2012 4:40 PM
> To: public-wai-evaltf@w3.org
>
> Cc: Eval TF
> Subject: Re: AW: Using AT for evaluation
>
> Hi list,
>
> I disagree with the proposed mandatory requirement to use a screen
> reader as part of WCAG-EM, for several reasons:
>
> 1. Inconclusive result from any one SR use
> 2. Difficulty of separating conformant behaviour from SR repair
> behaviour
> 3. Much higher professional prerequisite for conducting evaluations
>
> Let me quickly elaborate these three points.
>
> 1) Inconclusive result from any one SR use. Even for well known
> semantic constructs such as forms with legends or tables, SR
> implementation varies so widely that a well constructed form or table
> might be OK in one SR and can pose problems in another. What must
> surely count is whether content has been implemented correctly, not if
> some AT is able to use it as intended. For most cases, checks exist if
> semantics have been implemented according to spec or technique that do
> not require the use of a SR.
> To take the example of correct pronunciation of words mentioned by
> several: If a word is pronounced correctly will depend on the
> inclusion of the word in the SR dictionary and also, the options set
> by individual users. It will vary a lot across SR makes and versions.
> (I have met more than one SR user who willingly turned off other
> languages because he/he opted for speedy delivery over correct choice
> of language).
> Finally, leaving it open which SR to use makes results rather
> inconclusive. I think we cannot and should not mandate, say, the use
> of NVDA simply because it is free of charge. If anything, it should be
> the most frequently used SR (often JAWS) which costs quite a bit (see
> point 3).
>
> 2) Difficulty of separating conformant behaviour from SR repair
> behaviour. SR show a wide range of behaviour towards conformant and
> badly formatted content, which makes it difficult to rely on any one
> outcome in a conformance evaluation. Since the web is such a mess on
> the whole, SR are good in coping with that mess, but to varying
> degrees. Take a text rendered using Cufon. Jaws will read through it
> OK, other SRs will stutter, voiceover just reads image - image -
> image. So how does any one of these results help if we do not intend
> to cover a range (which I believe is impossible due to time and budget
> constraints)?
>
> 3) Much higher professional prerequisites for conducting evaluations.
> Requiring the use of a SR without making sure that it is used
> proficiently will lead to dubious results. SR have shortcuts and modes
> of operation that help expert users make sense of content. So if SR
> use becomes a mandatory part of the evaluation there should be a
> requirement that evaluators know their SR well enough to have
> meaningful results. This puts the bar a lot higher for folks who look
> for evaluation guidance but are not prepared for the time and effort
> to learn how to use a SR adequately.
> And adequate use will be difficult to define, especially if we do not
> mandate a SR.
>
> Having said all that, there are some points which are tricky to
> evaluate without a SR - for example, checking that when inserting
> content via DOM and setting the focus via a script, the SR focus will
> indeed move to the content inserted. But this touches on the question
> whether we should rely strongly on any observed actual AT
> implementation behaviour at any given point in time. If AT does not
> yet manage to deal adequately with correct code, the alternative would
> be to check whether implementation is done according to spec and put
> the burden on AT to catch up. Of course, checking for 'correct code'
> is difficult without being s scripting expert, it also raises the bar.
> This is a tricky one, and I am not really sure what's best here.
>
> Regards,
> Detlev
>
>
> On 4 Apr 2012, at 15:57, Michael S Elledge wrote:
>
>  Hi Everyone--
>>
>> My preference would be to require the use of a screen reader in
>> conducting an evaluation, either with or without a person with
>> disabilities. The option, in my mind, would be having it done by a  person
>> with disabilities.
>>
>> With the availability of free screen readers like NVDA, testers  ought to
>> be able to incorporate it in testing without incurring  unreasonable costs.
>> I realize this falls short of the ideal, which  is evaluation by a variety
>> of people with different disabilities,  but we've found it to be critical
>> in our testing for discovering  issues (such as pronunciation or
>> functionality) that would otherwise  be missed.
>>
>> I think another AT frequently used by persons with disabilities,  screen
>> enlargers like ZoomText, may be unnecessary in the testing  methodology, so
>> long as other methods are used to review an enlarged  screen. I bring that
>> up for discussion, because others may not agree  with me.
>>
>> Best regards,
>>
>> Mike
>>
>> On 4/2/2012 8:17 PM, Vivienne CONWAY wrote:
>>
>>>
>>> HI all
>>>
>>> I always use screen readers and am wary when clients say that they
>>>  don't need testing with AT because they don't have any disabled  users.
>>>  We never know the extent of employees' limitations - they  don't have to
>>> disclose everything.  Also, it often happens that  someone suffers an
>>> injury or illness while in employment only to  find that they can't use the
>>> system now that worked for them  previously.  The old 1 in 5 thought
>>> regarding disabilities applies  to those in employment as to the general
>>> public in their use of a  website.
>>>
>>> So... I would suggest that AT (at least the screen reader) be  strongly
>>> encouraged for all website/application testing.
>>>
>>>
>>> Regards
>>>
>>> Vivienne L. Conway, B.IT(Hons), MACS CT
>>> PhD Candidate & Sessional Lecturer, Edith Cowan University, Perth,  W.A.
>>> Director, Web Key IT Pty Ltd.
>>> v.conway@ecu.edu.au
>>> v.conway@webkeyit.com
>>> Mob: 0415 383 673
>>>
>>> This email is confidential and intended only for the use of the
>>> individual or entity named above. If you are not the intended  recipient,
>>> you are notified that any dissemination, distribution or  copying of this
>>> email is strictly prohibited. If you have received  this email in error,
>>> please notify me immediately by return email  or telephone and destroy the
>>> original message.
>>> ______________________________**__________
>>> From: RichardWarren [richard.warren@userite.com]
>>> Sent: Tuesday, 3 April 2012 2:28 AM
>>> To: Kerstin Probiesch
>>> Cc: 'Eval TF'
>>> Subject: Re: AW: Using AT for evaluation
>>>
>>> Dear Kerstin,
>>>
>>> I don't object too much if a "real user" (ie blind person) doesn't  use a
>>> screen reader to test the site - the most important thing is that a
>>> screen
>>> reader is used. Only a screen reader's audio output would demonstrate
>>> misspelled words and phone numbers (thanks Denis). You would not test
>>> without using a visual browser so you should also use an audio  browser.
>>>
>>> For an enclosed environment, such as an Intranet, it could be  possible
>>> to
>>> exclude testing with certain AT - only IF you know that the  technology
>>> will
>>> not be required. However this would then mean that the Intranet  could
>>> never
>>> be used by such a disabled person and could breach employment
>>> legislation.
>>>
>>> Richard
>>>
>>>
>>> -----Original Message-----
>>> From: Kerstin Probiesch
>>> Sent: Monday, April 02, 2012 3:51 PM
>>> To: 'Denis Boudreau' ; 'RichardWarren'
>>> Cc: 'Eval TF'
>>> Subject: AW: Using AT for evaluation
>>>
>>> Hi Richard, Denis, all,
>>>
>>> I also think that test with "real" (at least screen reader) users are
>>> important and that we should strongly recommend it but leave it
>>> optional. As
>>> I remember the discussion on our last telco there are two aspects:
>>>
>>> - testing with AT and
>>> - accessibility supported
>>>
>>> I think we have an intersection but also other aspects like: are
>>> technologies like PDF and Flash accessibility supported? Depending  upon
>>> the
>>> answer it will have probably different consequences for our  methodology.
>>> Also different use cases like internet and intranet (especially  when it
>>> comes to scripting for JAWS or other screen readers in closed
>>> environments)
>>> might have an impact. I'm thinking about if we could find for the  tests
>>> of
>>> intranets something better than just "optional" without reducing the
>>> audience of our methodology in whole.
>>>
>>> Best
>>>
>>> Kerstin
>>>
>>>
>>> Von: Denis Boudreau [mailto:dboudreau@**accessibiliteweb.com<dboudreau@accessibiliteweb.com>
>>> ]
>>> Gesendet: Montag, 2. April 2012 15:40
>>> An: RichardWarren
>>> Cc: Eval TF
>>> Betreff: Re: Using AT for evaluation
>>>
>>> Hi Richard,
>>>
>>> I would also like to weigh in with Richard here. All too often,  screen
>>> reader testing is considered a luxury that can be done without. I  am
>>> one of
>>> those who think that an evaluation cannot be considered complete  unless
>>> some
>>> screen reader testing has been conducted - and ideally, not only by a
>>> developer, but really by a "real" end user with a visual  impairment,
>>> using
>>> the assistive technology regularly. The same could be said of other  end
>>> user
>>> using other tools for other disabilities or limitations, but at the  very
>>> least, screen readers.
>>>
>>> There are always things that are brought up with AT testing that  cannot
>>> be
>>> flagged using only a checklist. Some of the things that come to  mind are
>>> links used for buttons that really should be coded as <button>, an
>>> overwhelming number of heading elements in a page (big menus and  fat
>>> footers
>>> anyone?) or quite obviously, any script that opens up or reveals
>>>  content in
>>> a page. I recently had big surprises simply with phone number  formats
>>> and
>>> how screen readers read them. That was another real eye opener (no  pun
>>> intended).
>>>
>>> This is why I tend to follow this pattern personally:
>>>
>>> * testing the web page with a screen reader
>>> * using an automatic checker for basic problems
>>> * running manual testing to complete the audit
>>>
>>> And whenever I am being offered the budget to do so, calling in a
>>> visually
>>> impaired friend or two who can push those tests much further that  my
>>> sighted
>>> self can push them.
>>>
>>> /Denis
>>>
>>>
>>>
>>>
>>> On 2012-03-29, at 6:48 PM, RichardWarren wrote:
>>>
>>>
>>> First – sorry I missed the last half of the teleconference – system
>>> crash.
>>>
>>> I wish to add to the discussion on using AT in evaluation. I  believe it
>>> is
>>> important to use a screen reader at the very least before  completing an
>>> evaluation. We do the normal stuff first (it is not fair to ask a  blind
>>> user
>>> to struggle if we already know that the site is impossible for  them).
>>> But as
>>> soon as we are happy that a site is reasonably good we always ask
>>> someone to
>>> check with their screen reader. Most times their comments re- inforce
>>> what we
>>>
>>> have found (often with better phrasing <G>). But just occasionally  they
>>> find
>>> something that our other systems do not pick up. For example the word
>>> “accesskeys” sound completely Russian unless it is written “access
>>>  keys”, or
>>> “access-keys”.
>>>
>>> I strongly believe that the audio output needs to be checked  properly.
>>> If
>>> you look at our outline procedure sent to this list on 26 Feb you  will
>>> see
>>> that we find someone who is new to the site to use a screen reader.  This
>>> approach gives us a high level of confidence in our final evaluation.
>>>
>>> Richard
>>>
>>> Technical Manager
>>> Website Auditing Limited
>>> http://www.userite.com
>>>
>>> This e-mail is confidential. If you are not the intended recipient  you
>>> must not disclose or use the information contained within. If  you have
>>> received it in error please return it to the sender via  reply e-mail and
>>> delete any record of it from your system. The  information contained within
>>> is not the opinion of Edith Cowan  University in general and the University
>>> accepts no liability for  the accuracy of the information provided.
>>>
>>> CRICOS IPC 00279B
>>>
>>>
>>>
>>>
>> --
>> Michael S. Elledge
>> Associate Director
>> Usability/Accessibility Research and Consulting
>> Michigan State University
>> Kellogg Center
>> 219 S. Harrison Rd Room 93
>> East Lansing, MI  48824
>> 517-353-8977
>>
>
> --
> Detlev Fischer
> testkreis - das Accessibility-Team von feld.wald.wiese
> c/o feld.wald.wiese
> Borselstraße 3-7 (im Hof)
> 22765 Hamburg
>
> Tel   +49 (0)40 439 10 68-3
> Mobil +49 (0)1577 170 73 84
> Fax   +49 (0)40 439 10 68-5
>
> http://www.testkreis.de
> Beratung, Tests und Schulungen für barrierefreie Websites
>
>
>
>
>
>


-- 
If you want to build a ship, don't drum up the people to gather wood,
divide the work, and give orders. Instead, teach them to yearn for the vast
and endless sea.
- Antoine De Saint-Exupéry, The Little Prince