Re: Conformance evaluations of IBM Home Page Reader

claws@us.ibm.com wrote:
> 
> Jon and Ian,
> 
> I was going to post this note on the UA listserv, but I wasn't sure if the
> attachments would work or if you liked attachments on the listserv. Feel free to
> redistribute this information.

I'm  sending this reply to the UAGL list.
 
> Over the past year and a half, IBM Special Needs has evaluated Home Page Reader
> several times against different drafts of the UA Guidelines to identify user
> requirements for new releases of HPR and to assess  our conformance to the
> guidelines. We have also assessed other user agents and assistive technology
> against the guidelines as part of our competitve analysis. Each time we have
> found that the guidelines were subject to interpretation.

That is, I believe, inevitable. The challenge for the Working Group
is to convey a requirement that is precise enough to be understood
and verifiable, but flexible enough to allow developer interpretation
and alternative techniques that still allow one to satisfy the
requirement.

> As we evaluated each
> guideline against the HPR product and other user agents, we asked ourselves the
> following questions:

These and other questions might be the basis of a FAQ for the
UA GL when they approach Recommendation.
 
> 1. What are some techniques and solutions that could be implemented to comply
> with each guideline? What are the best techniques? (We know the techniques
> document is attempting to address this question.)

Right, and we should rank them, even if we don't use a formal
system for doing so.

> 2. Do the techniques implemented in HPR and other user agents comply 100% with a
> guideline or are the implementations only sufficient or simply inadequate?

Today, our only options for conformance to a checkpoint
are "yes", "no", "n/a". I don't know whether we need more levels:
"yes, significantly"; "yes, a little", etc. I think that we cannot
eliminate all subjectivity. In the end, people should be able
to say "Yes, but not very much" and still count that as a Yes.

However, if there are checkpoints where you feel that conformance
is easy but no accessibility issues are addressed, please let us know.

> 3. If HPR/other UAs do not comply with the guideline at all, is the guideline
> important as a user requirement for the target market?

Testing, review, and experience with the guidelines will show what
is important, superfluous, under-specified, and misunderstood. 
In part, these reviews are bringing weaknesses and unclear points
to light and thus are invaluable before we go to Recommendation.

> 4. If you use HPR/other UAs with other mainstream browsers and assistive
> technology, does the total browser solution comply with the guideline?

No. The goal of the current conformance statement was to allow a piece
of software to conform without depending on another piece of software.

Which checkpoints do you think suggest that software in tandem
might conform but not individually?


> We are in the process of doing a conformance assessment for HPR 2.5 against the
> August 9, 1999, draft of the UA guidelines, and we will send that to you in a
> week or two. We hope that these evaluations provide some insight into how
> developers are and will be able to use the UA guidelines. Thanks.

Already received [1]. I have not reviewed the report yet, but I'll
respond to it on the list.

Thank you Cathy,

 - Ian

[1] http://lists.w3.org/Archives/Public/w3c-wai-ua/1999JulSep/0234.html


-- 
Ian Jacobs (jacobs@w3.org)   http://www.w3.org/People/Jacobs
Tel/Fax:                     +1 212 684-1814

Received on Friday, 27 August 1999 16:17:09 UTC