Re: Agreed recognition API?

On 5/20/2011 9:34 AM, Bjorn Bringert wrote:
>
> Work with browser vendors to add better user-modification features to 
> browsers. I haven't yet seen any evidence that API-level support is needed to 
> allow user modifications.

What's the nature of your disability?  it might help me to build a use case if I 
understood if it was your eyes or hands that were not functioning properly.

User modification features in a browser are only a small fraction of the 
solution. you can't modify application without understanding the internals.  For 
example, can you give me all the information I need to write my own user 
interface for all of the Google apps? Can you tell me how to access all of the 
capabilities and data displayed by gmail without using a mouse? Admittedly I 
haven't looked very closely at these applications but it's the kind questions we 
need to answer for accessibility implementations.

The API level support needed is an API to alter the data used by the application 
in order to perform a given task.  For example, I'm working on a tool for better 
for diabetes self-management. I'm building as a speech driven application. It 
looks kind of like a adding machine tape with a few field you can go back and 
correct. The code that performs all the various calculations for things like 
carbohydrates per meal per day etc. uses the same interface as the speech 
recognition code does for editing and correction. I will say something like 
"change carbohydrates <pause> 35" and the code will query the buffer look for 
the most recent carbohydrate line and then change and restore.

How would you do this in a regular text buffer? Well you go to the end of the 
file, search backwards for the right pattern, then you have to do something 
magic to isolate the numeric field for the carbohydrate especially if the field 
can be modified by hand so spacing might be off or there might not be the right 
characters (nonnumeric) cluttering up the field. Basically, you can't do it with 
normal editor commands. But you can, if you have access to the buffer, get the 
line, clean the junk off the end, insert the new carbohydrate value and rewrite 
the line to the buffer.

You can't do this unless you have a full-featured interface both at the speech 
recognizer level and public data access to internals of the application

Harvesting data through the GUI is like cooking food from the heat off your 
exhaust pipe of your local internal combustion engine. It will work but it's 
really not a good idea. listening to people working with text-to-speech screen 
scraper's, I came understand just how bad an idea it is to hook accessibility 
device off of a GUI for folks not using speech recognition.  Direct access to 
data is faster and more reliable than twiddling menus, tabs and then tabbing 
over to some GUI control. Yes, that's what we have to do and it fails on a 
regular basis. In a browser, it gets even worse because of all the sort of 
wonderful Java/JavaScript/flash-based UIs that are all sparkly pretty pink 
ponies but cannot be queried consistently without visual input.

the really sad thing is it easier to make all this information available than it 
is to build a half assed accessibility interface.  current model isn't working 
at least in my corner of the universe. The number of applications I use that are 
accessible easily with speech recognition have dropped over the past 15 years. 
Microsoft is the best in this regard and there are a lot of competitors for 
worst including a couple of companies you wouldn't expect. With a migration to 
web-based applications, if we don't deal with accessibility issues now and build 
them in correctly, it'll be another 15 years before we catch up to where we are 
today.

--- eric

Received on Friday, 20 May 2011 14:55:42 UTC