Re: Agreed recognition API?

On Fri, May 20, 2011 at 3:55 PM, Eric S. Johansson <esj@harvee.org> wrote:
> On 5/20/2011 9:34 AM, Bjorn Bringert wrote:
>>
>> Work with browser vendors to add better user-modification features to
>> browsers. I haven't yet seen any evidence that API-level support is needed
>> to allow user modifications.
>
> What's the nature of your disability?  it might help me to build a use case
> if I understood if it was your eyes or hands that were not functioning
> properly.
>
> User modification features in a browser are only a small fraction of the
> solution. you can't modify application without understanding the internals.
>  For example, can you give me all the information I need to write my own
> user interface for all of the Google apps? Can you tell me how to access all
> of the capabilities and data displayed by gmail without using a mouse?
> Admittedly I haven't looked very closely at these applications but it's the
> kind questions we need to answer for accessibility implementations.
>
> The API level support needed is an API to alter the data used by the
> application in order to perform a given task.  For example, I'm working on a
> tool for better for diabetes self-management. I'm building as a speech
> driven application. It looks kind of like a adding machine tape with a few
> field you can go back and correct. The code that performs all the various
> calculations for things like carbohydrates per meal per day etc. uses the
> same interface as the speech recognition code does for editing and
> correction. I will say something like "change carbohydrates <pause> 35" and
> the code will query the buffer look for the most recent carbohydrate line
> and then change and restore.
>
> How would you do this in a regular text buffer? Well you go to the end of
> the file, search backwards for the right pattern, then you have to do
> something magic to isolate the numeric field for the carbohydrate especially
> if the field can be modified by hand so spacing might be off or there might
> not be the right characters (nonnumeric) cluttering up the field. Basically,
> you can't do it with normal editor commands. But you can, if you have access
> to the buffer, get the line, clean the junk off the end, insert the new
> carbohydrate value and rewrite the line to the buffer.
>
> You can't do this unless you have a full-featured interface both at the
> speech recognizer level and public data access to internals of the
> application
>
> Harvesting data through the GUI is like cooking food from the heat off your
> exhaust pipe of your local internal combustion engine. It will work but it's
> really not a good idea. listening to people working with text-to-speech
> screen scraper's, I came understand just how bad an idea it is to hook
> accessibility device off of a GUI for folks not using speech recognition.
>  Direct access to data is faster and more reliable than twiddling menus,
> tabs and then tabbing over to some GUI control. Yes, that's what we have to
> do and it fails on a regular basis. In a browser, it gets even worse because
> of all the sort of wonderful Java/JavaScript/flash-based UIs that are all
> sparkly pretty pink ponies but cannot be queried consistently without visual
> input.
>
> the really sad thing is it easier to make all this information available
> than it is to build a half assed accessibility interface.  current model
> isn't working at least in my corner of the universe. The number of
> applications I use that are accessible easily with speech recognition have
> dropped over the past 15 years. Microsoft is the best in this regard and
> there are a lot of competitors for worst including a couple of companies you
> wouldn't expect. With a migration to web-based applications, if we don't
> deal with accessibility issues now and build them in correctly, it'll be
> another 15 years before we catch up to where we are today.

It sounds like you want general APIs for accessing data in web apps.
That sounds like a good idea, but doesn't really have very much to do
with speech as far as I can tell. To make this a bit more concrete,
perhaps you could propose some APIs that you would like web browsers
to implement?


-- 
Bjorn Bringert
Google UK Limited, Registered Office: Belgrave House, 76 Buckingham
Palace Road, London, SW1W 9TQ
Registered in England Number: 3977902

Received on Friday, 20 May 2011 15:07:41 UTC