Re: questions arising from AT vendor in regards to Mozillas implementation of the canvas subdom

On 2/20/12 11:45 PM, Dominic Mazzoni wrote:
> On Sun, Feb 19, 2012 at 5:42 PM, Charles Pritchard <chuck@jumis.com 
> <mailto:chuck@jumis.com>> wrote:
>
>     Those are the issues, not character boundary maps. Nobody passes
>     those to ATs. There is no AT that receives an integer array with
>     the location of every character on the screen.
>
>
> I don't think that's true - AT can call 
> IAccessibleText::characterExtents to query the exact bounding 
> rectangle of any arbitrary character on the screen, isn't that 
> basically the same thing?
>
> http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/interface_i_accessible_text.html#ab7f7ac52f4950d096856e59b0b008d1


The API does not generate an integer array. Someone using the API can 
build an array, as offsetAtPoint reports. OS X VoiceOver does show a 
character-by-character bounding box. It only does one character at a time.

We have setup a method to do caret browsing from inside the browser, but 
we have not setup a method to do it through pure IAccessible APIs.

We do not have a method to notify the scripting environment when the 
UA/AT changes the caret offset or selection. There's not enough 
scaffolding in DOM Events to handle it. Further, there are several "may 
or may not" conditions in IAccessible that may or may not hinder any 
kind of standardization. Richard has pointed out a few times that OS X 
And Windows handle selection and caret movement a little differently. 
There is still no stability in setting multiple ranges on the DOM: some 
browsers work well, some really just handle one range.

I've asked that onscroll be examined as a mechanism for notifying the 
author about the need for a redraw. It's similar to hooking into 
onresize. We may be able to cover a little more ground by having 
onscroll fired whenever the AT runs setCaretOffset or addSelection.

I don't see us having any success with items like scrollSubstringTo, 
offsetAtPoint and such. If it's absolutely vital, I'd like to see just 
how much feature we can pack into the onscroll event of a Canvas element.

There are two concerns I have here:
1. We can't be expected to do better than the rest of the DOM and the 
browser implementation. And we can only get the DOM updated when we can 
provide alternate use cases outside of Canvas.
2. We have been identifying real-world AT issues: while addressing the 
bulk of the issues in current ATs, some IAccessible interfaces may still 
be unimplemented. I believe setting priorities is a common practice, but 
I am worried that we'll be accused of a piecemeal approach.

I have not seen an AT working with an integer array of characterExtents. 
I've not seen offsetAtPoint used either, but it's something to be aware 
of. I'd imagine we'll see mouse events used instead of a direct call to 
offsetAtPoint. We have worked hard to enable virtual focus. Some of 
these APIs are akin to a virtual mouse and virtual ranges.

Though there have been experiments, I don't see any mainstream browsers 
that work with multiple mice. Chrome does work with virtual ranges in 
the form of the "find" tool. Character strings are highlighted when they 
match, they are shown in the scroll bar. They are not exposed to the 
scripting environment. The tool has unreliable results when the page 
flow is complex, such as overflow areas and non relative position boxes.

I have been able to capture "find next" through onscroll handlers of 
overflow containers.

It's more typically the case in an authoring tool that things like 
context menu and find are handled entirely by the scripting environment. 
This is another area where DOM simply is not feature-rich enough to 
close the gap.


-Charles