- From: Benjamin Hawkes-Lewis <bhawkeslewis@googlemail.com>
- Date: Sun, 16 Jan 2011 16:40:32 +0000
- To: Richard Schwerdtfeger <schwer@us.ibm.com>
- Cc: Frank Olivier <Frank.Olivier@microsoft.com>, "chuck@jumis.com" <chuck@jumis.com>, Cynthia Shelly <cyns@microsoft.com>, "david.bolter@gmail.com" <david.bolter@gmail.com>, "janina@rednote.net" <janina@rednote.net>, "oedipus@hicom.net" <oedipus@hicom.net>, "public-canvas-api@w3.org" <public-canvas-api@w3.org>, public-canvas-api-request@w3.org, "public-html-a11y@w3.org" <public-html-a11y@w3.org>, Shawn Warren <swarren@aisquared.com>, Tim Lalor <tlalor@aisquared.com>
On Sun, Jan 16, 2011 at 4:35 PM, Benjamin Hawkes-Lewis <bhawkeslewis@googlemail.com> wrote: > On Fri, Jan 14, 2011 at 5:37 PM, Richard Schwerdtfeger > <schwer@us.ibm.com> wrote: >> I think the IE team needs to look at the following: >> >> GTK client on HTML5 canvas: >> http://blogs.gnome.org/alexl/2010/11/23/gtk3-vs-html5/ >> >> noVNC - VNC client using HTML5 Canvas: >> http://kanaka.github.com/noVNC/ [snip] > I think canvas does have a potential role to play in addressing the > remote desktop environment use case, but I don't think the proposed > features help. I'll discuss that use case in a new thread. As far as I know, VNC and similar protocols tunnel pixel and sound in and keyboard presses, mouse events, and sound out. They do not tunnel in the information necessary to drive remote assistive technology or tunnel out the commands necessary to drive remote environments programmatically. Today's solutions to the problem seem to include: 1. Firing up a dedicated server on the remote system to tunnel accessibility traffic back and forth directly between the remote system and the local assistive technology. This is how JAWS provides remote access. 2. Firing up a remote instance of the assistive technology and pulling in sound and braille events into devices on the local system rather than trying to interact with the remote system via local AT. This is how Window-Eyes provides remote access to Windows systems. Generalizing the first model to the web is problematic. I can imagine producing a standardized accessibility API with mappings to desktop APIs. But Windows accessibility depends on a huge series of hacks that depend on specialized interpretations of the accessibility tree, video intercepts, specialized APIs for Office and the DOM, and so on. Consequently, simply converting the UI Automation accessibility tree to a standard format and then pushing it over the network would be fairly useless to client applications. Expecting developers of remote access servers to map not only the accessibility tree, but also all the other APIs that Windows accessibility applications depend on, into a standardized accessibility tree seems unrealistic. We could forget about supporting todays' Windows applications and move forward on the basis that desktop applications should make use of system widget sets, and when they make custom widgets use system accessibility features, rather than forcing AT to depend on hacks. But even then we're talking about a huge, politically complicated, technically tangled project. RTE support is a tiny part of the puzzle. On the other hand, Freedom Scientific's Remote Access software probably could work just as well alongside visuals served by a webapp VNC client as it does beside a desktop VNC client, and could depend on the remote server for selection and caret information. RTE features for HTML5 canvas don't really help here. In the short term, I think it would be more realistic to webify the second model. HTML has <canvas> for bitmapped visual output, <audio> for sound output, DOM event support for keyboard and mouse events. There has been discussion around adding features for visual and audio input (in particular for webcams) and other devices such as bluetooth. If we added support for braille output (a <braille> element maybe?), then webapp clients could register themselves as braille output devices for remote systems via remote access servers. Again, RTE support for canvas doesn't really help here. Thoughts? -- Benjamin Hawkes-Lewis
Received on Sunday, 16 January 2011 16:42:37 UTC