Proposal: Canvas accessibility and a media querries approach for alternative content

Based on feedback from David Singer and James Craig I am modifying my
previous proposal (this is not spec. ready) to see if we can get on a
similar page to what is being proposed for accessible media:

<source media="accessibility(caption:no, audio-description:yes)"
src="A">

Here are some challenges:

1. The existing CSS 3 media queries does not support the accessibility
properties provided. A suggestion would be to make an exception to allow
the user agent to provide the information either internal to the browser or
via HTML 5 local data storage. The current HTML 5 specification requires a
direct match.

2. The source element require a source file vs. a local fragment. Should we
consider integrating code fragment vs. entire files? If we are going to
pull in files then they should be embedded in an IFrame in the browser for
security reasons.

3. We need a strategy for best fit of CSS media queries for accessibility.
We may not get a perfect match but if we can get close we should try to.

With these caviats in mind here is a draft of what I am proposing that I
would like to discuss Thursday this week:

1. The canvas element shall support zero or more child elements
representative of the default accessible rendering (if possible) for the
visual canvas modality and alternative modal renderings based on media
types and accessibility properties. The media types should be based on
those currently specified in the HTML 5 specification in addition to a
subset of those provided in the the IMS Global Consortium Access for All
Meta Data V3 default model. The default mode would be representative of the
shadow DOM. In many instances a more usable solution may require an
alternative.
2. Create new media elements for canvas that support different modalities
applicable for accessibility.

Define new media elements that may be contained in canvas: called visual

- default - represents a shadow DOM alternative for the default media type.
The intent is to allow the script author to bind accessibility information
using standard HTML elements, with ARIA semantics, to the visual rendering
in canvas. The default element would be a child of canvas but would in no
way be rendered by the browser. All rendering would be on the canvas
through the use of Script provided by the author. canvas should be
considered a form of visual media element.

- visual - this is an alternative visual rendering for canvas that would be
used if the default visual view is inaccessible based on the accessibility
requirements requirements submitted by the author,

Both default and visual support aria-activedescendant where the id refers
to a valid id in the child DOM.

In the future, and as the web expands in its capability provide other media
types such as <tactile> and <olfactory>.

3. Allow canvas to provide an audio media type as a child if the the user
prefers an audio alternative - such as a self voicing application for a
visually impaired user.

4. Modify the source attribute for the media attribute (included with the
source element) as follows as defined in the Access For All version 3
specification:

4a. adaptationtype - Provide one or more of the following adaptation types:

audiodescription
caption
signlanguage
highcontrast
transcript
alternativeText
longDescription
haptic
e-book

4b. ATInteroperable - A boolean that content support interoperability with
operating system platforms through the use of strong host language
semantics or the use of WAI-ARIA for web content. If a plug-in is provided
(say a Java applet) the application must support interoperability through
the use of a recognized Accessibility API.

4c. languageOfAdaptation - value determined by [ISO 639-2:1998]

4d. EducationLevelOfAdaptiaon - Define a range based on the adapation
(Integer)

4e. ControlFlexibility: Provide one or more of fullKeyboardControl,
fullMouseControl

5e. DisplayTranformability:One or more of the following:

fontSize
fontFace
fontWeight
letterSpacing
wordSpacing
lineHeight
foregroundColour
backgroundColour
cursorPresentation
highlightPresentation
layout
structurePresentation

6e. Harzard: One ore more of the following:


flashing, sound, olfactory, motionSimulation


We can use a subset of these.

5. View Selection should be based on a user preference in the browser based
on the mode and type preference supported in HTML 5. If nothing is set, the
shadow DOM is used for the accessibility mapping when provided. The user
agent should select the first view that fits the user's request. Platform
accessibility API may also be used to configure the view selection. Note:
is no alternative view is provided to meet the preferences the view reverts
back to the canvas + shadow DOM view. The shadow DOM is also optional.

6. The canvas tag shall include a script method to set the bounding
rectangle for a given valid id in the shadow DOM. This would allow
the browser to provide the bounding rectangle for the id in the shadow DOM
when it receives focus. The rectangle should be relative to the location of
the canvas element and would be converted to screen coordinates in the
accessibility API and would allow the canvas to move without changing the
location of the shadow DOM element. This will allow screen magnifiers to
determine the visible focus. (Ideally it would be nice to also do this via
XPath statements vs. id)
7. All allowable HTML elements in the shadow DOM shall support the tabindex
content attribute and ARIA attributes.
8. The shadow DOM should include all renderable HTML elements save standard
input controls. The reasoning being is these
controls are user agent managed in terms of the keyboard and would likely
conflict with canvas keyboard management.


The format would look something along the following lines:
<canvas>
   <default aria-activedescendant="foo">  /*Since canvas is visual this is
the default mode. It contains the shadow DOM*/
     <div role="toolbar">
        <div role="tab" tabindex="-1" id="foo">
        </div>
     </div>
   </access>
   <visual type="text"> /*Here we could place a long description or an
alternative aria-enabled widget*/
   </access>
   <audio>
   </access>
   <tactile> /*This is in access for all and we could lose it for now for
obvious reasons*/
   </access>
   <olfactory> /*This is in access for all and we could lose it for now for
obvious reasons*/
   </access>
   <visual type="signlanguage">
   </access>
</canvas>


----- Forwarded by Richard Schwerdtfeger/Austin/IBM on 12/13/2009 01:55 PM
-----
                                                                           
             Richard                                                       
             Schwerdtfeger/Aus                                             
             tin/IBM                                                    To 
                                       public-canvas-api@w3.org,           
             11/17/2009 10:30          public-canvas-api-request@w3.org,   
             AM                        david.bolter@gmail.com, Steven      
                                       Faulkner                            
                                       <faulkner.steve@gmail.com>,         
                                       Alexander Surkov                    
                                       <surkov.alexander@gmail.com>        
                                                                        cc 
                                                                           
                                                                   Subject 
                                       Canvas accessibility draft proposal 
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           



So, I am listening to all this and would like to make a draft proposal for
the HTML working group meeting this week (Paul Cotton made a request):

1. Modify the canvas element to support aria-activedescendant where the id
refers to a valid id in a shadow DOM.
2. The canvas element shall support zero or more child access elements
representative of alternative views. The access element shall have a mode
attribute. Whose values are based on a subset of the IMS Global Consortium
Access for All Meta Data V3 default. The default mode would be
representative of the shadow DOM. In many instances a more usable solution
may require an alternative.
3. The mode content attribute values for access would be:
- default - representing the shadow DOM for the complex visual rendering of
<canvas>
- textual - Here we could place a long description or an alternative
accessible UI renderable using HTML 5 standard markup - including ARIA.
- auditory - an audio alternative. Some users may prefer this
- visual - This is really an alternative for audio but it would address
things like signLanguage

Note: access for all also supports "tactile" and "olfactory". I would
recommend ignoring those for now. These could added in future releases of
HTML.

4. The access element would include a type attribute for which we should
take a subset of the list (from AccessForAll): audioDescription, caption,
e-book, signLanguage, highContrast, transcript, alternativeText,
longDescription, haptic}

The e-book may be rendered with a plug-in. We can choose a subset of these
if necessary. At this point this is for discussion purposes.

5. View Selection should be based on a user preference in the browser based
on the mode and type preference supported in HTML 5. If nothing is set, the
shadow DOM is used for the accessibility mapping when provided. The user
agent should select the first view that fits the user's request. Platform
accessibility API may also be used to configure the view selection. Note:
is no alternative view is provided to meet the preferences the view reverts
back to the canvas + shadow DOM view. The shadow DOM is also optional.

6. The canvas tag shall include a script method to set the bounding
rectangle for a given valid id in the shadow DOM. This would allow
the browser to provide the bounding rectangle for the id in the shadow DOM
when it receives focus. The rectangle should be relative to the location of
the canvas element and would be converted to screen coordinates in the
accessibility API and would allow the canvas to move without changing the
location of the shadow DOM element. This will allow screen magnifiers to
determine the visible focus. (Ideally it would be nice to also do this via
XPath statements vs. id)
7. All allowable HTML elements in the shadow DOM shall support the tabindex
content attribute and ARIA attributes.
8. The shadow DOM should include all renderable HTML elements save standard
input controls. The reasoning being is these
controls are user agent managed in terms of the keyboard and would likely
conflict with canvas keyboard management.


The format would be along the following lines:
<canvas aria-activedescendant="foo">
   <access mode="default">  /*Since canvas is visual this is the default
mode*/
     <div role="toolbar">
        <div role="tab" tabindex="-1" id="foo">:
        </div>
     </div>
   </access>
   <access mode="textual"> /*Here we could place a long description or an
alternative aria-enabled widget*/
   </access>
   <access mode="auditory">
   </access>
   <access mode="tactile"> /*This is in access for all and we could lose it
for now for obvious reasons*/
   </access>
   <access mode="olfactory"> /*This is in access for all and we could lose
it for now for obvious reasons*/
   </access>
   <access mode="visual" type="signlanguage">
   </access>
</canvas>

Rich Schwerdtfeger
Distinguished Engineer, SWG Accessibility Architect/Strategist

Rich Schwerdtfeger
Distinguished Engineer, SWG Accessibility Architect/Strategist