Re: Proposal: Canvas accessibility and a media querries approach for alternative content (Action Item 6 in the HTML Accessibility Task Force)

Rich Schwerdtfeger
Distinguished Engineer, SWG Accessibility Architect/Strategist


                                                                           
             Maciej Stachowiak                                             
             <mjs@apple.com>                                               
             Sent by:                                                   To 
             public-canvas-api         Richard                             
             -request@w3.org           Schwerdtfeger/Austin/IBM@IBMUS      
                                                                        cc 
                                       Ian Hickson <ian@hixie.ch>, James   
             01/24/2010 09:03          Craig <jcraig@apple.com>,           
             PM                        public-canvas-api@w3.org, HTML WG   
                                       <public-html@w3.org>,               
                                       public-html-request@w3.org, David   
                                       Singer <singer@apple.com>           
                                                                   Subject 
                                       Re: Proposal: Canvas accessibility  
                                       and a media querries approach   for 
                                       alternative content (Action Item 6  
                                       in the HTML Accessibility Task      
                                       Force)                              
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           





On Jan 24, 2010, at 3:06 PM, Richard Schwerdtfeger wrote:




      Regading response to system colors, I like this approach as well,
      however the CSS working group was unable to provide standardized
      system information on the OS to a web application to tell the author
      the author requirements.


The way the multiple higher contrast modes work on Mac OS X, applications
do not need to know anything about it. Neither Web applications nor native
applications. They just draw as normal and contrast settings are
implemented by the window server.


<rich>


This is not the case on all operating systems. The web author does not know
what the settings are on each OS platform. On Windows they define system
settings for colors on a subset of controls. This is inconsistent from the
Mac. We would need a consistent browser mapping. The solution you are
proposing does not address the cross platform issue.


</rich>



      Regarding two users operating off the same content. I would agree to
      a point. However, I feel you may not be considering more challenging
      use cases and users. For example, a cognitively impaired user may not
      be able to process the complexit of a particular application. Rather,
      they may need one that is much more simplified that has to gradually
      introduce content at the users request.


At Apple, we strive to design applications that are easy for anyone to use,
and we encourage out ISVs to do the same. We believe you will rarely find
apps with UI so complex that it needs to be scaled back. I would tend to
assume that an application with too much UI complexity is a UI design
failure, not just an accessibility failure. That being said, I understand
that overly-complex UI (that can be scaled back, but isn't normally) is
certainly a possibility. If I were building a native application for
Windows or Mac OS X, how would allow selection of a simplified UI for the
cognitively impaired?


<rich>


well, take the system preferences on the mac. You provide a broad range of
options to modify at one time. A cognitively impaired user may this too
much information to deal with. For more cognitively impaired users they can
only deal with one option at at time. An alternative UI might be to show
one option, say print and fax, and have a button to choose next setting.


So, no I do not believe Apple has addressed complexity for all users. I
believe that given your selected user set you have met the needs of those
users.


</rich>



      Also, for may a subway map is often provided on some sites. It is not
      something that is operated. However, a blind user will need to take
      the information in the subway map, enter in data and generate route
      to take on the page. At the same time a cognitively impaired user may
      not be able to handle the complexities of a subway map as there is
      too much information provided. It is for these reasons that the
      learning and education space has developed Access For All to deliver
      content that best fits all users based on their needs.


I would expect the UI to enter data and generate a route to be exposed to
all users. It's not an accessible alternative, it is simply an additional
feature, in my opinion. For the cognitively impaired, what do you believe
should happen when they ask for a subway map? How would a native
application do it?


<rich>


Sure that could be a feature. It may also be that users select alternative
content based on a preference. So, a native application (like the IPhone).
allows the user to choose alternative UIs in the map app. To get to it,
however, I have to select a page peel image at the bottom. This may not
exactly be intuitive. How does one know that there is a list driving
direction on the alternative page? I mean this is the web where there are
no conventions for this. So, we are face with the user having to poke
around and find the alternative UI for the driving directions. It is in
fact a UI feature like you suggest. It is much easier to allow the user to
have a programmatic way of choosing the text equivalent for the map in much
the same way it is easier to select the alternative content for the subway
map.


I think we have an acceptable middlegrown here based on the last canvas
meeting which would be to define HTML 5 attributes that can be married to
CSS to choose the alternative automatically, This also allows for the
fallback content or other content  in a web page, or features if you will,
to be rendered based on a user defined style sheet.


What I am suggesting is to provide a way for the user to specify the
alternate content automatically.


</rich>



      > I can't imagine we would want a different or more complex model for

      > Web applications than we do for native applications. I can imagine
      > that in some rare cases, modality-specific UI may be truly
      > necessary. But encouraging that as a common or default model would
      > not be in line with our approach to accessibility. Our built-in
      > accessibility features in Mac OS X and iPhone have been praised for

      > their polish and ease of use, and we would not want to take a step
      > back from that on the WEb.

      A much as a bigot I am about Apple products I think you need to take
      a broader perspective than you are. So, please let me enlighten you
      further by providing you with a slightly different scenario: Even web
      content providers scale down content to fit devices to deliver the
      pertinent information based on the display real estate. This makes
      the content easier to use. Yet, in doing so two people cannot sit
      side by side (desktop and IPhone user) and directly operate the same
      application side by side. These principles are no different than
      providing reduced content to a cognitively impaired user. Making all
      that possible meant knowing the capabilities of the device the
      content is targeted at.


When content authors choose to do this, there is no mechanism for automatic
selection. Rather, they rely on being provided the information about the
user's browsing environment through HTTP headers, scripting interfaces, and
CSS media queries, and may use that to choose appropriate content. Other
sites give users the explicit choice in the page to select a "mobile" or
"full" version. But we didn't add an <iphone> tag for iPhone-specific
content, or anything like that. And the vast majority of sites do *not*
have an iPhone-specific or mobile-specific version. Much of the power of
the iPhone's browsing experience comes from the ability to handle content
that is not specially adapted.

Now, I think it's fine to provide the analogous mechanisms to detect the
user's accessibility needs and possibly adapt the content. In fact, CSS
media queries can already provide much of what is needed declaratively, and
using media().matchMedium(...) allows media queries to be evaluated from
script. I'm guessing we don't disagree on this part.


Here's where I think you and I part ways:

1) I disagree that an additional declarative mechanism specific to <canvas>
and only <canvas> is needed. The general mechanism of CSS media queries can
do the job. I don't see how a <canvas>-specific mechanism adds anything.

<rich>
I think we have a vehicle to address the middle ground from the last post
that through the use of attributes that could also make use of straight CSS
for selection with less of an impact on the spec. The group would like your
feedback. We had agreement from both Microsoft and Mozilla.
</rich>


2) I disagree that it should be our default assumption that much or most
content rendered with <canvas> will need multiple versions of the content.
I believe that many UI elements built out of canvas will be adaptable
simply based on their accessibility subdoms, just as most UI elements built
without using canvas. The primary approach that we take in native
application accessibility, and what I believe should be the primary
approach for Web applications, is to make a single interface that is
adaptable for a wide range of needs. I acknowledge there may be exceptions,
but they should not be the primary design focus of our accessibility
approach. Most developers are very willing to adapt their content with an
accessibility API, few are willing or able to implement multiple separate
UIs.

<rich>
Maciej, I deal with hundreds of products across SWG and the reality is that
in some cases you need alternative content. I do agree with that in most
cases an accessibility API adaptation (our shadow DOM approach) is
sufficient. I get involved with things like complex visualization for
Information Management of large amounts of data.  One example is a tag
cloud representation of large amounts of data. The larger fonts represent
the most frequently occuring tags. Having the user render that as an
accessible DOM to a blind user and have them process the data based on Font
size is unrealistic. An alternative rendering of tags and occurrences is
much easier to process. There are other examples.
</rich>


3) I don't think it really makes sense to have <canvas> be the locus of
selecting alternate interfaces. Many application-like interfaces use
<canvas> plus something else. In that case, having alternate renderings for
the <canvas> only does not properly provide the alternative content. You
may need to switch more than just the <canvas>. Or you may still want a
<canvas> but with something different drawn on it. How is selecting from
multiple <canvas> fallbacks going to help with that situation? For that
matter, what if you have a Web application that is too complicated for the
cognitively impaired, but it's made using a big pile o' <div>s rather than
<canvas>? Doesn't it have the same need to provide alternatives? We should
be designing mechanisms that can apply to any Web application, regardless
of what elements it uses to implement itself. CSS Media Queries and their
associated scripting API can fit that bill. I don't think a
<canvas>-specific mechanism for selecting alternate content works.

Thus, my conclusion is that we should focus on a general mechanism for
choosing alternate content, which authors can optionally choose to use. We
should not be designing something overly <canvas>-specific, nor should we
make alternate content selection the focus of how <canvas> accessibility
works.

<rich>
I agree with this and I think the solution we are now proposing which
should address this. It is less specific to canvas in its way of providing
alternative content. Thanks to both you and Ian for pushing harder on this.
</rich>



      The same should be the case for a user's needs.

      Going forward, we are going to find that making content more usable
      will also require adaptation to the environment the user is operating
      in. This means lighting conditions, background noise, etc. Content
      will need to be context aware and adaptable.


I could imagine operating systems or devices adapting for lighting
conditions or background noise (for example by adjusting brightness or
volume automatically). Every individual application doing this does not
strike me as a likely scenario.


<rich>


So, for example. Significant background noise may be a trigger to an
application that they should turn on captioning. The point at which
background noise is a problem for the user is based on the user's hearing
capability. There needs to be a vehicle that says "captioning on now"
whether the OS determines it or the application based on information
retrieved from the browser.


</rich>


      I think the IPhone is a fantastic tool but even Apple can improve on
      what it is doing. This is not a step backward for Apple.



Regards,
Maciej