Re: Proposal: Canvas accessibility and a media querries approach for alternative content (Action Item 6 in the HTML Accessibility Task Force)

On Jan 24, 2010, at 3:06 PM, Richard Schwerdtfeger wrote:

> 
> Regading response to system colors, I like this approach as well, however the CSS working group was unable to provide standardized system information on the OS to a web application to tell the author the author requirements. 
> 
The way the multiple higher contrast modes work on Mac OS X, applications do not need to know anything about it. Neither Web applications nor native applications. They just draw as normal and contrast settings are implemented by the window server.

> Regarding two users operating off the same content. I would agree to a point. However, I feel you may not be considering more challenging use cases and users. For example, a cognitively impaired user may not be able to process the complexit of a particular application. Rather, they may need one that is much more simplified that has to gradually introduce content at the users request.
> 
At Apple, we strive to design applications that are easy for anyone to use, and we encourage out ISVs to do the same. We believe you will rarely find apps with UI so complex that it needs to be scaled back. I would tend to assume that an application with too much UI complexity is a UI design failure, not just an accessibility failure. That being said, I understand that overly-complex UI (that can be scaled back, but isn't normally) is certainly a possibility. If I were building a native application for Windows or Mac OS X, how would allow selection of a simplified UI for the cognitively impaired?

> Also, for may a subway map is often provided on some sites. It is not something that is operated. However, a blind user will need to take the information in the subway map, enter in data and generate route to take on the page. At the same time a cognitively impaired user may not be able to handle the complexities of a subway map as there is too much information provided. It is for these reasons that the learning and education space has developed Access For All to deliver content that best fits all users based on their needs. 
> 
I would expect the UI to enter data and generate a route to be exposed to all users. It's not an accessible alternative, it is simply an additional feature, in my opinion. For the cognitively impaired, what do you believe should happen when they ask for a subway map? How would a native application do it?

> > I can't imagine we would want a different or more complex model for 
> > Web applications than we do for native applications. I can imagine 
> > that in some rare cases, modality-specific UI may be truly 
> > necessary. But encouraging that as a common or default model would 
> > not be in line with our approach to accessibility. Our built-in 
> > accessibility features in Mac OS X and iPhone have been praised for 
> > their polish and ease of use, and we would not want to take a step 
> > back from that on the WEb.
> 
> A much as a bigot I am about Apple products I think you need to take a broader perspective than you are. So, please let me enlighten you further by providing you with a slightly different scenario: Even web content providers scale down content to fit devices to deliver the pertinent information based on the display real estate. This makes the content easier to use. Yet, in doing so two people cannot sit side by side (desktop and IPhone user) and directly operate the same application side by side. These principles are no different than providing reduced content to a cognitively impaired user. Making all that possible meant knowing the capabilities of the device the content is targeted at.
> 
When content authors choose to do this, there is no mechanism for automatic selection. Rather, they rely on being provided the information about the user's browsing environment through HTTP headers, scripting interfaces, and CSS media queries, and may use that to choose appropriate content. Other sites give users the explicit choice in the page to select a "mobile" or "full" version. But we didn't add an <iphone> tag for iPhone-specific content, or anything like that. And the vast majority of sites do *not* have an iPhone-specific or mobile-specific version. Much of the power of the iPhone's browsing experience comes from the ability to handle content that is not specially adapted.

Now, I think it's fine to provide the analogous mechanisms to detect the user's accessibility needs and possibly adapt the content. In fact, CSS media queries can already provide much of what is needed declaratively, and using media().matchMedium(...) allows media queries to be evaluated from script. I'm guessing we don't disagree on this part.


Here's where I think you and I part ways:

1) I disagree that an additional declarative mechanism specific to <canvas> and only <canvas> is needed. The general mechanism of CSS media queries can do the job. I don't see how a <canvas>-specific mechanism adds anything.

2) I disagree that it should be our default assumption that much or most content rendered with <canvas> will need multiple versions of the content. I believe that many UI elements built out of canvas will be adaptable simply based on their accessibility subdoms, just as most UI elements built without using canvas. The primary approach that we take in native application accessibility, and what I believe should be the primary approach for Web applications, is to make a single interface that is adaptable for a wide range of needs. I acknowledge there may be exceptions, but they should not be the primary design focus of our accessibility approach. Most developers are very willing to adapt their content with an accessibility API, few are willing or able to implement multiple separate UIs.

3) I don't think it really makes sense to have <canvas> be the locus of selecting alternate interfaces. Many application-like interfaces use <canvas> plus something else. In that case, having alternate renderings for the <canvas> only does not properly provide the alternative content. You may need to switch more than just the <canvas>. Or you may still want a <canvas> but with something different drawn on it. How is selecting from multiple <canvas> fallbacks going to help with that situation? For that matter, what if you have a Web application that is too complicated for the cognitively impaired, but it's made using a big pile o' <div>s rather than <canvas>? Doesn't it have the same need to provide alternatives? We should be designing mechanisms that can apply to any Web application, regardless of what elements it uses to implement itself. CSS Media Queries and their associated scripting API can fit that bill. I don't think a <canvas>-specific mechanism for selecting alternate content works.

Thus, my conclusion is that we should focus on a general mechanism for choosing alternate content, which authors can optionally choose to use. We should not be designing something overly <canvas>-specific, nor should we make alternate content selection the focus of how <canvas> accessibility works.

> The same should be the case for a user's needs.
> 
> Going forward, we are going to find that making content more usable will also require adaptation to the environment the user is operating in. This means lighting conditions, background noise, etc. Content will need to be context aware and adaptable. 
> 
I could imagine operating systems or devices adapting for lighting conditions or background noise (for example by adjusting brightness or volume automatically). Every individual application doing this does not strike me as a likely scenario.
> I think the IPhone is a fantastic tool but even Apple can improve on what it is doing. This is not a step backward for Apple.
> 


Regards,
Maciej