Re: HTML 5 Canvas Accessibility Call on January 10

On 1/16/2011 2:02 PM, Benjamin Hawkes-Lewis wrote:
> On Sun, Jan 16, 2011 at 5:34 PM, Charles Pritchard<chuck@jumis.com>  wrote:
>> On 1/16/2011 8:35 AM, Benjamin Hawkes-Lewis wrote:
>>> The idea is that people use canvas to build RTEs to solve a series of
>> Fine with that: provided the opportunity cost is actually analyzed.
> Sure.
With several vendors categorically -against- supporting canvas-based RTE,
I hope there is room for cost analysis.

While I'm comfortable with vendors stating, that company wide, they are 
discouraging
the use of Canvas RTE; it seems to be an overstretch of authority to 
actively discourage
the rest of programming world, by intentionally obscuring and/or 
omitting vital properties.
>> As you know, I'm a big fan of serialization.
>>
>> Currently, contentEditable is something of a black box, with a few features
>> thrown in, borrowed from Microsoft's early work on the feature.
> "black box" in what sense?
Opaque. In relation to the scripting environment: many input behaviors 
are UA-specific, and the
scripting environment is not notified. I believe there's consensus that 
contentEditable could use
far more development, in the specs process.

>>> We're dealing here with at least four broad use cases:
>>>
>>> Â  Â 1. Collaborative coding in an integrated development environment in
>>> Â  Â  Â  the cloud (Bespin).
>>> Â  Â 2. Enabling users to enter rich text in multiple languages (Google
>>> IME).
>>> Â  Â 3. Using a cross-platform widget set to write an application once and
>>> Â  Â  Â  deploy it in multiple environments (GTK client).
>>> Â  Â 4. Providing access to a remote desktop environment (VNC client).
>> Yes, some of the use cases are a little artificial, as my original intent
>> was more technical:
> I don't understand the distinction you're making, but I think we should
> discount "artificial" use cases and focus on real ones like the above.
These are particular use-cases: we're talking about actual, live software.
Broad use cases are: serialization, unit testing, a11y WYSIWYG authoring.

Canvas-based RTE as a valid use-case has been rejected, by several vendors.
They seem to negate item #1 and #2, as well as one of my software 
projects (a 10k LOC JS project).

Here's a use case to help out, which uses img instead of canvas:

Scanned books (ex. Google Books) . It's a raster based document service 
which would benefit
from integration with mouse and cursor selection, magnification, and 
other services.

>> and so that it may interface with AT.
> I think there's general agreement that web RTEs should be accessible.
It's attached with a rider, that web RTEs should be implemented by the UA.
We're all aware that it's the responsibility of the OS to implement 
accessibility,
and the UA to hand-off the proper data to IAccessible APIs.

There's push-back here, discouraging authors from sending a11y 
information through the UA,
for fear that they will misuse the interfaces.

>> I'd like the RTE to work to the standard that users expect.
> Which standard is that?
The quality that users expect from a rich text editor; multiple font styles,
mouse, keyboard and ink input.

>> 1. This started from technical use cases: serialization, unit testing.
> Re unit testing: I cannot find the previous discussion of unit testing
> in this context. Would you mind linking to it? Has anyone analyzed what
Hironori Bono mentions automated IME compliance testing.
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-January/029776.html

More and more, unit tests are handled in the scripting environment, but done
so in a "trusted" zone, where proprietary methods can be used (such as 
Mozilla drawWindow).

That becomes a pain when you are targeting multiple platforms.

> Re serialization: use cases are about user needs. "Serialization" is a
> feature but it's not a use case. I could say I need to serialize the
> names of my cats, but that's not a use case.
It's a reasonable use case with mice, and other input devices.

> If by "technical use case", you mean a "use case" that is not directly
> tied back to meeting end-user requirements, I think we should discount
> such "use cases" in order to make sure we're solving real user problems.

This seems to be another rich point in our communication.

APIs are consumed by ATs and app developers.

My end-user does not interface with the scripting API.
I'm the consumer here.

I'm on these lists, because there are deficiencies in the browser 
standards, which
impact my ability to develop software targeting end-users.

>> The "cloud" IDE is just something to point to.
> Are you saying we should ignore this use case? If so, I disagree.
>
I'm saying that it's a very general example, and hasn't received much 
traction,
especially in the context of Canvas RTE.

I think it's a valid use case, and should not be ignored.
>> 2. Enabling authors to develop rich text alternatives. Yes, it may be
>> for transliteration, it could be for handwriting recognition -- it
>> could be for an artificial language, or it could be a UI for an AT
>> solution. It could also be for design: think Adobe Illustrator, where
>> the user can edit text in place.
> These all sound like use cases, although obviously we'd need to
> elaborate them in detail to work out what features would support them
> best and how much urgency to assign to them.
>
> Transliteration: Not sure what you mean or how the proposed features
> help.
See here:
http://lists.w3.org/Archives/Public/www-international/2010OctDec/0095.html

> Handwriting recognition: I don't understand why we need the proposed
> features for handwriting recognition.
Selection of text that's been entered, interaction between the keyboard 
and displayed text.

> Artificial languages: I can't see implementors treating this as a strong
> motivation for adding new features. I think the web stack already
> supports outputting these using a series of SVG, canvas, or img
> elements.
When I mention existing languages that aren't yet supported, the 
response is,
we'll support them, or use Web Fonts.

There are non-standard glyphs, languages, dialects, and especially so 
when it comes to accessibility.

Yes, SVG like canvas, can support the display of arbitrary glyphs, and 
SVG Fonts tries hard to make it more efficient,
but often, SVG suffers the same inefficiencies as VML, when the 
situation requires falling back to multiple <use> elements.
> UI for an AT solution: Not sure what you mean or how the proposed
> features help.

Accessibility could mean a user interface that works based on the 
pressing of
a minimal set of buttons. Alternate user interfaces exist for the 
mentally disabled,
physically handicap, visually impaired.

These alternative UIs are created to serve a need. That's generally why 
software is created.

The current path, in working against author-developed IMEs means
requiring plug-ins to the UA, unnecessary system permissions, and forcing
developers to wait for the UA to target minority user bases.

> Vector graphics editing: Given that Adobe Illustrator can output SVG,
> it seems like SVG is already a great fit for this, so I don't see why
> we need the new features to support this.
Adobe Illustrator is a great solution for SVG editing. It's an even 
better editor for PDF editing.

As a software developer, I design editors that run in the browser.

Aviary designed a reasonable SVG editor. It's not sufficient as a vector 
editor.
The SVG spec is not sufficiently implemented, either. When it comes to 
real world use.

>> I've brought up legibility, to Mozilla, and been denied.
>> I've brought up text alignment (putting two fonts on the same baseline), and
>> been denied.
> Hammer vendors have their own ideas about the purposes of hammers,
> which don't include turning them into spanners.
Meaning what?

I put forward a specific example.
Your metaphor does not speak to the merit of the solution, and says 
little about the example.

Are you saying that fillText should be used to "hammer" text, but not 
"span" it?

>> It's a miracle that the Shadow DOM ( drawFocusRing ) was allowed
>> through. That said, WebKit is still lagging on allowing focus to
>> elements in the shadow DOM. Chromium developers are having broad
>> issues with JAWS, with "non-focusable" elements.
> I haven't heard about these difficulties. Do you have any links to
> further information?
http://www.chromium.org/user-experience/screen-reader-support
https://bugs.webkit.org/show_bug.cgi?id=50126

>>> The reasons given for creating an IME in canvas are all /stronger/
>>> reasons for implementing good IMEs directly in the browser:
>>>
>>>
>>> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-January/029759.html
>> There's a basic fallacy in that sentiment: that all IMEs should be written
>> directly in the browser.
> My sentiment does not depend on such a strong claim. I'm saying the
> reasons mentioned are all better reasons to improve browser (and system)
> IMEs than to require browser developers to add features for authors to
> roll their own.
We've been discussing exposing existing features. The issue has not come 
to an analysis
as to how much work would be required of browser developers.

We're stuck on the "benefits" part of the cost / benefit analysis.

My hope is that by exposing more data for hand-rolled IMEs, we'll see some
creativity emerge from the general Internet.

It's creativity from those prototypes which will inspire improvements in 
UA IMEs.

>> One of the purposes of the scripting environment is to allow authors to
>> develop alternatives.
> It's a way authors use the environment, but it's inefficient to require
> the diversion of development resources that would be better spent
> elsewhere.
The cost hasn't been weighed, and I'm coming up against push-back 
against the very
idea, not the cost.

I can't send in a patch to Mozilla, for some items, because the bugs 
themselves have
been rejected.
>> I've said it many many times -- language is not static, and programming is a
>> form of expression.
>>
>> Restricting IME to that which is supplied ONLY by browser vendors goes
>> against free speech.
> Authors and browser vendors are not different types of people, they are
> just people (not that infrequently the same people) playing different
> roles.
Browser vendors develop virtual machines, and part scene graphs, to display
that content to end users. They expose OS-level and user profile APIs to 
authors.

Authors consume those APIs to create programs which run across browsers 
(various versions and vendors)
and other supporting hosts.

> We are talking about what constraints we can reasonably place upon
> conforming HTML5/ARIA implementations. We cannot ultimately constrain
> what developers do on the open web, so "Restricting IME to that which is
> supplied ONLY by browser vendors goes against free speech" is a
> strawman.
Multiple vendors have suggested as much: that text IME should remain in 
the realm of the UA.
As for the reasonableness of these constraints: isn't that something to 
be borne out by actual
analysis of the costs? By and large, we're not talking about developer 
time here, we're talking
about browser vendors intentionally obfuscating access to necessary data.

Any addition to the specs should be carefully weighed, as it does have 
an impact on vendors.
I agree with that, and I understand that they have limited time and 
resources. I do my best
to find the least costly solutions to propose to vendors, to address 
defects in their product lines.

>>> I think canvas does have a potential role to play in addressing the
>>> remote desktop environment use case, but I don't think the proposed
>>> features help. I'll discuss that use case in a new thread.
>> Yes, current remote desktop environments use pixels.
>>
>> There are some basic implementations of Flash using Canvas: they provide
>> something of a demonstration of using an API, instead of pixels, for
>> display.
> Today's desktop environments do not draw using Flash, so I don't see the
> relevance.
I don't think we're communicating well on this topic.

I do think it's a significant help, to expose a11y apis through remote 
desktop access.

My example was intended to say that: though VNC, and many remote desktop 
programs just
send pixels down the pipeline, there are use cases where bytecode / APIs 
are sent. In either case,
sharing data, about elements, when available, is  a positive.

>>> The web standards community did advise against *depending* on JS; that
>>> was good advice at the time and remains good advice today. There are use
>>> cases where it's impractical advice; text entry isn't one of them but
>>> real-time gaming, instant messaging, and remote desktop access are.
>> WebApps have a very different use case than hypertext.
> I suspect we're going to keep disagreeing on that point. :)
Something special happens when you go from writing a hypertext blog,
to writing a rich web application. Apart from the complexity; you start 
writing UI widgets,
memory and state management, and so on.

Hypertext is a word doc,
Apps is a word doc viewer / editor.

>> It's not reasonable to expect all app developers to have a fully
>> accessible non-graphical UI to their application.
> I think where the technology exists to create multiple representations
> of the same application flow from the same blueprint, it's not an
> unreasonable user expectation. I favour expanding the parts of the web
> stack that support that ability. That's not the same as saying that
> users have an ethical right to *demand* such alternate representations.
This is one of the highlights of ARIA, and what brought to its doorsteps:
we've redundant implementations of many UI widgets, using HTML, and 
using Canvas,
and ARIA works wonderfully to define their roles.


>> This is [if I'm not going to far], the right way to do a canvas RTE:
>> <canvas><div contentEditable>Editable Content</div></canvas>
>>
> If there's a right way to do it, that's probably it, but it still
> requires bolt-on accessibility so it's still subject to the historical
> lesson of Flash.
Flash doesn't use the same DOM as the scripting environment, when hosted.
Seems apples to oranges in that regard.

Yes, Flash is more complicated to program than adding HTML tags and css,
and authors have made more mistakes, in that space.

I'm not comfortable with the implication that historically, authors are 
"incompetent",
be they Flash authors or other.

>> The issue at hand, is that it can not be discussed, without being
>> chilled by authority figures who have no experience nor stake in the
>> matter.
> I think "chilled" is the wrong word to use for a situation where
> hammer vendors are not keen on making the spanner you want.
We're not talking about the creation of new tools, not in these proposals.
This is about exposing existing APIs.

I'm not keen on hammer vendors telling me I need to wear a blindfold
to use their product.

>> Opportunity costs are not being weighed.
>>
>> Proposals which cost very little are not rationally analyzed as their use
>> case is labeled "Canvas RTE".
> I disagree that the proposals "cost very little" and "are not being
> rationally analyzed". Which is not to say that the analysis cannot be
> improved.
I over-stated that second part, there has been rational analysis, 
however prejudiced it may be.

I haven't seen evidence that there are large costs, for most of the 
issues I've mentioned.
There is some evidence for "very little" costs, as some of them have 
been implemented,
but intentionally obscured from easy access.


>> When a browser vendor like Mozilla outright states that things have
>> been obfuscated, intentionally, it means there's Zero room for sending
>> in patches to fix the issue,
> I don't think the statements of individual Mozilla developers mean
> there is zero room for sending in patches.
>
> Anyway, Gecko's a FOSS code base. Disagree with a decision from on high?
> Fork it.
I'm not interested in forking Gecko. I have work-arounds. I'm interested 
in not-needing
workarounds in future releases.

Yes, the statements of individual Moz developers do mean there's zero room,
as they do reflect a group consensus from the vendor.

WebKit is a very different culture, but unfortunately, their decisions 
are impacted by those
from Moz, and so here I am, on the standards list, trying to unblock 
progress in these areas.

>>> It's been a struggle to get AT vendors who are not *also* browser
>>> vendors (Apple, Opera, Microsoft) involved in the standards process,
>>> so I'd like to complement you on getting AI Squared involved. It's
>>> really good news. :)
>> Bad style to complain... but still... Apple has been absurd in this
>> area.  They control their stack, from top to bottom.
> Minding positive iPhone and Mac experiences enjoyed by blind
> acquaintances, I disagree that's a bad thing.
>
> http://blog.fawny.org/2010/11/25/badtastehasacost/
>
> http://behindthecurtain.us/2010/06/12/my-first-week-with-the-iphone/
>
> Please note: As a long-term user of *both* stacks, I also value the
> ultimate freedom to DIY offered by FOSS stacks like Linux-Gnome-Orca.
> I'm glad we have an ecosystem capable of supporting both types of stack,
> and people working on accessibility in both stacks.
Thanks for the links.

I'm also glad to see a diverse ecosystem.

>> With other vendors, the standard vendor reply is, "well that's something
>> that should be handled automatically by the UA, not by the author."
> Indeed. Hammer vendors like making hammers that hit nails, rather than
> hammers that require you to get the nail vendor to make their own
> special attachment for your hammer in order to hammer in their brand of
> nails.
When it comes to the browser chrome, sure, they're hammer vendors.

I don't think that the APIs exposed by browsers can be considered hammers.
They're a massive collection of parts intended to be combined by 
programmers,
in arbitrary and interesting ways.

>> I've been told, repeatedly, that authors don't know what they're doing, so
>> vendors can't expose additional APIs. Even at the cost of a11y.
...
> Built-in accessibility means that ignorant authors can still produce
> accessible content and functionality, that knowledgable authors
> don't end up trading in accessibility to cut development time, and
> that you don't need to retrofit accessibility after corners are cut.
We're looking at exposing built-in accessibility, not tossing it out the 
window, or requiring new parts.
> There are use cases where built-in accessibility seems very difficult to
> achieve, such as providing audio descriptions or captions for media. I
> don't think RTE is one of those use cases.
AISquared has reminded me, that even the more straightforward use cases, 
such as RTE,
are difficult to support due to deficiencies in existing standards.

They work on screen magnifiers, with access to system data, and are 
still forced to
using pixel-based heuristics on simple text editing.

But that's a problem for another time. Right now, my focus is on 
exposing a handful
of methods and properties that are already available on all target 
platforms.


>> Microsoft is generally silent, though I appreciate their growing public
>> attendance.
> I agree the growing involvement of Microsoft in web standards is a good
> thing. :)
Their attention to accessibility is superb, and I was very pleased to 
see our large (50k LOC)
mostly-Canvas based application boot up in their preview releases. The 
rendering quality is quite wonderful,
and the speed is certainly up to par.

-Charles

Received on Wednesday, 19 January 2011 21:27:52 UTC