Response on canvas-requirement survey

In my role as the W3C team contact for the HTML working group, I'm
responding here to the following WBS survey:

  Accept requirement for immediate mode graphics a la canvas element?
  http://www.w3.org/2002/09/wbs/40318/req-gapi-canvas/

  "Do use cases such as games, shared whiteboards, and yahoo pipes and
  others[1] in the ESW wiki motivate a requirement that HTML 5 provide
  an immediate mode graphics API and canvas element?

  This is a proposal to close ISSUE-15[2] immediate-mode-graphics.

  Possible impact on the charter has been moved to a separate survey[3].

  [1] http://esw.w3.org/topic/HTML/AddedElementCanvas
  [2] http://www.w3.org/html/wg/tracker/issues/15
  [3] http://www.w3.org/2002/09/wbs/40318/tactics-gapi-canvas/results

I submitted a "Yes" vote.

Note that I don't have any special decision-making authority to
control the final result of this survey; I am representing my
organization and get one vote (just like any other member of the
group) and it's the responsibility of the chairs to weigh that
vote along with the other 60 or so votes submitted.

What follows below are (very) detailed comments related to my
response. The comments are quite long. I'm sorry I was not able to
make them shorter. But I put a Summary section at the end, so if
you don't care to read through all the details (or run out of time
and/or patience reading through them), feel free to skip to the end.

-----------------------------------------------------------------
Detailed Comments
-----------------------------------------------------------------
There is a range of views on the W3C team with regard to the
canvas element and its associated immediate-mode graphics API.

A number of team members have expressed serious concerns about
accessibility considerations (or lack of) in the canvas
specification. In some discussions, it has been suggested that
canvas is fundamentally broken by design with regard to
accessibility considerations and unfixable in that regard, and
that SVG should be used instead, because canvas by its natures
cannot provide access to semantics of the content -- access that
is necessary for providing accessible alternatives to the content.
That same view has been expressed in at least one message posted
to this list; from Rich Schwerdtfeger on Nov 21:

  http://lists.w3.org/Archives/Public/public-html/2007Nov/0320.html
  "I object to having <canvas> in HTML 5. There is no vehicle to
  apply accessibility semantics to the markup - unlike SVG."

  http://lists.w3.org/Archives/Public/public-html/2007Nov/0328.html
  "With Canvas you draw and it goes into the bit bucket. There is
  no way to capture and access the semantics of what you are
  doing... I am concerned that we would advocate using canvas over
  SVG where we would have an opportunity to apply semantics to the
  base drawing."

But also note in that same thread the replies to Rich's message,
including this one from Anne van Kesteren:

  http://lists.w3.org/Archives/Public/public-html/2007Nov/0322.html
  Anne cites some part of the canvas spec that provides for
  fallback content:
  http://www.w3.org/html/wg/html5/#canvas
  "When authors use the canvas element, they should also provide
  content  that, when presented to the user, conveys essentially
  the same function or  purpose as the bitmap canvas. This content
  may be placed as content of the  canvas element. The contents of
  the canvas element, if any, are the  element's fallback content."

Rich's response regarding fallback content:

  http://lists.w3.org/Archives/Public/public-html/2007Nov/0328.html
  With a fallback, I may have a text alternative for say a map
  which is a set of directions. This is fine for someone who is
  blind. For someone with a cognitive impairment I may need
  something else.

Other team members have expressed concerns that the HTML5 spec is
simply not the appropriate place for an immediate-mode graphics
API and have suggested that if we spec an immediate-mode graphics
API at all it should in a separate spec of its own and be managed
by a working group under the W3C Graphics Activity, with graphics
experts driving in its development. Along with that concern, it
has been pointed out that we already have good open APIs for
graphics support -- such as Open GL ES -- support for which is
already implemented even in mobile phones, with hardware
acceleration even. And there are some who have suggested that it
would be better to put resources into work on SVG -- including
working on the SVG JavaScript API -- rather than progressing
any further with canvas.

So some of the discussions (both about the accessibility
considerations and about use of resources to work on graphics APIs
within the W3C) have become in part "canvas vs. SVG" discussions,
or to put it into more general terms "immediate-mode vs.
retained-mode graphics" discussions, and along with that
discussions about whether it is necessary for the W3C to spec an
immediate-mode graphics API (given that there are existing APIs
for immediate-mode graphics that are already widely deployed).

With regard to the canvas vs. SVG issue, we have had similar
discussions within the HTML working group itself and embedded in
the comments in the responses to this particular survey; for
example, in his response, Gregory Rosmaita expressed concern about

  whether CANVAS is being forced upon users by developers loathe
  to provide the same functionality using an extant, proven,
  standardized and accessible graphical markup mechanism such as SVG"

But others have responded to suggest that there is a need for both
canvas and SVG; for example, this comment from Matthew Raymond in
his response to the survey:

  Memory concerns, performance considerations and the need for
  programmable effects will naturally lend themselves to an
  immediate-mode graphics API. The idea that SVG and <canvas> are
  competing technologies is in the large a fallacy, and only true
  in a specific, limited set of use cases... There is no evidence
  to support that SVG will be displaced by <canvas>. Opera,
  Mozilla and Apple all support _BOTH_ SVG and <canvas> natively,
  while IE currently supports neither natively."

Looking more at the responses to the survey, there seems to me to
be consensus in the working group that there is a need for a spec
for immediate-mode graphics -- with even some of those responding
"No" to the survey commenting that it would "good" to have a
standardized immediate-mode graphics API but questioning whether
it is in scope of this working group to produce it; for example,
this comment response from Sam Ruby in his response:

  An immediate mode graphics API and canvas element would clearly
  be a good thing; my only issue is the scope question...

And this comment from Gregory Rosmaita:

  I do not believe that it falls under the HTML WG's charter to
  define immediate mode graphics and CANVAS -- this is properly
  the domain of the W3C's graphics activity

And related comment from Matthew Raymond again in his response:

  The idea that the API is out of scope is weak, given the fact
  "UI widgets" and "Editing APIs and user-driven WYSIWYG editing
  features" are already within scope per the charter. At best, you
  could argue that the API should be JOINTLY developed with
  another group, but since there are already at least four
  existing implementations "in the wild" with few compatibility
  issues between them, such an arrangement would largely be an
  exercise in bureaucracy. What's more, for compatibility, we can
  always just keep the existing <canvas> "2d" context as-is and
  create a "w3c-2d" context at a later date.

Along with that comment that 'there are already at least four
existing implementations "in the wild" with few compatibility
issues between them', there were a number of similar comments in
the survey and in discussions on this list. And while it is
arguable whether it's accurate to say that there are "few
compatibility issues between them", that assertion is itself at
least testable, given that a relatively extensive set of test
cases (565 tests) exist online for testing any implementation:

  http://canvex.lazyilluminati.com/tests/tests/

A test report on various implementations is also available:

  http://canvex.lazyilluminati.com/tests/tests/results.html

Also, canvas is already in use in a significant amount of existing
content on the Web -- most famously in Yahoo Pipes, but also
apparently "average developers", as evidenced by things such as
bug reports that get filed if the regression happens in the canvas
API in version of a particular UA; for example:

  Mozilla bug #405584: Canvas.drawImage method is not working
  https://bugzilla.mozilla.org/show_bug.cgi?id=405584

So in regard to the maturity of the canvas spec, we have:

  - existence of a testsuite for canvas
  - multiple implementations of canvas in widely deployed
    products
  - canvas already in significant use on the Web, both in large
    sites and by working developers

Those are all things that we usually don't expect to see until a
spec reaches Candidate Recommendation or even after. The fact that
canvas already has all of those argues strongly for it already
being well ready for publication as a First Public Working Draft
-- given that a key purpose of publishing a spec as a FPWD is to
solicit comments on that spec from as wide a public readership as
possible; the canvas spec clearly seems ready for public review.

Along with that we have a market reality to consider: the vendors
who have already implemented support for the canvas and canvas API
specifications are not going to remove that support from their
products, regardless of what we (the members of this working group
and the W3C) decide to do with the canvas specification. So not
spec'ing canvas at all is not a practical option, because if we
know the feature is going to continue to be part of those products
and of existing websites -- and thus in essence actually part of
the Web -- the only choices we have to either ignore it completely
(which some might describe as us effectively sticking our heads
into the sand) or to work on spec'ing it in such a way (as thoroughly
and precisely as possible) so as to help ensure interoperable
implementations, and to continue on working to improve the spec as
much as possible in areas where limitations are identified (in
particular, with regard to accessibility considerations).

Another reality we will continue to be faced with is a certain
number of Web developers and sites will choose to make use of
canvas regardless of whatever any of us or our organizations might
do to promote or advocate SVG as a superior solution. And implicit
in the scope of the work we are tasked with doing is the goal of
providing Web developers with open-standards alternatives to
proprietary technologies controlled by single vendors. Supporting
canvas as a choice to developers helps us meet that goal. It seems
to help us quite a lot, in fact, given that we have developers who
have expressed a lot of enthusiasm about it and are very glad to
have it as a choice.

Also implicit in the work we are doing is the related goal of
making it easier for developers to create Web applications that
work interoperably across conformant browsers. If developers write
Web applications that they want to work as expected across a range
of browsers, we want should want to ensure that the set of
standards-based features available in those browsers is common
across them and is as well specified as possible. canvas, even
with its limitations, seems to have already proven itself as a
viable feature that many developers would like to have as a choice
in building applications that work across browsers.

So providing a standard canvas specification to support the
choices that Web developers are already making seems to clearly
align with the documented goals of this group.

-----------------------------------------------------------------
Summary
-----------------------------------------------------------------
So, in summary, as a representative of the W3C team in this group
and in particular the W3C Interaction domain and the Director --
and as a W3C staff member one of whose primary responsibilities is
to help ensure the success of the work of this group -- and after
discussion with other members of the team and extensive review of
the discussions within this group and in the comments submitted to
the survey, I am supporting the decision to provide the canvas
element and its associated API in the HTML5 specification.

That said, I want to note that going forward, I plan to work with
the W3C Web Accessibility Initiative Protocols and Formats working
group -- in particular, with that group's chair, Al Gilman -- to
make sure that the HTML5 specification and the canvas
specification in particular are reviewed thoroughly by members of
that group and that the issues identified get communicated back to
the HTML working group and addressed. I will also work to ensure
that the Status section of the HTML5 specification when it gets
published as a First Public Working Draft contains an
as-accurate-as-possible description of the open concerns and
objections to its content, and that the discussions around those
concerns and objections will continue, and that the group is
committed to address those concerns and objections before we
request transition to CR (which it says in our charter we are
scheduled to publish in Q3 of next year).

I also want to say that as I understand it at least, a decision to
retain the canvas API in HTML5 at this point does not necessarily
bind the group to keeping it in the spec forever. I think for now
it in part just helps to clear the way for us to go ahead with
publishing a FPWD of HTML5 with the canvas spec included, and
encouraging the public review of the spec, which is something I
think the group should really want to have right now. But
existence of the canvas spec in the FPWD of HTML5 does not
preclude the group from deciding to move the canvas immediate-mode
graphics API out into a separate spec later -- if and when it
seems like we have the circumstances necessary for making that a
practical option. For more on that, see the comments (from me and
others) in the "How should work on the canvas element and
immediate mode graphics API proceed?" survey:

  http://www.w3.org/2002/09/wbs/40318/tactics-gapi-canvas/results

That's it. If you managed to read this whole message from
beginning to end, well, God bless you.

  --Mike

-- 
Michael(tm) Smith
http://people.w3.org/mike/
http://sideshowbarker.net/