Re: hit testing and retained graphics

On Wed, Jul 6, 2011 at 3:11 PM, Charles Pritchard <chuck@jumis.com> wrote:
> On 7/6/2011 2:25 PM, Tab Atkins Jr. wrote:
>> On Sat, Jul 2, 2011 at 9:01 AM, Charles Pritchard<chuck@jumis.com>  wrote:
>>>
>>> On 7/1/2011 12:15 PM, Tab Atkins Jr. wrote:
>>>>
>>>> Honestly, attempting to reimplement HTML in<canvas>    sounds like a
>>>> great way to give your competitors a free head-start against you.
>>>
>>> There are several implementations for HTML rendering for output to PDF
>>> and other document formats. Companies have built products based on it.
>>
>> I'm not sure how writing an HTML-to-PDF converter is relevant here.
>> PDFs are useful outside the web.  We're talking about writing webapps
>
> PDF rendering and generation are useful inside the web as well.
> PDF rendering is included in some browsers, and otherwise made
> available via plugin for others.
>
> I brought up pdf.js as another example of Canvas being used
> for rich application purposes.

Note that pdf.js is using <canvas> currently because it was faster to
prototype in.  They plan to switch over to using SVG, possibly with
canvas rendering retained for a quicker first-paint time (that is,
first rendering the PDF in canvas, then constructing it in SVG and
swapping the canvas for the SVG when it's ready).


> Web Apps -- or "Applications" may work with a wide variety of formats;
> PDF is a good example of a rich format.

Are you trying to say that webapps should run inside of PDF documents,
or that webapps may include PDF as part of their operation?  The
former seems silly, and the latter seems unnecessary - PDFs advantages
are generally all offline or print-related.


>>>> (Imo, we should continue to make it difficult to do full-featured text
>>>> and UI in canvas, so that it stays painful.  Then we can make it less
>>>> painful to do things the right way, and natural incentives will help
>>>> people make the right choice.)
>>>
>>> Why would you say that out loud Tab? The world is hard enough as it is.
>>> It pains me to see this kind of thinking applied to such important
>>> decisions.
>>>
>>> I should not have to defend free enterprise to developers on a web
>>> standards mailing list.
>>
>> [continued in the following reply]
>>>
>>> IMO, if your strategy includes actively or passively creating or
>>> maintaining
>>> "painful" situations for developers, you are doing it wrong.
>>
>> I... I don't know how you achieved such a confusion over my goals or
>> my proposed solutions.
>
> You've proposed to 1) create a new retained mode API or 2) use SVG, abandon
> Canvas.

I did not propose 2.  <canvas> is very useful for a wide variety of
things.  However, for the use-cases that have been brought forward in
this discussion, it appears that SVG is a good solution.


>> In this specific topic, the right way to get bounds information for
>> objects, associate drawn images with captions, keep text around so it
>> can be read later by a machine, etc. is to use a retained-mode API.
>> The wrong way is to use an immediate-mode one.
>
> The Canvas shadow DOM is a retained mode API -- the DOM is retained-mode.

Irrelevant, as you don't draw with the subtree.  All of the use-cases
presented so far involve interacting with the drawing somehow.


> 3) Canvas 2d is an immediate mode API; WebGL is mixed; both
> rely on DOM Core as a pre-requisite.
>
> We need to allow authors to update the shadow DOM with the current path
> from Canvas. We'd like to allow authors to pull a string representing
> the current path, as a DOMString based on SVG paths. We'd also like
> to have a Path object, which would make management of multiple
> paths a little more efficient in the UA / rendering engine
> and little easier on developers.

No, you don't "need" to do that.  That's a component of one possible
solution to the use-cases presented.  It is not an essential part of
the problem to be solved.  (This is why I keep harping on use-cases,
so the conversation doesn't conflate problems and solutions.)


>> the two approaches are somewhat incompatible.  Or, we can promote
>> existing examples of the former (like SVG), find out what problems
>
> Canvas requires a subset of DOM and CSS, it does not need SVG,
> and requiring it would be big burden on implementers. It's overkill.

Rendering the open web requires SVG more and more as time goes on, and
the general feel of a lot of people in the community is that this is a
good thing; weighing its benefits against its faults, SVG is a
worthwhile technology.  All browsers support SVG, and new implementors
who want to support the web need to support it as well.

I'm completely uninterested in helping implementors that aren't
invested in the web.


>> they have that make them difficult to use, and fix them.  Or, we can
>> come up with new retained-mode solutions that are designed up-front to
>> solve the problems we know of and be easy to use.
>
> This is what we've done, by specifying the Canvas shadow DOM and learning
> that Canvas developers would like pointer events to bubbling from the shadow
> DOM.

This use-case still hasn't been fleshed out well.  I know that, for
example, in games you don't need or want pointer events to retarget
into the subtree, because you don't have anything useful in there.

A *possible* use-case that I could imagine has a problem that could be
solved by this solution is an author trying to give form widgets a
fancier appearance via canvas (basically, wrapping an <input> with a
<canvas>).  In this case, it might make sense to retarget clicks from
the canvas to the input, so you can just listen on the input.  This
can be equally solved by listening on the <canvas>, though, if it's a
bubbling event (and all the user-interaction events are, I believe).
In the future, a combination of the element() function and the
'content' property would make this solveable in CSS without a single
<canvas> in the page content, via code something like this:

<input type=radio>
<style>
input[type=radio] {
  content: element(radioCanvas) replaced;
}
</style>
<script>
var radioCanvas = document.createElement('canvas');
document.CSSElementMap.radioCanvas = radioCanvas;
var radioCanvasContext = radioCanvas.getContext('2d');
...drawing code here...
</script>

If the author is trying to use <canvas> to draw an entire <form>, they
might need a more complex solution, but this runs into the "you're
doing it wrong" problem.


>> I'm on the side of one of the latter solutions, but only because I
>> think it has a better chance of solving accessibility problems
>> "automatically".  I am *strongly opposed* to any solution that is
>> completely optional, as that will leave the large majority of
>> applications using the relevant technology less accessible.
>
> Canvas accessibility will always be completely optional: authors can at
> their choosing decide not to use the DOM, and treat the canvas as a
> semantically opaque image.
>
> Accessibility is still "optional" on the web; and it's not "automatic",
> it just happens that, with authoring tools or good practices, developers
> realize that there are opportunities expose semantic data.
>
> Companies and developers can set their own policies as to what
> they feel is -optional- and what is not -- Authoring Tools have
> a strong obligation to include accessibility in the work-flow.

This is incorrect.  HTML is designed such that you get a lot of
accessibility "automatically" simply by authoring good markup, which
is usually the cleanest and prettiest code anyway.  A lot of features
have nice automatic accessibility as well; for example, <video> offers
<track> for captions rather than recommending baked-in captions.

The fact that we have a pixel-pushing API does mean that authors can
always choose to render everything manually.  We can make it easy to
use solutions with accessibility baked in for some problems, though,
so the author doesn't have to think very hard (or, ideally, at all)
about whether or not it's worthwhile to add accessibility.

~TJ

Received on Thursday, 7 July 2011 00:08:57 UTC