Re: [w3ctag/design-reviews] Incubation: [HTML] html-in-canvas (Issue #1204)

matatk left a comment (w3ctag/design-reviews#1204)

We are very supportive of the goals of this proposal, which is well written. It's good to see privacy considerations addressed. But the proposal requires more work.

This is just initial feedback, which has three parts: (1) specific questions; (2) general concerns around accessibility; and (3) some thoughts on how this may address concerns we have with other proposals.

# Specifics

1. What changes can trigger the paint event? The explainer says when "any canvas children has changed" but then claims "changing the transform does not cause the paint event to fire". This sounds handwavy to me, for example:

   - What if the transform on a non-direct descendant changes?

   - What if the scroll offset of a descendant changes?

   - What about another canvas? This might lead to a ton of other issues though

   - And there might be an interop risk. For certain types of changes, a UA implementation may keep them completely off the main thread, and doesn't even run update the rendering steps for them. Examples include:

     - Video

     - Animated images

     - A blinking caret (as in the basic example)

2. Maintaining the interactability of the child elements seems to be a goal of this proposal, according to:

   > Browser features like hit testing, intersection observer, and accessibility rely on an element's DOM location. To ensure these work, the element's transform property should be updated so that the DOM location matches the drawn location.

   This looks very hacky and is probably not enough. For example:

   - What if the canvas has multiple child elements whose `drawElementImage()` calls are not in the DOM order? This might mess up hit testing even with their `transform` property updated to match the drawn location.

   - What if the same child element is drawn for multiple times?

   We'd like to see a more holistic solution.

3. Bikeshed idea: the naming of the `paint` event. It sounds too general while its actual meaning is element image changed.

# Some accessibility concerns

We are still looking into the various threads relating to the accessibility of the proposal. The overall goal and direction are very welcome.

We note that there are some significant unresolved issues that could quite significantly alter the accessibility of the end result, the way that the UA and AT need to work, and the DX.

One example is https://github.com/WICG/html-in-canvas/issues/134 but there are others too. We will come back with further feedback on this and other issues, as appropriate, in due course.

# Some possible wins over concerns raised in other proposals

There has been a lot of work done on rendering decorative text in the canvas, i.e. https://tag-github-bot.w3.org/gh/w3ctag/design-reviews/1095

We were [not satisfied with the canvas text metrics proposal](https://github.com/w3ctag/design-reviews/issues/1095#issuecomment-3872160989), because it introduces at first a complex, imperative API that requires the developer to implement accessibility themselves. We would've preferred to see, as is common on the platform, a simpler, declarative API, that has accessibility built in, come along first, so that most developers can just use that.

We felt that it would be possible to add such a simple declarative API as part of the HTML-in-canvas work. What do you think? This could solve an accessibility barrier whilst supporting more artistic uses of the canvas.

There have been two recent developments in this area, which we're looking at:

* A proposal to make text drawing in canvas accessible by exposing the drawn text to AT - there are significant concerns with this (it would be very noisy and potentially unreliable for AT users).

* [An update to the Text Metrics proposal was shared at a WHATNOT meeting](https://github.com/whatwg/html/issues/10677#issuecomment-4847744218). We're not yet aware of what's changed, but are looking into it.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3ctag/design-reviews/issues/1204#issuecomment-5278762405
You are receiving this because you are subscribed to this thread.

Message ID: <w3ctag/design-reviews/issues/1204/5278762405@github.com>