Re: Preliminary walk through of HitRegions ahead of Canvas call

On Sun, Mar 2, 2014 at 3:09 PM, Mark Sadecki <mark@w3.org> wrote:

> I walked myself through the Hit Regions spec and attempted to identify
> sections that were a high priority for L1, those that would be nice to have
> in L1 and those that could be delayed until L2.  During this walk through,
> a few questions emerged.  The answers to these questions will most likely
> affect what goes in L1 and what gets delayed until L2:
>
> 1. Do we plan on supporting nested regions with hierarchical/ancestor
> relationships in L1? (Assuming so)
>

Maybe. They don't seem too hard to implement.


> 2. Do we plan on supporting custom mouse cursors?
>

No.


> 3. Do we need fill rule?
>

Yes, L1


> Assumptions I made:
>
> 1. We will not be supporting unbacked region descriptions like label or
> ARIA roles for L1.
> 2. We will be supporting removeHitRegion
> 3. We will be supporting the MouseEvent interface
>

Yes


> Below are my notes.  We can use these as talking points during the Canvas
> call on 03 March 2014.
>
> Mark
>
> > ###### 1.1.15 Hit regions
> >
> > A hit region list is a list of hit regions for a bitmap.
>
> I imagine we are still supporting hit region lists.
> >
> > Each hit region consists of the following information:
> >
> >   * A set of pixels on the bitmap for which this region is responsible.
> >
> >   * A bounding circumference on the bitmap that surrounds the hit
> region's set of pixels as they stood when it was created.
> >
> >   * Optionally, a non-empty string representing an ID for distinguishing
> the region from others.
> >
> >   * Optionally, a reference to another region that acts as the parent
> for this one.
> >
> >   * A count of regions that have this one as their parent, known as the
> hit region's child count.
>
> Do we have plans to support nested regions with hierarchical/ancestor
> relationships?
> >
> >   * A cursor specification, in the form of either a CSS cursor value, or
> the string "`inherit`" meaning that the cursor of the hit region's parent,
> if any, or of the `canvas` element, if not, is to be used instead.
>
> If this is not hard to implement, it could be nice to have.  Lower
> priority though.
>

yes, let's do it for L2


> >
> >   * Optionally, either a control, or an unbacked region description.
>
> Control for L1 and unbacked region description for L2?
>

yes


> > A control is just a reference to an `Element` node, to which, in certain
> > conditions, the user agent will route events, and from which the user
> agent
> > will determine the state of the hit region for the purposes of
> accessibility
> > tools. (The control is ignored when it is not a descendant of the
> `canvas`
> > element.)
> >
> > An unbacked region description consists of the following:
> >
> >     * Optionally, a label.
> >
> >     * An ARIA role, which, if the unbacked region description also has a
> label, could be the empty string.
> >
> > context . `addHitRegion`(options)
>
> I assume this is for L2?
>

yes


> >
> >
> >
> > Adds a hit region to the bitmap. The argument is an object with the
> following
> > members:
> >
> > `path` (default null)
> >
> >     A `Path` object that describes the pixels that form part of the
> region. If this member is not provided or is set to null, the current
> default path is used instead.
> > `fillRule` (default "`nonzero`")
>
> I understand we are only going to support the current default path at this
> time.  How should we include that in the spec?
>

Maybe the method should throw notsupported if you put pass in a path. (same
with label and role)


> >
> >     The fill rule to use when determining which pixels are inside the
> path.
> > `id` (default empty string)
>
> >From my understanding, fill rule will be needed for Hit Testing.  Is that
> correct?  Is it a necessary part or can it wait until L2?  Seems like Rik
> has basic Hit Testing support in the Firefox Nightly.  Not sure if it uses
> fill rule at all.
> >
> >     The ID to use for this region. This is used in `MouseEvent` events
> on the `canvas` (`event.region`) and as a way to reference this region in
> later calls to `addHitRegion()`.
> > `parentID` (default null)
>
> Assuming we will support this.
> >
> >     The ID of the parent region, for purposes of navigation by
> accessibility tools and for cursor fallback.
> > `cursor` (default "`inherit`")
>
> L1
> >
> >     The cursor to use when the mouse is over this region. The value
> "`inherit`" means to use the cursor for the parent region (as specified by
> the `parentID` member), if any, or to use the `canvas` element's cursor if
> the region has no parent.
> > `control` (default null)
>
> Lower priority for L1
>

Maybe your message content is a bit out of sync.
Content is needed for L1.


> >
> >     An element (that is a descendant of the `canvas`) to which events
> are to be routed, and which accessibility tools are to use as a surrogate
> for describing and interacting with this region.
> > `label` (default null)
>
> L1
>

No, that should go for L2.


> >
> >     A text label for accessibility tools to use as a description of this
> region, if there is no control.
> > `role` (default null)
>
> L2
> >
> >     An ARIA role for accessibility tools to use to determine how to
> represent this region, if there is no control.
>
> L2
> >
> > Hit regions can be used for a variety of purposes:
> >
> >   * With an ID, they can make hit detection easier by having the user
> agent check which region the mouse is over and include the ID in the mouse
> events.
> >   * With a control, they can make routing events to DOM elements
> automatic, allowing e.g. clicks on a `canvas` to automatically submit a
> form via a `button` element.
> >   * With a label, they can make it easier for users to explore a
> `canvas` without seeing it, e.g. by touch on a mobile device.
> >   * With a cursor, they can make it easier for different regions of the
> `canvas` to have different cursors, with the user agent automatically
> switching between them.
> > context . `removeHitRegion`(id)
> >
> >
> >
> > Removes a hit region (and all its descendants) from the canvas bitmap.
> The
> > argument is the ID of a region added using `addHitRegion()`.
>
> L1
> >
> > The pixels that were covered by this region and its descendants are
> > effectively cleared by this operation, leaving the regions
> non-interactive. In
> > particular, regions that occupied the same pixels before the removed
> regions
> > were added, overlapping them, do not resume their previous role.
> >
> > A hit region A is an ancestor region of a hit region B if B has a parent
> and
> > its parent is either A or another hit region for which A is an ancestor
> > region.
>
> See above regarding support for nested/hierarchical relationships.
> >
> > The region identified by the ID ID in a bitmap bitmap is the value
> returned by
> > the following algorithm (which can return a hit region or nothing):
> >
> >   1. If ID is null, return nothing and abort these steps.
> >
> >   2. Let list be the hit region list associated with bitmap.
> >
> >   3. If there is a hit region in list whose ID is a case-sensitive match
> for ID, then return that hit region and abort these steps.
> >
> >   4. Otherwise, return nothing.
> >
> > The region representing the control control for a bitmap bitmap is the
> value
> > returned by the following algorithm (which can return a hit region or
> > nothing):
> >
> >   1. Let list be the hit region list associated with bitmap.
> >
> >   2. If there is a hit region in list whose control is control, then
> return that hit region and abort these steps.
> >
> >   3. Otherwise, return nothing.
> >
> > The control represented by a region region for a `canvas` element
> ancestor is
> > the value returned by the following algorithm (which can return an
> element or
> > nothing):
> >
> >   1. If region has no control, return nothing and abort these steps.
> >
> >   2. Let control be region's control.
> >
> >   3. If control is not a descendant of ancestor, then return nothing and
> abort these steps.
> >
> >   4. Otherwise, return control.
> >
> > The cursor for a hit region region of a `canvas` element ancestor is the
> value
> > returned by the following algorithm:
> >
> >   1. _Loop_: If region has a cursor specification other than
> "`inherit`", then return that hit region's cursor specification and abort
> these steps.
> >
> >   2. If region has a parent, then let region be that hit region's
> parent, and return to the step labeled _loop_.
> >
> >   3. Otherwise, return the used value of the 'cursor' property for the
> `canvas` element, if any; if there isn't one, return 'auto'. [CSSUI]
> >
> > The region for a pixel pixel on a bitmap bitmap is the value returned by
> the
> > following algorithm (which can return a hit region or nothing):
> >
> >   1. Let list be the hit region list associated with bitmap.
> >
> >   2. If there is a hit region in list whose set of pixels contains
> pixel, then return that hit region and abort these steps.
> >
> >   3. Otherwise, return nothing.
> >
> > To clear regions that cover the pixels pixels on a bitmap bitmap, the
> user
> > agent must run the following steps:
> >
> >   1. Let list be the hit region list associated with bitmap.
> >
> >   2. Remove all pixels in pixels from the set of pixels of each hit
> region in list.
> >
> >   3. Garbage-collect the regions of bitmap.
> >
> > To garbage-collect the regions of a bitmap bitmap, the user agent must
> run the
> > following steps:
> >
> >   1. Let list be the hit region list associated with bitmap.
> >
> >   2. _Loop_: Let victim be the first hit region in list to have an empty
> set of pixels and a zero child count, if any. If there is no such hit
> region, abort these steps.
> >
> >   3. If victim has a parent, then decrement that hit region's child
> count by one.
> >
> >   4. Remove victim from list.
> >
> >   5. Jump back to the step labeled _loop_.
> >
> > Adding a new region and calling `clearRect()` are the two ways this
> clearing
> > algorithm can be invoked. The hit region list itself is also reset when
> the
> > rendering context is reset, e.g. when a `CanvasRenderingContext2D`
> object is
> > bound to or unbound from a `canvas`, or when the dimensions of the
> bitmap are
> > changed.
> >
> > * * *
> >
> > When the `addHitRegion()` method is invoked, the user agent must run the
> > following steps:
> >
> >   1. Let arguments be the dictionary object provided as the method's
> argument.
> >
> >   2. If the arguments object's `path` member is not null, let source
> path be the `path` member's value. Otherwise, let it be the
> `CanvasRenderingContext2D` object's current default path.
>
> I assume this will be something like: "The source path will be the
> `CanvasRenderingContext2D` object's current default path."
> >
> >   3. Transform all the coordinates and lines in source path by the
> current transform matrix, if the arguments object's `path` member is not
> null.
>
> L2
> >
> >   4. Let specified pixels be the pixels contained in source path, using
> the fill rule indicated by the `fillRule` member.
>
> L2?
>

No, L1


> >
> >   5. If the arguments object's `id` member is the empty string, let it
> be null instead.
>
> L1
> >
> >   6. If the arguments object's `id` member is not null, then let
> previous region for this ID be the region identified by the ID given by the
> `id` member's value in this scratch bitmap, if any. If the `id` member is
> null or no such region currently exists, let previous region for this ID be
> null.
> L1
> >
> >   7. If the arguments object's `parent` member is the empty string, let
> it be null instead.
> L1?
> >
> >   8. If the arguments object's `parent` member is not null, then let
> parent region be the region identified by the ID given by the `parent`
> member's value in the scratch bitmap, if any. If the `parent` member is
> null or no such region currently exists, let parent region be null.
> L1?
> >
> >   9. If the arguments object's `label` member is the empty string, let
> it be null instead.
> L1
> >
> >   10. If any of the following conditions are met, throw a
> `NotSupportedError` exception and abort these steps.
> >
> >     * The arguments object's `control` and `label` members are both
> non-null.
> >     * The arguments object's `control` and `role` members are both
> non-null.
> >     * The arguments object's `role` member's value is the empty string,
> and the `label` member's value is either null or the empty string.
> Not necessary in L1 unless we support label and role.
> >     * The specified pixels has no pixels.
> >     * The arguments object's `control` member is not null but is neither
> an `a` element that represents a hyperlink, a `button` element, an `input`
> element whose `type` attribute is in one of the Checkbox or Radio Button
> states, nor an `input` element that is a button.
> I believe we do not need the above if the control is not limited.
> >     * The parent region is not null but has a control.
> >     * The previous region for this ID is the same hit region as the
> parent region.
> >     * The previous region for this ID is an ancestor region of the
> parent region.
> >   11. If the `parent` member is not null but parent region is null, then
> throw a `NotFoundError` exception and abort these steps.
> >
> >   12. If any of the following conditions are met, throw a `SyntaxError`
> exception and abort these steps.
> >
> >     * The arguments object's `cursor` member is not null but is neither
> an ASCII case-insensitive match for the string "`inherit`", nor a valid CSS
> 'cursor' property value. [CSSUI]
> Depends on if we support cursors in L1
> >     * The arguments object's `role` member is not null but its value is
> not an ordered set of unique space-separated tokens whose tokens are all
> case-sensitive matches for names of non-abstract WAI-ARIA roles. [ARIA]
> L2
> >   13. Let region be a newly created hit region, with its information
> configured as follows:
> >
> > Hit region's set of pixels
> >
> >
> >
> > The specified pixels
> >
> > Hit region's bounding circumference
> >
> >
> >
> > A user-agent-defined shape that wraps the pixels contained in source
> path. (In
> > the simplest case, this can just be the bounding rectangle; this
> specification
> > allows it to be any shape in order to allow other interfaces.)
> L1
> >
> > Hit region's ID
> >
> >
> >
> > If the arguments object's `id` member is not null: the value of the `id`
> > member. Otherwise, region has no id.
> L1
> >
> > Hit region's parent
> >
> >
> >
> > If parent region is not null: parent region. Otherwise, region has no
> parent.
> L1?
> >
> > Hit region's child count
> >
> >
> >
> > Initially zero.
>
> L1?
> >
> > Hit region's cursor specification
> L1?
>

No, let's do L2 for cursors


> >
> >
> >
> > If parent region is not null: parent region. Otherwise, region has no
> parent.
> >
> > Hit region's control
> >
> >
> >
> > If the arguments object's `control` member is not null: the value of the
> > `control` member. Otherwise, region has no control.
> L1
> >
> > Hit region's label
> >
> >
> >
> > If the arguments object's `label` member is not null: the value of the
> `label`
> > member. Otherwise, region has no label.
> L2
> >
> > Hit region's ARIA role
> >
> >
> >
> > If the arguments object's `role` member is not null: the value of the
> `role`
> > member (which might be the empty string). Otherwise, if the arguments
> object's
> > `label` member is not null: the empty string. Otherwise, region has no
> ARIA
> > role.
> L2
> >
> >   14. If the arguments object's `cursor` member is not null, then act as
> if a CSS rule for the `canvas` element setting its 'cursor' property had
> been seen, whose value was the hit region's cursor specification.
> L2?
> >
> > For example, if the user agent prefetches cursor values, this would
> cause that
> > to happen in response to an appropriately-formed `addHitRegion()` call.
> >
> >   15. If the arguments object's `control` member is not null, then let
> previous region for the control be the region representing the control
> given by the `control` member's value for this scratch bitmap, if any. If
> the `control` member is null or no such region currently exists, let
> previous region for the control be null.
> L1
> >
> >   16. If there is a previous region with this control, remove it from
> the scratch bitmap's hit region list; then, if it had a parent region,
> decrement that hit region's child count by one.
> L1
> >
> >   17. If there is a previous region with this ID, remove it, and all hit
> regions for which it is an ancestor region, from the scratch bitmap's hit
> region list; then, if it had a parent region, decrement that hit region's
> child count by one.
> L1
> >
> >   18. If there is a parent region, increment its hit region's child
> count by one.
> L1?
> >
> >   19. Clear regions that cover the pixels in region's set of pixels on
> this scratch bitmap.
> >
> >   20. Add region to the scratch bitmap's element's hit region list.
> >
> > When the `removeHitRegion()` method is invoked, the user agent must run
> the
> > following steps:
> >
> >   1. Let region be the region identified by the ID given by the method's
> argument in the rendering context's scratch bitmap. If no such region
> currently exists, abort these steps.
> >
> > If the method's argument is the empty string, then no region will match.
> >
> >   2. Remove region, and all hit regions for which it is an ancestor
> region, from the rendering context's scratch bitmap's hit region list;
> then, if it had a parent region, decrement that hit region's child count by
> one.
> >
> >   3. Garbage-collect the regions of the rendering context's scratch
> bitmap.
>
> L1?
> >
> > * * *
> >
> > The `MouseEvent` interface is extended to support hit regions:
> >
> >
> >     partial interface MouseEvent {
> >       readonly attribute DOMString? region;
> >     };
> >
> >     partial dictionary MouseEventInit {
> >       DOMString? region;
> >     };
> >
> > event . `region`
> >
> >
> >
> > If the mouse was over a hit region, then this returns the hit region's
> ID, if
> > it has one.
> >
> > Otherwise, returns null.
> >
> > The `region` attribute on `MouseEvent` objects must return the value it
> was
> > initialized to. When the object is created, this attribute must be
> initialized
> > to null. It represents the hit region's ID if the mouse was over a hit
> region
> > when the event was fired.
> L1
> >
> > When a `MouseEvent` is to be fired at a `canvas` element by the user
> agent in
> > response to a pointing device action, if the `canvas` element has a hit
> region
> > list, the user agent must instead follow these steps. If these steps say
> to
> > _act as normal_, that means that the event must be fired as it would
> have had
> > these requirements not been applied.
> >
> >   1. If the pointing device is not indicating a pixel on the `canvas`,
> act as normal and abort these steps.
> >
> >   2. Let pixel be the pixel indicated by the pointing device.
> >
> >   3. Let region be the hit region that is the region for the pixel pixel
> on this `canvas` element's bitmap, if any.
> >
> >   4. If there is no region, then act as normal and abort these steps.
> >
> >   5. Let id be the region's ID, if any.
> >
> >   6. If there is an id, then initialize the event object's `region`
> attribute to id.
> >
> >   7. Let control be the control represented by region for this `canvas`
> element, if any.
> >
> >   8. If there is a control, then target the event object at control
> instead of the `canvas` element.
> >
> >   9. Continue dispatching the event, but with the updated event object
> and target as given in the above steps.
> >
> > * * *
> >
> > When a user's pointing device cursor is positioned over a `canvas`
> element,
> > user agents should render the pointing device cursor according to the
> cursor
> > specification described by the cursor for the hit region that is the
> region
> > for the pixel that the pointing device designates on the `canvas`
> element's
> > bitmap.
>
> L1?
> >
> > * * *
> >
> > User agents are encouraged to make use of the information present in a
> > `canvas` element's hit region list to improve the accessibility of
> `canvas`
> > elements.
> >
> > Each hit region should be handled in a fashion equivalent to a node in a
> > virtual DOM tree rooted at the `canvas` element. The hierarchy of this
> virtual
> > DOM tree must match the hierarchy of the hit regions, as described by the
> > parent of each region. Regions without a parent must be treated as
> children of
> > the `canvas` element for the purpose of this virtual DOM tree. For each
> node
> > in such a DOM tree, the hit region's bounding circumference gives the
> region
> > of the screen to use when representing the node (if appropriate).
> >
> > The semantics of a hit region for the purposes of this virtual DOM tree
> are
> > those of the the control represented by the region, if it has one,
>
> L1
> or else of
> > a non-interactive element whose ARIA role, if any, is that given by the
> hit
> > region's ARIA role, and whose textual representation, if any, is given
> by the
> > hit region's label.
>
> L2
> >
> > For the purposes of accessibility tools, when an element C is a
> descendant of
> > a `canvas` element and there is a region representing the control C for
> that
> > `canvas` element's bitmap, then the element's position relative to the
> > document should be presented as if it was that region in the `canvas`
> > element's virtual DOM tree.
> L1
> >
> > The semantics of a hit region for the purposes of this virtual DOM tree
> are
> > those of the the control represented by the region, if it has one,
> L1
> or else of
> > a non-interactive element whose ARIA role, if any, is that given by the
> hit
> > region's ARIA role, and whose textual representation, if any, is given
> by the
> > hit region's label.
> L2
> >
> > Thus, for instance, a user agent on a touch-screen device could provide
> haptic
> > feedback when the user croses over a hit region's bounding
> circumference, and
> > then read the hit region's label to the user. Similarly, a desktop user
> agent
> > with a virtual accessibility focus separate from the keyboard input focus
> > could allow the user to navigate through the hit regions, using the
> virtual
> > DOM tree described above to enable hierarchical navigation. When an
> > interactive control inside the `canvas` element is focused, if the
> control has
> > a corresponding region, then that hit region's bounding circumference
> could be
> > used to determine what area of the display to magnify.
> L2?
> >
>
> --
> Mark Sadecki
> Web Accessibility Engineer
> World Wide Web Consortium, Web Accessibility Initiative
> Telephone: +1.617.715.4017
> Email: mark@w3.org
> Web: http://w3.org/People/mark
>
>