Re: Preliminary walk through of HitRegions ahead of Canvas call

Where is Dominic on this? We need a second implementation in Chrome or we
need to have something in Webkit.


Rich Schwerdtfeger



From:	Rik Cabanier <cabanier@gmail.com>
To:	Mark Sadecki <mark@w3.org>
Cc:	"public-canvas-api@w3.org" <public-canvas-api@w3.org>, Jatinder
            Mann <jmann@microsoft.com>, Jay Munro <jaymunro@microsoft.com>,
            "Rik Cabanier (cabanier@adobe.com)" <cabanier@adobe.com>,
            Richard Schwerdtfeger/Austin/IBM@IBMUS
Date:	03/02/2014 10:02 PM
Subject:	Re: Preliminary walk through of HitRegions ahead of Canvas call






On Sun, Mar 2, 2014 at 3:09 PM, Mark Sadecki <mark@w3.org> wrote:
  I walked myself through the Hit Regions spec and attempted to identify
  sections that were a high priority for L1, those that would be nice to
  have in L1 and those that could be delayed until L2.  During this walk
  through, a few questions emerged.  The answers to these questions will
  most likely affect what goes in L1 and what gets delayed until L2:

  1. Do we plan on supporting nested regions with hierarchical/ancestor
  relationships in L1? (Assuming so)

Maybe. They don't seem too hard to implement.

  2. Do we plan on supporting custom mouse cursors?

No.

  3. Do we need fill rule?

Yes, L1

  Assumptions I made:

  1. We will not be supporting unbacked region descriptions like label or
  ARIA roles for L1.
  2. We will be supporting removeHitRegion
  3. We will be supporting the MouseEvent interface

Yes

  Below are my notes.  We can use these as talking points during the Canvas
  call on 03 March 2014.

  Mark

  > ###### 1.1.15 Hit regions
  >
  > A hit region list is a list of hit regions for a bitmap.

  I imagine we are still supporting hit region lists.
  >
  > Each hit region consists of the following information:
  >
  >   * A set of pixels on the bitmap for which this region is responsible.
  >
  >   * A bounding circumference on the bitmap that surrounds the hit
  region's set of pixels as they stood when it was created.
  >
  >   * Optionally, a non-empty string representing an ID for
  distinguishing the region from others.
  >
  >   * Optionally, a reference to another region that acts as the parent
  for this one.
  >
  >   * A count of regions that have this one as their parent, known as the
  hit region's child count.

  Do we have plans to support nested regions with hierarchical/ancestor
  relationships?
  >
  >   * A cursor specification, in the form of either a CSS cursor value,
  or the string "`inherit`" meaning that the cursor of the hit region's
  parent, if any, or of the `canvas` element, if not, is to be used
  instead.

  If this is not hard to implement, it could be nice to have.  Lower
  priority though.

yes, let's do it for L2

  >
  >   * Optionally, either a control, or an unbacked region description.

  Control for L1 and unbacked region description for L2?

yes

  > A control is just a reference to an `Element` node, to which, in
  certain
  > conditions, the user agent will route events, and from which the user
  agent
  > will determine the state of the hit region for the purposes of
  accessibility
  > tools. (The control is ignored when it is not a descendant of the
  `canvas`
  > element.)
  >
  > An unbacked region description consists of the following:
  >
  >     * Optionally, a label.
  >
  >     * An ARIA role, which, if the unbacked region description also has
  a label, could be the empty string.
  >
  > context . `addHitRegion`(options)

  I assume this is for L2?

yes

  >
  >
  >
  > Adds a hit region to the bitmap. The argument is an object with the
  following
  > members:
  >
  > `path` (default null)
  >
  >     A `Path` object that describes the pixels that form part of the
  region. If this member is not provided or is set to null, the current
  default path is used instead.
  > `fillRule` (default "`nonzero`")

  I understand we are only going to support the current default path at
  this time.  How should we include that in the spec?

Maybe the method should throw notsupported if you put pass in a path. (same
with label and role)

  >
  >     The fill rule to use when determining which pixels are inside the
  path.
  > `id` (default empty string)

  >From my understanding, fill rule will be needed for Hit Testing.  Is
  that correct?  Is it a necessary part or can it wait until L2?  Seems
  like Rik has basic Hit Testing support in the Firefox Nightly.  Not sure
  if it uses fill rule at all.
  >
  >     The ID to use for this region. This is used in `MouseEvent` events
  on the `canvas` (`event.region`) and as a way to reference this region in
  later calls to `addHitRegion()`.
  > `parentID` (default null)

  Assuming we will support this.
  >
  >     The ID of the parent region, for purposes of navigation by
  accessibility tools and for cursor fallback.
  > `cursor` (default "`inherit`")

  L1
  >
  >     The cursor to use when the mouse is over this region. The value
  "`inherit`" means to use the cursor for the parent region (as specified
  by the `parentID` member), if any, or to use the `canvas` element's
  cursor if the region has no parent.
  > `control` (default null)

  Lower priority for L1

Maybe your message content is a bit out of sync.
Content is needed for L1.

  >
  >     An element (that is a descendant of the `canvas`) to which events
  are to be routed, and which accessibility tools are to use as a surrogate
  for describing and interacting with this region.
  > `label` (default null)

  L1

No, that should go for L2.

  >
  >     A text label for accessibility tools to use as a description of
  this region, if there is no control.
  > `role` (default null)

  L2
  >
  >     An ARIA role for accessibility tools to use to determine how to
  represent this region, if there is no control.

  L2
  >
  > Hit regions can be used for a variety of purposes:
  >
  >   * With an ID, they can make hit detection easier by having the user
  agent check which region the mouse is over and include the ID in the
  mouse events.
  >   * With a control, they can make routing events to DOM elements
  automatic, allowing e.g. clicks on a `canvas` to automatically submit a
  form via a `button` element.
  >   * With a label, they can make it easier for users to explore a
  `canvas` without seeing it, e.g. by touch on a mobile device.
  >   * With a cursor, they can make it easier for different regions of the
  `canvas` to have different cursors, with the user agent automatically
  switching between them.
  > context . `removeHitRegion`(id)
  >
  >
  >
  > Removes a hit region (and all its descendants) from the canvas bitmap.
  The
  > argument is the ID of a region added using `addHitRegion()`.

  L1
  >
  > The pixels that were covered by this region and its descendants are
  > effectively cleared by this operation, leaving the regions
  non-interactive. In
  > particular, regions that occupied the same pixels before the removed
  regions
  > were added, overlapping them, do not resume their previous role.
  >
  > A hit region A is an ancestor region of a hit region B if B has a
  parent and
  > its parent is either A or another hit region for which A is an ancestor
  > region.

  See above regarding support for nested/hierarchical relationships.
  >
  > The region identified by the ID ID in a bitmap bitmap is the value
  returned by
  > the following algorithm (which can return a hit region or nothing):
  >
  >   1. If ID is null, return nothing and abort these steps.
  >
  >   2. Let list be the hit region list associated with bitmap.
  >
  >   3. If there is a hit region in list whose ID is a case-sensitive
  match for ID, then return that hit region and abort these steps.
  >
  >   4. Otherwise, return nothing.
  >
  > The region representing the control control for a bitmap bitmap is the
  value
  > returned by the following algorithm (which can return a hit region or
  > nothing):
  >
  >   1. Let list be the hit region list associated with bitmap.
  >
  >   2. If there is a hit region in list whose control is control, then
  return that hit region and abort these steps.
  >
  >   3. Otherwise, return nothing.
  >
  > The control represented by a region region for a `canvas` element
  ancestor is
  > the value returned by the following algorithm (which can return an
  element or
  > nothing):
  >
  >   1. If region has no control, return nothing and abort these steps.
  >
  >   2. Let control be region's control.
  >
  >   3. If control is not a descendant of ancestor, then return nothing
  and abort these steps.
  >
  >   4. Otherwise, return control.
  >
  > The cursor for a hit region region of a `canvas` element ancestor is
  the value
  > returned by the following algorithm:
  >
  >   1. _Loop_: If region has a cursor specification other than
  "`inherit`", then return that hit region's cursor specification and abort
  these steps.
  >
  >   2. If region has a parent, then let region be that hit region's
  parent, and return to the step labeled _loop_.
  >
  >   3. Otherwise, return the used value of the 'cursor' property for the
  `canvas` element, if any; if there isn't one, return 'auto'. [CSSUI]
  >
  > The region for a pixel pixel on a bitmap bitmap is the value returned
  by the
  > following algorithm (which can return a hit region or nothing):
  >
  >   1. Let list be the hit region list associated with bitmap.
  >
  >   2. If there is a hit region in list whose set of pixels contains
  pixel, then return that hit region and abort these steps.
  >
  >   3. Otherwise, return nothing.
  >
  > To clear regions that cover the pixels pixels on a bitmap bitmap, the
  user
  > agent must run the following steps:
  >
  >   1. Let list be the hit region list associated with bitmap.
  >
  >   2. Remove all pixels in pixels from the set of pixels of each hit
  region in list.
  >
  >   3. Garbage-collect the regions of bitmap.
  >
  > To garbage-collect the regions of a bitmap bitmap, the user agent must
  run the
  > following steps:
  >
  >   1. Let list be the hit region list associated with bitmap.
  >
  >   2. _Loop_: Let victim be the first hit region in list to have an
  empty set of pixels and a zero child count, if any. If there is no such
  hit region, abort these steps.
  >
  >   3. If victim has a parent, then decrement that hit region's child
  count by one.
  >
  >   4. Remove victim from list.
  >
  >   5. Jump back to the step labeled _loop_.
  >
  > Adding a new region and calling `clearRect()` are the two ways this
  clearing
  > algorithm can be invoked. The hit region list itself is also reset when
  the
  > rendering context is reset, e.g. when a `CanvasRenderingContext2D`
  object is
  > bound to or unbound from a `canvas`, or when the dimensions of the
  bitmap are
  > changed.
  >
  > * * *
  >
  > When the `addHitRegion()` method is invoked, the user agent must run
  the
  > following steps:
  >
  >   1. Let arguments be the dictionary object provided as the method's
  argument.
  >
  >   2. If the arguments object's `path` member is not null, let source
  path be the `path` member's value. Otherwise, let it be the
  `CanvasRenderingContext2D` object's current default path.

  I assume this will be something like: "The source path will be the
  `CanvasRenderingContext2D` object's current default path."
  >
  >   3. Transform all the coordinates and lines in source path by the
  current transform matrix, if the arguments object's `path` member is not
  null.

  L2
  >
  >   4. Let specified pixels be the pixels contained in source path, using
  the fill rule indicated by the `fillRule` member.

  L2?

No, L1

  >
  >   5. If the arguments object's `id` member is the empty string, let it
  be null instead.

  L1
  >
  >   6. If the arguments object's `id` member is not null, then let
  previous region for this ID be the region identified by the ID given by
  the `id` member's value in this scratch bitmap, if any. If the `id`
  member is null or no such region currently exists, let previous region
  for this ID be null.
  L1
  >
  >   7. If the arguments object's `parent` member is the empty string, let
  it be null instead.
  L1?
  >
  >   8. If the arguments object's `parent` member is not null, then let
  parent region be the region identified by the ID given by the `parent`
  member's value in the scratch bitmap, if any. If the `parent` member is
  null or no such region currently exists, let parent region be null.
  L1?
  >
  >   9. If the arguments object's `label` member is the empty string, let
  it be null instead.
  L1
  >
  >   10. If any of the following conditions are met, throw a
  `NotSupportedError` exception and abort these steps.
  >
  >     * The arguments object's `control` and `label` members are both
  non-null.
  >     * The arguments object's `control` and `role` members are both
  non-null.
  >     * The arguments object's `role` member's value is the empty string,
  and the `label` member's value is either null or the empty string.
  Not necessary in L1 unless we support label and role.
  >     * The specified pixels has no pixels.
  >     * The arguments object's `control` member is not null but is
  neither an `a` element that represents a hyperlink, a `button` element,
  an `input` element whose `type` attribute is in one of the Checkbox or
  Radio Button states, nor an `input` element that is a button.
  I believe we do not need the above if the control is not limited.
  >     * The parent region is not null but has a control.
  >     * The previous region for this ID is the same hit region as the
  parent region.
  >     * The previous region for this ID is an ancestor region of the
  parent region.
  >   11. If the `parent` member is not null but parent region is null,
  then throw a `NotFoundError` exception and abort these steps.
  >
  >   12. If any of the following conditions are met, throw a `SyntaxError`
  exception and abort these steps.
  >
  >     * The arguments object's `cursor` member is not null but is neither
  an ASCII case-insensitive match for the string "`inherit`", nor a valid
  CSS 'cursor' property value. [CSSUI]
  Depends on if we support cursors in L1
  >     * The arguments object's `role` member is not null but its value is
  not an ordered set of unique space-separated tokens whose tokens are all
  case-sensitive matches for names of non-abstract WAI-ARIA roles. [ARIA]
  L2
  >   13. Let region be a newly created hit region, with its information
  configured as follows:
  >
  > Hit region's set of pixels
  >
  >
  >
  > The specified pixels
  >
  > Hit region's bounding circumference
  >
  >
  >
  > A user-agent-defined shape that wraps the pixels contained in source
  path. (In
  > the simplest case, this can just be the bounding rectangle; this
  specification
  > allows it to be any shape in order to allow other interfaces.)
  L1
  >
  > Hit region's ID
  >
  >
  >
  > If the arguments object's `id` member is not null: the value of the
  `id`
  > member. Otherwise, region has no id.
  L1
  >
  > Hit region's parent
  >
  >
  >
  > If parent region is not null: parent region. Otherwise, region has no
  parent.
  L1?
  >
  > Hit region's child count
  >
  >
  >
  > Initially zero.

  L1?
  >
  > Hit region's cursor specification
  L1?

No, let's do L2 for cursors

  >
  >
  >
  > If parent region is not null: parent region. Otherwise, region has no
  parent.
  >
  > Hit region's control
  >
  >
  >
  > If the arguments object's `control` member is not null: the value of
  the
  > `control` member. Otherwise, region has no control.
  L1
  >
  > Hit region's label
  >
  >
  >
  > If the arguments object's `label` member is not null: the value of the
  `label`
  > member. Otherwise, region has no label.
  L2
  >
  > Hit region's ARIA role
  >
  >
  >
  > If the arguments object's `role` member is not null: the value of the
  `role`
  > member (which might be the empty string). Otherwise, if the arguments
  object's
  > `label` member is not null: the empty string. Otherwise, region has no
  ARIA
  > role.
  L2
  >
  >   14. If the arguments object's `cursor` member is not null, then act
  as if a CSS rule for the `canvas` element setting its 'cursor' property
  had been seen, whose value was the hit region's cursor specification.
  L2?
  >
  > For example, if the user agent prefetches cursor values, this would
  cause that
  > to happen in response to an appropriately-formed `addHitRegion()` call.
  >
  >   15. If the arguments object's `control` member is not null, then let
  previous region for the control be the region representing the control
  given by the `control` member's value for this scratch bitmap, if any. If
  the `control` member is null or no such region currently exists, let
  previous region for the control be null.
  L1
  >
  >   16. If there is a previous region with this control, remove it from
  the scratch bitmap's hit region list; then, if it had a parent region,
  decrement that hit region's child count by one.
  L1
  >
  >   17. If there is a previous region with this ID, remove it, and all
  hit regions for which it is an ancestor region, from the scratch bitmap's
  hit region list; then, if it had a parent region, decrement that hit
  region's child count by one.
  L1
  >
  >   18. If there is a parent region, increment its hit region's child
  count by one.
  L1?
  >
  >   19. Clear regions that cover the pixels in region's set of pixels on
  this scratch bitmap.
  >
  >   20. Add region to the scratch bitmap's element's hit region list.
  >
  > When the `removeHitRegion()` method is invoked, the user agent must run
  the
  > following steps:
  >
  >   1. Let region be the region identified by the ID given by the
  method's argument in the rendering context's scratch bitmap. If no such
  region currently exists, abort these steps.
  >
  > If the method's argument is the empty string, then no region will
  match.
  >
  >   2. Remove region, and all hit regions for which it is an ancestor
  region, from the rendering context's scratch bitmap's hit region list;
  then, if it had a parent region, decrement that hit region's child count
  by one.
  >
  >   3. Garbage-collect the regions of the rendering context's scratch
  bitmap.

  L1?
  >
  > * * *
  >
  > The `MouseEvent` interface is extended to support hit regions:
  >
  >
  >     partial interface MouseEvent {
  >       readonly attribute DOMString? region;
  >     };
  >
  >     partial dictionary MouseEventInit {
  >       DOMString? region;
  >     };
  >
  > event . `region`
  >
  >
  >
  > If the mouse was over a hit region, then this returns the hit region's
  ID, if
  > it has one.
  >
  > Otherwise, returns null.
  >
  > The `region` attribute on `MouseEvent` objects must return the value it
  was
  > initialized to. When the object is created, this attribute must be
  initialized
  > to null. It represents the hit region's ID if the mouse was over a hit
  region
  > when the event was fired.
  L1
  >
  > When a `MouseEvent` is to be fired at a `canvas` element by the user
  agent in
  > response to a pointing device action, if the `canvas` element has a hit
  region
  > list, the user agent must instead follow these steps. If these steps
  say to
  > _act as normal_, that means that the event must be fired as it would
  have had
  > these requirements not been applied.
  >
  >   1. If the pointing device is not indicating a pixel on the `canvas`,
  act as normal and abort these steps.
  >
  >   2. Let pixel be the pixel indicated by the pointing device.
  >
  >   3. Let region be the hit region that is the region for the pixel
  pixel on this `canvas` element's bitmap, if any.
  >
  >   4. If there is no region, then act as normal and abort these steps.
  >
  >   5. Let id be the region's ID, if any.
  >
  >   6. If there is an id, then initialize the event object's `region`
  attribute to id.
  >
  >   7. Let control be the control represented by region for this `canvas`
  element, if any.
  >
  >   8. If there is a control, then target the event object at control
  instead of the `canvas` element.
  >
  >   9. Continue dispatching the event, but with the updated event object
  and target as given in the above steps.
  >
  > * * *
  >
  > When a user's pointing device cursor is positioned over a `canvas`
  element,
  > user agents should render the pointing device cursor according to the
  cursor
  > specification described by the cursor for the hit region that is the
  region
  > for the pixel that the pointing device designates on the `canvas`
  element's
  > bitmap.

  L1?
  >
  > * * *
  >
  > User agents are encouraged to make use of the information present in a
  > `canvas` element's hit region list to improve the accessibility of
  `canvas`
  > elements.
  >
  > Each hit region should be handled in a fashion equivalent to a node in
  a
  > virtual DOM tree rooted at the `canvas` element. The hierarchy of this
  virtual
  > DOM tree must match the hierarchy of the hit regions, as described by
  the
  > parent of each region. Regions without a parent must be treated as
  children of
  > the `canvas` element for the purpose of this virtual DOM tree. For each
  node
  > in such a DOM tree, the hit region's bounding circumference gives the
  region
  > of the screen to use when representing the node (if appropriate).
  >
  > The semantics of a hit region for the purposes of this virtual DOM tree
  are
  > those of the the control represented by the region, if it has one,

  L1
  or else of
  > a non-interactive element whose ARIA role, if any, is that given by the
  hit
  > region's ARIA role, and whose textual representation, if any, is given
  by the
  > hit region's label.

  L2
  >
  > For the purposes of accessibility tools, when an element C is a
  descendant of
  > a `canvas` element and there is a region representing the control C for
  that
  > `canvas` element's bitmap, then the element's position relative to the
  > document should be presented as if it was that region in the `canvas`
  > element's virtual DOM tree.
  L1
  >
  > The semantics of a hit region for the purposes of this virtual DOM tree
  are
  > those of the the control represented by the region, if it has one,
  L1
  or else of
  > a non-interactive element whose ARIA role, if any, is that given by the
  hit
  > region's ARIA role, and whose textual representation, if any, is given
  by the
  > hit region's label.
  L2
  >
  > Thus, for instance, a user agent on a touch-screen device could provide
  haptic
  > feedback when the user croses over a hit region's bounding
  circumference, and
  > then read the hit region's label to the user. Similarly, a desktop user
  agent
  > with a virtual accessibility focus separate from the keyboard input
  focus
  > could allow the user to navigate through the hit regions, using the
  virtual
  > DOM tree described above to enable hierarchical navigation. When an
  > interactive control inside the `canvas` element is focused, if the
  control has
  > a corresponding region, then that hit region's bounding circumference
  could be
  > used to determine what area of the display to magnify.
  L2?
  >

  --
  Mark Sadecki
  Web Accessibility Engineer
  World Wide Web Consortium, Web Accessibility Initiative
  Telephone: +1.617.715.4017
  Email: mark@w3.org
  Web: http://w3.org/People/mark