Re: Preliminary walk through of HitRegions ahead of Canvas call

Last week Dominic said he wants the hit regions to actually have hit
behavior. This means that event routing should kept in the spec.
I looked at the firefox code and at first glance this seems fairly easy to
implement.

Hopefully he will find time to implement this before we go to PR so we have
2 independent implementations.


On Mon, Mar 3, 2014 at 6:20 AM, Richard Schwerdtfeger <schwer@us.ibm.com>wrote:

> Where is Dominic on this? We need a second implementation in Chrome or we
> need to have something in Webkit.
>
>
> Rich Schwerdtfeger
>
> [image: Inactive hide details for Rik Cabanier ---03/02/2014 10:02:22
> PM---On Sun, Mar 2, 2014 at 3:09 PM, Mark Sadecki <mark@w3.org> w]Rik
> Cabanier ---03/02/2014 10:02:22 PM---On Sun, Mar 2, 2014 at 3:09 PM, Mark
> Sadecki <mark@w3.org> wrote: > I walked myself through the Hit
>
> From: Rik Cabanier <cabanier@gmail.com>
> To: Mark Sadecki <mark@w3.org>
> Cc: "public-canvas-api@w3.org" <public-canvas-api@w3.org>, Jatinder Mann <
> jmann@microsoft.com>, Jay Munro <jaymunro@microsoft.com>, "Rik Cabanier (
> cabanier@adobe.com)" <cabanier@adobe.com>, Richard
> Schwerdtfeger/Austin/IBM@IBMUS
> Date: 03/02/2014 10:02 PM
> Subject: Re: Preliminary walk through of HitRegions ahead of Canvas call
> ------------------------------
>
>
>
>
>
>
> On Sun, Mar 2, 2014 at 3:09 PM, Mark Sadecki <*mark@w3.org* <mark@w3.org>>
> wrote:
>
>    I walked myself through the Hit Regions spec and attempted to identify
>    sections that were a high priority for L1, those that would be nice to have
>    in L1 and those that could be delayed until L2.  During this walk through,
>    a few questions emerged.  The answers to these questions will most likely
>    affect what goes in L1 and what gets delayed until L2:
>
>    1. Do we plan on supporting nested regions with hierarchical/ancestor
>    relationships in L1? (Assuming so)
>
>
> Maybe. They don't seem too hard to implement.
>
>
>    2. Do we plan on supporting custom mouse cursors?
>
>
> No.
>
>
>    3. Do we need fill rule?
>
>
> Yes, L1
>
>
>    Assumptions I made:
>
>    1. We will not be supporting unbacked region descriptions like label
>    or ARIA roles for L1.
>    2. We will be supporting removeHitRegion
>    3. We will be supporting the MouseEvent interface
>
>
> Yes
>
>
>    Below are my notes.  We can use these as talking points during the
>    Canvas call on 03 March 2014.
>
>    Mark
>
>    > ###### 1.1.15 Hit regions
>    >
>    > A hit region list is a list of hit regions for a bitmap.
>
>    I imagine we are still supporting hit region lists.
>    >
>    > Each hit region consists of the following information:
>    >
>    >   * A set of pixels on the bitmap for which this region is
>    responsible.
>    >
>    >   * A bounding circumference on the bitmap that surrounds the hit
>    region's set of pixels as they stood when it was created.
>    >
>    >   * Optionally, a non-empty string representing an ID for
>    distinguishing the region from others.
>    >
>    >   * Optionally, a reference to another region that acts as the
>    parent for this one.
>    >
>    >   * A count of regions that have this one as their parent, known as
>    the hit region's child count.
>
>    Do we have plans to support nested regions with hierarchical/ancestor
>    relationships?
>    >
>    >   * A cursor specification, in the form of either a CSS cursor
>    value, or the string "`inherit`" meaning that the cursor of the hit
>    region's parent, if any, or of the `canvas` element, if not, is to be used
>    instead.
>
>    If this is not hard to implement, it could be nice to have.  Lower
>    priority though.
>
>
> yes, let's do it for L2
>
>
>    >
>    >   * Optionally, either a control, or an unbacked region description.
>
>    Control for L1 and unbacked region description for L2?
>
>
> yes
>
>
>    > A control is just a reference to an `Element` node, to which, in
>    certain
>    > conditions, the user agent will route events, and from which the
>    user agent
>    > will determine the state of the hit region for the purposes of
>    accessibility
>    > tools. (The control is ignored when it is not a descendant of the
>    `canvas`
>    > element.)
>    >
>    > An unbacked region description consists of the following:
>    >
>    >     * Optionally, a label.
>    >
>    >     * An ARIA role, which, if the unbacked region description also
>    has a label, could be the empty string.
>    >
>    > context . `addHitRegion`(options)
>
>    I assume this is for L2?
>
>
> yes
>
>
>    >
>    >
>    >
>    > Adds a hit region to the bitmap. The argument is an object with the
>    following
>    > members:
>    >
>    > `path` (default null)
>    >
>    >     A `Path` object that describes the pixels that form part of the
>    region. If this member is not provided or is set to null, the current
>    default path is used instead.
>    > `fillRule` (default "`nonzero`")
>
>    I understand we are only going to support the current default path at
>    this time.  How should we include that in the spec?
>
>
> Maybe the method should throw notsupported if you put pass in a path.
> (same with label and role)
>
>
>    >
>    >     The fill rule to use when determining which pixels are inside
>    the path.
>    > `id` (default empty string)
>
>    >From my understanding, fill rule will be needed for Hit Testing.  Is
>    that correct?  Is it a necessary part or can it wait until L2?  Seems like
>    Rik has basic Hit Testing support in the Firefox Nightly.  Not sure if it
>    uses fill rule at all.
>    >
>    >     The ID to use for this region. This is used in `MouseEvent`
>    events on the `canvas` (`event.region`) and as a way to reference this
>    region in later calls to `addHitRegion()`.
>    > `parentID` (default null)
>
>    Assuming we will support this.
>    >
>    >     The ID of the parent region, for purposes of navigation by
>    accessibility tools and for cursor fallback.
>    > `cursor` (default "`inherit`")
>
>    L1
>    >
>    >     The cursor to use when the mouse is over this region. The value
>    "`inherit`" means to use the cursor for the parent region (as specified by
>    the `parentID` member), if any, or to use the `canvas` element's cursor if
>    the region has no parent.
>    > `control` (default null)
>
>    Lower priority for L1
>
>
> Maybe your message content is a bit out of sync.
> Content is needed for L1.
>
>
>    >
>    >     An element (that is a descendant of the `canvas`) to which
>    events are to be routed, and which accessibility tools are to use as a
>    surrogate for describing and interacting with this region.
>    > `label` (default null)
>
>    L1
>
>
> No, that should go for L2.
>
>
>    >
>    >     A text label for accessibility tools to use as a description of
>    this region, if there is no control.
>    > `role` (default null)
>
>    L2
>    >
>    >     An ARIA role for accessibility tools to use to determine how to
>    represent this region, if there is no control.
>
>    L2
>    >
>    > Hit regions can be used for a variety of purposes:
>    >
>    >   * With an ID, they can make hit detection easier by having the
>    user agent check which region the mouse is over and include the ID in the
>    mouse events.
>    >   * With a control, they can make routing events to DOM elements
>    automatic, allowing e.g. clicks on a `canvas` to automatically submit a
>    form via a `button` element.
>    >   * With a label, they can make it easier for users to explore a
>    `canvas` without seeing it, e.g. by touch on a mobile device.
>    >   * With a cursor, they can make it easier for different regions of
>    the `canvas` to have different cursors, with the user agent automatically
>    switching between them.
>    > context . `removeHitRegion`(id)
>    >
>    >
>    >
>    > Removes a hit region (and all its descendants) from the canvas
>    bitmap. The
>    > argument is the ID of a region added using `addHitRegion()`.
>
>    L1
>    >
>    > The pixels that were covered by this region and its descendants are
>    > effectively cleared by this operation, leaving the regions
>    non-interactive. In
>    > particular, regions that occupied the same pixels before the removed
>    regions
>    > were added, overlapping them, do not resume their previous role.
>    >
>    > A hit region A is an ancestor region of a hit region B if B has a
>    parent and
>    > its parent is either A or another hit region for which A is an
>    ancestor
>    > region.
>
>    See above regarding support for nested/hierarchical relationships.
>    >
>    > The region identified by the ID ID in a bitmap bitmap is the value
>    returned by
>    > the following algorithm (which can return a hit region or nothing):
>    >
>    >   1. If ID is null, return nothing and abort these steps.
>    >
>    >   2. Let list be the hit region list associated with bitmap.
>    >
>    >   3. If there is a hit region in list whose ID is a case-sensitive
>    match for ID, then return that hit region and abort these steps.
>    >
>    >   4. Otherwise, return nothing.
>    >
>    > The region representing the control control for a bitmap bitmap is
>    the value
>    > returned by the following algorithm (which can return a hit region or
>    > nothing):
>    >
>    >   1. Let list be the hit region list associated with bitmap.
>    >
>    >   2. If there is a hit region in list whose control is control, then
>    return that hit region and abort these steps.
>    >
>    >   3. Otherwise, return nothing.
>    >
>    > The control represented by a region region for a `canvas` element
>    ancestor is
>    > the value returned by the following algorithm (which can return an
>    element or
>    > nothing):
>    >
>    >   1. If region has no control, return nothing and abort these steps.
>    >
>    >   2. Let control be region's control.
>    >
>    >   3. If control is not a descendant of ancestor, then return nothing
>    and abort these steps.
>    >
>    >   4. Otherwise, return control.
>    >
>    > The cursor for a hit region region of a `canvas` element ancestor is
>    the value
>    > returned by the following algorithm:
>    >
>    >   1. _Loop_: If region has a cursor specification other than
>    "`inherit`", then return that hit region's cursor specification and abort
>    these steps.
>    >
>    >   2. If region has a parent, then let region be that hit region's
>    parent, and return to the step labeled _loop_.
>    >
>    >   3. Otherwise, return the used value of the 'cursor' property for
>    the `canvas` element, if any; if there isn't one, return 'auto'. [CSSUI]
>    >
>    > The region for a pixel pixel on a bitmap bitmap is the value
>    returned by the
>    > following algorithm (which can return a hit region or nothing):
>    >
>    >   1. Let list be the hit region list associated with bitmap.
>    >
>    >   2. If there is a hit region in list whose set of pixels contains
>    pixel, then return that hit region and abort these steps.
>    >
>    >   3. Otherwise, return nothing.
>    >
>    > To clear regions that cover the pixels pixels on a bitmap bitmap,
>    the user
>    > agent must run the following steps:
>    >
>    >   1. Let list be the hit region list associated with bitmap.
>    >
>    >   2. Remove all pixels in pixels from the set of pixels of each hit
>    region in list.
>    >
>    >   3. Garbage-collect the regions of bitmap.
>    >
>    > To garbage-collect the regions of a bitmap bitmap, the user agent
>    must run the
>    > following steps:
>    >
>    >   1. Let list be the hit region list associated with bitmap.
>    >
>    >   2. _Loop_: Let victim be the first hit region in list to have an
>    empty set of pixels and a zero child count, if any. If there is no such hit
>    region, abort these steps.
>    >
>    >   3. If victim has a parent, then decrement that hit region's child
>    count by one.
>    >
>    >   4. Remove victim from list.
>    >
>    >   5. Jump back to the step labeled _loop_.
>    >
>    > Adding a new region and calling `clearRect()` are the two ways this
>    clearing
>    > algorithm can be invoked. The hit region list itself is also reset
>    when the
>    > rendering context is reset, e.g. when a `CanvasRenderingContext2D`
>    object is
>    > bound to or unbound from a `canvas`, or when the dimensions of the
>    bitmap are
>    > changed.
>    >
>    > * * *
>    >
>    > When the `addHitRegion()` method is invoked, the user agent must run
>    the
>    > following steps:
>    >
>    >   1. Let arguments be the dictionary object provided as the method's
>    argument.
>    >
>    >   2. If the arguments object's `path` member is not null, let source
>    path be the `path` member's value. Otherwise, let it be the
>    `CanvasRenderingContext2D` object's current default path.
>
>    I assume this will be something like: "The source path will be the
>    `CanvasRenderingContext2D` object's current default path."
>    >
>    >   3. Transform all the coordinates and lines in source path by the
>    current transform matrix, if the arguments object's `path` member is not
>    null.
>
>    L2
>    >
>    >   4. Let specified pixels be the pixels contained in source path,
>    using the fill rule indicated by the `fillRule` member.
>
>    L2?
>
>
> No, L1
>
>
>    >
>    >   5. If the arguments object's `id` member is the empty string, let
>    it be null instead.
>
>    L1
>    >
>    >   6. If the arguments object's `id` member is not null, then let
>    previous region for this ID be the region identified by the ID given by the
>    `id` member's value in this scratch bitmap, if any. If the `id` member is
>    null or no such region currently exists, let previous region for this ID be
>    null.
>    L1
>    >
>    >   7. If the arguments object's `parent` member is the empty string,
>    let it be null instead.
>    L1?
>    >
>    >   8. If the arguments object's `parent` member is not null, then let
>    parent region be the region identified by the ID given by the `parent`
>    member's value in the scratch bitmap, if any. If the `parent` member is
>    null or no such region currently exists, let parent region be null.
>    L1?
>    >
>    >   9. If the arguments object's `label` member is the empty string,
>    let it be null instead.
>    L1
>    >
>    >   10. If any of the following conditions are met, throw a
>    `NotSupportedError` exception and abort these steps.
>    >
>    >     * The arguments object's `control` and `label` members are both
>    non-null.
>    >     * The arguments object's `control` and `role` members are both
>    non-null.
>    >     * The arguments object's `role` member's value is the empty
>    string, and the `label` member's value is either null or the empty string.
>    Not necessary in L1 unless we support label and role.
>    >     * The specified pixels has no pixels.
>    >     * The arguments object's `control` member is not null but is
>    neither an `a` element that represents a hyperlink, a `button` element, an
>    `input` element whose `type` attribute is in one of the Checkbox or Radio
>    Button states, nor an `input` element that is a button.
>    I believe we do not need the above if the control is not limited.
>    >     * The parent region is not null but has a control.
>    >     * The previous region for this ID is the same hit region as the
>    parent region.
>    >     * The previous region for this ID is an ancestor region of the
>    parent region.
>    >   11. If the `parent` member is not null but parent region is null,
>    then throw a `NotFoundError` exception and abort these steps.
>    >
>    >   12. If any of the following conditions are met, throw a
>    `SyntaxError` exception and abort these steps.
>    >
>    >     * The arguments object's `cursor` member is not null but is
>    neither an ASCII case-insensitive match for the string "`inherit`", nor a
>    valid CSS 'cursor' property value. [CSSUI]
>    Depends on if we support cursors in L1
>    >     * The arguments object's `role` member is not null but its value
>    is not an ordered set of unique space-separated tokens whose tokens are all
>    case-sensitive matches for names of non-abstract WAI-ARIA roles. [ARIA]
>    L2
>    >   13. Let region be a newly created hit region, with its information
>    configured as follows:
>    >
>    > Hit region's set of pixels
>    >
>    >
>    >
>    > The specified pixels
>    >
>    > Hit region's bounding circumference
>    >
>    >
>    >
>    > A user-agent-defined shape that wraps the pixels contained in source
>    path. (In
>    > the simplest case, this can just be the bounding rectangle; this
>    specification
>    > allows it to be any shape in order to allow other interfaces.)
>    L1
>    >
>    > Hit region's ID
>    >
>    >
>    >
>    > If the arguments object's `id` member is not null: the value of the
>    `id`
>    > member. Otherwise, region has no id.
>    L1
>    >
>    > Hit region's parent
>    >
>    >
>    >
>    > If parent region is not null: parent region. Otherwise, region has
>    no parent.
>    L1?
>    >
>    > Hit region's child count
>    >
>    >
>    >
>    > Initially zero.
>
>    L1?
>    >
>    > Hit region's cursor specification
>    L1?
>
>
> No, let's do L2 for cursors
>
>
>    >
>    >
>    >
>    > If parent region is not null: parent region. Otherwise, region has
>    no parent.
>    >
>    > Hit region's control
>    >
>    >
>    >
>    > If the arguments object's `control` member is not null: the value of
>    the
>    > `control` member. Otherwise, region has no control.
>    L1
>    >
>    > Hit region's label
>    >
>    >
>    >
>    > If the arguments object's `label` member is not null: the value of
>    the `label`
>    > member. Otherwise, region has no label.
>    L2
>    >
>    > Hit region's ARIA role
>    >
>    >
>    >
>    > If the arguments object's `role` member is not null: the value of
>    the `role`
>    > member (which might be the empty string). Otherwise, if the
>    arguments object's
>    > `label` member is not null: the empty string. Otherwise, region has
>    no ARIA
>    > role.
>    L2
>    >
>    >   14. If the arguments object's `cursor` member is not null, then
>    act as if a CSS rule for the `canvas` element setting its 'cursor' property
>    had been seen, whose value was the hit region's cursor specification.
>    L2?
>    >
>    > For example, if the user agent prefetches cursor values, this would
>    cause that
>    > to happen in response to an appropriately-formed `addHitRegion()`
>    call.
>    >
>    >   15. If the arguments object's `control` member is not null, then
>    let previous region for the control be the region representing the control
>    given by the `control` member's value for this scratch bitmap, if any. If
>    the `control` member is null or no such region currently exists, let
>    previous region for the control be null.
>    L1
>    >
>    >   16. If there is a previous region with this control, remove it
>    from the scratch bitmap's hit region list; then, if it had a parent region,
>    decrement that hit region's child count by one.
>    L1
>    >
>    >   17. If there is a previous region with this ID, remove it, and all
>    hit regions for which it is an ancestor region, from the scratch bitmap's
>    hit region list; then, if it had a parent region, decrement that hit
>    region's child count by one.
>    L1
>    >
>    >   18. If there is a parent region, increment its hit region's child
>    count by one.
>    L1?
>    >
>    >   19. Clear regions that cover the pixels in region's set of pixels
>    on this scratch bitmap.
>    >
>    >   20. Add region to the scratch bitmap's element's hit region list.
>    >
>    > When the `removeHitRegion()` method is invoked, the user agent must
>    run the
>    > following steps:
>    >
>    >   1. Let region be the region identified by the ID given by the
>    method's argument in the rendering context's scratch bitmap. If no such
>    region currently exists, abort these steps.
>    >
>    > If the method's argument is the empty string, then no region will
>    match.
>    >
>    >   2. Remove region, and all hit regions for which it is an ancestor
>    region, from the rendering context's scratch bitmap's hit region list;
>    then, if it had a parent region, decrement that hit region's child count by
>    one.
>    >
>    >   3. Garbage-collect the regions of the rendering context's scratch
>    bitmap.
>
>    L1?
>    >
>    > * * *
>    >
>    > The `MouseEvent` interface is extended to support hit regions:
>    >
>    >
>    >     partial interface MouseEvent {
>    >       readonly attribute DOMString? region;
>    >     };
>    >
>    >     partial dictionary MouseEventInit {
>    >       DOMString? region;
>    >     };
>    >
>    > event . `region`
>    >
>    >
>    >
>    > If the mouse was over a hit region, then this returns the hit
>    region's ID, if
>    > it has one.
>    >
>    > Otherwise, returns null.
>    >
>    > The `region` attribute on `MouseEvent` objects must return the value
>    it was
>    > initialized to. When the object is created, this attribute must be
>    initialized
>    > to null. It represents the hit region's ID if the mouse was over a
>    hit region
>    > when the event was fired.
>    L1
>    >
>    > When a `MouseEvent` is to be fired at a `canvas` element by the user
>    agent in
>    > response to a pointing device action, if the `canvas` element has a
>    hit region
>    > list, the user agent must instead follow these steps. If these steps
>    say to
>    > _act as normal_, that means that the event must be fired as it would
>    have had
>    > these requirements not been applied.
>    >
>    >   1. If the pointing device is not indicating a pixel on the
>    `canvas`, act as normal and abort these steps.
>    >
>    >   2. Let pixel be the pixel indicated by the pointing device.
>    >
>    >   3. Let region be the hit region that is the region for the pixel
>    pixel on this `canvas` element's bitmap, if any.
>    >
>    >   4. If there is no region, then act as normal and abort these steps.
>    >
>    >   5. Let id be the region's ID, if any.
>    >
>    >   6. If there is an id, then initialize the event object's `region`
>    attribute to id.
>    >
>    >   7. Let control be the control represented by region for this
>    `canvas` element, if any.
>    >
>    >   8. If there is a control, then target the event object at control
>    instead of the `canvas` element.
>    >
>    >   9. Continue dispatching the event, but with the updated event
>    object and target as given in the above steps.
>    >
>    > * * *
>    >
>    > When a user's pointing device cursor is positioned over a `canvas`
>    element,
>    > user agents should render the pointing device cursor according to
>    the cursor
>    > specification described by the cursor for the hit region that is the
>    region
>    > for the pixel that the pointing device designates on the `canvas`
>    element's
>    > bitmap.
>
>    L1?
>    >
>    > * * *
>    >
>    > User agents are encouraged to make use of the information present in
>    a
>    > `canvas` element's hit region list to improve the accessibility of
>    `canvas`
>    > elements.
>    >
>    > Each hit region should be handled in a fashion equivalent to a node
>    in a
>    > virtual DOM tree rooted at the `canvas` element. The hierarchy of
>    this virtual
>    > DOM tree must match the hierarchy of the hit regions, as described
>    by the
>    > parent of each region. Regions without a parent must be treated as
>    children of
>    > the `canvas` element for the purpose of this virtual DOM tree. For
>    each node
>    > in such a DOM tree, the hit region's bounding circumference gives
>    the region
>    > of the screen to use when representing the node (if appropriate).
>    >
>    > The semantics of a hit region for the purposes of this virtual DOM
>    tree are
>    > those of the the control represented by the region, if it has one,
>
>    L1
>    or else of
>    > a non-interactive element whose ARIA role, if any, is that given by
>    the hit
>    > region's ARIA role, and whose textual representation, if any, is
>    given by the
>    > hit region's label.
>
>    L2
>    >
>    > For the purposes of accessibility tools, when an element C is a
>    descendant of
>    > a `canvas` element and there is a region representing the control C
>    for that
>    > `canvas` element's bitmap, then the element's position relative to
>    the
>    > document should be presented as if it was that region in the `canvas`
>    > element's virtual DOM tree.
>    L1
>    >
>    > The semantics of a hit region for the purposes of this virtual DOM
>    tree are
>    > those of the the control represented by the region, if it has one,
>    L1
>    or else of
>    > a non-interactive element whose ARIA role, if any, is that given by
>    the hit
>    > region's ARIA role, and whose textual representation, if any, is
>    given by the
>    > hit region's label.
>    L2
>    >
>    > Thus, for instance, a user agent on a touch-screen device could
>    provide haptic
>    > feedback when the user croses over a hit region's bounding
>    circumference, and
>    > then read the hit region's label to the user. Similarly, a desktop
>    user agent
>    > with a virtual accessibility focus separate from the keyboard input
>    focus
>    > could allow the user to navigate through the hit regions, using the
>    virtual
>    > DOM tree described above to enable hierarchical navigation. When an
>    > interactive control inside the `canvas` element is focused, if the
>    control has
>    > a corresponding region, then that hit region's bounding
>    circumference could be
>    > used to determine what area of the display to magnify.
>    L2?
>    >
>
>    --
>    Mark Sadecki
>    Web Accessibility Engineer
>    World Wide Web Consortium, Web Accessibility Initiative
>    Telephone: *+1.617.715.4017* <%2B1.617.715.4017>
>    Email: *mark@w3.org* <mark@w3.org>
>    Web: *http://w3.org/People/mark* <http://w3.org/People/mark>
>
>
>