Re: Preliminary walk through of HitRegions ahead of Canvas call

On Sun, Mar 2, 2014 at 8:01 PM, Rik Cabanier <cabanier@gmail.com> wrote:

>
>
>
> On Sun, Mar 2, 2014 at 3:09 PM, Mark Sadecki <mark@w3.org> wrote:
>
>> I walked myself through the Hit Regions spec and attempted to identify
>> sections that were a high priority for L1, those that would be nice to have
>> in L1 and those that could be delayed until L2.  During this walk through,
>> a few questions emerged.  The answers to these questions will most likely
>> affect what goes in L1 and what gets delayed until L2:
>>
>> 1. Do we plan on supporting nested regions with hierarchical/ancestor
>> relationships in L1? (Assuming so)
>>
>
> Maybe. They don't seem too hard to implement.
>
>
>> 2. Do we plan on supporting custom mouse cursors?
>>
>
> No.
>
>
>> 3. Do we need fill rule?
>>
>
> Yes, L1
>

I noticed that the w3c version of canvas doesn't have support for winding
rules (even though all browsers implemented it) so this can be dropped.


>
>
>> Assumptions I made:
>>
>> 1. We will not be supporting unbacked region descriptions like label or
>> ARIA roles for L1.
>> 2. We will be supporting removeHitRegion
>> 3. We will be supporting the MouseEvent interface
>>
>
> Yes
>
>
>> Below are my notes.  We can use these as talking points during the Canvas
>> call on 03 March 2014.
>>
>> Mark
>>
>> > ###### 1.1.15 Hit regions
>> >
>> > A hit region list is a list of hit regions for a bitmap.
>>
>> I imagine we are still supporting hit region lists.
>> >
>> > Each hit region consists of the following information:
>> >
>> >   * A set of pixels on the bitmap for which this region is responsible.
>> >
>> >   * A bounding circumference on the bitmap that surrounds the hit
>> region's set of pixels as they stood when it was created.
>> >
>> >   * Optionally, a non-empty string representing an ID for
>> distinguishing the region from others.
>> >
>> >   * Optionally, a reference to another region that acts as the parent
>> for this one.
>> >
>> >   * A count of regions that have this one as their parent, known as the
>> hit region's child count.
>>
>> Do we have plans to support nested regions with hierarchical/ancestor
>> relationships?
>> >
>> >   * A cursor specification, in the form of either a CSS cursor value,
>> or the string "`inherit`" meaning that the cursor of the hit region's
>> parent, if any, or of the `canvas` element, if not, is to be used instead.
>>
>> If this is not hard to implement, it could be nice to have.  Lower
>> priority though.
>>
>
> yes, let's do it for L2
>
>
>> >
>> >   * Optionally, either a control, or an unbacked region description.
>>
>> Control for L1 and unbacked region description for L2?
>>
>
> yes
>
>
>> > A control is just a reference to an `Element` node, to which, in certain
>> > conditions, the user agent will route events, and from which the user
>> agent
>> > will determine the state of the hit region for the purposes of
>> accessibility
>> > tools. (The control is ignored when it is not a descendant of the
>> `canvas`
>> > element.)
>> >
>> > An unbacked region description consists of the following:
>> >
>> >     * Optionally, a label.
>> >
>> >     * An ARIA role, which, if the unbacked region description also has
>> a label, could be the empty string.
>> >
>> > context . `addHitRegion`(options)
>>
>> I assume this is for L2?
>>
>
> yes
>
>
>> >
>> >
>> >
>> > Adds a hit region to the bitmap. The argument is an object with the
>> following
>> > members:
>> >
>> > `path` (default null)
>> >
>> >     A `Path` object that describes the pixels that form part of the
>> region. If this member is not provided or is set to null, the current
>> default path is used instead.
>> > `fillRule` (default "`nonzero`")
>>
>> I understand we are only going to support the current default path at
>> this time.  How should we include that in the spec?
>>
>
> Maybe the method should throw notsupported if you put pass in a path.
> (same with label and role)
>
>
>> >
>> >     The fill rule to use when determining which pixels are inside the
>> path.
>> > `id` (default empty string)
>>
>> >From my understanding, fill rule will be needed for Hit Testing.  Is
>> that correct?  Is it a necessary part or can it wait until L2?  Seems like
>> Rik has basic Hit Testing support in the Firefox Nightly.  Not sure if it
>> uses fill rule at all.
>> >
>> >     The ID to use for this region. This is used in `MouseEvent` events
>> on the `canvas` (`event.region`) and as a way to reference this region in
>> later calls to `addHitRegion()`.
>> > `parentID` (default null)
>>
>> Assuming we will support this.
>> >
>> >     The ID of the parent region, for purposes of navigation by
>> accessibility tools and for cursor fallback.
>> > `cursor` (default "`inherit`")
>>
>> L1
>> >
>> >     The cursor to use when the mouse is over this region. The value
>> "`inherit`" means to use the cursor for the parent region (as specified by
>> the `parentID` member), if any, or to use the `canvas` element's cursor if
>> the region has no parent.
>> > `control` (default null)
>>
>> Lower priority for L1
>>
>
> Maybe your message content is a bit out of sync.
> Content is needed for L1.
>
>
>> >
>> >     An element (that is a descendant of the `canvas`) to which events
>> are to be routed, and which accessibility tools are to use as a surrogate
>> for describing and interacting with this region.
>> > `label` (default null)
>>
>> L1
>>
>
> No, that should go for L2.
>
>
>> >
>> >     A text label for accessibility tools to use as a description of
>> this region, if there is no control.
>> > `role` (default null)
>>
>> L2
>> >
>> >     An ARIA role for accessibility tools to use to determine how to
>> represent this region, if there is no control.
>>
>> L2
>> >
>> > Hit regions can be used for a variety of purposes:
>> >
>> >   * With an ID, they can make hit detection easier by having the user
>> agent check which region the mouse is over and include the ID in the mouse
>> events.
>> >   * With a control, they can make routing events to DOM elements
>> automatic, allowing e.g. clicks on a `canvas` to automatically submit a
>> form via a `button` element.
>> >   * With a label, they can make it easier for users to explore a
>> `canvas` without seeing it, e.g. by touch on a mobile device.
>> >   * With a cursor, they can make it easier for different regions of the
>> `canvas` to have different cursors, with the user agent automatically
>> switching between them.
>> > context . `removeHitRegion`(id)
>> >
>> >
>> >
>> > Removes a hit region (and all its descendants) from the canvas bitmap.
>> The
>> > argument is the ID of a region added using `addHitRegion()`.
>>
>> L1
>> >
>> > The pixels that were covered by this region and its descendants are
>> > effectively cleared by this operation, leaving the regions
>> non-interactive. In
>> > particular, regions that occupied the same pixels before the removed
>> regions
>> > were added, overlapping them, do not resume their previous role.
>> >
>> > A hit region A is an ancestor region of a hit region B if B has a
>> parent and
>> > its parent is either A or another hit region for which A is an ancestor
>> > region.
>>
>> See above regarding support for nested/hierarchical relationships.
>> >
>> > The region identified by the ID ID in a bitmap bitmap is the value
>> returned by
>> > the following algorithm (which can return a hit region or nothing):
>> >
>> >   1. If ID is null, return nothing and abort these steps.
>> >
>> >   2. Let list be the hit region list associated with bitmap.
>> >
>> >   3. If there is a hit region in list whose ID is a case-sensitive
>> match for ID, then return that hit region and abort these steps.
>> >
>> >   4. Otherwise, return nothing.
>> >
>> > The region representing the control control for a bitmap bitmap is the
>> value
>> > returned by the following algorithm (which can return a hit region or
>> > nothing):
>> >
>> >   1. Let list be the hit region list associated with bitmap.
>> >
>> >   2. If there is a hit region in list whose control is control, then
>> return that hit region and abort these steps.
>> >
>> >   3. Otherwise, return nothing.
>> >
>> > The control represented by a region region for a `canvas` element
>> ancestor is
>> > the value returned by the following algorithm (which can return an
>> element or
>> > nothing):
>> >
>> >   1. If region has no control, return nothing and abort these steps.
>> >
>> >   2. Let control be region's control.
>> >
>> >   3. If control is not a descendant of ancestor, then return nothing
>> and abort these steps.
>> >
>> >   4. Otherwise, return control.
>> >
>> > The cursor for a hit region region of a `canvas` element ancestor is
>> the value
>> > returned by the following algorithm:
>> >
>> >   1. _Loop_: If region has a cursor specification other than
>> "`inherit`", then return that hit region's cursor specification and abort
>> these steps.
>> >
>> >   2. If region has a parent, then let region be that hit region's
>> parent, and return to the step labeled _loop_.
>> >
>> >   3. Otherwise, return the used value of the 'cursor' property for the
>> `canvas` element, if any; if there isn't one, return 'auto'. [CSSUI]
>> >
>> > The region for a pixel pixel on a bitmap bitmap is the value returned
>> by the
>> > following algorithm (which can return a hit region or nothing):
>> >
>> >   1. Let list be the hit region list associated with bitmap.
>> >
>> >   2. If there is a hit region in list whose set of pixels contains
>> pixel, then return that hit region and abort these steps.
>> >
>> >   3. Otherwise, return nothing.
>> >
>> > To clear regions that cover the pixels pixels on a bitmap bitmap, the
>> user
>> > agent must run the following steps:
>> >
>> >   1. Let list be the hit region list associated with bitmap.
>> >
>> >   2. Remove all pixels in pixels from the set of pixels of each hit
>> region in list.
>> >
>> >   3. Garbage-collect the regions of bitmap.
>> >
>> > To garbage-collect the regions of a bitmap bitmap, the user agent must
>> run the
>> > following steps:
>> >
>> >   1. Let list be the hit region list associated with bitmap.
>> >
>> >   2. _Loop_: Let victim be the first hit region in list to have an
>> empty set of pixels and a zero child count, if any. If there is no such hit
>> region, abort these steps.
>> >
>> >   3. If victim has a parent, then decrement that hit region's child
>> count by one.
>> >
>> >   4. Remove victim from list.
>> >
>> >   5. Jump back to the step labeled _loop_.
>> >
>> > Adding a new region and calling `clearRect()` are the two ways this
>> clearing
>> > algorithm can be invoked. The hit region list itself is also reset when
>> the
>> > rendering context is reset, e.g. when a `CanvasRenderingContext2D`
>> object is
>> > bound to or unbound from a `canvas`, or when the dimensions of the
>> bitmap are
>> > changed.
>> >
>> > * * *
>> >
>> > When the `addHitRegion()` method is invoked, the user agent must run the
>> > following steps:
>> >
>> >   1. Let arguments be the dictionary object provided as the method's
>> argument.
>> >
>> >   2. If the arguments object's `path` member is not null, let source
>> path be the `path` member's value. Otherwise, let it be the
>> `CanvasRenderingContext2D` object's current default path.
>>
>> I assume this will be something like: "The source path will be the
>> `CanvasRenderingContext2D` object's current default path."
>> >
>> >   3. Transform all the coordinates and lines in source path by the
>> current transform matrix, if the arguments object's `path` member is not
>> null.
>>
>> L2
>> >
>> >   4. Let specified pixels be the pixels contained in source path, using
>> the fill rule indicated by the `fillRule` member.
>>
>> L2?
>>
>
> No, L1
>
>
>> >
>> >   5. If the arguments object's `id` member is the empty string, let it
>> be null instead.
>>
>> L1
>> >
>> >   6. If the arguments object's `id` member is not null, then let
>> previous region for this ID be the region identified by the ID given by the
>> `id` member's value in this scratch bitmap, if any. If the `id` member is
>> null or no such region currently exists, let previous region for this ID be
>> null.
>> L1
>> >
>> >   7. If the arguments object's `parent` member is the empty string, let
>> it be null instead.
>> L1?
>> >
>> >   8. If the arguments object's `parent` member is not null, then let
>> parent region be the region identified by the ID given by the `parent`
>> member's value in the scratch bitmap, if any. If the `parent` member is
>> null or no such region currently exists, let parent region be null.
>> L1?
>> >
>> >   9. If the arguments object's `label` member is the empty string, let
>> it be null instead.
>> L1
>> >
>> >   10. If any of the following conditions are met, throw a
>> `NotSupportedError` exception and abort these steps.
>> >
>> >     * The arguments object's `control` and `label` members are both
>> non-null.
>> >     * The arguments object's `control` and `role` members are both
>> non-null.
>> >     * The arguments object's `role` member's value is the empty string,
>> and the `label` member's value is either null or the empty string.
>> Not necessary in L1 unless we support label and role.
>> >     * The specified pixels has no pixels.
>> >     * The arguments object's `control` member is not null but is
>> neither an `a` element that represents a hyperlink, a `button` element, an
>> `input` element whose `type` attribute is in one of the Checkbox or Radio
>> Button states, nor an `input` element that is a button.
>> I believe we do not need the above if the control is not limited.
>> >     * The parent region is not null but has a control.
>> >     * The previous region for this ID is the same hit region as the
>> parent region.
>> >     * The previous region for this ID is an ancestor region of the
>> parent region.
>> >   11. If the `parent` member is not null but parent region is null,
>> then throw a `NotFoundError` exception and abort these steps.
>> >
>> >   12. If any of the following conditions are met, throw a `SyntaxError`
>> exception and abort these steps.
>> >
>> >     * The arguments object's `cursor` member is not null but is neither
>> an ASCII case-insensitive match for the string "`inherit`", nor a valid CSS
>> 'cursor' property value. [CSSUI]
>> Depends on if we support cursors in L1
>> >     * The arguments object's `role` member is not null but its value is
>> not an ordered set of unique space-separated tokens whose tokens are all
>> case-sensitive matches for names of non-abstract WAI-ARIA roles. [ARIA]
>> L2
>> >   13. Let region be a newly created hit region, with its information
>> configured as follows:
>> >
>> > Hit region's set of pixels
>> >
>> >
>> >
>> > The specified pixels
>> >
>> > Hit region's bounding circumference
>> >
>> >
>> >
>> > A user-agent-defined shape that wraps the pixels contained in source
>> path. (In
>> > the simplest case, this can just be the bounding rectangle; this
>> specification
>> > allows it to be any shape in order to allow other interfaces.)
>> L1
>> >
>> > Hit region's ID
>> >
>> >
>> >
>> > If the arguments object's `id` member is not null: the value of the `id`
>> > member. Otherwise, region has no id.
>> L1
>> >
>> > Hit region's parent
>> >
>> >
>> >
>> > If parent region is not null: parent region. Otherwise, region has no
>> parent.
>> L1?
>> >
>> > Hit region's child count
>> >
>> >
>> >
>> > Initially zero.
>>
>> L1?
>> >
>> > Hit region's cursor specification
>> L1?
>>
>
> No, let's do L2 for cursors
>
>
>> >
>> >
>> >
>> > If parent region is not null: parent region. Otherwise, region has no
>> parent.
>> >
>> > Hit region's control
>> >
>> >
>> >
>> > If the arguments object's `control` member is not null: the value of the
>> > `control` member. Otherwise, region has no control.
>> L1
>> >
>> > Hit region's label
>> >
>> >
>> >
>> > If the arguments object's `label` member is not null: the value of the
>> `label`
>> > member. Otherwise, region has no label.
>> L2
>> >
>> > Hit region's ARIA role
>> >
>> >
>> >
>> > If the arguments object's `role` member is not null: the value of the
>> `role`
>> > member (which might be the empty string). Otherwise, if the arguments
>> object's
>> > `label` member is not null: the empty string. Otherwise, region has no
>> ARIA
>> > role.
>> L2
>> >
>> >   14. If the arguments object's `cursor` member is not null, then act
>> as if a CSS rule for the `canvas` element setting its 'cursor' property had
>> been seen, whose value was the hit region's cursor specification.
>> L2?
>> >
>> > For example, if the user agent prefetches cursor values, this would
>> cause that
>> > to happen in response to an appropriately-formed `addHitRegion()` call.
>> >
>> >   15. If the arguments object's `control` member is not null, then let
>> previous region for the control be the region representing the control
>> given by the `control` member's value for this scratch bitmap, if any. If
>> the `control` member is null or no such region currently exists, let
>> previous region for the control be null.
>> L1
>> >
>> >   16. If there is a previous region with this control, remove it from
>> the scratch bitmap's hit region list; then, if it had a parent region,
>> decrement that hit region's child count by one.
>> L1
>> >
>> >   17. If there is a previous region with this ID, remove it, and all
>> hit regions for which it is an ancestor region, from the scratch bitmap's
>> hit region list; then, if it had a parent region, decrement that hit
>> region's child count by one.
>> L1
>> >
>> >   18. If there is a parent region, increment its hit region's child
>> count by one.
>> L1?
>> >
>> >   19. Clear regions that cover the pixels in region's set of pixels on
>> this scratch bitmap.
>> >
>> >   20. Add region to the scratch bitmap's element's hit region list.
>> >
>> > When the `removeHitRegion()` method is invoked, the user agent must run
>> the
>> > following steps:
>> >
>> >   1. Let region be the region identified by the ID given by the
>> method's argument in the rendering context's scratch bitmap. If no such
>> region currently exists, abort these steps.
>> >
>> > If the method's argument is the empty string, then no region will match.
>> >
>> >   2. Remove region, and all hit regions for which it is an ancestor
>> region, from the rendering context's scratch bitmap's hit region list;
>> then, if it had a parent region, decrement that hit region's child count by
>> one.
>> >
>> >   3. Garbage-collect the regions of the rendering context's scratch
>> bitmap.
>>
>> L1?
>> >
>> > * * *
>> >
>> > The `MouseEvent` interface is extended to support hit regions:
>> >
>> >
>> >     partial interface MouseEvent {
>> >       readonly attribute DOMString? region;
>> >     };
>> >
>> >     partial dictionary MouseEventInit {
>> >       DOMString? region;
>> >     };
>> >
>> > event . `region`
>> >
>> >
>> >
>> > If the mouse was over a hit region, then this returns the hit region's
>> ID, if
>> > it has one.
>> >
>> > Otherwise, returns null.
>> >
>> > The `region` attribute on `MouseEvent` objects must return the value it
>> was
>> > initialized to. When the object is created, this attribute must be
>> initialized
>> > to null. It represents the hit region's ID if the mouse was over a hit
>> region
>> > when the event was fired.
>> L1
>> >
>> > When a `MouseEvent` is to be fired at a `canvas` element by the user
>> agent in
>> > response to a pointing device action, if the `canvas` element has a hit
>> region
>> > list, the user agent must instead follow these steps. If these steps
>> say to
>> > _act as normal_, that means that the event must be fired as it would
>> have had
>> > these requirements not been applied.
>> >
>> >   1. If the pointing device is not indicating a pixel on the `canvas`,
>> act as normal and abort these steps.
>> >
>> >   2. Let pixel be the pixel indicated by the pointing device.
>> >
>> >   3. Let region be the hit region that is the region for the pixel
>> pixel on this `canvas` element's bitmap, if any.
>> >
>> >   4. If there is no region, then act as normal and abort these steps.
>> >
>> >   5. Let id be the region's ID, if any.
>> >
>> >   6. If there is an id, then initialize the event object's `region`
>> attribute to id.
>> >
>> >   7. Let control be the control represented by region for this `canvas`
>> element, if any.
>> >
>> >   8. If there is a control, then target the event object at control
>> instead of the `canvas` element.
>> >
>> >   9. Continue dispatching the event, but with the updated event object
>> and target as given in the above steps.
>> >
>> > * * *
>> >
>> > When a user's pointing device cursor is positioned over a `canvas`
>> element,
>> > user agents should render the pointing device cursor according to the
>> cursor
>> > specification described by the cursor for the hit region that is the
>> region
>> > for the pixel that the pointing device designates on the `canvas`
>> element's
>> > bitmap.
>>
>> L1?
>> >
>> > * * *
>> >
>> > User agents are encouraged to make use of the information present in a
>> > `canvas` element's hit region list to improve the accessibility of
>> `canvas`
>> > elements.
>> >
>> > Each hit region should be handled in a fashion equivalent to a node in a
>> > virtual DOM tree rooted at the `canvas` element. The hierarchy of this
>> virtual
>> > DOM tree must match the hierarchy of the hit regions, as described by
>> the
>> > parent of each region. Regions without a parent must be treated as
>> children of
>> > the `canvas` element for the purpose of this virtual DOM tree. For each
>> node
>> > in such a DOM tree, the hit region's bounding circumference gives the
>> region
>> > of the screen to use when representing the node (if appropriate).
>> >
>> > The semantics of a hit region for the purposes of this virtual DOM tree
>> are
>> > those of the the control represented by the region, if it has one,
>>
>> L1
>> or else of
>> > a non-interactive element whose ARIA role, if any, is that given by the
>> hit
>> > region's ARIA role, and whose textual representation, if any, is given
>> by the
>> > hit region's label.
>>
>> L2
>> >
>> > For the purposes of accessibility tools, when an element C is a
>> descendant of
>> > a `canvas` element and there is a region representing the control C for
>> that
>> > `canvas` element's bitmap, then the element's position relative to the
>> > document should be presented as if it was that region in the `canvas`
>> > element's virtual DOM tree.
>> L1
>> >
>> > The semantics of a hit region for the purposes of this virtual DOM tree
>> are
>> > those of the the control represented by the region, if it has one,
>> L1
>> or else of
>> > a non-interactive element whose ARIA role, if any, is that given by the
>> hit
>> > region's ARIA role, and whose textual representation, if any, is given
>> by the
>> > hit region's label.
>> L2
>> >
>> > Thus, for instance, a user agent on a touch-screen device could provide
>> haptic
>> > feedback when the user croses over a hit region's bounding
>> circumference, and
>> > then read the hit region's label to the user. Similarly, a desktop user
>> agent
>> > with a virtual accessibility focus separate from the keyboard input
>> focus
>> > could allow the user to navigate through the hit regions, using the
>> virtual
>> > DOM tree described above to enable hierarchical navigation. When an
>> > interactive control inside the `canvas` element is focused, if the
>> control has
>> > a corresponding region, then that hit region's bounding circumference
>> could be
>> > used to determine what area of the display to magnify.
>> L2?
>> >
>>
>> --
>> Mark Sadecki
>> Web Accessibility Engineer
>> World Wide Web Consortium, Web Accessibility Initiative
>> Telephone: +1.617.715.4017
>> Email: mark@w3.org
>> Web: http://w3.org/People/mark
>>
>>
>