Re: Preliminary walk through of HitRegions ahead of Canvas call

On 3/3/14, 7:09 PM, Jay Munro wrote:
> It sounds like we’re getting a lot closer on what we want/need for L1 spec. When we’re ready to add hit regions back into L1, I could do that. However, I’ll need you implementation experts to go through the existing (whatwg or w3c) L2 spec and make notes on what goes in and what doesn’t.

I'm not an implementation expert, but I attempted to create a draft of what we want in L1 based on discussions we had on Monday night.  I commented out sections of the spec that I thought should be deferred until L2.  They are clearly commented out with the following convention:
`<!-- Deferred until L2
`...
`-->
I also made some editorial changes, mostly in regard to the Path object.  I also commented out the section that limited the elements that could be specified as objects for fallback content.  We may want to include text the indicates the object should be focusable (noted in comments).

Attached are 3 documents:
* HitRegionsL2.html (the original W3C spec last time HitRegions were included)
* HitRegionsL1.html (the edited version)
* HitRegions.patch (diff of the two)

Interested in your feedback.  

Mark
> 
> There was also some discussion about clarifying. If anyone wants to take a crack at clarifying the more challenging sections, that would also be great.
> 
> I was thinking as we were discussing parent/child relationships, that a diagram might be good to explain the order things are hit and bubbling. We might want to think visually on any of these as a supplement to the prose on what we settle on for the spec. If I understand the relationship or concept we want to get across, I can work up some diagrams if needed.
> 
>  
> 
> -Jay
> 
>  
> 
>  
> 
>  
> 
> *From:*Rik Cabanier [mailto:cabanier@gmail.com]
> *Sent:* Monday, March 3, 2014 9:25 AM
> *To:* Richard Schwerdtfeger
> *Cc:* Rik Cabanier (cabanier@adobe.com); Jay Munro; Jatinder Mann; Mark Sadecki; public-canvas-api@w3.org
> *Subject:* Re: Preliminary walk through of HitRegions ahead of Canvas call
> 
>  
> 
> Last week Dominic said he wants the hit regions to actually have hit behavior. This means that event routing should kept in the spec.
> 
> I looked at the firefox code and at first glance this seems fairly easy to implement.
> 
>  
> 
> Hopefully he will find time to implement this before we go to PR so we have 2 independent implementations.
> 
>  
> 
> On Mon, Mar 3, 2014 at 6:20 AM, Richard Schwerdtfeger <schwer@us.ibm.com <mailto:schwer@us.ibm.com>> wrote:
> 
> Where is Dominic on this? We need a second implementation in Chrome or we need to have something in Webkit.
> 
> 
> Rich Schwerdtfeger
> 
> Inactive hide details for Rik Cabanier ---03/02/2014 10:02:22 PM---On Sun, Mar 2, 2014 at 3:09 PM, Mark Sadecki <mark@w3.org> wRik Cabanier ---03/02/2014 10:02:22 PM---On Sun, Mar 2, 2014 at 3:09 PM, Mark Sadecki <mark@w3.org <mailto:mark@w3.org>> wrote: > I walked myself through the Hit
> 
> From: Rik Cabanier <cabanier@gmail.com <mailto:cabanier@gmail.com>>
> To: Mark Sadecki <mark@w3.org <mailto:mark@w3.org>>
> Cc: "public-canvas-api@w3.org <mailto:public-canvas-api@w3.org>" <public-canvas-api@w3.org <mailto:public-canvas-api@w3.org>>, Jatinder Mann <jmann@microsoft.com <mailto:jmann@microsoft.com>>, Jay Munro <jaymunro@microsoft.com <mailto:jaymunro@microsoft.com>>, "Rik Cabanier (cabanier@adobe.com <mailto:cabanier@adobe.com>)" <cabanier@adobe.com <mailto:cabanier@adobe.com>>, Richard Schwerdtfeger/Austin/IBM@IBMUS
> Date: 03/02/2014 10:02 PM
> Subject: Re: Preliminary walk through of HitRegions ahead of Canvas call
> 
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> 
> 
> 
> 
> 
> 
> 
> On Sun, Mar 2, 2014 at 3:09 PM, Mark Sadecki <mark@w3.org <mailto:mark@w3.org>> wrote:
> 
> I walked myself through the Hit Regions spec and attempted to identify sections that were a high priority for L1, those that would be nice to have in L1 and those that could be delayed until L2.  During this walk through, a few questions emerged.  The answers to these questions will most likely affect what goes in L1 and what gets delayed until L2:
> 
> 1. Do we plan on supporting nested regions with hierarchical/ancestor relationships in L1? (Assuming so)
> 
>  
> Maybe. They don't seem too hard to implement.
>  
> 
> 2. Do we plan on supporting custom mouse cursors?
> 
>  
> No.
>  
> 
> 3. Do we need fill rule?
> 
> 
> Yes, L1
>  
> 
> Assumptions I made:
> 
> 1. We will not be supporting unbacked region descriptions like label or ARIA roles for L1.
> 2. We will be supporting removeHitRegion
> 3. We will be supporting the MouseEvent interface
> 
> 
> Yes
>  
> 
> Below are my notes.  We can use these as talking points during the Canvas call on 03 March 2014.
> 
> Mark
> 
>> ###### 1.1.15 Hit regions
>>
>> A hit region list is a list of hit regions for a bitmap.
> 
> I imagine we are still supporting hit region lists.
>>
>> Each hit region consists of the following information:
>>
>>   * A set of pixels on the bitmap for which this region is responsible.
>>
>>   * A bounding circumference on the bitmap that surrounds the hit region's set of pixels as they stood when it was created.
>>
>>   * Optionally, a non-empty string representing an ID for distinguishing the region from others.
>>
>>   * Optionally, a reference to another region that acts as the parent for this one.
>>
>>   * A count of regions that have this one as their parent, known as the hit region's child count.
> 
> Do we have plans to support nested regions with hierarchical/ancestor relationships?
>>
>>   * A cursor specification, in the form of either a CSS cursor value, or the string "`inherit`" meaning that the cursor of the hit region's parent, if any, or of the `canvas` element, if not, is to be used instead.
> 
> If this is not hard to implement, it could be nice to have.  Lower priority though.
> 
> 
> yes, let's do it for L2
>  
> 
>>
>>   * Optionally, either a control, or an unbacked region description.
> 
> Control for L1 and unbacked region description for L2?
> 
> 
> yes
>  
> 
>> A control is just a reference to an `Element` node, to which, in certain
>> conditions, the user agent will route events, and from which the user agent
>> will determine the state of the hit region for the purposes of accessibility
>> tools. (The control is ignored when it is not a descendant of the `canvas`
>> element.)
>>
>> An unbacked region description consists of the following:
>>
>>     * Optionally, a label.
>>
>>     * An ARIA role, which, if the unbacked region description also has a label, could be the empty string.
>>
>> context . `addHitRegion`(options)
> 
> I assume this is for L2?
> 
> 
> yes
>  
> 
>>
>>
>>
>> Adds a hit region to the bitmap. The argument is an object with the following
>> members:
>>
>> `path` (default null)
>>
>>     A `Path` object that describes the pixels that form part of the region. If this member is not provided or is set to null, the current default path is used instead.
>> `fillRule` (default "`nonzero`")
> 
> I understand we are only going to support the current default path at this time.  How should we include that in the spec?
> 
> 
> Maybe the method should throw notsupported if you put pass in a path. (same with label and role) 
>  
> 
>>
>>     The fill rule to use when determining which pixels are inside the path.
>> `id` (default empty string)
> 
>>From my understanding, fill rule will be needed for Hit Testing.  Is that correct?  Is it a necessary part or can it wait until L2?  Seems like Rik has basic Hit Testing support in the Firefox Nightly.  Not sure if it uses fill rule at all.
>>
>>     The ID to use for this region. This is used in `MouseEvent` events on the `canvas` (`event.region`) and as a way to reference this region in later calls to `addHitRegion()`.
>> `parentID` (default null)
> 
> Assuming we will support this.
>>
>>     The ID of the parent region, for purposes of navigation by accessibility tools and for cursor fallback.
>> `cursor` (default "`inherit`")
> 
> L1
>>
>>     The cursor to use when the mouse is over this region. The value "`inherit`" means to use the cursor for the parent region (as specified by the `parentID` member), if any, or to use the `canvas` element's cursor if the region has no parent.
>> `control` (default null)
> 
> Lower priority for L1
> 
> 
> Maybe your message content is a bit out of sync.
> Content is needed for L1.
>  
> 
>>
>>     An element (that is a descendant of the `canvas`) to which events are to be routed, and which accessibility tools are to use as a surrogate for describing and interacting with this region.
>> `label` (default null)
> 
> L1
> 
> 
> No, that should go for L2.
>  
> 
>>
>>     A text label for accessibility tools to use as a description of this region, if there is no control.
>> `role` (default null)
> 
> L2
>>
>>     An ARIA role for accessibility tools to use to determine how to represent this region, if there is no control.
> 
> L2
>>
>> Hit regions can be used for a variety of purposes:
>>
>>   * With an ID, they can make hit detection easier by having the user agent check which region the mouse is over and include the ID in the mouse events.
>>   * With a control, they can make routing events to DOM elements automatic, allowing e.g. clicks on a `canvas` to automatically submit a form via a `button` element.
>>   * With a label, they can make it easier for users to explore a `canvas` without seeing it, e.g. by touch on a mobile device.
>>   * With a cursor, they can make it easier for different regions of the `canvas` to have different cursors, with the user agent automatically switching between them.
>> context . `removeHitRegion`(id)
>>
>>
>>
>> Removes a hit region (and all its descendants) from the canvas bitmap. The
>> argument is the ID of a region added using `addHitRegion()`.
> 
> L1
>>
>> The pixels that were covered by this region and its descendants are
>> effectively cleared by this operation, leaving the regions non-interactive. In
>> particular, regions that occupied the same pixels before the removed regions
>> were added, overlapping them, do not resume their previous role.
>>
>> A hit region A is an ancestor region of a hit region B if B has a parent and
>> its parent is either A or another hit region for which A is an ancestor
>> region.
> 
> See above regarding support for nested/hierarchical relationships.
>>
>> The region identified by the ID ID in a bitmap bitmap is the value returned by
>> the following algorithm (which can return a hit region or nothing):
>>
>>   1. If ID is null, return nothing and abort these steps.
>>
>>   2. Let list be the hit region list associated with bitmap.
>>
>>   3. If there is a hit region in list whose ID is a case-sensitive match for ID, then return that hit region and abort these steps.
>>
>>   4. Otherwise, return nothing.
>>
>> The region representing the control control for a bitmap bitmap is the value
>> returned by the following algorithm (which can return a hit region or
>> nothing):
>>
>>   1. Let list be the hit region list associated with bitmap.
>>
>>   2. If there is a hit region in list whose control is control, then return that hit region and abort these steps.
>>
>>   3. Otherwise, return nothing.
>>
>> The control represented by a region region for a `canvas` element ancestor is
>> the value returned by the following algorithm (which can return an element or
>> nothing):
>>
>>   1. If region has no control, return nothing and abort these steps.
>>
>>   2. Let control be region's control.
>>
>>   3. If control is not a descendant of ancestor, then return nothing and abort these steps.
>>
>>   4. Otherwise, return control.
>>
>> The cursor for a hit region region of a `canvas` element ancestor is the value
>> returned by the following algorithm:
>>
>>   1. _Loop_: If region has a cursor specification other than "`inherit`", then return that hit region's cursor specification and abort these steps.
>>
>>   2. If region has a parent, then let region be that hit region's parent, and return to the step labeled _loop_.
>>
>>   3. Otherwise, return the used value of the 'cursor' property for the `canvas` element, if any; if there isn't one, return 'auto'. [CSSUI]
>>
>> The region for a pixel pixel on a bitmap bitmap is the value returned by the
>> following algorithm (which can return a hit region or nothing):
>>
>>   1. Let list be the hit region list associated with bitmap.
>>
>>   2. If there is a hit region in list whose set of pixels contains pixel, then return that hit region and abort these steps.
>>
>>   3. Otherwise, return nothing.
>>
>> To clear regions that cover the pixels pixels on a bitmap bitmap, the user
>> agent must run the following steps:
>>
>>   1. Let list be the hit region list associated with bitmap.
>>
>>   2. Remove all pixels in pixels from the set of pixels of each hit region in list.
>>
>>   3. Garbage-collect the regions of bitmap.
>>
>> To garbage-collect the regions of a bitmap bitmap, the user agent must run the
>> following steps:
>>
>>   1. Let list be the hit region list associated with bitmap.
>>
>>   2. _Loop_: Let victim be the first hit region in list to have an empty set of pixels and a zero child count, if any. If there is no such hit region, abort these steps.
>>
>>   3. If victim has a parent, then decrement that hit region's child count by one.
>>
>>   4. Remove victim from list.
>>
>>   5. Jump back to the step labeled _loop_.
>>
>> Adding a new region and calling `clearRect()` are the two ways this clearing
>> algorithm can be invoked. The hit region list itself is also reset when the
>> rendering context is reset, e.g. when a `CanvasRenderingContext2D` object is
>> bound to or unbound from a `canvas`, or when the dimensions of the bitmap are
>> changed.
>>
>> * * *
>>
>> When the `addHitRegion()` method is invoked, the user agent must run the
>> following steps:
>>
>>   1. Let arguments be the dictionary object provided as the method's argument.
>>
>>   2. If the arguments object's `path` member is not null, let source path be the `path` member's value. Otherwise, let it be the `CanvasRenderingContext2D` object's current default path.
> 
> I assume this will be something like: "The source path will be the `CanvasRenderingContext2D` object's current default path."
>>
>>   3. Transform all the coordinates and lines in source path by the current transform matrix, if the arguments object's `path` member is not null.
> 
> L2
>>
>>   4. Let specified pixels be the pixels contained in source path, using the fill rule indicated by the `fillRule` member.
> 
> L2?
> 
> 
> No, L1
>  
> 
>>
>>   5. If the arguments object's `id` member is the empty string, let it be null instead.
> 
> L1
>>
>>   6. If the arguments object's `id` member is not null, then let previous region for this ID be the region identified by the ID given by the `id` member's value in this scratch bitmap, if any. If the `id` member is null or no such region currently exists, let previous region for this ID be null.
> L1
>>
>>   7. If the arguments object's `parent` member is the empty string, let it be null instead.
> L1?
>>
>>   8. If the arguments object's `parent` member is not null, then let parent region be the region identified by the ID given by the `parent` member's value in the scratch bitmap, if any. If the `parent` member is null or no such region currently exists, let parent region be null.
> L1?
>>
>>   9. If the arguments object's `label` member is the empty string, let it be null instead.
> L1
>>
>>   10. If any of the following conditions are met, throw a `NotSupportedError` exception and abort these steps.
>>
>>     * The arguments object's `control` and `label` members are both non-null.
>>     * The arguments object's `control` and `role` members are both non-null.
>>     * The arguments object's `role` member's value is the empty string, and the `label` member's value is either null or the empty string.
> Not necessary in L1 unless we support label and role.
>>     * The specified pixels has no pixels.
>>     * The arguments object's `control` member is not null but is neither an `a` element that represents a hyperlink, a `button` element, an `input` element whose `type` attribute is in one of the Checkbox or Radio Button states, nor an `input` element that is a button.
> I believe we do not need the above if the control is not limited.
>>     * The parent region is not null but has a control.
>>     * The previous region for this ID is the same hit region as the parent region.
>>     * The previous region for this ID is an ancestor region of the parent region.
>>   11. If the `parent` member is not null but parent region is null, then throw a `NotFoundError` exception and abort these steps.
>>
>>   12. If any of the following conditions are met, throw a `SyntaxError` exception and abort these steps.
>>
>>     * The arguments object's `cursor` member is not null but is neither an ASCII case-insensitive match for the string "`inherit`", nor a valid CSS 'cursor' property value. [CSSUI]
> Depends on if we support cursors in L1
>>     * The arguments object's `role` member is not null but its value is not an ordered set of unique space-separated tokens whose tokens are all case-sensitive matches for names of non-abstract WAI-ARIA roles. [ARIA]
> L2
>>   13. Let region be a newly created hit region, with its information configured as follows:
>>
>> Hit region's set of pixels
>>
>>
>>
>> The specified pixels
>>
>> Hit region's bounding circumference
>>
>>
>>
>> A user-agent-defined shape that wraps the pixels contained in source path. (In
>> the simplest case, this can just be the bounding rectangle; this specification
>> allows it to be any shape in order to allow other interfaces.)
> L1
>>
>> Hit region's ID
>>
>>
>>
>> If the arguments object's `id` member is not null: the value of the `id`
>> member. Otherwise, region has no id.
> L1
>>
>> Hit region's parent
>>
>>
>>
>> If parent region is not null: parent region. Otherwise, region has no parent.
> L1?
>>
>> Hit region's child count
>>
>>
>>
>> Initially zero.
> 
> L1?
>>
>> Hit region's cursor specification
> L1?
> 
> 
> No, let's do L2 for cursors
>  
> 
>>
>>
>>
>> If parent region is not null: parent region. Otherwise, region has no parent.
>>
>> Hit region's control
>>
>>
>>
>> If the arguments object's `control` member is not null: the value of the
>> `control` member. Otherwise, region has no control.
> L1
>>
>> Hit region's label
>>
>>
>>
>> If the arguments object's `label` member is not null: the value of the `label`
>> member. Otherwise, region has no label.
> L2
>>
>> Hit region's ARIA role
>>
>>
>>
>> If the arguments object's `role` member is not null: the value of the `role`
>> member (which might be the empty string). Otherwise, if the arguments object's
>> `label` member is not null: the empty string. Otherwise, region has no ARIA
>> role.
> L2
>>
>>   14. If the arguments object's `cursor` member is not null, then act as if a CSS rule for the `canvas` element setting its 'cursor' property had been seen, whose value was the hit region's cursor specification.
> L2?
>>
>> For example, if the user agent prefetches cursor values, this would cause that
>> to happen in response to an appropriately-formed `addHitRegion()` call.
>>
>>   15. If the arguments object's `control` member is not null, then let previous region for the control be the region representing the control given by the `control` member's value for this scratch bitmap, if any. If the `control` member is null or no such region currently exists, let previous region for the control be null.
> L1
>>
>>   16. If there is a previous region with this control, remove it from the scratch bitmap's hit region list; then, if it had a parent region, decrement that hit region's child count by one.
> L1
>>
>>   17. If there is a previous region with this ID, remove it, and all hit regions for which it is an ancestor region, from the scratch bitmap's hit region list; then, if it had a parent region, decrement that hit region's child count by one.
> L1
>>
>>   18. If there is a parent region, increment its hit region's child count by one.
> L1?
>>
>>   19. Clear regions that cover the pixels in region's set of pixels on this scratch bitmap.
>>
>>   20. Add region to the scratch bitmap's element's hit region list.
>>
>> When the `removeHitRegion()` method is invoked, the user agent must run the
>> following steps:
>>
>>   1. Let region be the region identified by the ID given by the method's argument in the rendering context's scratch bitmap. If no such region currently exists, abort these steps.
>>
>> If the method's argument is the empty string, then no region will match.
>>
>>   2. Remove region, and all hit regions for which it is an ancestor region, from the rendering context's scratch bitmap's hit region list; then, if it had a parent region, decrement that hit region's child count by one.
>>
>>   3. Garbage-collect the regions of the rendering context's scratch bitmap.
> 
> L1?
>>
>> * * *
>>
>> The `MouseEvent` interface is extended to support hit regions:
>>
>>
>>     partial interface MouseEvent {
>>       readonly attribute DOMString? region;
>>     };
>>
>>     partial dictionary MouseEventInit {
>>       DOMString? region;
>>     };
>>
>> event . `region`
>>
>>
>>
>> If the mouse was over a hit region, then this returns the hit region's ID, if
>> it has one.
>>
>> Otherwise, returns null.
>>
>> The `region` attribute on `MouseEvent` objects must return the value it was
>> initialized to. When the object is created, this attribute must be initialized
>> to null. It represents the hit region's ID if the mouse was over a hit region
>> when the event was fired.
> L1
>>
>> When a `MouseEvent` is to be fired at a `canvas` element by the user agent in
>> response to a pointing device action, if the `canvas` element has a hit region
>> list, the user agent must instead follow these steps. If these steps say to
>> _act as normal_, that means that the event must be fired as it would have had
>> these requirements not been applied.
>>
>>   1. If the pointing device is not indicating a pixel on the `canvas`, act as normal and abort these steps.
>>
>>   2. Let pixel be the pixel indicated by the pointing device.
>>
>>   3. Let region be the hit region that is the region for the pixel pixel on this `canvas` element's bitmap, if any.
>>
>>   4. If there is no region, then act as normal and abort these steps.
>>
>>   5. Let id be the region's ID, if any.
>>
>>   6. If there is an id, then initialize the event object's `region` attribute to id.
>>
>>   7. Let control be the control represented by region for this `canvas` element, if any.
>>
>>   8. If there is a control, then target the event object at control instead of the `canvas` element.
>>
>>   9. Continue dispatching the event, but with the updated event object and target as given in the above steps.
>>
>> * * *
>>
>> When a user's pointing device cursor is positioned over a `canvas` element,
>> user agents should render the pointing device cursor according to the cursor
>> specification described by the cursor for the hit region that is the region
>> for the pixel that the pointing device designates on the `canvas` element's
>> bitmap.
> 
> L1?
>>
>> * * *
>>
>> User agents are encouraged to make use of the information present in a
>> `canvas` element's hit region list to improve the accessibility of `canvas`
>> elements.
>>
>> Each hit region should be handled in a fashion equivalent to a node in a
>> virtual DOM tree rooted at the `canvas` element. The hierarchy of this virtual
>> DOM tree must match the hierarchy of the hit regions, as described by the
>> parent of each region. Regions without a parent must be treated as children of
>> the `canvas` element for the purpose of this virtual DOM tree. For each node
>> in such a DOM tree, the hit region's bounding circumference gives the region
>> of the screen to use when representing the node (if appropriate).
>>
>> The semantics of a hit region for the purposes of this virtual DOM tree are
>> those of the the control represented by the region, if it has one,
> 
> L1
> or else of
>> a non-interactive element whose ARIA role, if any, is that given by the hit
>> region's ARIA role, and whose textual representation, if any, is given by the
>> hit region's label.
> 
> L2
>>
>> For the purposes of accessibility tools, when an element C is a descendant of
>> a `canvas` element and there is a region representing the control C for that
>> `canvas` element's bitmap, then the element's position relative to the
>> document should be presented as if it was that region in the `canvas`
>> element's virtual DOM tree.
> L1
>>
>> The semantics of a hit region for the purposes of this virtual DOM tree are
>> those of the the control represented by the region, if it has one,
> L1
> or else of
>> a non-interactive element whose ARIA role, if any, is that given by the hit
>> region's ARIA role, and whose textual representation, if any, is given by the
>> hit region's label.
> L2
>>
>> Thus, for instance, a user agent on a touch-screen device could provide haptic
>> feedback when the user croses over a hit region's bounding circumference, and
>> then read the hit region's label to the user. Similarly, a desktop user agent
>> with a virtual accessibility focus separate from the keyboard input focus
>> could allow the user to navigate through the hit regions, using the virtual
>> DOM tree described above to enable hierarchical navigation. When an
>> interactive control inside the `canvas` element is focused, if the control has
>> a corresponding region, then that hit region's bounding circumference could be
>> used to determine what area of the display to magnify.
> L2?
>>
> 
> --
> Mark Sadecki
> Web Accessibility Engineer
> World Wide Web Consortium, Web Accessibility Initiative
> Telephone: +1.617.715.4017 <tel:%2B1.617.715.4017>
> Email: mark@w3.org <mailto:mark@w3.org>
> Web: http://w3.org/People/mark
> 
>  
> 
>  
> 


-- 
Mark Sadecki
Web Accessibility Engineer
World Wide Web Consortium, Web Accessibility Initiative
Telephone: +1.617.715.4017
Email: mark@w3.org
Web: http://w3.org/People/mark