W3C home > Mailing lists > Public > public-canvas-api@w3.org > October to December 2011

RE: draft of hit testing on active regions (actually paths) in canvas

From: Richard Schwerdtfeger <schwer@us.ibm.com>
Date: Tue, 1 Nov 2011 10:46:04 -0500
To: Frank Olivier <Frank.Olivier@microsoft.com>
Cc: "chucko@jumis.com" <chucko@jumis.com>, Cynthia Shelly <cyns@microsoft.com>, "david.bolter@gmail.com" <david.bolter@gmail.com>, "janina@rednote.net" <janina@rednote.net>, "jbrewer@w3.org" <jbrewer@w3.org>, Jonas Sicking <jonas@sicking.cc>, "public-canvas-api@w3.org" <public-canvas-api@w3.org>, "public-html-a11y@w3.org" <public-html-a11y@w3.org>
Message-ID: <OF6E72F1F5.20B0FEDD-ON8625793B.00568F70-8625793B.00569D58@us.ibm.com>
Thanks Frank, I will write up a mod. with the new API for us to poke at.

Rich Schwerdtfeger
CTO Accessibility Software Group

From:	Frank Olivier <Frank.Olivier@microsoft.com>
To:	Richard Schwerdtfeger/Austin/IBM@IBMUS, Jonas Sicking
            <jonas@sicking.cc>, "public-canvas-api@w3.org"
            <public-canvas-api@w3.org>, "public-html-a11y@w3.org"
Cc:	"chucko@jumis.com" <chucko@jumis.com>, Cynthia Shelly
            <cyns@microsoft.com>, "david.bolter@gmail.com"
            <david.bolter@gmail.com>, "janina@rednote.net"
            <janina@rednote.net>, "jbrewer@w3.org" <jbrewer@w3.org>
Date:	10/31/2011 06:47 PM
Subject:	RE: draft of hit testing on active regions (actually paths) in

A preview of that draft, based on notes I took during the meeting:

These new methods manage the association of pixels drawn with canvas
fallback elements:
beginElement(Element element);
endElement(Element element);
eraseElement(element) ;

So, code example:

For a html page with this content:
  <input type=checkbox id=checkboxA >Checkbox A</input>
  <input type=checkbox id=checkboxB >Checkbox B</input>

By default, all pixels on a canvas are ‘unassociated’. The author would use
these new methods to associate pixels with fallback elements.

When drawing the checkbox:
// Rendering code
ctx.beginElement(checkboxA);     //Tell the canvas context that all
non-fully-transparent pixels that we are rendering from this point on
represents the clickable area for the checkbox UI
// Draw a background for the checkbox
ctx.fillStyle = 'white’;
ctx.rect(0, 0, 100, 20);

// Draw the actual checkbox
ctx.rect(5, 5, 10, 10);

if (checkboxA.checked)
  ctx.fillStyle = 'black';
ctx.fillText(‘checkbox’, 20, 10);
endElement(checkboxA); //We are done with element rendering

The UA/magnifier would associate any pixels rendered from beginElement() to
endElement() with the fallback dom element; the fallback dom element ‘owns’
those pixels on the screen. To resolve overlap, last write wins.
(Implementation note: A UA could use a bitmap to associate pixels with
fallback dom elements. A value of 0 in that bitmap could mean ‘no
association’; value 1 could mean ‘first element in the fallback dom element
lookup table’, etc.)

Between beginElement() and endElement(), the UA would keep track of the
extent of the pixels that are touched by drawing operations; it would use
this to compute the bounding rectangle when a magnifier asks for the
element’s extent.

At some point in the future, when my canvas bitmap no longer has that
checkbox visible:

               ctx.eraseElement(checkboxA); //Clear all pixels ‘owned’ by
               (This imo assumes that you are removing or disabling the
corresponding DOM checkbox input element)

All the pixels that were associated with checkboxA would now be
unassociated with any fallback dom element.

eraseElements() does the same, but for all fallback dom elements.

I could (potentially) also clear some of the pixels that are associated
with a corresponding canvas element:
ctx.beginElement(null);   //Tell the canvas context that all
non-fully-transparent pixels that we are rendering from this point on
should not be part of any element
ctx.fillStyle = 'white’;
ctx.rect(0, 0, 100, 100);
endElement(checkboxA); //We are done with element rendering

From: Richard Schwerdtfeger [mailto:schwer@us.ibm.com]
Sent: Monday, October 31, 2011 4:32 PM
To: Jonas Sicking; public-canvas-api@w3.org; public-html-a11y@w3.org
Cc: chucko@jumis.com; Cynthia Shelly; david.bolter@gmail.com; Frank
Olivier; janina@rednote.net; jbrewer@w3.org
Subject: Re: draft of hit testing on active regions (actually paths) in


We discussed your points on the PF call today. Apple was present. I will
write up a revised draft that associated drawing to the elements per your

Thanks for you feedback.


Rich Schwerdtfeger
CTO Accessibility Software Group

Inactive hide details for Richard Schwerdtfeger---10/31/2011 11:03:00
AM---Rich Schwerdtfeger CTO Accessibility Software GroupRichard
Schwerdtfeger---10/31/2011 11:03:00 AM---Rich Schwerdtfeger CTO
Accessibility Software Group

From: Richard Schwerdtfeger/Austin/IBM@IBMUS
To: Jonas Sicking <jonas@sicking.cc>,
Cc: chucko@jumis.com, cyns@exchange.microsoft.com, david.bolter@gmail.com,
franko@microsoft.com, janina@rednote.net, jbrewer@w3.org,
public-canvas-api@w3.org, public-html-a11y@w3.org
Date: 10/31/2011 11:03 AM
Subject: Re: draft of hit testing on active regions (actually paths) in

Rich Schwerdtfeger
CTO Accessibility Software Group

Jonas Sicking <jonas@sicking.cc> wrote on 10/27/2011 07:42:40 PM:

> From: Jonas Sicking <jonas@sicking.cc>
> To: Richard Schwerdtfeger/Austin/IBM@IBMUS,
> Cc: franko@microsoft.com, chucko@jumis.com, david.bolter@gmail.com,
> cyns@exchange.microsoft.com, public-html-a11y@w3.org,
> janina@rednote.net, jbrewer@w3.org, public-canvas-api@w3.org
> Date: 10/27/2011 07:43 PM
> Subject: Re: draft of hit testing on active regions (actually paths)in
> Some feedback:
> Limiting to only allowing paths seems like a unfortunate limitation.
> For example it misses calls to drawImage which I think is quite
> common. I'd rather prefer a call where you say "all drawing operations
> from this point should be considered drawing for element X", then let
> the page do arbitrary drawing operations, then have a second call
> which says "i'm now drawing for element Y" or "I'm now drawing for no
> element".

I see. So what would you use compute the bounds? Do you use the all the
points to compute a single rectangle for hit testing? ... or do you compute
multiple paths for an element to do hit testing on?

> What is the use case for the zIndex argument? The actual pixel drawing
> operations hasn't had a need for that so far and instead rely on the
> painters algorithm. It seems better to me to have a direct mapping
> between the drawing operations and the accessibility API.
zIndex: Performance and authors may have overlapping drawing objects.

> Why return false rather than throw an exception if the element doesn't
> exist? Also what do you mean by "doesn't exist". If the element
> doesn't exist then how could the page have a reference to it and pass
> that reference to the function? Do you mean "element isn't inside the
> canvas"?
Throwing an exception makes sense. I meant the element did not exist. I am
thinking that the author may have content outside fallback such as a drop
down menu. We could limit the elements to the fallback content. I would
like your feedback on that.

> In general I think I prefer the API I proposed in [1]. Can you
> describe what problems you were trying to solve with your changes
> compared to that proposal?
> [1]

> / Jonas
> On Thu, Oct 27, 2011 at 4:56 PM, Richard Schwerdtfeger
> <schwer@us.ibm.com> wrote:
> > Hi Frank,
> >
> > Thank you for the call today. I am glad we are on the same page onthe
> > basic methods for hit testing. I took the basic methods that Jonas had
> > out earlier on the list and the discussion we had in San Diego and put
> > together an active regions and hit testing section. I also agree with
> > this is for when we have multiple interactive elements within canvas. I
> > glad to see that customers are starting to come to Microsoft to talk
> > building more interactive widgets with canvas - not because I want them
> > but more because it shows we are making the right moves to address
> > accessibility.
> >
> > Jonas, Charles, Frank, David, Cynthia
> >
> > This is a first draft and the section I would like you to look at is
> > 16. We can change the names to whatever we want. I have not yet added
> > information as to how existing functions like scrollElementIntoView
> > work on these elements as they should be included in the layout engine
> > positioning per Jonas earlier feedback. I have also not stated how we
> > convert the paths to rectangles for the underlying OS platform
> > APIs. We could either do this in the canvas spec. or in the
> > API mapping guide that Cynthia is leading. We just need to decide.
> > Personally, I prefer the one stop shopping.
> >
> > I am sure Ian will want to tweak or rewrite the algorithm section
> > as he has his own consistent way of doing that but I took my best shot
> > it.
> >
> > Please review section 16.
> >
> > (See attached file: clickableregion.html)
> >
> > Since, like many of you I am going to TPAC, I will not have time to
> > changes prior to the face to face. Janina please put a link to this
> > in your canvas agenda for TPAC.
> >
> > See you all in California.
> >
> >
> > Rich Schwerdtfeger
> >

(image/gif attachment: graycol.gif)

Received on Tuesday, 1 November 2011 15:52:14 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:31:54 UTC