Re: Request to re-open issue 131 -USE CASES, USE CASES, USE CASES

Hi Rich,

Frank wrote provided info here: http://www.w3.org/wiki/Canvas_hit_testing

Is that enough to work with or do we need frank to provide more details?

regards
steve

On 8 January 2012 17:57, Richard Schwerdtfeger <schwer@us.ibm.com> wrote:

>  We just need Frank to write up a formal change proposal. Apparently, we
> are now creating a new document for accessibility extensions to Canvas 2D
> API for which Steve and I will maintain. As we gain implementations the
> chairs can decide how best to integrate the implemented work back in.
>
>
> Rich Schwerdtfeger
>
> [image: Inactive hide details for David Bolter ---12/20/2011 10:19:28
> AM---Hi all, I think Frank's proposal is reasonable and I haven't]David
> Bolter ---12/20/2011 10:19:28 AM---Hi all, I think Frank's proposal is
> reasonable and I haven't come up with a better idea. It think it
>
> From: David Bolter <dbolter@mozilla.com>
>
> To: Steve Faulkner <faulkner.steve@gmail.com>,
> Cc: Richard Schwerdtfeger/Austin/IBM@IBMUS, chuck@jumis.com, Cynthia
> Shelly <cyns@microsoft.com>, david bolter <david.bolter@gmail.com>,
> franko@microsoft.com, Maciej Stachowiak <mjs@apple.com>, Paul Cotton <
> Paul.Cotton@microsoft.com>, public-canvas-api@w3.org, public-html@w3.org,
> public-html-a11y@w3.org, Sam Ruby <rubys@intertwingly.net>, Jonas Sicking
> <jonas@sicking.cc>
> Date: 12/20/2011 10:19 AM
> Subject: Re: Request to re-open issue 131 -USE CASES, USE CASES, USE CASES
> ------------------------------
>
>
>
> Hi all,
>
> I think Frank's proposal is reasonable and I haven't come up with a better
> idea. It think it is definitely worth a thoughtful and careful read.
>
> Note I recall Paul Bakaus (from Zynga) was following the hit testing
> discussion in the Summer. Paul I'd be curious to hear your fresh thoughts
> on what you think of Frank's proposal. I'm also curious this approach would
> make the canvas sub-dom more attractive to you guys.
>
> Cheers,
> David
>
>
> ----- Original Message -----
> > From: "Steve Faulkner" <faulkner.steve@gmail.com>
> > To: "Jonas Sicking" <jonas@sicking.cc>
> > Cc: "Richard Schwerdtfeger" <schwer@us.ibm.com>, chuck@jumis.com,
> "Cynthia Shelly" <cyns@microsoft.com>, "david
> > bolter" <david.bolter@gmail.com>, dbolter@mozilla.com,
> franko@microsoft.com, "Maciej Stachowiak" <mjs@apple.com>,
> > "Paul Cotton" <Paul.Cotton@microsoft.com>, public-canvas-api@w3.org,
> public-html@w3.org, public-html-a11y@w3.org, "Sam
> > Ruby" <rubys@intertwingly.net>
> > Sent: Tuesday, December 20, 2011 5:35:00 AM
> > Subject: Re: Fw: Request to re-open issue 131 -USE CASES, USE CASES, USE
> CASES
> > Hi Jonas,
> >
> > you wrote:
> >
> > > I honestly have lost track of what the latest proposal is at this
> > > point.
> >
> > Frank's proposal:
> >
> > http://www.w3.org/wiki/Canvas_hit_testing
> >
> > here it is inline:
> >
> > IMO We need a 'general purpose' hit testing solution here (to assist
> > in author uptake) with a very simple method that allows authors to see
> > what path/pixels are actually being set for hit testing:
> >
> > boolean setElementPath(in Element element);
> >
> > I would define this as: (Additional spec text for
> >
> http://dev.w3.org/html5/canvas-extensions/Overview.html#focus-management-1
> )
> >
> > When a canvas is interactive, authors should include focusable
> > elements in the element's fallback content corresponding to each
> > focusable part of the canvas.
> >
> > When multiple focusable elements are added, authors should use
> > setElementPath() to set the focus ring of each individual focusable
> > element. If the focus ring is not set with setElementPath(), the focus
> > ring of a focusable element in the fallback content is the bounding
> > rectangle of the parent canvas element. [This improves accessibility
> > for the case where the entire canvas element represents a single
> > interactive control (think very simple custom-drawn checkbox), and
> > fallback element click handling is being handled entirely by the
> > author. [The single checkbox case.]]
> >
> > When setElementPath() is called, the drawing path is used to form the
> > focus ring provided that drawing path contains a closed path. The
> > drawing path is used to form a best fit bounding rectangle in screen
> > coordinates. The bounding rectangle and drawing path may be used to
> > enhance accessibility properties [ARIA] for the targeted element.
> >
> > User agents should use the information set by setElementPath() to
> > create accessible user experiences. For example, a screen reader may
> > read the fallback element's details when the user indicates interest
> > in that region of the canvas.
> >
> > The setElementPath(element) method, when invoked, must run the
> > following steps: 1. If the element is not a descendant of the canvas
> > element with whose context the method is associated, then return false
> > and abort these steps.
> >
> > 2. If supporting an accessibility API, user agents may use the drawing
> > path to form a best fit rectangle in screen coordinates and apply it
> > to the bounding rectangle of the associated accessible object. The
> > focus ring should be subject to the clipping region.
> >
> > 3. Return true.
> >
> > When the user interacts with the canvas, the user agent should forward
> > the event to the fallback element.
> >
> > If two or more elements have overlapping paths (set via
> > setElementPath()) the last call to setElementPath() wins.
> >
> > regards
> > Stevef
> >
> > On 20 December 2011 10:22, Jonas Sicking <jonas@sicking.cc> wrote:
> > > On Mon, Dec 19, 2011 at 1:19 PM, Richard Schwerdtfeger
> > > <schwer@us.ibm.com>
> > > wrote:
> > >>
> > >> Jonas,
> > >>
> > >> For purposes of agreement getting some consensus I would like to
> > >> put the
> > >> text discussion and focus on this use case which you had agreed we
> > >> should
> > >> support while at TPAC:
> > >>
> > >>
> > >>
> > >> 1. Hit Testing and the bounds of an object
> > >>
> > >> USE CASE: Regarding hit testing, it is very, very simple. In ALL
> > >> operating
> > >> systems that support an accessibility API it is ESSENTIAL that a
> > >> magnifier
> > >> be able to determine the location of an accessible object on the
> > >> screen so
> > >> that a user may zoom to it. It has absolutely nothing to do with
> > >> rich text
> > >> editing other than the fact that like all other objects we would
> > >> need to
> > >> find the text box to zoom to it. You and I, who can see, can scan a
> > >> page and
> > >> find what we want. Yet, a magnifier user may only be able to see,
> > >> say a text
> > >> box, which has focus and a few characters as the screen my be
> > >> magnified by a
> > >> factor of 10. The few characters in the text box may be all they
> > >> see on the
> > >> screen. So, to zoom to something else they will ask their assistive
> > >> technology to do things like find an object and zoom to it - or
> > >> they may ask
> > >> it to read from the beginning of an application at the first
> > >> accessible
> > >> object and maintain a magnification point around the object
> > >>
> > >> Unlike HTML accessible canvas object reside in fallback content
> > >> which is
> > >> NOT visible. So, the screen location of these objects can NOT be
> > >> found
> > >> without programmatic intervention. In ALL accessible GUI OS
> > >> platforms the
> > >> bound so the drawing object are acquired from the device context
> > >> which is
> > >> mapped ultimately to the drawing object and then to the
> > >> corresponding
> > >> accessible object. The screen location is typically the same
> > >> location used
> > >> in hit testing.
> > >>
> > >> USE CASE: USE Braille devices also use the bounding information to
> > >> assist
> > >> in line breaks on Braille displays.
> > >>
> > >> How do I know these things? I built the offscreen model for the
> > >> first GUI
> > >> screen readers for the PC. I was hip deep in the graphics engine
> > >> and
> > >> windowing systems for both OS/2 and Windows. I also worked on one
> > >> of the
> > >> first screen magnifiers the PC - Screen Magnifier/2.
> > >>
> > >> So, there are your use cases. There is NO invention here and the
> > >> text
> > >> editor case is really a red herring as it is not the essential
> > >> reason why we
> > >> need the bounds and hit testing.
> > >>
> > >> USE CASE: The use case for hit testing is it pushes the load off
> > >> the
> > >> author to the user agent. Imagine you having to do all the GUI hit
> > >> testing
> > >> manually for your Windows app. Also, now, pointing device handling
> > >> occurs at
> > >> the canvas element while the keyboard handling is handled at an
> > >> element in
> > >> fallback content.
> > >>
> > >> Here is the accessibility API for UNIX Systems that needs the
> > >> bounds (see
> > >> BoundingBox) of an object:
> > >>
> http://people.gnome.org/~billh/at-spi-idl/html/classAccessibility_1_1Component.html
> > >> Here is the accessibility API (see accLocation) for MSAA which is
> > >> used
> > >> both Chrome and Firefox on Windows:
> > >> http://msdn.microsoft.com/en-us/library/dd318466.aspx
> > >> Here it the accessibility API (see Bounding Box) for an UIA
> > >> provider:
> > >> http://msdn.microsoft.com/en-us/library/ms726714(v=VS.85).aspx
> > >>
> > >> Right now, without a change to canvas we cannot supply this
> > >> information to
> > >> assistive technologies.
> > >
> > >
> > > Yes, I definitely support the ability to associate an area of the
> > > canvas
> > > with a element in the sub-dom (sorry, forget what the official name
> > > is, if
> > > there is one) of the canvas element. This will enable things like
> > > hit-testing, driving screen magnifiers, implementing
> > > scrolling-to-part-of-canvas, etc.
> > >
> > > I apologize if I gave the impression of otherwise.
> > >
> > >>
> > >> Do you support Frank moving forward with the setElementPath/hit
> > >> test
> > >> proposal for the working group to review and are you still
> > >> supportive of
> > >> having such an API for canvas?
> > >
> > >
> > > I honestly have lost track of what the latest proposal is at this
> > > point. The
> > > main goal I have is to create an API which is simple enough to use
> > > for
> > > people to want to do their own canvas hit-testing using the API we
> > > provide.
> > > That is how we can get the most number of people to use these APIs,
> > > and thus
> > > create the most accessible web.
> > >
> > > / Jonas
>
>
>


-- 
with regards

Steve Faulkner
Technical Director - TPG

www.paciellogroup.com | www.HTML5accessibility.com |
www.twitter.com/stevefaulkner
HTML5: Techniques for providing useful text alternatives -
dev.w3.org/html5/alt-techniques/
Web Accessibility Toolbar - www.paciellogroup.com/resources/wat-ie-about.html