minutes: CANVAS subteam of HTML5 A11y TF telecon 2011-01-10 [draft]

aloha!

minutes from today's canvas subteam teleconference can be accessed as
hypertext at:

http://www.w3.org/2011/01/10-html-a11y-minutes.html

as an IRC log at:

http://www.w3.org/2011/01/10-html-a11y-irc

and as plain text following this announcement -- as usual, please report
any errors, omissions, mis-attributions, clarifications, and the like
by replying-to this announcement on-list

thanks to TimL and ShawnW for participating in today's meeting to 
provide us with insight into what is needed for screen magnification
support in CANVASand for making yourselves available to the canvas
subteam...

gregory.

    _________________________________________________________

                             - DRAFT -

            HTML Accessibility Task Force Teleconference

10 Jan 2011

Agenda
http://lists.w3.org/Archives/Public/public-canvas-api/2011JanMar/0002.html

  See also: IRC log - http://www.w3.org/2011/01/10-html-a11y-irc

Attendees

  Present
         Chuck_Pritchard, Gregory_Rosmaita, Rich, Shawn_Warren,
         Tim_Lalor

  Regrets
         Frank_Oliver

  Chair
         Rich

  Scribe
         Gregory_Rosmaita

Contents

    * Topics
        1. State of Canvas A11y
        2. What is needed for Screen Magnification Support for
           CANVAS?
    * Summary of Action Items
    _________________________________________________________

  <trackbot> Date: 10 January 2011

  <richardschwerdtfe> Meeting: HTML Canvas Accessibility subteam
  meeting

  <richardschwerdtfe> can you call in Gregory?

  i'm on the phone

  <scribe> scribe: Gregory_Rosmaita

  <scribe> scribenick: oedipus

State of Canvas A11y

  Tim Lalor: senior engineer, AISquared (make ZoomText)

  Shawn Warren from AI Squared

  HTML_Canvas_2DContext120910.html:
 
http://lists.w3.org/Archives/Public/public-canvas-api/2011JanMar/att-0002/HTML_Canvas_2DContext120910.html

  CanvasEditor.html:
 
http://lists.w3.org/Archives/Public/public-canvas-api/2011JanMar/att-0002/CanvasEditor.html

  HTML_Canvas_Element.html:
 
http://lists.w3.org/Archives/Public/public-canvas-api/2011JanMar/att-0002/HTML_Canvas_Element.html

  CP: magnification brings unique challanges -- doing zoom doesn't
  garuntee magnification

  RS: what canvas is in HTML5
  ... canvas isn't a lot different from windows -- drawing to canvas
  in windows environment -- big issue for a11y -- how to convey info
  to an accessibility API on any given platform
  ... got fairly long way along
  ... created inside of CANVAS (uses javascript to draw items on
  canvas)
  ... inside of CANVAS to make accessible, created sub-tree in UA so
  that can create HTML elements representative of objects drawn to
  canvas can be mapped to a11y API -- could alos markup with ARIA --
  all would populate a11y API
  ... magnification problems: DOM MVC via javascript, what is in
  sub-tree isn't drawn on canvas -- positions of elements don't map to
  what one actually sees on canvas
  ... no a11y API where js can do stuff like that
  ... what info is absolutely essential for screen magnification?
  ... proposal - FocusRing would be trackable from a11y APIs
  ... caret positioning proposal -- is this enough for magnification?
  ... clipping of text and exposition of clipped text -- issue for
  magnification if no positioning info -- other strategies?
  ... move focus position to content -- moves focus position on
  screen?
  ... trying to walk fine line: what is necessary for author to do and
  AT to have

  TL: looks like CANVAS is opaque to us right now
  ... few pieces: focus events are key -- need to know where to track
  the view -- if can capture focus event, will help us understand
  where viewport of magnifier should focus on -- FocusRing
  ... caret tracking also fundamental for screen mag -- maintain in
  view and wrap in locator

  RS: have to get WG sign-off on FocusRing proposal -- waiting for MS
  feedback
  ... changes to spec that refer to that -- example of 2 checkboxes
  (don't have FocusRing drawing in API)

  CanvasEditor.html:
 
http://lists.w3.org/Archives/Public/public-canvas-api/2011JanMar/att-0002/CanvasEditor.html

What is needed for Screen Magnification Support for CANVAS?

  TL: used ARIA-selection of "no-width" for caret

  RS: for caret istelf, yes -- proposal is caret selection would know
  if RTE
  ... added API to 2D Canvas context
 
http://lists.w3.org/Archives/Public/public-canvas-api/2011JanMar/att-0002/HTML_Canvas_2DContext120910.html
  ... use actual textfield or checkbox in subtree -- process as usual,
  provide positioning info

  TL: scrolling into view as content consumed and presented to user,
  need to highlight words
  ... imagine canvas a single sentence clipped by bounding rectange;
  need to drive presentation of canvas (scroll to left) -- 2 parts:
  position info; being able to drive viewport of canvas itself

  RS: charles is our resident canvas expert
  ... can't markup position of text in sub-tree

  CP: can't do with textarea -- no API for this

  SW: have ScroolTo -- API not there -- nothing to model on to write
  in standards
  ... done RTE with scroll bars; know where cursor is; TEXTAREA
  element is quite basic; contenteditable also very basic

  RS: question: if have ability to move caret position thorugh a11y
  APIs, would that be picked up by app for positioning

  SW: will work if trying to navigate between text; but if want to
  scroll down 20 pixels, how does one do that?
  ... single-line text editing with input box laid over picture -- if
  type a lot, shifts to left -- HTML forms a "bit" deficient in this
  area

  TL: static content and dynamic content
  ... is there way to scroll data content so can use highlight as
  march across section; scroll when reach right edge

  SW: setting caret position good enough for most of that -- can be
  recalculated before scroll -- automatic word-wrapping a BIG problem
  ... looking at windows API -- magnifier is a sub-window that
  accesses same content at diff zoom level and may have diff selection
  objects
  ... another issue: when zoom in using UA zoom, CANVAS doesn't know
  how to handle content -- ensure not blurry -- pixel offset needs to
  be tackled

  TL: selection and caret location: 1 distinct case is diff controls
  and selection events for multiple characters and words selected
  ... raised as events? caret move with selection? or mouse selection
  across text

  CP: have OnSelectStart
  ... related to input method edititors
  ... onSelectStart only selection event -- more would be helpful
  ... some push-back on input method editinig -- should it be up to
  OS, browser vendor? when UA vendor hasn't implemented, what to do?
  ... this is a good use case -- can't get anywhere without additions
  to APIs
  ... will help counter push-back
  ... bare essentials necessary
  ... need scroll events and selection events that work in TEXTAREA

  TL: help us in dynamic area - -when user typing to know what
  selected -- could span more than is visible -- helps to know what
  user is actually selecting so can keep scrolling view and can tell
  user what is happening
  ... watch text and background colors
  ... when API available, use it; when not available, have heuristics

  CP: list of API holes?
  ... or incomplete implementation of API

  RS: AT immunity -- have to be cognizant of propritary methods

  TL: use open standards -- focus on APIs not heuristics

  CP: heuristics too hairy for me -- you guys know the standards -- if
  anything you can share for input method would be appreciated

  TL: using MSAA standard Windows events and UIA
  ... evaluating UIA's integrity

  GJR: IAccessible2?

  TL: playing with it--in roadmap -- other higher priorities

  RS: can you post what window events and APIs you are using?

  TL: window events is a kind of alchemy
  ... other APIs pretty self-explanatory

  RS: use MSAA standard windows events for showing/hiding?

  TL: selection: MSAA provides data on current selection, comparison
  to cached version,

  RS: IA2 does selcetion changing
  ... selection change events at high level -- how tell if something
  scrolled off screen?

  TL: those are used in some scenarios, but use API to compute
  visability
  ... couple of levels of visability: visible on page; rendered but
  clipped; not rendered anywhere (on or off screen)
  ... where heuristics come into play -- if user scrolls, then render

  CP: requestAnimationFrame similar
  ... what would the API be if one were available?
  ... API that reduces need for heuristics?
  ... UA devs need to know what needs to be there

  RS: take content and select it -- aria-selected -- could mark area
  -- receive notification events under the covers
  ... positioning info

  CP: what positioning info is needed?

  TL: CANVAS completely contains text, would like to highlight each
  word inside of canvas in a sequence -- speak word and highlight even
  if not editable -- positioning critical to displaying highlight

  CP: can you get away with next word?

  TL: if way to enumerate can wrap rectangle around it

  CP: as long as going word-to-word is feasible
  ... problem with older verions of IE -- but enumerating works

  RS: suggesting notification from AT to enumerate words?

  CP: getNextElement or getNextChild --- from current position go to
  next word

  RS: how AT call into UA to get info?
  ... can provide functionality via javascript -- need to get to next
  word and speak it and highlight it

  CP: when highlighting interacting with DOM or IME

  TL: when highlighting may leave UA rendering alone before write
  anything to screen

  RS: canvas author draws pre-rasterized text

  CP: optimized to be pre-rastorized

  RS: how to highlight once happens?

  CP: going to need to determine if caret moves

  TL: know where next word is or next sentence -- in our own way, can
  do highlighting as goes to screen -- flatten image, so canvas not
  effective -- can't use canvas to do stuff inside canvas -- have own
  tech to highlight regardless
  ... hesistant to move cursor around -- prefer to not move cursor

  CP: FF allows multiple selection ranges
  ... 2 people may have 2 diff cursors moving -- virtual or secondary
  cursor use case?
  ... like concept of bringing that in -- here is virtual cursor, what
  is next word?
  ... need L,W,H for same purpose?

  RS: each a11y API on windows has notion of bounding recatangles --
  can compute rectangle and then highlight it
  ... can't expect author to write in position info for all text --
  need to have second virtual cursor?

  CP: for best case, need virtual window to do scroll and adjust to
  new pixel ratio -- make sense if virtual window had cursor,
  scrolling and own DPI ratio

  RS: how to do for spec?

  <Downchuck> AbstractView

  CP: got to go all the way to AbstractView
  ... none of this has to do strictly with canvas, right?
  ... using AbstractView with CANVAS, but isn't "of canvas"

  RS: 1-to-1 mapping in a11y APIs
  ... 1-to-1 mapping tying into layout engine -- canvas is seperate
  ... view of canvas is separate from DOM -- not wired together -- can
  forsee problem with SVG as well
  ... when deal with MS Office, most AT vendors access office through
  proprietary DOM --- does AISquared do that?

  <Downchuck> http://www.w3.org/TR/DOM-Level-2-Views/views.html

  TL: use Word Object Model to get content -- have to use others to
  get application support
  ... toolbars, etc. reflected in MSAA
  ... if object model too slow, then might do own analysis -- a lot of
  techs to get same info -- can talk to object model, subscribe to
  MSSAA event or graphics stack
  ... use "content model" to get content of document

  RS: how map through layers?
  ... if select text, how map positioning info -- look at cached
  drawing calls or OSM?

  <Downchuck>
  http://msdn.microsoft.com/en-us/library/kw65a0we(v=vs.80).aspx

  TL: in Excel had to do translations and calculations based on
  mapping mode
  ... convert between object model and pixels -- may have to go
  through hoops to get to screen

  RS: similar situation with CANVAS -- providing a DOM
  ... does Office object model provide positioning? device context?

  TL: have had challenges with PDF document -- depending on how query,
  may get "page relevant" -- each DOM needs a bit of massaging

  RS: what do we need to do in canvas -- how to get info to AT dev

  TL: if there is a virtual canvas, real-life canvas may only be a
  portion of that -- don't want author to provide positioning --
  coordinates on virtual canvas and calculate to render in viewport

  RS: is that consistent mapping?

  CP: larger bitmap area or virtual coordinate system?

  TL: can think of it both ways --

  CP: optimal not to have bitmap
  ... AbstactView for canvas seems a good fit
  ... coordinate is x,y in virtual canvas (may be outside of bitmap)

  RS: with canvas sub-tree how to give abstract coordinates?

  CP: top left of any canvas is 0,0; after that, best model is SVG
  coordination -- what need in CANVAS

  TL: use virtual for topmost level

  RS: dev pushback?

  CP: AbstactView for canvas may be politically tricky but needed;
  already in SVG, but can't zoom into SVG with magnifier

  TL: seems that this is right solution

  <Downchuck>
 
http://msdn.microsoft.com/en-us/library/microsoft.office.interop.word.selection.startof.aspx

  <Downchuck> somewhat interesting link on word selection

  RS: would like to get AISquared feedback at meetings - email is ok,
  but being on call is very helpful

  <Downchuck>
  https://03.chat.mibbit.com/?url=irc%3A%2F%2Firc.w3.org%3A6665

  <Downchuck> i just used that one for quick access

  RS: CP please float AbstactView by WG?

  CP: may be best to float before SVG WG before going to HTML WG

  RS: alternate proposals for caret and FocusRing proposals
  ... FrankO says "drop"
  ... IME stuff happening at google --people will use canvas for this
  (editing)

  CP: guy working on IME not dropping it

  RS: concern: don't want to have HTML5 go out and say "we'll fix in
  next point release"

  CP: good to get methods for canvas in there -- IME in canvas drives
  people nuts
  ... APIs for canvas are simple -- bit of tweaking and feedback
  should work
  ... treading warily in WHAT WG space

  RS: FocusRing a no-brainer

  TL: can't mange input="text" -- scrolls itself

  RS: thanks to TL and SW for attending

  [adjourned]

Summary of Action Items

  [End of minutes]
    _________________________________________________________