Re: What I think we need

Thanks for putting this together, Jay.  This should help facilitate a
productive meeting on Monday.   Looking forward to talking more about his
then.  A few comments:

On Thu, Sep 4, 2014 at 8:29 PM, Jay Munro <jaymunro@microsoft.com> wrote:

> Here's where I see things with the testing, gleaned from our discussions
> previously.
>
>  What am I missing?
>
> 1. There are some tests for drawFocusIfNeeded in the canvas collection.
> According to Philippe, they seem to be up-to-date. Are they enough?
> 2. Mark wrote some tests for Hit Regions, but mentioned that he needed to
> update them. Those aren't in the collection. We need hitRegions,
> clearHitRegions, addHitRegion, and removeHitRegion.
> 2. Rich posted a link to the test harness they use for WAI-ARIA -
> http://www.w3.org/WAI/PF/testharness/ and Michael Cooper added that it
> has public read access, but write access is restricted to members of the
> PFWG (I'm not sure what PF is). Can this harness be used to test?
>
I think we can use web-platform-tests, which supports manual tests (just
append -manual to the end of the file name). The test runner will pause and
ask for confirmation of pass/fail from tester.  Better than using a
different test suite only because the results will be consolidated in a
single report.  One concern is that we won't be able to indicate what
Inspector and/or AAPI we used to perform the test.

> 3. Rich gave a set of details for testing the features:
>         - Covering multiple hit regions
>         - Covering fallback content that uses a standard basic HTML5
> control (I would choose a button), a link, and an aria enabled button or
> control. We then validate the regions map to the bounds   on the
> accessibility API
>         - Make we can see the bounds drawn as we move focus using the
> drawFocusIfNeeded function to render the visible focus.
>         - Make sure a magnifier can follow the keyboard focus location on
> the screen
>         - Test it with a screen reader
>         - Test it with an inspection tool.
>         - test to make sure the id of the object in the hit matches the
> element you are hovering over.
>         - Move the browser window or scroll the window to make sure that
> the new location on the screen reflects the new bounds location.
>
> 4. We'll need to figure out:
>         a. Who can write or update the tests. Sounds like Rich and Mark
> have this pretty much under control.
>
Maybe.  May need input from others.

>         b. Someone to run the tests (more than one). Determine if the
> drawFocusIfNeeded tests are current, and run any others. It's still open to
> whether these tests can be automated.
>
I can help run tests.

>         c. Define where the results get reported.
>
We should try to use web-platform-tests and wptrunner, what is used for the
rest of canvas.  If that doesn't work, we can look at the PF (Protocols and
Formats) test harness which is optimized for manual/AAPI testing.

Mark

>         d. A summary.
>
>
>
> Jay Munro  Content Developer 2 Internet Explorer     85/2275
> 425-703-2242
>
>