Re: ACTION-2133: Make sure the content at https://www.w3.org/wiki/aria_1.1_testable_statements fully covers the changes indicated at https://www.w3.org/wai/aria/wiki/aria_1.1_changes (Accessible Rich Internet Applications Working Group)

Hey Michael, all.

Comments inline.

On 02/10/2017 09:48 PM, Michael Cooper wrote:

> https://www.w3.org/WAI/ARIA/wiki/ARIA_1.1_Test_Case_Coverage

Michael: You are the Best. Staff Contact. EVER. Seriously. Thank you for
doing that!! I found it extremely helpful in reviewing the testable
statements page and adding placeholders for missing tests.

All: As hinted at in the agenda for Thursday's meeting, I did a complete
cleanup and reorganization of the testable statements page. Hopefully
the new groupings (and consistency in contents, and per-attribute
statements of what changed in ARIA 1.1, and links to the related
sections in the ARIA and Core AAM specs) will make things easier for all
of us going forward.

A side effect of my work is that Michael's links are now currently
broken. I plan to address that next. In the meantime, however, I wanted
to bring something to everyone's attention ASAP, namely:

> If this analysis is correct, we are maybe half done with defining test
> cases.

I believe your analysis is indeed correct. But if we're both correct,
and some assumptions I'm making are also correct, I'm afraid we're not
half done. Applying my assumptions (which I will state momentarily) to
1.1 changes, I came up with ~170 tests that need to be written. If bogus
assumptions on my part mean we can halve that number, then it's only ~85
and we're maybe one-third done.

The good news is that I believe many of these to-be-written tests can be
created via copy and paste, so it's not *quite* as bad as it sounds....

As for my assumptions:

1. For new, *non-global* properties: We need a test for each role that
   supports that new property.

   Rationale: We cannot assume that user agents use our taxonomy
   programmatically in their implementations. Thus for non-global
   properties, there's an excellent chance that we won't get the
   implementation "for free"; instead, user agents will likely have
   to add new support on a per-role basis.

2. For new, *global* properties: We just need to test one representative
   role, plus any role for which there is special/custom behavior
   related to that property.

   Rationale: Global properties without role-specific special/custom
   behavior is something we *can* assume user agents will not be
   implementing on a per-role basis. Thus because the implementation
   itself is likely to be global, exhaustive testing is not needed.
   Hurray!

3. For new roles: We just need to test non-global properties supported
   on those roles:

   Rationale: See rationales 1 and 2 above.

Put it all together and you wind up with:
https://www.w3.org/wiki/ARIA_1.1_Testable_Statements

If, prior to Thursday's meeting, anyone suspects that my assumptions
stated above are bogus: Please speak up now. If we have consensus on
bogusity, I can modify the testable statements page accordingly prior to
Thursday's meeting so we can proceed to divvying up testable statement
writing and discussing any outstanding questions.

Sorry to be the (potential) bearer of bad news. And thanks in advance
for your feedback!

--joanie

Received on Wednesday, 15 February 2017 15:59:43 UTC