Accessibility Conformance Testing (ACT) Rules Format 1.0 Feedback

Hi,

Level Access’s feedback on Accessibility Conformance Testing (ACT) Rules Format 1.0, based on our experience / knowledge now gained from attempting its real-life implementation.

General

Suggestion 1: We need to tighten the outcome data format – possibly restricting it to one form.  The examples suggested for data-format are not trimmed to only those pieces of the format which are 100% necessary.  For example, we likely need the id of the coded test being run, the testing engine name + version, the id of the rule being claimed as supported, the id of the unit test and the outcome from the coded test.
Reason: People are already starting to implement tools which read this outcome format.  It would be best to promote a single format ASAP, which of course could be a cut-down form of EARL.   You’ll notice, that both Alfa and aXe already report using different structural versions of EARL (and, Deque’s earl context seems different to the earl-act.json context) – with differences in the additional information presented too.  Without clarity more variants from more tool developers will follow.

Under 3. ACT Rule Types

Suggestion 2: We might consider the removal of the sentence “Composite rules cannot contain other composite rules. Any time a nested composite rule would be needed, all of the relevant atomic rules can be combined directly into the new composite rule.”
Reason: We might need a Composite rule which covers an entire SC, in order to generate outcomes as described under 4.4.1 Outcome Mapping.  What we are noticing is that our tests are way more atomic than the ACT-R rules.  Which means we already use a pseudo “Composite” rule structure to replicate most ACT-R rules.  If we were to present our collection of single, more atomic, rules formally to the ACT-TF as say a replacement for a larger over-arching ACT-R rule we would almost certainly end up having to have Composite rules of Composite rules when covering an entire SC.

Suggestion 3: We might consider asking authors to ensure their rules are as atomic as possible.
Reason: For example, the ACT-R rule “Buttons should have an accessible name” under our testing methods is actually composed of five atomic rules.

Under 4.5 Rule Input

Suggestion 4: We might consider extending the list under each example to include “ECMAScript” and “Shadow DOM”.
Reason: Both must now be assumed to be present as default when testing.

Suggestion 5: Tighten the sentence “For input aspects that are not well defined, an ACT Rule MUST include either a detailed description of the aspect in question, or a reference to a well defined description.”
Reason: We’ve noticed that the ACT-R Rules point to testing the Accessibility Tree.  There is no, as far as we understand, formal Accessibility Tree standard.  This means it’s implementation by each different tool developer might be a cause for divergence.  We might consider creating a detailed, shared understanding of some of these terms.

Alistair

---

Alistair Garrison
Director of Accessibility Research
Level Access