Re: [aria-testsuite] spec violations in ARIA testsuite

Hi James,


in the first instance i would suggest you simply file bugs against the html
spec detailing where you think the mappings are in error and provide
examples of widgets that use the pattern + any other relevant info.

If this does not result in agreeable changes, then it would be time to get
PF involved.

--

Regards

SteveF
HTML 5.1 <http://www.w3.org/html/wg/drafts/html/master/>


On 20 August 2013 20:00, James Craig <jcraig@apple.com> wrote:

> Michael (Cooper) and Janina.
>
> I think we need some PF/HTML intervention for these first three "strong
> native semantics" issues.
>
>
> On Aug 20, 2013, at 8:18 AM, Michael[tm] Smith <mike@w3.org> wrote:
>
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/_functional/tree/ariatree.html
> ":30:21:
> >>> error: Bad value “group” for attribute “role” on element “ul”.
> >> ...
> >> Error in the validator. The group role definitely exists and it's
> properly used in this tree example.
> >> http://www.w3.org/WAI/PF/aria/complete#group
> >> ...
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/presentation-role/528.html
> ":9:23:
> >>> error: Bad value “checkbox” for attribute “role” on element “ul”.
> >>
> >> Granted, it's a weird test case to set the role on the UL, but: Error
> in the validator.
> >
> > The validator's in conformance with the HTML spec in reporting those as
> > errors, because the HTML spec currently doesn't allow ul@role="group" or
> > ul@role="checkbox":
> >
> >
> http://www.w3.org/html/wg/drafts/html/master/dom.html#sec-implicit-aria-semantics
> >
> > So allowing ul@role=group and ul@role="checkbox" would require a change
> to
> > the HTML spec.
>
> Janina and Michael. PF needs to push back against the HTML WG here.
> There's not valid reason to enforce strong native semantics on UL. It's
> just a block container, not a form control. I don't care about the checkbox
> role, but "group" and "menu" roles *required* for trees, treegrids, menus,
> lists, and other common ARIA patterns.
>
> If HTML doesn't change this, many complex ARIA widgets will not validate.
>
>
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/name-computation-input/548.html
> ":9:59:
> >>> error: Bad value “menu” for attribute “role” on element “select”.
> >> ...
> >> Error in the validator.
> >
> > The validator's in conformance with the HTML spec in reporting that as an
> > error, because the HTML spec currently doesn't allow select@role="menu":
> >
> >
> http://www.w3.org/html/wg/drafts/html/master/dom.html#sec-strong-native-semantics
> >
> > As far as I can tell, the only role value allowed by the HTML spec for
> the
> > select element is select@role=listbox. So allowing select@role=menu
> would
> > require a change to the HTML spec.
>
> Need to push the HTML WG on this one, too.
>
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/name-computation-input/548.html
> ":10:60:
> >>> error: Bad value “menuitem” for attribute “role” on element “option”.
> >> ...
> >> Error in the validator.
> >
> > The validator's in conformance with the HTML spec in reporting that as an
> > error, because the HTML spec currently doesn't allow option@role
> ="menuitem":
> >
> >
> http://www.w3.org/html/wg/drafts/html/master/dom.html#sec-strong-native-semantics
> >
> > The only role value allowed by the HTML spec for the option element is
> > option@role=option. So allowing option@role=menuitem would require a
> change
> > to the HTML spec.
>
> Ditto.
>
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/_/828.htm
> ":1:59:
> >>> error: Bad value “text all” for attribute “aria-relevant” on element
> “div”.
> >>
> >> Error in the validator. A value of "text all" is redundant (should just
> >> be "all") but it's not explicitly disallowed or invalid.
> >
> > hmm.. The definition of the value of the aria-relevant attribute in the
> > ARIA spec currently reads:
> >
> >  The attribute is represented as a space delimited list of the following
> >  values: additions, removals, text; or a single catch-all value all.
> >
> > I read that as an either/or, with the spec requiring that it be only one
> of
> > the following two choices:
> >
> >  A. A list of one or more of "additions", "removals", "text".
> >
> >    OR
> >
> >  B. A single value, "all" (instead of a list of values).
> >
> > By that reading, "text all" would be invalid.
> >
> > So if I'm misreading the spec here, and "text all" is actually intended
> to
> > be valid, then I'd think other readers are likely to be misreading the
> spec
> > in the same way, so the spec should be refined to make it more clear.
>
> Okay. I can accept that reading. Test case error.
>
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/host-language/role-2tokens-first.html
> ":8:39:
> >>> error: Bad value “dialog foo” for attribute “role” on element “div”.
> >> ...
> >> This should be a warning ("unrecognized role 'foo'"), not an error.
> >
> > That current implementation of ARIA checking in the validator doesn't
> allow
> > checking of role values that contain multiple tokens. Making the
> validator
> > support role values containing multiple tokens would basically require me
> > to completely re-write the entirety of the ARIA-checking support in the
> > validator, in a much more complex way then the already-extremely-complex
> > way in which it's now implemented even just for the single-value-role
> case.
> > I'm not going to do that. See my comments here:
> >
> >
> http://lists.w3.org/Archives/Public/public-pfwg-comments/2013JulSep/0007.html
> >
> > I'm not aware of any good reason why a conforming document would ever
> need
> > to contain something like role="dialog foo".
>
> An example:
> role="switch checkbox"
>
> Switch is an ARIA 1.1 role that inherits from checkbox. If the UA knows
> about the switch role, if can communicate the "on/off" nature of the
> switch. Otherwise if will fall back to the more general "checked/unchecked"
> nature of a generic ARIA 1.0 checkbox role.
>
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/namefromauthor-requ/874.html
> ":9:54:
> >>> error: Bad value “ marquee” for attribute “role” on element “div”.
> >> Error in the validator.
> >> http://www.w3.org/WAI/PF/aria/complete#marquee
> >
> > The problem here is that the validator implementation currently doesn't
> > allow spaces around role names in the value of the role attribute. I
> guess
> > I can update it easily enough to just allow spaces around role names
> > (though not space-separated lists of multiple tokens), but I think the
> > better solution is instead for the ARIA spec to require that role
> > attributes not contain multiple tokens, nor any spaces.
>
> That's a deal breaker, I believe.
>
> From the spec: http://www.w3.org/WAI/PF/aria/complete#host_general_role
> """
> An implementing host language will provide an attribute with the following
> characteristics:
>
>         • The attribute name MUST be role;
>         • The attribute value MUST allow a token list as the value;
>         • The appearance of the name literal of any concrete WAI-ARIA role
> as one of these tokens MUST NOT in and of itself make the attribute value
> illegal in the host-language syntax; and
>         • The first name literal of a non-abstract WAI-ARIA role in the
> list of tokens in the role attribute defines the role according to which
> the user agent MUST process the element. User Agent processing for roles is
> defined in the WAI-ARIA User Agent Implementation Guide
> [ARIA-IMPLEMENTATION].
>
> """
>
> This is a necessary part of ARIA 1.0 and will become more necessary in
> subsequent versions. For example, in ARIA 2.0 we're probably going to add a
> partial Element interface that includes a reflected "role" property taking
> a DOMTokenList, and another property that holds the ~"computedRole" or
> something to indicate to the author script which of the tokens is being
> used as the role.
>
>
> >>>
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/properties-global-norole/properties-global-norole-aria-invalid-false.html
> >>> was supposed to be invalid but was not.
> >> ...
> >> I'm not sure what this error message means. Why doesn't the validator
> >> trust the author's indication that the content is invalid?
> >
> > That message is being emitted because I have the test runner in in the
> > validator configured to expect that any files with the string "-invalid-"
> > in their filenames are intentionally invalid. But I think what those
> > particular files in the ARIA testsuite are actually intended to test is
> the
> > "aria-invalid" attribute. So I guess I may need to come up with a
> different
> > filename string for identifying documents that are intentionally invalid.
> > Maybe "-not-valid-" instead?
> >
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/roles-plain-abstract/roles-plain-abstract-command.html
> ":8:36:
> >>> error: Bad value “command” for attribute “role” on element “div”.
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/roles-plain-abstract/roles-plain-abstract-composite.html
> ":8:38:
> >>> error: Bad value “composite” for attribute “role” on element “div”.
> >> ...
> >> These are intended to be invalid. It's testing the abstract roles.
> >
> > OK, so it would be helpful to me and maybe to others as well if we could
> > follow some file-naming convention for documents that are intentionally
> > invalid. Maybe "-not-valid-"?
>
> I don't have a strong preference.
>
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/_/554.htm
> ":3:69:
> >>> error: Element “input” is missing required attribute “alt”.
> >> ...
> >>
> >> This is an error in the test case, but the validator output should
> >> probably be more explicit in that @alt is required on
> input[type="image"]
> >> not just "input"…
> >> Should be ~: Element “input” with attribute “type” whose value is
> “image”
> >> must have non-empty attribute “alt”.
> >> ...
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/namefromauthor-requ/862.html
> ":7:53:
> >>> error: Element “div” is missing required attribute “aria-checked”.
> >> ...
> >> Validator should be more explicit here.
> >> Element “div” with attribute “role” whose value is “checkbox” must have
> >> non-empty attribute “aria-checked”.
> >> ...
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/namefromauthor-requ/885.html
> ":8:74:
> >>> error: Element “div” is missing required attribute “aria-valuemax”.
> >> ...
> >> Validator message should be:
> >> Element “…” with attribute “role” whose value is “…” must have
> non-empty attribute “aria-…”.
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/presentation-role/530.html
> ":8:55:
> >>> error: Element “li” is missing required attribute “role”.
> >> ...
> >> Validator message should be:
> >> Element “…” with attribute “aria-…” whose value is “…” must have
> non-empty attribute “role”.
> >> ...
> >>> "
> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/presentation-role/531.html
> ":7:75:
> >>> error: Attribute “aria-checked” not allowed on element “ul” in this
> context.
> >> ...
> >> Validator message should be:
> >> Attribute “aria-checked” not allowed on element “ul” with attribute
> “role” whose value is “presentation”.
> >
> > I agree it'd be better for the error messages for those cases to be more
> > precise. I'll consider refining them. But it's actually not possible to
> > make them more precise if the checking for them continues to be handled
> in
> > the place where it's currently handled in the validator, in the RelaxNG
> > grammar. Making them more specific would require moving the checks out
> of the
> > RelaxNG grammar and instead writing some custom Java code to handle them.
>
> Okay. Thanks.
>
>
>
>
>