Re: [aria-testsuite] spec violations in ARIA testsuite

Michael (Cooper) and Janina. 

I think we need some PF/HTML intervention for these first three "strong native semantics" issues.


On Aug 20, 2013, at 8:18 AM, Michael[tm] Smith <mike@w3.org> wrote:

>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/_functional/tree/ariatree.html":30:21:
>>> error: Bad value “group” for attribute “role” on element “ul”.
>> ...
>> Error in the validator. The group role definitely exists and it's properly used in this tree example.
>> http://www.w3.org/WAI/PF/aria/complete#group
>> ...
>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/presentation-role/528.html":9:23:
>>> error: Bad value “checkbox” for attribute “role” on element “ul”.
>> 
>> Granted, it's a weird test case to set the role on the UL, but: Error in the validator.
> 
> The validator's in conformance with the HTML spec in reporting those as
> errors, because the HTML spec currently doesn't allow ul@role="group" or 
> ul@role="checkbox":
> 
>  http://www.w3.org/html/wg/drafts/html/master/dom.html#sec-implicit-aria-semantics
> 
> So allowing ul@role=group and ul@role="checkbox" would require a change to
> the HTML spec.

Janina and Michael. PF needs to push back against the HTML WG here. There's not valid reason to enforce strong native semantics on UL. It's just a block container, not a form control. I don't care about the checkbox role, but "group" and "menu" roles *required* for trees, treegrids, menus, lists, and other common ARIA patterns. 

If HTML doesn't change this, many complex ARIA widgets will not validate.


>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/name-computation-input/548.html":9:59:
>>> error: Bad value “menu” for attribute “role” on element “select”.
>> ...
>> Error in the validator.
> 
> The validator's in conformance with the HTML spec in reporting that as an
> error, because the HTML spec currently doesn't allow select@role="menu":
> 
>  http://www.w3.org/html/wg/drafts/html/master/dom.html#sec-strong-native-semantics
> 
> As far as I can tell, the only role value allowed by the HTML spec for the
> select element is select@role=listbox. So allowing select@role=menu would
> require a change to the HTML spec.

Need to push the HTML WG on this one, too.

>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/name-computation-input/548.html":10:60:
>>> error: Bad value “menuitem” for attribute “role” on element “option”.
>> ...
>> Error in the validator.
> 
> The validator's in conformance with the HTML spec in reporting that as an
> error, because the HTML spec currently doesn't allow option@role="menuitem":
> 
>  http://www.w3.org/html/wg/drafts/html/master/dom.html#sec-strong-native-semantics
> 
> The only role value allowed by the HTML spec for the option element is
> option@role=option. So allowing option@role=menuitem would require a change
> to the HTML spec.

Ditto.

>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/_/828.htm":1:59:
>>> error: Bad value “text all” for attribute “aria-relevant” on element “div”.
>> 
>> Error in the validator. A value of "text all" is redundant (should just
>> be "all") but it's not explicitly disallowed or invalid.
> 
> hmm.. The definition of the value of the aria-relevant attribute in the
> ARIA spec currently reads:
> 
>  The attribute is represented as a space delimited list of the following
>  values: additions, removals, text; or a single catch-all value all.
> 
> I read that as an either/or, with the spec requiring that it be only one of
> the following two choices:
> 
>  A. A list of one or more of "additions", "removals", "text".
> 
>    OR
> 
>  B. A single value, "all" (instead of a list of values).
> 
> By that reading, "text all" would be invalid.
> 
> So if I'm misreading the spec here, and "text all" is actually intended to
> be valid, then I'd think other readers are likely to be misreading the spec
> in the same way, so the spec should be refined to make it more clear.

Okay. I can accept that reading. Test case error.

>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/host-language/role-2tokens-first.html":8:39:
>>> error: Bad value “dialog foo” for attribute “role” on element “div”.
>> ...
>> This should be a warning ("unrecognized role 'foo'"), not an error.
> 
> That current implementation of ARIA checking in the validator doesn't allow
> checking of role values that contain multiple tokens. Making the validator
> support role values containing multiple tokens would basically require me
> to completely re-write the entirety of the ARIA-checking support in the
> validator, in a much more complex way then the already-extremely-complex
> way in which it's now implemented even just for the single-value-role case.
> I'm not going to do that. See my comments here:
> 
>  http://lists.w3.org/Archives/Public/public-pfwg-comments/2013JulSep/0007.html
> 
> I'm not aware of any good reason why a conforming document would ever need
> to contain something like role="dialog foo".

An example:
role="switch checkbox"

Switch is an ARIA 1.1 role that inherits from checkbox. If the UA knows about the switch role, if can communicate the "on/off" nature of the switch. Otherwise if will fall back to the more general "checked/unchecked" nature of a generic ARIA 1.0 checkbox role.

>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/namefromauthor-requ/874.html":9:54:
>>> error: Bad value “ marquee” for attribute “role” on element “div”.
>> Error in the validator.
>> http://www.w3.org/WAI/PF/aria/complete#marquee
> 
> The problem here is that the validator implementation currently doesn't
> allow spaces around role names in the value of the role attribute. I guess
> I can update it easily enough to just allow spaces around role names
> (though not space-separated lists of multiple tokens), but I think the
> better solution is instead for the ARIA spec to require that role
> attributes not contain multiple tokens, nor any spaces.

That's a deal breaker, I believe. 

From the spec: http://www.w3.org/WAI/PF/aria/complete#host_general_role
"""
An implementing host language will provide an attribute with the following characteristics:

	• The attribute name MUST be role;
	• The attribute value MUST allow a token list as the value;
	• The appearance of the name literal of any concrete WAI-ARIA role as one of these tokens MUST NOT in and of itself make the attribute value illegal in the host-language syntax; and
	• The first name literal of a non-abstract WAI-ARIA role in the list of tokens in the role attribute defines the role according to which the user agent MUST process the element. User Agent processing for roles is defined in the WAI-ARIA User Agent Implementation Guide [ARIA-IMPLEMENTATION].

"""

This is a necessary part of ARIA 1.0 and will become more necessary in subsequent versions. For example, in ARIA 2.0 we're probably going to add a partial Element interface that includes a reflected "role" property taking a DOMTokenList, and another property that holds the ~"computedRole" or something to indicate to the author script which of the tokens is being used as the role.


>>> https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/properties-global-norole/properties-global-norole-aria-invalid-false.html
>>> was supposed to be invalid but was not.
>> ...
>> I'm not sure what this error message means. Why doesn't the validator
>> trust the author's indication that the content is invalid?
> 
> That message is being emitted because I have the test runner in in the
> validator configured to expect that any files with the string "-invalid-"
> in their filenames are intentionally invalid. But I think what those
> particular files in the ARIA testsuite are actually intended to test is the
> "aria-invalid" attribute. So I guess I may need to come up with a different
> filename string for identifying documents that are intentionally invalid.
> Maybe "-not-valid-" instead?
> 
>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/roles-plain-abstract/roles-plain-abstract-command.html":8:36:
>>> error: Bad value “command” for attribute “role” on element “div”.
>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/roles-plain-abstract/roles-plain-abstract-composite.html":8:38:
>>> error: Bad value “composite” for attribute “role” on element “div”.
>> ...
>> These are intended to be invalid. It's testing the abstract roles.
> 
> OK, so it would be helpful to me and maybe to others as well if we could
> follow some file-naming convention for documents that are intentionally
> invalid. Maybe "-not-valid-"?

I don't have a strong preference.

>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/_/554.htm":3:69:
>>> error: Element “input” is missing required attribute “alt”.
>> ...
>> 
>> This is an error in the test case, but the validator output should
>> probably be more explicit in that @alt is required on input[type="image"]
>> not just "input"…
>> Should be ~: Element “input” with attribute “type” whose value is “image”
>> must have non-empty attribute “alt”.
>> ...
>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/namefromauthor-requ/862.html":7:53:
>>> error: Element “div” is missing required attribute “aria-checked”.
>> ...
>> Validator should be more explicit here. 
>> Element “div” with attribute “role” whose value is “checkbox” must have
>> non-empty attribute “aria-checked”.
>> ...
>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/namefromauthor-requ/885.html":8:74:
>>> error: Element “div” is missing required attribute “aria-valuemax”.
>> ...
>> Validator message should be:
>> Element “…” with attribute “role” whose value is “…” must have non-empty attribute “aria-…”.
>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/presentation-role/530.html":8:55:
>>> error: Element “li” is missing required attribute “role”.
>> ...
>> Validator message should be:
>> Element “…” with attribute “aria-…” whose value is “…” must have non-empty attribute “role”.
>> ...
>>> "https://dvcs.w3.org/hg/pfwg/raw-file/tip/ARIA/1.0/tests/test-files/presentation-role/531.html":7:75:
>>> error: Attribute “aria-checked” not allowed on element “ul” in this context.
>> ...
>> Validator message should be:
>> Attribute “aria-checked” not allowed on element “ul” with attribute “role” whose value is “presentation”. 
> 
> I agree it'd be better for the error messages for those cases to be more
> precise. I'll consider refining them. But it's actually not possible to
> make them more precise if the checking for them continues to be handled in
> the place where it's currently handled in the validator, in the RelaxNG
> grammar. Making them more specific would require moving the checks out of the
> RelaxNG grammar and instead writing some custom Java code to handle them.

Okay. Thanks.

Received on Tuesday, 20 August 2013 19:00:39 UTC