W3C home > Mailing lists > Public > public-css-testsuite@w3.org > July 2013

Re: [css3-ui] Tests submitted for directional focus navigation properties

From: Gérard Talbot <css21testsuite@gtalbot.org>
Date: Tue, 16 Jul 2013 22:45:24 -0400
Message-ID: <7a494599d504bd95b396b2ecadd9e8a1.squirrel@ed-sh-cp3.entirelydigital.com>
To: "Rebecca Hauck" <rhauck@adobe.com>
Cc: "Leif Arne Storset" <lstorset@opera.com>, "public-css-testsuite@w3.org" <public-css-testsuite@w3.org>, "Tantek Çelik" <tantek@cs.stanford.edu>, "Jorrit Vermeiren" <jorritv@opera.com>, "Giuseppe Pascale" <giuseppep@opera.com>

Le Mar 16 juillet 2013 21:04, Rebecca Hauck a écrit :
> Hi Gérard,
> On 7/16/13 4:42 PM, ""Gérard Talbot"" <css21testsuite@gtalbot.org>
> wrote:
>>Le Mar 16 juillet 2013 16:24, Rebecca Hauck a écrit :
>>> Hi Leif,
>>> I took a look and added comments on the commit in Github:
>>> 96f2e6
>>> Other than the few minor changes I suggested, they look good.
>>> Cheers,
>>> -Rebecca
>>This test passes for me on a UA that does not support this property.
>> You
>>can add an intermediate step to first verify the property is supported.
>>I have in the past also criticized some tests for not being designed to
>>fail when the test is testing an invalid syntax (or invalid
>> declaration)
>>and when an UA does not support a property. I think this issue should
>> be
> What exactly should be clarified?

I think it should be stated in the Design Requirements section of Test
that if the [property or declaration or feature] being tested isn't
supported, then the test should always fail.

Right now, there is only "The test will not pass inadvertently." in
CSS Test Review Checklist, Test Design
which is a not-so-well-known wiki page.

> I think that for any spec test, if
> the
> thing you're testing isn't supported, the test should always fail.

In the past, Florian and I had an email exchange on this precise issue




and I think the Test Format Guidelines


should explicitly cover such test design situation. Because there is a
minority of tests that will "pass" when the UA does not support the
[property or declaration or feature] being tested.

Eg. UAs that do not support border-radius will pass these 8 tests









If you look more, you can easily find more like these.

> if
> the
> thing you're testing isn't supported, the test should always fail.
> Running
> such as suite (particularly in an automated engine) would give the
> impression of partial support for that feature when there is none.


> For
> these type of tests that test invalid or incomplete syntax, the
> reference
> file should not be designed to render whatever the default behavior is.
> For this case, however, I (maybe mistakenly) used Tab and Shift+Tab to
> test directional navigation, which causes them to pass on Chrome, which
> does not support this, for example.
> We're in the process of a W3C testing documentation project [1], which
> will be rolled out sometime soon. When I edit the Review Process doc
> [2],
> I can articulate this point there. I will surely solicit your feedback
> when it's ready.
>>shows that IE10 scores 21% but, in reality, the only tests IE10 passes
>>are those testing invalid syntax causing parsing errors.
>>Some CSS2.1 and CSS3 tests will actually be passed by buggy or old
>>browsers (say, Netscape 4 or IE4). I think this is wrong.
> We are in agreement. Sounds like there might be some poorly designed
> tests
> here.
>>Opera has now 2 accounts:
>>will return "No Assets Found." while
>>will return "15 Test Cases Found".
>>I think there should be only 1 Opera account.
> There are many tests under both accounts [3][4],


> so I'm not sure what we
> can do about that.
> It looks like there was one account created through
> the Shepherd UI under the name "Opera Software ASA" and tests submitted
> with that author title map to that account. Other tests were submitted
> under just "Opera Software", which did not match anything so Shepherd
> created a new account with the underscoring method. You're right, this
> could be confusing, so I'll check with Peter how to handle this.
>>First, use directional navigation to navigate the focus to the "START"
>>link below.
>>Test passes if navigating left once moves the focus to the "FINISH"
>> link.
>>I have no idea if this is possible... but is it possible to make the
>>wording of pass-fail conditions sentences in tests like those to be
>> less
>>technical? I am asking because I really do not know this.
> This was my first thought as well, but then I looked at the spec, which
> explicitly says: "This specification does not define which keys of a
> device are directional navigational keys,"


> so this is really the only
> way
> to express the steps here. The steps aren't so much technical as they
> are
> vague because so is the spec - intentionally.  One must assume the
> implementor running these test knows which keys you can use to navigate
> (though there is a note in the spec stating that arrow keys are a likely
> choice as well as shift+tab for reverse direction).
>>I am using Opera 12.16 and I have no idea what exactly I must do in
>> such
> Opera's implementation and how to use it is described here [5].

Structural elements within a document are also navigable. The previous
and next header on the page can be found using W and S. The keys E and D
do the same for text elements.

The keys Q and A allow you to jump to the previous or next link in the
document. (Ctrl+Down and Ctrl+Up do the same.)
Use Opera without a mouse: Document Navigation

I am still somewhat unsure on how exactly I should be "doing" these
tests as a tester using Opera 12.16. I have found and checked the box
marked "Enable single-key shortcuts"... And so Q and A keys navigate, E
and D keys navigate and Ctrl+Down and Ctrl+Up navigate.

> I don't
> think this specific instruction should be in the test itself though
> because as I mentioned above the spec makes no keyboard requirements.
> It
> might have been nice if this info were provided with the review was
> solicited, but it was easy enough to find :)

As test creator, I would provide such kind of info (the keyboard
combination for various browsers) inside <!-- comments --> in the source

>>your comments in
>>are not "echoed" into Shepherd system. I believe comments on tests
>>should not be scattered around.
> Yes, I know. When these tests are resubmitted with reviewer links, I
> will
> then add a link to the review in Github when I set them to Approved. I
> figured there would be a fairly quick turnaround with Leif and his team,
> so I was waiting for their resubmission in the next few days.
> Keep in mind that Peter is doing work to integrate Shepherd with Github
> so
> this will happen automatically when that work is done.  Right now, we're
> favoring Github's review tools to Shepherd's for many reasons (more
> widely
> used and understood, support for inline comments, email notifications,
> subscriptions, etc.). I'd like to move away from having all of the
> review
> comments done on this mailing list, where the comments are so far away
> from the code.
> In this interim period, you can continue to do things as you have been.
> When we're fully integrated with Github (and with the rest of the w3c
> test
> suites), I'll propose this change in process and make sure it's
> documented
> in the 'Reviewing Tests' doc.

Oh well... I like using Shepherd ... but anyway...


> [1] https://github.com/w3c/testtwf-website/issues?milestone=1&state=open
> (note that the closed issues are in the process
> [2] https://github.com/w3c/testtwf-website/issues/13
> [3] http://test.csswg.org/shepherd/search/author/opera_software/
> [4] http://test.csswg.org/shepherd/search/author/opera/
> [5] http://www.opera.com/help/tutorials/nomouse/

Contributions to the CSS 2.1 test suite:

CSS 2.1 Test suite RC6, March 23rd 2011:

CSS 2.1 test suite harness:

Contributing to to CSS 2.1 test suite:
Received on Wednesday, 17 July 2013 02:46:02 UTC

This archive was generated by hypermail 2.4.0 : Friday, 20 January 2023 19:58:19 UTC