- From: Scott González <scott.gonzalez@gmail.com>
- Date: Sun, 22 Sep 2013 08:20:36 -0400
- To: Tobie Langel <tobie@w3.org>
- Cc: public-test-infra <public-test-infra@w3.org>
- Message-ID: <CAO8i3ie5bR4GSn5ssfdfz9R9UMrjjbEb=xHTwCkpD1oVP7x2ww@mail.gmail.com>
#3 sounds like the best option to me, with a simple toggle for whether the SHOULD tests are run. Perhaps something like the Skip JSHint option in the jQuery UI test suites: http://view.jqueryui.com/master/tests/unit/progressbar/progressbar.html Would the SHOULD tests run by default or would you need to toggle them on? On Sun, Sep 22, 2013 at 5:26 AM, Tobie Langel <tobie@w3.org> wrote: > Hi all, > > A pull request[1] on the main repo brought up the issue of how to handle > optional normative requirements (SHOULD, MAY, etc.). It's not the first > time this issue was discussed. I found, for example, a rather long thread > on this topic[2] on the public-webapps mailing list. I couldn't find, > however a recommended practice on how we should handle this. I'd like us to > agree on one and document it. > > Here's a number of propositions: > > 1) We only test MUST normative requirements. > > 2) We test all normative requirements, and rely on result interpretation > to determine whether an implementation conforms to the spec (an > implementation can fully conform even though it fails a number tests, as > long as those are determined to be SHOULD/MAY tests). > > 3) We test all normative requirements but add meta data to those tests > that aren't MUST requirements. This allows running subset of tests when > SHOULD requirements don't make sense. E.g. avoid running media capture > tests on a device that doesn't have a camera. > > I'd be inclined to go with 3), but I'm eager to hear other's thoughts on > the subject. > > Thanks, > > --tobie > --- > [1]: https://github.com/w3c/web-platform-tests/pull/306 > [2]: > http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/0053.html > > > >
Received on Sunday, 22 September 2013 12:21:03 UTC