- From: HU, BIN <bh526r@att.com>
- Date: Mon, 23 Sep 2013 03:33:09 +0000
- To: Tobie Langel <tobie@w3.org>, public-test-infra <public-test-infra@w3.org>
3) is reasonable. I also suggest that the meta-data can also distinguish "SHOULD" type v.s. "MAY" type. Thanks Bin -----Original Message----- From: Tobie Langel [mailto:tobie@w3.org] Sent: Sunday, September 22, 2013 2:26 AM To: public-test-infra Subject: Should we test SHOULD? Hi all, A pull request[1] on the main repo brought up the issue of how to handle optional normative requirements (SHOULD, MAY, etc.). It's not the first time this issue was discussed. I found, for example, a rather long thread on this topic[2] on the public-webapps mailing list. I couldn't find, however a recommended practice on how we should handle this. I'd like us to agree on one and document it. Here's a number of propositions: 1) We only test MUST normative requirements. 2) We test all normative requirements, and rely on result interpretation to determine whether an implementation conforms to the spec (an implementation can fully conform even though it fails a number tests, as long as those are determined to be SHOULD/MAY tests). 3) We test all normative requirements but add meta data to those tests that aren't MUST requirements. This allows running subset of tests when SHOULD requirements don't make sense. E.g. avoid running media capture tests on a device that doesn't have a camera. I'd be inclined to go with 3), but I'm eager to hear other's thoughts on the subject. Thanks, --tobie --- [1]: https://github.com/w3c/web-platform-tests/pull/306 [2]: http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/0053.html
Received on Monday, 23 September 2013 03:33:53 UTC