webtv-ISSUE-61: Support external testing/certification organizations [testing]

webtv-ISSUE-61: Support external testing/certification organizations [testing]

http://www.w3.org/2011/webtv/track/issues/61

Raised by: Mark Vickers
On product: testing

Build a single, consistent web site to run all W3C specification tests.
- Central: Provide one home for all W3C tests.
- Interactive or scripted: Must be able to run tests either interactively or completely scripted.
- Configurable: Enable choice of which specs and/or individual tests to run.
- One click/script: Any mix of specification tests should all be runnable from one click or a single script.
- Automated: All tests must be as fully automated as possible.
- Results: Test runs must create a detailed pass/fail report.

Motivation:
- HTML5 being built into external specs: DLNA, OIPF, DTG, HbbTV, etc.
- Many of these organizations provide testing and/or certification. A consistent W3C test suite ensures all organizations measure W3C specs consistently.
- Current W3C tests difficult to use by external organizations due to inconsistencies of test frameworks, test locations, coverage and lack of maintenance.

Dependencies:
- Involves changes and additions to test suites and frameworks across W3C.

What needs to be standardized:
Two external HTML5-related test suites meet the goals above and can be used as examples and/or to leverage technology:
1. Khronos WebGL Conformance Test Suite http://www.khronos.org/webgl/wiki/Testing/Conformance
2. ECMA Test262 http://test262.ecmascript.org

Received on Wednesday, 27 March 2013 21:34:44 UTC