- From: Giuseppe Pascale <giuseppep@opera.com>
- Date: Tue, 22 Apr 2014 18:22:36 +0200
- To: "SULLIVAN, BRYAN L" <bs3131@att.com>, Robin Berjon <robin@w3.org>, Tobie Langel <tobie@w3.org>
- Cc: "public-test-infra@w3.org" <public-test-infra@w3.org>, "public-web-and-tv@w3.org" <public-web-and-tv@w3.org>
- Message-ID: <CANiD0kreSgS=LZ3uhnGWDG-amXzwEZy9zxYAJbFjG6nn9BWy=Q@mail.gmail.com>
Bryan, thanks for starting this. Robin,Tobie, testing folks, during the workshop that Bryan mentioned, we discussed testing of we specs, and various organizations and company, while agreeing that having a common pool of test cases would be ideal, wanted to first get a better understanding of the of the W3C testing effort, to make sure this meets the requirements of those organizations/companies and that any effort they would put into contributing to it wouldn't be wasted. In no particular order, here a set of questions I've heard from various people+some comments from me. Can you help address them? I would like to invite other IG participants to chime in if I forgot something: 1. the first question was about "where to find information on the W3C testing setup and material". Bryan tried to answer with the mail below. In short it seems to me that the starting point is http://testthewebforward.org/docs/. Please chime in if anything needs to be added 2. TTWF website points to this post ( http://testthewebforward.org/blog/2013/02/20/testing-the-open-web-platform.html) under "Want to learn more about our plans?". Is that post still 100% accurate? If not, would be good to get an update post/page about what TTWF is and what the plans around testing are. 3. One other question raised was about process: how you submit tests, how do their get reviewed approved etc. AFAIKS this is answered here: http://testthewebforward.org/docs/review-process.html. One thing that is not clear though is timing information and who does what. If an organization/company was to submit 100s of test, who would be reviewing them? Is there any guarantee those would be reviewed at all? Or would your recommend to whoever intends to contribute tests to also contribute reviews? 4. Let's assume some organizations/companies decide the contribute to the W3C effort. What are the plans when it comes to maintaining the test that gets submitted? Are there processes in place to make sure that if spec changes, tests are "invalidated"? In other words, how can I know, at any given time, if a test suite for a given spec is still valid? And who is in charge to check that tests are still valid when a spec gets updated? Also, are there way to "challenge" a test, i.e. to say that a given (approved) test is in fact invalid? 5. IIRC not all WGs are using the process/tools from TTWF. Is this documented somewhere? Will these other groups continue with their tool for the time being or is there any plan to merge the various efforts at some point? 6. Do the test include metadata that easily allow to (at the very list) extract relevant test for a given spec? Are these mandatory/checked/maintained? 7. Some people expressed an interest to organize a TTWF event (or something similar) dedicated for TV, to better discuss and understand the needs of the TV industry. Do you think this would be doable? Is there already a calendar or next events and/or open dates? How would you recommend we go about this? (and do you think it would be useful in the first place? /g On Wed, Apr 16, 2014 at 4:23 PM, SULLIVAN, BRYAN L <bs3131@att.com> wrote: > Hi all, > > > > In the 4th Web & TV Workshop, one of the action items that I said I would > take was to help disseminate useful information to WebTV members that want > to get engaged in the testing effort at W3C, as a first step toward closing > the gaps in WebTV-usecase test assets and methodology noted in the > workshop. This is addressing a specific need of the WebTV IG, but should be > useful for any community of interest (e.g. WebMob) or new contributors at > large. > > > > This information will be provided on the test-infra list ( > http://lists.w3.org/Archives/Public/public-test-infra/), and for more > convenient/long-term use on one or more of: > > · the Testing Wiki (https://www.w3.org/wiki/Testing) – note the > “not maintained” banner for now, but I expect that it will get maintained > soon as part of this effort, unless the places below take precendence > > · the TTWF site (http://testthewebforward.org/) if/once I figure > out how we can add communities of interest (e.g. WebTV, Mobile, etc) to > TTWF. TTWF should be the preferred public home for such info I think. > > · The W3C github repository (http://testthewebforward.org/) in > particular the readme.md ( > https://github.com/w3c/web-platform-tests/blob/master/README.md) or some > other place on the github site, for more detailed help in getting engaged. > > > > The point of putting this info on one of these more permanent/usable > places is to prevent people who want to get engaged, from having to > rediscover this info each time (similar to forum FAQ/sticky posts) and bug > the list etc with newbie questions. We can also add a “who to ask” list if > questions aren’t answered there and direct people to the test-infra list as > a backup. > > > > Our intent to do this is not to put a management layer on the testing > effort, rather to make it easier for people to get engaged and to organize > as a community of interest around closing the gaps that matter to them. > This may include reviewing existing tests, adding new ones, building > metadata e.g. test runner scripts that focus test runs on specific features > or link tests to specific features/assertions where that info is missing > from tests, sharing test results, etc. > > > > Any other info that people have on “how to get engaged” info and where > that is documented, is requested as a response to this post. > > > > Thanks, > > Bryan Sullivan | Service Standards | AT&T > > >
Received on Tuesday, 22 April 2014 16:23:28 UTC