<trackbot> Date: 19 December 2012
<olivier> Scribe: chrislowis
olivier: Seems like the best dates are 26-27th March.
... we are planning to hold the meeting in SF.
... I'm talking to jer and chris about hosting at apple or google respectively.
... I'm hearing no objections so pencil in 26-27th March 2012 as the date for our next face-to-face.
<ChrisWilson> Is this in San Francisco, or in Mountain View area, or undecided?
ChrisWilson: undecided so far.
olivier: I'd like to announce that Chris Lowis is to take on the co-chair role.
chrislowis: is pleased to take up the co-chair role.
olivier: we're going to move the call from this time on Wednesdays to this time on Thursdays, every two weeks.
... quick show of hands for either 10th or 17th for the next call.
chris: I may be taking vacation around that time. It could be one or the other possibly both.
ChrisWilson: slight preference for the 10th.
olivier: in that case we'll go for the 10th.
RESOLUTION: 10th January for the next teleconference.
<olivier> http://w3c.github.com/testing-how-to/#(1)%22
The URL of the presentation plh is about to give is above.
I won't be scribing the presentation.
<olivier> please feel free to ask questions - use q+ if you want to be added to the queue
olivier: has anybody built a tool to extract MUST statements from prose, or do we do that by hand?
<gmandyam> +q
plh: there has been some tools but what we've seen that it's not to bad to go through the spec and extract them.
olivier: when we've talked about testing in the past, not all the algorithms that are specified in our spec are entirely described as they tend to be "well known" from the signal processing literature.
<Zakim> olivier, you wanted to ask if there is any tool extracting MUSTs from prose
plh: some test suites are testing specifications to help get the spec approved, but are not so useful for product testing. Product testing is a level above that.
... the level above *that* is to write tests to find bugs in implementations.
... and there you need much more detail.
<gmandyam> Giri Mandyam speaking ...
<tmichel> +q
gmandyam: if a WG has a use cases document, is there no obligation to write tests for those use cases?
plh: no, there is no obligation.
gmandyam: so the MUSTs in the normative specification are the only things that matter, in the sense they supercede the use cases?
plh: that is case.
... creating a test suite for CR (?) is the first step. You can put stronger requirements on yourself.
tmichel: looking at the charter of the audio WG we are chartered to provide a test suite in order to exit CR.
shepazu: we have recommendations yet we still keep producing tests because interoperability is very important. That is our ultimate goal even if we decide first to produce the minimum set of tests required for CR.
<olivier> testharness.js is at http://w3c-test.org/resources/testharness.js
olivier: you mention that a file can contain many tests. Is there a best practice?
plh: it's a good idea to keep related tests in section. Tests need to run on all devices including mobile. So a file with too many tests might run into trouble.
<jussi> Agh, sorry, I forgot we rescheduled back to the old slot
chris: Is there any notion of a test which is like a manual test but where a screenshot is taken and a human being compares a test with a screenshot?
plh: no, Reftests are used for that - you provide a separate file for reference.
chris: would a possible reference test be used for example for the canvas2d api? Comparing a rendered canvas with a png file for example?
plh: yes, that would be a reftest.
chris: and that image could be different on IE and Firefox for example?
plh: at the moment we don't have the concept of an "approximation" in the reftest, it either matches or doesn't.
shepazu: we reftest any given implementation is self-consistent to the library/platform etc, but may not be different between implementations.
chris: good, if that's the case that's what we call reference tests in webkit.
olivier: Just a point of reference Alan has posted the start of a test harness for us using testharness.js
https://github.com/alankligman/webaudio-conformance/blob/master/sdk/incoming/destination.html
olivier: thank you to plh for joining us today.
... we can continue discussion on the mailing list.
<olivier> http://lists.w3.org/Archives/Public/public-test-infra/
shepazu: I want to officially welcome Chris Lowis as co-chair, the email has now been sent and is official.
olivier: great. We'll talk on the 10th, Thursday at the same time.