Re: W3C Testing How To slides

On Mon, 2012-09-17 at 21:05 +0200, James Graham wrote:
> On Sun, 16 Sep 2012, Philippe Le Hegaret wrote:
> 
> > Mike and I put slides together on how to do testing at W3C:
> > http://w3c.github.com/testing-how-to/
> 
> Nice :)
> 
> A few points below:
> 
> == Technical feedback ==
> 
> assert_true(!!MyE.firstElementChild); is an abuse of assert_true; that's 
> why it looks so ugly :) If you wanted to write a real test here you would 
> use assert_idl_attribute(MyE, "firstChild"); But that would likely fail in 
> some browsers due to them having the IDL attributes in the wrong part of 
> the prototype chain. I suggest that it would be clearer to think of a 
> different test here. For example a test like assert_equals(document.title, 
> "the title element value") or similar would be quite straightforward.

I changed the test to use document.title.

> The paragraph explaining the test is unnecessary and slightly harmful 
> (it's better to write *more* tests than to write long descriptions in each 
> test). It also doesn't scale to > 1 test per page. Descriptive test titles 
> are more important (and using <title></title> to describe the whole file).

The description was also there to provide content for the
firstElementChild test itself btw. I removed it since it's now using
document.title.

> The async test should be something that is actually async. For example you 
> could test that the load event fires:
> 
> var t = async_test("Load event fires");
> onload = function(e) {
>    t.step(function() {
>      assert_equals(e.type, "load");
>      t.done();
>    });
> }

Good point. done. I didn't put done() inside step() btw since it seemed
wrong to me (ie done() would be invoked before step() is over).

> This would then be usful to demonstrate how t.step_func(); can reduce 
> boilerplate for this type of test.

Yes, I skipped step_func in the slides since it seemed more advanced.

> The demo for the manual test doesn't seem to work very well. It might fail 
> in the browser that I'm testing, but that wasn't at all clear from the 
> text. It would be nice to find an example that has a clearer pass/fail 
> condition.

Added on my todo.

> Many more test types will need client and server side code than just 
> XHR/WebSockets. For example with HTML5 navigation it's hard to write some 
> of the tests without precise control over the timing of various loads. So 
> I expect requiring at least a subset of tests to have a server side 
> component to be the common case.

Agreed. XHR and WebSockets are the only examples we have so far. We
could list more once we have them for sure.

> The github repos are read-only at the moment afaik.

Hum, that's not the intent. Quickly looking through the github
interface, I don't see an obvious setting to change. I'll talk to Mike
about it.

> == General Feedback ==
> 
> I agree with Tobie that these slides have rather a large amount of text; 
> more than I would like as an audience member. It could be good to trim 
> down the bits to be projected and put the other material into an adjunct 
> page that could be used as a virtual handout.

I'll need more than 30 minutes to do that so I'm putting that one on my
todo list as well unless someone beats to it.

Thanks for the feedback!

Philippe

Received on Wednesday, 19 September 2012 00:02:34 UTC