Re: Request for Review: single-touch tests; deadline Sept 24

On 9/23/12 10:32 PM, ext Rick Byers wrote:
> 1. We still want some test that validates 'instanceof TouchList', 
> right?  This represents a legitimate bug in WebKit, right?  Or is it 
> OK for TouchList to not actually be a defined symbol?

These are all good questions and I'd like to hear from others.

> 2. I agree with Cathy that the large number of tests makes things a 
> little unwieldy (eg. having 10+ 'Pass: touch point is a Touch 
> object' makes the output pretty cluttered and hard to skim, similarly 
> there's little value to repeating over and over that TouchList is 
> missing identifiedTouch).  But I also don't think we want to combine 
> all the checks into a single test case (since, as Cathy says, that'll 
> cause errors - like WebKit's missing identifiedTouch - to halt the 
> whole test).  Perhaps the best tradeoff here is to just test the 
> properties of a given type once.  I.e. once we've verified one Touch 
> object, it's really overkill to verify all the others (I think it's 
> pretty unlikely, for example, that any implementation would have a 
> 'screenX' on some Touch instances but not others).  Or maybe we put 
> these behind an 'exhaustive mode' or something.
>
> Alternately, we could be a little fancier and get the best of both 
> worlds by establishing a baseline for a Touch object once (with one 
> test per property) - keeping track of which tests pass, and in 
> subsequent tests just assert that all touches match the same baseline. 
>  Let me know if this sounds worthwhile and I can code it up.

I don't have a really strong preference here and am inclined for us to 
do the least amount of work that is necessary to exit CR. (This testing 
effort is mostly about testing the spec as opposed to product release 
testing so after CR, I don't know how much use the test suite will 
actually get.)


> 3. I added tests for some additional things that aren't currently 
> covered by the test assertions:

Excellent!

>   * createTouchList and createTouch work as defined
>       o Note that this uncovered a new bug in the WebKit
>         implementation - as spec'd createTouchList is supposed to take
>         an array of Touch objects, but WebKit doesn't handle this -
>         instead supporting a variable number of Touch arguments (as
>         can be seen in the WebKit layout test for this API
>         <http://code.google.com/searchframe#OAMlx_jo-ck/src/content/test/data/layout_tests/LayoutTests/fast/events/touch/script-tests/document-create-touch-list.js&exact_package=chromium&q=createTouchList&type=cs&l=17>).
>          I've filed https://bugs.webkit.org/show_bug.cgi?id=97418
>         against myself to track.
>

OK.

> I forked Art's version of the tests on gitHub here: 
> https://github.com/RByers/WebEvents.  The additions above are here 
> <https://github.com/RByers/WebEvents/commit/b2717c4925fc5f4a7724ad4e9e59be37d8276857#single-touch.html> 
> and here 
> <https://github.com/RByers/WebEvents/commit/dc21ef44950fe11f38d91a6b85195c43a3d7e8e5#single-touch.html>. 
>  Art, if these look OK, do you want to pull them into your copy (or 
> even have a single repo we all have permission to push to?)

I went ahead and pulled your changes into mine because they all look 
good to me, and I added you as a contributor.

At some point the tests will need to be copied into Hg to use the W3C's 
Testing Framework <http://w3c-test.org/framework/>. It may be easier to 
continue to develop them via GH and then when we're ready, one of us can 
copy them into something like 
<http://w3c-test.org/webevents/tests/touch-events-v1/submissions/Google-Nokia/>.

[BTW, the above would be consistent with what some members of the 
Testing list (e.g. 
<http://lists.w3.org/Archives/Public/public-test-infra/2012JulSep/0079.html>) 
have been promoting ... To lower the barrier for non-W3C members to 
easily contribute tests (and not have to mess with Hg), GH is used to 
create and develop tests and then when the tests are "baked", a member 
of the W3C/WG can then move them into Hg.]


> With these few additions, I think we've pretty thoroughly covered this 
> simple single-touch scenario.  I can see a couple additional things we 
> may want to test in this scenario:
>
>   * non-zero scroll offset (just add some padding to the top, bottom
>     and probably left, then have JS set the scroll position so the
>     page looks the same as it does now by default).
>   * touch point actually lines up with the user's finger - eg. draw a
>     small circle on touchstart/touchend for manual verification?
>
> I'm not sure if either is really worth it though - thoughts?

Unless someone feels real strongly and is willing to contribute code, 
I'm inclined to move towards "we're done" re single touch.

> There are a couple other single-touch scenarios I could imagine 
> testing (in separate test case files), eg:
>
>   * impact of preventDefault on scrolling
>       o I think many of the details here are implementation dependent
>         though - eg. if PD isn't called, do you still get touchmove
>         events during a scroll operation?  It may be hard to write a
>         useful test that isn't dependent on at least some
>         implementation details.
>   * impact of preventDefault on synthesized mouse events
>       o but since these are optional - we'd probably technically have
>         to make the test pass if no mouse events are detected at all
>   * generation of touchcancel
>       o this is so platform dependent that I'm not sure what we'd want
>         here really - but a simple test that lets a user confirm it is
>         possible to get a touchcancel may be good.
>       o While playing with this in chrome, I discovered an interesting
>         bug
>         <http://code.google.com/p/chromium/issues/detail?id=151871>where
>         instead of getting touchcancel, I can get into a nasty state
>         where I get illegal events.
>   * interaction of touch events with iframes
>       o this is completely unspecified and up to the UA, right?
>
> I'm not sure which of these are actually worth testing though - thoughts?

(See above. I'm inclined to say we only need tests for MUST assertions 
in the spec and we do not need tests for any implementation-dependent 
feature.)


> I think I could whip up simple tests for the first three pretty 
> quickly - but I'm not sure I could make the PD ones completely free of 
> any implementation detail bias.
>
> And of course we have the mutl-touch scenarios to finish.

Yes and I haven't reviewed what Olli submitted so I'm "wondering out 
loud" (read that as an uneducated guess ;-)) if we only need to test 
those TAs in Cathy's doc that only have a Yes for multi-touch (and not 
create any new tests if there is a Yes for both single and multi). Would 
that be acceptable?

-AB

Received on Monday, 24 September 2012 17:49:34 UTC