RE: Request for Review: single-touch tests; deadline Sept 24

Thanks Art and Rick for the updates.

I ran the latest version at
https://github.com/AFBarstow/WebEvents/blob/master/single-touch.html on a
couple devices/browsers. Here are the results.

1. Nokia N9 stock WebKit browser passes 235/259 tests. The failing tests are

a. "document.createTouchList exists and correctly creates a TouchList from a
single Touch"; "assert_equals: touchList.length is 1"; "expected 1 but got
0"
b. "TouchList.identifiedTouch attribute exists"; "assert_true:
identifiedTouch attribute in TouchList"; "expected true got false"
c. "TouchList.identifiedTouch attribute is of type function";
"assert_equals: identifiedTouch attribute of type long"; "expected
'function' but got 'undefined'"
d. "document.createTouchList exists and correctly creates a TouchList from a
Touch array"; "assert_equals: touchList.length is 1"; "expected 1 but got 0"
e. "touchstart: touches.identifiedTouch same as first touch point id";
"'undefined' is not a function"
f. "touchend: touch screen/screen is consistent with clientX/clientY";
"assert_true: touch.screenX is less than windows.screen.width"; "expected
true got false"

2. Firefox on Nokia N9 (version 15.0) passes 275/276 tests. The failing test
is
a. "document.createTouch exists and creates a Touch object with requested
properties"; "assert_equals: touch.clientX is
touch.pageX-window.pageXOffset."; "expected 15 but got 0"

3. Opera Mobile on Nokia N8 (version 12.00.2254) passes 236/259 tests. The
failing tests are the same as items a through e with the N9 stock browser.

Regards, Cathy.


> -----Original Message-----
> From: Barstow Art (Nokia-CIC/Boston)
> Sent: Monday, September 24, 2012 1:49 PM
> To: ext Rick Byers
> Cc: public-webevents@w3.org
> Subject: Re: Request for Review: single-touch tests; deadline Sept 24
> 
> On 9/23/12 10:32 PM, ext Rick Byers wrote:
> > 1. We still want some test that validates 'instanceof TouchList',
> > right?  This represents a legitimate bug in WebKit, right?  Or is it
> > OK for TouchList to not actually be a defined symbol?
> 
> These are all good questions and I'd like to hear from others.
> 
> > 2. I agree with Cathy that the large number of tests makes things a
> > little unwieldy (eg. having 10+ 'Pass: touch point is a Touch object'
> > makes the output pretty cluttered and hard to skim, similarly there's
> > little value to repeating over and over that TouchList is missing
> > identifiedTouch).  But I also don't think we want to combine all the
> > checks into a single test case (since, as Cathy says, that'll cause
> > errors - like WebKit's missing identifiedTouch - to halt the whole
> > test).  Perhaps the best tradeoff here is to just test the properties
> > of a given type once.  I.e. once we've verified one Touch object, it's
> > really overkill to verify all the others (I think it's pretty
> > unlikely, for example, that any implementation would have a 'screenX'
> > on some Touch instances but not others).  Or maybe we put these behind
> > an 'exhaustive mode' or something.
> >
> > Alternately, we could be a little fancier and get the best of both
> > worlds by establishing a baseline for a Touch object once (with one
> > test per property) - keeping track of which tests pass, and in
> > subsequent tests just assert that all touches match the same baseline.
> >  Let me know if this sounds worthwhile and I can code it up.
> 
> I don't have a really strong preference here and am inclined for us to do
the
> least amount of work that is necessary to exit CR. (This testing effort is
> mostly about testing the spec as opposed to product release testing so
after
> CR, I don't know how much use the test suite will actually get.)
> 
> 
> > 3. I added tests for some additional things that aren't currently
> > covered by the test assertions:
> 
> Excellent!
> 
> >   * createTouchList and createTouch work as defined
> >       o Note that this uncovered a new bug in the WebKit
> >         implementation - as spec'd createTouchList is supposed to take
> >         an array of Touch objects, but WebKit doesn't handle this -
> >         instead supporting a variable number of Touch arguments (as
> >         can be seen in the WebKit layout test for this API
> >         <http://code.google.com/searchframe#OAMlx_jo-
> ck/src/content/test/data/layout_tests/LayoutTests/fast/events/touch/scrip
> t-tests/document-create-touch-
> list.js&exact_package=chromium&q=createTouchList&type=cs&l=17>).
> >          I've filed https://bugs.webkit.org/show_bug.cgi?id=97418
> >         against myself to track.
> >
> 
> OK.
> 
> > I forked Art's version of the tests on gitHub here:
> > https://github.com/RByers/WebEvents.  The additions above are here
> >
> <https://github.com/RByers/WebEvents/commit/b2717c4925fc5f4a7724ad4
> e9e
> > 59be37d8276857#single-touch.html>
> > and here
> >
> <https://github.com/RByers/WebEvents/commit/dc21ef44950fe11f38d91a6
> b85195c43a3d7e8e5#single-touch.html>.
> >  Art, if these look OK, do you want to pull them into your copy (or
> > even have a single repo we all have permission to push to?)
> 
> I went ahead and pulled your changes into mine because they all look good
> to me, and I added you as a contributor.
> 
> At some point the tests will need to be copied into Hg to use the W3C's
> Testing Framework <http://w3c-test.org/framework/>. It may be easier to
> continue to develop them via GH and then when we're ready, one of us can
> copy them into something like <http://w3c-
> test.org/webevents/tests/touch-events-v1/submissions/Google-Nokia/>.
> 
> [BTW, the above would be consistent with what some members of the
> Testing list (e.g.
>
<http://lists.w3.org/Archives/Public/public-test-infra/2012JulSep/0079.html>
)
> have been promoting ... To lower the barrier for non-W3C members to easily
> contribute tests (and not have to mess with Hg), GH is used to create and
> develop tests and then when the tests are "baked", a member of the
> W3C/WG can then move them into Hg.]
> 
> 
> > With these few additions, I think we've pretty thoroughly covered this
> > simple single-touch scenario.  I can see a couple additional things we
> > may want to test in this scenario:
> >
> >   * non-zero scroll offset (just add some padding to the top, bottom
> >     and probably left, then have JS set the scroll position so the
> >     page looks the same as it does now by default).
> >   * touch point actually lines up with the user's finger - eg. draw a
> >     small circle on touchstart/touchend for manual verification?
> >
> > I'm not sure if either is really worth it though - thoughts?
> 
> Unless someone feels real strongly and is willing to contribute code, I'm
> inclined to move towards "we're done" re single touch.
> 
> > There are a couple other single-touch scenarios I could imagine
> > testing (in separate test case files), eg:
> >
> >   * impact of preventDefault on scrolling
> >       o I think many of the details here are implementation dependent
> >         though - eg. if PD isn't called, do you still get touchmove
> >         events during a scroll operation?  It may be hard to write a
> >         useful test that isn't dependent on at least some
> >         implementation details.
> >   * impact of preventDefault on synthesized mouse events
> >       o but since these are optional - we'd probably technically have
> >         to make the test pass if no mouse events are detected at all
> >   * generation of touchcancel
> >       o this is so platform dependent that I'm not sure what we'd want
> >         here really - but a simple test that lets a user confirm it is
> >         possible to get a touchcancel may be good.
> >       o While playing with this in chrome, I discovered an interesting
> >         bug
> >
> <http://code.google.com/p/chromium/issues/detail?id=151871>where
> >         instead of getting touchcancel, I can get into a nasty state
> >         where I get illegal events.
> >   * interaction of touch events with iframes
> >       o this is completely unspecified and up to the UA, right?
> >
> > I'm not sure which of these are actually worth testing though -
thoughts?
> 
> (See above. I'm inclined to say we only need tests for MUST assertions in
the
> spec and we do not need tests for any implementation-dependent
> feature.)
> 
> 
> > I think I could whip up simple tests for the first three pretty
> > quickly - but I'm not sure I could make the PD ones completely free of
> > any implementation detail bias.
> >
> > And of course we have the mutl-touch scenarios to finish.
> 
> Yes and I haven't reviewed what Olli submitted so I'm "wondering out loud"
> (read that as an uneducated guess ;-)) if we only need to test those TAs
in
> Cathy's doc that only have a Yes for multi-touch (and not create any new
> tests if there is a Yes for both single and multi). Would that be
acceptable?
> 
> -AB
>