Re: Request for Review: single-touch tests; deadline Sept 24

Thanks Cathy!  Inline.

On Mon, Sep 24, 2012 at 1:36 PM, <Cathy.Chan@nokia.com> wrote:

> Thanks Art and Rick for the updates.
>
> I ran the latest version at
> https://github.com/AFBarstow/WebEvents/blob/master/single-touch.html on a
> couple devices/browsers. Here are the results.
>
> 1. Nokia N9 stock WebKit browser passes 235/259 tests. The failing tests
> are
>
> a. "document.createTouchList exists and correctly creates a TouchList from
> a
> single Touch"; "assert_equals: touchList.length is 1"; "expected 1 but got
> 0"
>

Sounds like a bug.  Current webkit didn't show this problem, perhaps
Nokia-specific?


> b. "TouchList.identifiedTouch attribute exists"; "assert_true:
> identifiedTouch attribute in TouchList"; "expected true got false"
> c. "TouchList.identifiedTouch attribute is of type function";
> "assert_equals: identifiedTouch attribute of type long"; "expected
> 'function' but got 'undefined'


Yep, identifiedTouch missing is a known WebKit
bug<https://bugs.webkit.org/show_bug.cgi?id=96294>- I'm planning on
fixing this in ToT sometime soon.

"
> d. "document.createTouchList exists and correctly creates a TouchList from
> a
> Touch array"; "assert_equals: touchList.length is 1"; "expected 1 but got
> 0"
>

This might be the same WebKit
bug<https://bugs.webkit.org/show_bug.cgi?id=97418>that I just found
with Chrome.  Or it may be the same bug as c).

e. "touchstart: touches.identifiedTouch same as first touch point id";
> "'undefined' is not a function"

f. "touchend: touch screen/screen is consistent with clientX/clientY";

"assert_true: touch.screenX is less than windows.screen.width"; "expected
> true got false"
>

I think this is a bug in my test wrt. viewport scaling - I'll poke around
and make sure this passes on iOS and Android at least.


> 2. Firefox on Nokia N9 (version 15.0) passes 275/276 tests. The failing
> test
> is
> a. "document.createTouch exists and creates a Touch object with requested
> properties"; "assert_equals: touch.clientX is
> touch.pageX-window.pageXOffset."; "expected 15 but got 0"
>
> 3. Opera Mobile on Nokia N8 (version 12.00.2254) passes 236/259 tests. The
> failing tests are the same as items a through e with the N9 stock browser.
>
> Regards, Cathy.
>
>
> > -----Original Message-----
> > From: Barstow Art (Nokia-CIC/Boston)
> > Sent: Monday, September 24, 2012 1:49 PM
> > To: ext Rick Byers
> > Cc: public-webevents@w3.org
> > Subject: Re: Request for Review: single-touch tests; deadline Sept 24
> >
> > On 9/23/12 10:32 PM, ext Rick Byers wrote:
> > > 1. We still want some test that validates 'instanceof TouchList',
> > > right?  This represents a legitimate bug in WebKit, right?  Or is it
> > > OK for TouchList to not actually be a defined symbol?
> >
> > These are all good questions and I'd like to hear from others.
> >
> > > 2. I agree with Cathy that the large number of tests makes things a
> > > little unwieldy (eg. having 10+ 'Pass: touch point is a Touch object'
> > > makes the output pretty cluttered and hard to skim, similarly there's
> > > little value to repeating over and over that TouchList is missing
> > > identifiedTouch).  But I also don't think we want to combine all the
> > > checks into a single test case (since, as Cathy says, that'll cause
> > > errors - like WebKit's missing identifiedTouch - to halt the whole
> > > test).  Perhaps the best tradeoff here is to just test the properties
> > > of a given type once.  I.e. once we've verified one Touch object, it's
> > > really overkill to verify all the others (I think it's pretty
> > > unlikely, for example, that any implementation would have a 'screenX'
> > > on some Touch instances but not others).  Or maybe we put these behind
> > > an 'exhaustive mode' or something.
> > >
> > > Alternately, we could be a little fancier and get the best of both
> > > worlds by establishing a baseline for a Touch object once (with one
> > > test per property) - keeping track of which tests pass, and in
> > > subsequent tests just assert that all touches match the same baseline.
> > >  Let me know if this sounds worthwhile and I can code it up.
> >
> > I don't have a really strong preference here and am inclined for us to do
> the
> > least amount of work that is necessary to exit CR. (This testing effort
> is
> > mostly about testing the spec as opposed to product release testing so
> after
> > CR, I don't know how much use the test suite will actually get.)
> >
> >
> > > 3. I added tests for some additional things that aren't currently
> > > covered by the test assertions:
> >
> > Excellent!
> >
> > >   * createTouchList and createTouch work as defined
> > >       o Note that this uncovered a new bug in the WebKit
> > >         implementation - as spec'd createTouchList is supposed to take
> > >         an array of Touch objects, but WebKit doesn't handle this -
> > >         instead supporting a variable number of Touch arguments (as
> > >         can be seen in the WebKit layout test for this API
> > >         <http://code.google.com/searchframe#OAMlx_jo-
> > ck/src/content/test/data/layout_tests/LayoutTests/fast/events/touch/scrip
> > t-tests/document-create-touch-
> > list.js&exact_package=chromium&q=createTouchList&type=cs&l=17>).
> > >          I've filed https://bugs.webkit.org/show_bug.cgi?id=97418
> > >         against myself to track.
> > >
> >
> > OK.
> >
> > > I forked Art's version of the tests on gitHub here:
> > > https://github.com/RByers/WebEvents.  The additions above are here
> > >
> > <https://github.com/RByers/WebEvents/commit/b2717c4925fc5f4a7724ad4
> > e9e
> > > 59be37d8276857#single-touch.html>
> > > and here
> > >
> > <https://github.com/RByers/WebEvents/commit/dc21ef44950fe11f38d91a6
> > b85195c43a3d7e8e5#single-touch.html>.
> > >  Art, if these look OK, do you want to pull them into your copy (or
> > > even have a single repo we all have permission to push to?)
> >
> > I went ahead and pulled your changes into mine because they all look good
> > to me, and I added you as a contributor.
> >
> > At some point the tests will need to be copied into Hg to use the W3C's
> > Testing Framework <http://w3c-test.org/framework/>. It may be easier to
> > continue to develop them via GH and then when we're ready, one of us can
> > copy them into something like <http://w3c-
> > test.org/webevents/tests/touch-events-v1/submissions/Google-Nokia/>.
> >
> > [BTW, the above would be consistent with what some members of the
> > Testing list (e.g.
> >
> <
> http://lists.w3.org/Archives/Public/public-test-infra/2012JulSep/0079.html
> >
> )
> > have been promoting ... To lower the barrier for non-W3C members to
> easily
> > contribute tests (and not have to mess with Hg), GH is used to create and
> > develop tests and then when the tests are "baked", a member of the
> > W3C/WG can then move them into Hg.]
> >
> >
> > > With these few additions, I think we've pretty thoroughly covered this
> > > simple single-touch scenario.  I can see a couple additional things we
> > > may want to test in this scenario:
> > >
> > >   * non-zero scroll offset (just add some padding to the top, bottom
> > >     and probably left, then have JS set the scroll position so the
> > >     page looks the same as it does now by default).
> > >   * touch point actually lines up with the user's finger - eg. draw a
> > >     small circle on touchstart/touchend for manual verification?
> > >
> > > I'm not sure if either is really worth it though - thoughts?
> >
> > Unless someone feels real strongly and is willing to contribute code, I'm
> > inclined to move towards "we're done" re single touch.
> >
> > > There are a couple other single-touch scenarios I could imagine
> > > testing (in separate test case files), eg:
> > >
> > >   * impact of preventDefault on scrolling
> > >       o I think many of the details here are implementation dependent
> > >         though - eg. if PD isn't called, do you still get touchmove
> > >         events during a scroll operation?  It may be hard to write a
> > >         useful test that isn't dependent on at least some
> > >         implementation details.
> > >   * impact of preventDefault on synthesized mouse events
> > >       o but since these are optional - we'd probably technically have
> > >         to make the test pass if no mouse events are detected at all
> > >   * generation of touchcancel
> > >       o this is so platform dependent that I'm not sure what we'd want
> > >         here really - but a simple test that lets a user confirm it is
> > >         possible to get a touchcancel may be good.
> > >       o While playing with this in chrome, I discovered an interesting
> > >         bug
> > >
> > <http://code.google.com/p/chromium/issues/detail?id=151871>where
> > >         instead of getting touchcancel, I can get into a nasty state
> > >         where I get illegal events.
> > >   * interaction of touch events with iframes
> > >       o this is completely unspecified and up to the UA, right?
> > >
> > > I'm not sure which of these are actually worth testing though -
> thoughts?
> >
> > (See above. I'm inclined to say we only need tests for MUST assertions in
> the
> > spec and we do not need tests for any implementation-dependent
> > feature.)
> >
> >
> > > I think I could whip up simple tests for the first three pretty
> > > quickly - but I'm not sure I could make the PD ones completely free of
> > > any implementation detail bias.
> > >
> > > And of course we have the mutl-touch scenarios to finish.
> >
> > Yes and I haven't reviewed what Olli submitted so I'm "wondering out
> loud"
> > (read that as an uneducated guess ;-)) if we only need to test those TAs
> in
> > Cathy's doc that only have a Yes for multi-touch (and not create any new
> > tests if there is a Yes for both single and multi). Would that be
> acceptable?
> >
> > -AB
> >
>
>