Re: Request for Review: single-touch tests; deadline Sept 24

With the following two changes, and Art's guidance that these tests should
focus on testing the spec, not interoperability between implementations,
I'm now happy with the state of single-touch.html

On the multiple test results for the same problem issue, how about a simple
pragmatic trade-off: collapse all failures about identifiedTouch into one.
 This addresses my main use case (wanting to be able to glance at the
failures and make sure they're all understood) without adding much
complexity.  I've committed a proposed change for that here:
https://github.com/AFBarstow/WebEvents/commit/eef7fbdb9e08bc5c901d0678d2f7ae35b35f7071.
 Seem OK?

And for the screenX/Y failures on mobile devices: given the above and the
complexity of the various scale factors applied in different scenarios
(browser zoom, vs. device scale, etc), I don't think it's worth trying to
validate these values.  I've removed them:
https://github.com/AFBarstow/WebEvents/commit/86d50fc931a97f408234e937c4d45d3fbb03ff6d

Rick

On Mon, Sep 24, 2012 at 6:59 PM, Rick Byers <rbyers@google.com> wrote:

> Thanks Cathy!  Inline.
>
> On Mon, Sep 24, 2012 at 1:36 PM, <Cathy.Chan@nokia.com> wrote:
>
>> Thanks Art and Rick for the updates.
>>
>> I ran the latest version at
>> https://github.com/AFBarstow/WebEvents/blob/master/single-touch.html on a
>> couple devices/browsers. Here are the results.
>>
>> 1. Nokia N9 stock WebKit browser passes 235/259 tests. The failing tests
>> are
>>
>> a. "document.createTouchList exists and correctly creates a TouchList
>> from a
>> single Touch"; "assert_equals: touchList.length is 1"; "expected 1 but got
>> 0"
>>
>
> Sounds like a bug.  Current webkit didn't show this problem, perhaps
> Nokia-specific?
>
>
>> b. "TouchList.identifiedTouch attribute exists"; "assert_true:
>> identifiedTouch attribute in TouchList"; "expected true got false"
>> c. "TouchList.identifiedTouch attribute is of type function";
>> "assert_equals: identifiedTouch attribute of type long"; "expected
>> 'function' but got 'undefined'
>
>
> Yep, identifiedTouch missing is a known WebKit bug<https://bugs.webkit.org/show_bug.cgi?id=96294>- I'm planning on fixing this in ToT sometime soon.
>
> "
>> d. "document.createTouchList exists and correctly creates a TouchList
>> from a
>> Touch array"; "assert_equals: touchList.length is 1"; "expected 1 but got
>> 0"
>>
>
> This might be the same WebKit bug<https://bugs.webkit.org/show_bug.cgi?id=97418>that I just found with Chrome.  Or it may be the same bug as c).
>
> e. "touchstart: touches.identifiedTouch same as first touch point id";
>> "'undefined' is not a function"
>
> f. "touchend: touch screen/screen is consistent with clientX/clientY";
>
> "assert_true: touch.screenX is less than windows.screen.width"; "expected
>> true got false"
>>
>
> I think this is a bug in my test wrt. viewport scaling - I'll poke around
> and make sure this passes on iOS and Android at least.
>
>
>> 2. Firefox on Nokia N9 (version 15.0) passes 275/276 tests. The failing
>> test
>> is
>> a. "document.createTouch exists and creates a Touch object with requested
>> properties"; "assert_equals: touch.clientX is
>> touch.pageX-window.pageXOffset."; "expected 15 but got 0"
>>
>> 3. Opera Mobile on Nokia N8 (version 12.00.2254) passes 236/259 tests. The
>> failing tests are the same as items a through e with the N9 stock browser.
>>
>> Regards, Cathy.
>>
>>
>> > -----Original Message-----
>> > From: Barstow Art (Nokia-CIC/Boston)
>> > Sent: Monday, September 24, 2012 1:49 PM
>> > To: ext Rick Byers
>> > Cc: public-webevents@w3.org
>> > Subject: Re: Request for Review: single-touch tests; deadline Sept 24
>> >
>> > On 9/23/12 10:32 PM, ext Rick Byers wrote:
>> > > 1. We still want some test that validates 'instanceof TouchList',
>> > > right?  This represents a legitimate bug in WebKit, right?  Or is it
>> > > OK for TouchList to not actually be a defined symbol?
>> >
>> > These are all good questions and I'd like to hear from others.
>> >
>> > > 2. I agree with Cathy that the large number of tests makes things a
>> > > little unwieldy (eg. having 10+ 'Pass: touch point is a Touch object'
>> > > makes the output pretty cluttered and hard to skim, similarly there's
>> > > little value to repeating over and over that TouchList is missing
>> > > identifiedTouch).  But I also don't think we want to combine all the
>> > > checks into a single test case (since, as Cathy says, that'll cause
>> > > errors - like WebKit's missing identifiedTouch - to halt the whole
>> > > test).  Perhaps the best tradeoff here is to just test the properties
>> > > of a given type once.  I.e. once we've verified one Touch object, it's
>> > > really overkill to verify all the others (I think it's pretty
>> > > unlikely, for example, that any implementation would have a 'screenX'
>> > > on some Touch instances but not others).  Or maybe we put these behind
>> > > an 'exhaustive mode' or something.
>> > >
>> > > Alternately, we could be a little fancier and get the best of both
>> > > worlds by establishing a baseline for a Touch object once (with one
>> > > test per property) - keeping track of which tests pass, and in
>> > > subsequent tests just assert that all touches match the same baseline.
>> > >  Let me know if this sounds worthwhile and I can code it up.
>> >
>> > I don't have a really strong preference here and am inclined for us to
>> do
>> the
>> > least amount of work that is necessary to exit CR. (This testing effort
>> is
>> > mostly about testing the spec as opposed to product release testing so
>> after
>> > CR, I don't know how much use the test suite will actually get.)
>> >
>> >
>> > > 3. I added tests for some additional things that aren't currently
>> > > covered by the test assertions:
>> >
>> > Excellent!
>> >
>> > >   * createTouchList and createTouch work as defined
>> > >       o Note that this uncovered a new bug in the WebKit
>> > >         implementation - as spec'd createTouchList is supposed to take
>> > >         an array of Touch objects, but WebKit doesn't handle this -
>> > >         instead supporting a variable number of Touch arguments (as
>> > >         can be seen in the WebKit layout test for this API
>> > >         <http://code.google.com/searchframe#OAMlx_jo-
>> >
>> ck/src/content/test/data/layout_tests/LayoutTests/fast/events/touch/scrip
>> > t-tests/document-create-touch-
>> > list.js&exact_package=chromium&q=createTouchList&type=cs&l=17>).
>> > >          I've filed https://bugs.webkit.org/show_bug.cgi?id=97418
>> > >         against myself to track.
>> > >
>> >
>> > OK.
>> >
>> > > I forked Art's version of the tests on gitHub here:
>> > > https://github.com/RByers/WebEvents.  The additions above are here
>> > >
>> > <https://github.com/RByers/WebEvents/commit/b2717c4925fc5f4a7724ad4
>> > e9e
>> > > 59be37d8276857#single-touch.html>
>> > > and here
>> > >
>> > <https://github.com/RByers/WebEvents/commit/dc21ef44950fe11f38d91a6
>> > b85195c43a3d7e8e5#single-touch.html>.
>> > >  Art, if these look OK, do you want to pull them into your copy (or
>> > > even have a single repo we all have permission to push to?)
>> >
>> > I went ahead and pulled your changes into mine because they all look
>> good
>> > to me, and I added you as a contributor.
>> >
>> > At some point the tests will need to be copied into Hg to use the W3C's
>> > Testing Framework <http://w3c-test.org/framework/>. It may be easier to
>> > continue to develop them via GH and then when we're ready, one of us can
>> > copy them into something like <http://w3c-
>> > test.org/webevents/tests/touch-events-v1/submissions/Google-Nokia/>.
>> >
>> > [BTW, the above would be consistent with what some members of the
>> > Testing list (e.g.
>> >
>> <
>> http://lists.w3.org/Archives/Public/public-test-infra/2012JulSep/0079.html
>> >
>> )
>> > have been promoting ... To lower the barrier for non-W3C members to
>> easily
>> > contribute tests (and not have to mess with Hg), GH is used to create
>> and
>> > develop tests and then when the tests are "baked", a member of the
>> > W3C/WG can then move them into Hg.]
>> >
>> >
>> > > With these few additions, I think we've pretty thoroughly covered this
>> > > simple single-touch scenario.  I can see a couple additional things we
>> > > may want to test in this scenario:
>> > >
>> > >   * non-zero scroll offset (just add some padding to the top, bottom
>> > >     and probably left, then have JS set the scroll position so the
>> > >     page looks the same as it does now by default).
>> > >   * touch point actually lines up with the user's finger - eg. draw a
>> > >     small circle on touchstart/touchend for manual verification?
>> > >
>> > > I'm not sure if either is really worth it though - thoughts?
>> >
>> > Unless someone feels real strongly and is willing to contribute code,
>> I'm
>> > inclined to move towards "we're done" re single touch.
>> >
>> > > There are a couple other single-touch scenarios I could imagine
>> > > testing (in separate test case files), eg:
>> > >
>> > >   * impact of preventDefault on scrolling
>> > >       o I think many of the details here are implementation dependent
>> > >         though - eg. if PD isn't called, do you still get touchmove
>> > >         events during a scroll operation?  It may be hard to write a
>> > >         useful test that isn't dependent on at least some
>> > >         implementation details.
>> > >   * impact of preventDefault on synthesized mouse events
>> > >       o but since these are optional - we'd probably technically have
>> > >         to make the test pass if no mouse events are detected at all
>> > >   * generation of touchcancel
>> > >       o this is so platform dependent that I'm not sure what we'd want
>> > >         here really - but a simple test that lets a user confirm it is
>> > >         possible to get a touchcancel may be good.
>> > >       o While playing with this in chrome, I discovered an interesting
>> > >         bug
>> > >
>> > <http://code.google.com/p/chromium/issues/detail?id=151871>where
>> > >         instead of getting touchcancel, I can get into a nasty state
>> > >         where I get illegal events.
>> > >   * interaction of touch events with iframes
>> > >       o this is completely unspecified and up to the UA, right?
>> > >
>> > > I'm not sure which of these are actually worth testing though -
>> thoughts?
>> >
>> > (See above. I'm inclined to say we only need tests for MUST assertions
>> in
>> the
>> > spec and we do not need tests for any implementation-dependent
>> > feature.)
>> >
>> >
>> > > I think I could whip up simple tests for the first three pretty
>> > > quickly - but I'm not sure I could make the PD ones completely free of
>> > > any implementation detail bias.
>> > >
>> > > And of course we have the mutl-touch scenarios to finish.
>> >
>> > Yes and I haven't reviewed what Olli submitted so I'm "wondering out
>> loud"
>> > (read that as an uneducated guess ;-)) if we only need to test those TAs
>> in
>> > Cathy's doc that only have a Yes for multi-touch (and not create any new
>> > tests if there is a Yes for both single and multi). Would that be
>> acceptable?
>> >
>> > -AB
>> >
>>
>>
>