- From: Fuqiao Xue <xfq@w3.org>
- Date: Wed, 30 May 2018 22:23:48 +0800
- To: public-wicg@w3.org
https://www.w3.org/2018/05/30-gesture-minutes.html
text extract follows:
[1]W3C
[1] https://www.w3.org/
– DRAFT –
Gesture Events
30 May 2018
[2]IRC log
[2] https://www.w3.org/2018/05/30-gesture-irc
Attendees
Present
AnQing, Chaals, Fuqiao, Max_Dapeng, Navid,
patrick_h_lauke, Xiaozhen, YanDong
Regrets
Chair
Max
Scribe
chaals
Contents
* [3]Meeting Minutes
1. [4]AnQing introduce gesture events proposal
2. [5]Discussion points - usage of gestures?
Meeting Minutes
[6]basic proposal
[6] https://github.com/JuntaoPeng/GestureEvents/blob/master/GestureEvents.md
ML: This is the second meeting of the CG - the first meeting
happened last year, before launching the CG.
… Today we have a proposal from Alibaba.
AnQing introduce gesture events proposal
[slides being shown on Webex]
AQ: Let's look at the motivation, and what to do. Basically we
want standards for gesture events on mobile phones, coming from
the smartphone OS.
… purpose is to specify common interactions like pinch, swipe,
or double-tap.
… we propose to define gesture events. The motivation is
simple, W3C defines touch events and pointer events but these
describe limited operations. In daily life we have to use more
events.
… So Web developers write JS to implement gesture detection
from a sequence of touch/pointer events. This is very
inefficient.
… There is also a lot of learning to make this work.
… enabling gesture events will improve performance, making user
interaction work more smoothly.
Navid: You want to do gesture detection, so you expect the OS
to find the gesture and pass it to the browser by name?
<patrick_h_lauke> everybody's muted
<NavidZ_> I lost audio too
<patrick_h_lauke> i think the question was: do you expect these
events to be generated like touch/pointer events (by the UA),
or by the OS
<patrick_h_lauke> referring to the "originate from smartphone
OS" line
<patrick_h_lauke> but i believe that line is in the slide more
as a "gestures are a thing that started gathering momentum/be a
thing on smartphones"
<patrick_h_lauke> rather than "it's generated by the OS"
<NavidZ_> Thanks Patrick :)
<patrick_h_lauke> anytime :)
AQ: We want to handle these on the browser, but the
functionality originates in the OS.
<patrick_h_lauke> (i think the answer to the question is: it'll
be a JS API similar to touch/pointer events, generated by the
browser)
<patrick_h_lauke> (unless the browser can HOOK into the OS' own
gesture recognizer)
<NavidZ_> But does the detection happen in the browser? do you
expect the gesture event to come from the OS?
<NavidZ_> like a tap event from the Android?
<NavidZ_> I don't think Android has such a thing
<NavidZ_> it has some helper classes for that
<NavidZ_> which detect the gesture based on the raw input
<patrick_h_lauke> for platform-agnosticity you probably want
the browser to have its own recognizer
AQ: We want to standardise some common gesture events [see
slide]
<patrick_h_lauke> but to leverage any helper available at
platform/OS level perhaps?
<xfq> yeah, that's what I think too
<patrick_h_lauke> (random: [7]https://twitter.com/
patrick_h_lauke/status/778558022381600768) ;)
[7] https://twitter.com/patrick_h_lauke/status/778558022381600768)
Discussion points - usage of gestures?
AQ: [shows some demos from the Web]
<patrick_h_lauke> i think in general, both the use cases and
the fact that user and authors want gestures is undisputed. the
crux of the problem is getting agreement, AND getting all the
players to agree that this can be standardised in light of IPR
concerns
AQ: There is maybe 20% touch-based scrolling, 15% of sites
using touchstart, from chrome telemetry metrics.
<patrick_h_lauke> chaals: if i have an iphone, and turn
voiceover on, double-tap and swipe and possibly others are
taken up by voiceover. if a site relies on gestures, it won't
work in this scenario
<Zakim> chaals, you wanted to note that some events are handled
differently e.g. with screenreaders running.
<patrick_h_lauke> chaals: it would be good to have an
abstraction - of what the user WANTS to do, rather than what
gesture they are executing
PHL: Chaals' point was that it would be good to avoid having
event names that are tied to specific user movement on the
platform - e.g. double-tap, or swipe'left.
<NavidZ_> patrick_h_lauke: Do you refer to say like click to
indicate activation and we do fire that as well on tap?
PHL: instead it would be good to have events that describe the
user intent - zoom-in, pan-up, etc.
… The proposal has fairly high level things already, buut we
could go a bit further.
… Chaals' other point. Voiceover swallows e.g. double-tap,
swipe, ...
<Max> ok, thanks Chaals.
PHL: so if the site relies on that, the user will not be able
to generate them. Or if the site overrides the input then the
user will not be able to use their own interface.
<patrick_h_lauke> (i hope i encapsulated chaals' concern plus
my take on it)
PHL: Having standardised events, even at the level of
"double-tap" or "swipe" means the platform can intervene, and
enable users of e.g. VoiceOver to do *something* that generates
a named double-tap, even if their actual interaction movement
is different. So this would be better than the current
situation.
<patrick_h_lauke> (and that i didn't crack up)
[+1 to what Patrick said]
<patrick_h_lauke> and yes, taking the abstraction even further,
would be Indie UI Events
<patrick_h_lauke> but that was abandoned/died
[Note that if the browser is generating these events, it needs
to make sure there is a way for the platform to intermediate
and generate events with a different gesture. This will fail
unless there is real implementation of the requirement.]
[And the Web has a history of getting this wrong ;( ]
AQ: We will return to this question...
<patrick_h_lauke> i personally think the proposed events
(tapevent, panevent, pinchevent, longpressevent [which would be
context click?], doubletapevent, rotateevent) are already
fairly high level
<patrick_h_lauke> they're at least one step up from processing
was touch/pointer coordinates to divine a gesture
<patrick_h_lauke> and Indie UI Events style events would be
even more abstract. but those fell down a rabbit hole of trying
to capture all possible user intents
AQ: We see other usage data with lower numbers, but it is
clearly something that developers are trying to achieve.
… having these events will reduce page size, increase
efficiency.
<patrick_h_lauke> in short: i personally think these gesture
events as proposed here are a good stepping stone, as it's
something authors are doing right now/are wanting to do, and it
avoids potential for needing to constantly invent new abstract
events that encapsulate intents
YD: The point is to bring a mobile need to the W3C standard.
[like Patrick, I agree that we need something in this space,
but we need to make sure that it doesn't break because of
assuming that a user gesture always means the same thing...]
YD: We are trying to bring Web technology to enable the same
power as mobile applications.
<patrick_h_lauke> but the meaning of a gesture is handled by
the developer, so it's not our problem?
<patrick_h_lauke> for reference, apple's proprietary gesture
events [8]https://developer.apple.com/library/content/
documentation/AppleApplications/Reference/SafariWebContent/
HandlingEvents/HandlingEvents.html#//apple_ref/doc/uid/
TP40006511-SW23
[8] https://developer.apple.com/library/content/documentation/AppleApplications/Reference/SafariWebContent/HandlingEvents/HandlingEvents.html#//apple_ref/doc/uid/TP40006511-SW23
<patrick_h_lauke> and microsoft's gesture events [9]https://
docs.microsoft.com/en-us/previous-versions/windows/
internet-explorer/ie-developer/dev-guides/
hh673557(v=vs.85)#Gesture_object_events
[9] https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/dev-guides/hh673557(v=vs.85)#Gesture_object_events
<patrick_h_lauke> [10]https://docs.microsoft.com/en-us/
previous-versions/windows/internet-explorer/ie-developer/
dev-guides/hh673557(v=vs.85)#gesture-object-and-events
[10] https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/dev-guides/hh673557(v=vs.85)#gesture-object-and-events
YD: For the browser implementation, we built on a Weex runtime.
The events are needed by developers.
<patrick_h_lauke> and note, both apple and microsoft have not
expressed any interest in standardising this
<patrick_h_lauke> as there are IPR issues
<patrick_h_lauke> for reference, Indie UI Events [11]https://
dvcs.w3.org/hg/IndieUI/raw-file/default/src/
indie-ui-events.html
[11] https://dvcs.w3.org/hg/IndieUI/raw-file/default/src/indie-ui-events.html
YD: We think the abstraction like pinch and double-tap are
reflecting the user intent, and this is helpful.
FQ: double-tap is a user movement. That is one way. Another is
that we standardise a set of user events - ZoomIn, Zoomout. The
user interaction might be double-tap, or pinch...
YD: OK, understood. From the developer, we think the meaning of
a gesture depends on the application.
… double-tap can sometimes mean zoom in, sometimes mean zoom
out...
… the meaning of the intent will be best left to the
application developer
PHL: In an ideal world we would have something like IndieUI
tried. It would be good to have intent events instead of events
that describe specific interaction.
… But it tried to capture a large number of user intents, and
ended up with a very small set that could work, and nobody
actually implemented the idea in the 3 years since it has been
released.
… So there is some problem making that happen. As new inputs
and new devices appear, there is a risk that we need to add new
event types to deal with new intents user can have, and that
seems like it would be very slow. So I don't think the
high-level abstract definition that we would really like will
actually occur.
… The proposal here is not reflecting user intent, but it does
seem like a step forward from what we have now.
… I will emphasise that the main problem in this space has been
IPR issues.
Max: This is not a new requirement, and we know it will bring
benefits. The IPR issue is also important.
AQ: As PHL noted, Microsoft and Apple are unwilling to
participate for IPR reasons. Is there a path forward for
dealing with this issue?
<patrick_h_lauke> (as a side note, mentioning that even for
standardisation of Touch Events apple raised a concern /
initated a PAG)
<patrick_h_lauke> (if that's the right terminology...)
Max: If we just define the API, does that get around the IPR
issue?
<patrick_h_lauke> [12]https://www.w3.org/2012/te-pag/
[12] https://www.w3.org/2012/te-pag/
<patrick_h_lauke> :)
[An API definition is not likely to infringe, but implementing
it probably will...
... so if you have a proposal that nobody can implement, we
didn't achieve anything useful....]
[So we need to look at the actual IPR. If there are expired
patents in this area, that means there are things you can
implement....]
<patrick_h_lauke> and beyond being ABLE to implement, there
also needs to be the DESIRE to implement
<patrick_h_lauke> noting how Apple has refused so far to do
anything in the Pointer Events space
Max: maybe there are multiple ways to implement an API. Whether
that infringes something depends what specific patent covers
… Does W3C have resources to use for studying patents in this
area?
NZ: I think it would be useful to talk to the implementors,
since they are the ones who are most concerned.
Max: Problem is that the implementors are not IPR experts. E.g.
in W3C Working Groups, if there are IPR issues a PAG will be
formed with IPR experts, who are different people.
… how can we get these resources in the CG.
… We need to understand what the IPR is related to this
proposal, and the status of that IPR.
<Max> q
PHL: As well as IPR, this won't work unless browsers *want* to
implement this. Working with pointer events, there was a time
where Google were not keen to implement them even though they
were involved in developing it.
… because Apple refused to even talk about this. Google's
thought was that this would not be interoperable so the
implementation time would be wasted
[Yandex felt the same way, from the developer side, wondering
why they should spend all the time to make their sites work
using pointer events when they would still have to make them
work with plain touch events because browsers would not accept
pointer events].
PHL: If there isn't interoperable implementation, we either
have to accept what e.g. Apple does which may have IPR risks
and other technical issues, or we will force developers to do
double work to deal with interoperability problems.
Max: Important point. From developer perspective,
interoperability is important. Browser vendors may have their
own thoughts...
… This seems like a complex problem, unless browser vendors
reach a common agreement. Otherwise this will be difficult to
move forward.
<patrick_h_lauke> i would say, as Rick Byers and other
mentioned in the email discussion leading up to this, a more
productive approach would be to see if a polyfill/library could
be developed/maintained collaboratively
<xfq> [13]https://lists.w3.org/Archives/Public/public-wicg/
2018May/0005.html
[13] https://lists.w3.org/Archives/Public/public-wicg/2018May/0005.html
<patrick_h_lauke> as both a proof of concept, and as possible
incentive for standardisation talk
[I note that e.g. Alibaba can work with UCBrowser and find a
few more high-profile people prepared to make the investment as
a proof of concept - if it is good enough, developers might
start to create pressure for interoperable implementation,
which is sort of what happened for Pointer Events]
<patrick_h_lauke> AQ: [shows examples of some technical
problems, like gestures that cross iframe boundaries and being
able to detect these]
AQ: Handling e.g. pinch across iframes is difficult.
[Yes this is difficult, do you have any concrete proposals to
handle it?]
<patrick_h_lauke> (i think that question came from Rick or
Navid, not me)
[this applies to any multi-touch gesture - rotate, even swipe]
[It also happens where there isn't a clear indication of what
is the iframe boundary - e.g. a user starts a rotate with two
finers each near the border between an iframe and the parent.
Should the iframe get the rotate, or the parent?]
PHL: Probably the next steps would be looking at some of the
polyfill libraries, maybe making a proof of concept to show how
these things could look.
… maybe as chaals said you can get enough momentum to ship a
proof of concept implementation, that might motivate further
discussion and development.
… Hammer seems to be abandoned, but developing on something
like that would be helpful to show why this work would be
useful
<xfq> [14]Hammer.js
[14] https://hammerjs.github.io/
PHL: (or at least triggering the lawyers who might think they
hold patents over this)
… Max: So next steps might be proof-of-concept
implementations...
<patrick_h_lauke> "flying a kite for the lawyers ;) "
<Max> [15]https://weex.incubator.apache.org/
[15] https://weex.incubator.apache.org/
YD: We tried to implement this in weex runtime. I think next
time we will have some demos based on this.
s/Wix?/Weex/
YD: Maybe the blocking issue is IPR. We cann raise the question
and try to involve people who know more about this - or
developers who might have good use cases to move this work
forward.
[thanks to all, adjourned]
<patrick_h_lauke> good stuff, thank you
<Max> thanks all!
Received on Wednesday, 30 May 2018 14:23:58 UTC