- From: François Daoust <fd@w3.org>
- Date: Thu, 15 Jul 2021 13:10:09 +0000
- To: public-media-wg@w3.org
Hi all,
The minutes of this week's call are available at:
https://www.w3.org/2021/07/13-mediawg-minutes.html
... and copied as raw text below.
Call was mostly focused on WebCodecs and on whether to expose the API in
Window environments. A follow-up Media WG call is scheduled in a couple
of weeks from now.
Thanks,
Francois.
-----
Media WG Call
13 July 2021
[2]Agenda. [3]IRC log.
[2]
https://github.com/w3c/media-wg/blob/main/meetings/2021-07-13-Media_Working_Group_Teleconference-agenda.md
[3] https://www.w3.org/2021/07/13-mediawg-irc
Attendees
Present
Bernard Aboba, Chris Cunningham, Chris Needham, Dale
Curtis, Francois Daoust, Gary Katsevman, Greg Freedman,
Jan-Ivar Bruaroey, Jer Noble, Joey Parrish, Mark Watson,
Matt Wolenetz, Peng Liu, Tess O'Connor, Thomas Guilbert,
Zachary Cava
Chair
Chris, Jer
Scribe
Jer, Francois
Contents
1. [4]Rechartering and Open UI collaboration - tidoust
2. [5]CfC: Should WebCodecs be exposed in Window environments?
Meeting minutes
Rechartering and Open UI collaboration - tidoust
tidoust: During AC meeting review of the draft charter, it was
suggested to add a coordination to Open UI collaboration CG
tidoust: Open UI CG interest is in harmonizing native controls
across browsers, establishing guidelines in UIs
tidoust: Interest in media controls as well
tidoust: suggestion is to coordinate with them as they seek
input from media experts
tidoust: Draft charter is under final approval and should be
complete very soon
CfC: Should WebCodecs be exposed in Window environments?
[6]Should WebCodecs be exposed in Window environments?
[6] https://github.com/w3c/webcodecs/issues/211
bernard: Q about the interaction between WebCodecs and machine
learning
bernard: haven't seen much data about that interaction
chcunningham: if ML wants to ingest video, we can expose the
video frame wherever its needed
bernard: the Q is about, e.g., does WebNN work well with
WebCodecs
dale: In Chrome, we have been working with WebGPU to ensure our
implementation will work well with WebCodecs
dale: we should make sure we're aligned there, but it's
slightly orthogonal
cpn: thanks everyone for your replies and patience w.r.t. the
CfC process
cpn: summarizing my own view, the availability of other apis in
a worker context has been raised as a concern, things like
WebAudio or OffscreenCanvas are unavailable or unavailable on
workers could hurt adoption
cpn: w.r.t. the difficulties of using workers in and of
themselves, restricting to a worker makes it harder to use the
API
jan-ivar: what's the result of the CfC?
cpn: The replies have not brought up new information; the
positions are similar as already stated
cpn: we've heard from all the implementors and many application
developers (the overwhelming majority of which support exposing
to worker)
cpn: the primary views supporting deferring the decision were
Apple and Mozilla
jean-ivar: Mozilla's position stands; we have data to share
about performance concerns we'd be willing to share
jean-ivar: [7]our data shows jank in the test case provided by
dale
[7]
https://docs.google.com/document/d/18OFTfCuZvcEbq_Xmt9VxMqEflV02Z8_2YODDXD_XdFg/edit#heading=h.4fmosb1x3t5w
dale: those sites with lots of scrolling are not the primary
targets of web codecs
dale: think it's unlikely that sites will choose WebCodecs as
their primary media playback engine
dale: our job is not to police the entire internet, and keep
people from doing bad things
jean-ivar: i think it was a lack of ambition if we say that
only SPA apps and simple pages will use WebCodecs
jean-ivar: I'm sure advertisers are going to use this as well;
it will be used in many places, and we would like that to not
be janky
dale: we would like the bad cases to be in the 95th percentile
dale: "you can't save everyone"
jean-ivar: it depends on how you measure "everyone". Major
sites will get this right, but there's going to be a long tail
of sites that will use this
jernoble: One concern I have with exposing in Window is that it
doesn't to be a use case that we have.
… We could try to mitigate some of those for sites that need to
run on main thread.
… If we do decide as a group that Window is a target that we
want, I think we need to do a better job at queueing
Dale: For clarity, there is a queueing mechanism.
… The only problem on main thread is throughput, maintaining
60fps, but queues are being used.
jernoble: OK, I wasn't aware of it.
… Then some amount of jitter buffer is doable
dale: Yes, completely.
… Single-frame latency queue, no one is doing that, 3-4 frames
are common.
Bernard: All of our issues relate to pre-processing and
post-processing, you can't just take WebCodecs into accounts.
… WebCodecs is a tiny piece of the whole processing workflow.
… If you tell developers that there's a wrong way to do things
early on, they will do it the right way.
jean-ivar: Google announced an intent to ship of a new API, the
question of whether to expose that API to window was tied to
this issue
jean-ivar: i would question whether it is correct to tie those
decisions together
chcunningham: the googlers here are not the same people who
worked on that intent-to-ship
chcunningham: lets not stall or increase the scope of this
conversation by pulling in that intent-to-ship into this WG
jan-ivar: that makes sense, and that's why i wanted to bring
this up; others in the WG might not have been aware of it
chcunningham: the two APIs are tied together in origin trials,
but they're separate Apis and implementations in our process
cpn: in terms of the WG, Insertable Streams is not a
deliverable of this WG; it doesn't have a home or an official
status
hober: my understanding is the choice before the group is to
either expose on window&worker, or to start by exposing on
worker and expose window later, correct?
hober: from an architectural standpoint, it's always easier to
add features rather than remove them; the more cautious
approach would be to expose on workers first
… we have time to learn from that experience and decide how
best to expose it on window
… pragmatically, it would be sensible to delay for now, not
saying that's no forever
cpn: there is a question we put to the TAG w.r.t. design
principles; if we go down the route for worker-only, is that
unprecedented
… it would be interesting to hear from the TAG about overall
criteria about when APIs are exposed to worker vs. window.
hober: we don't offhand have a case where APIs are exposed in
worker only, we do have the other way around
… the fact that some of those APIs are exposed only on window,
because we don't want things complicating or interfering with
worker work
jan-ivar: one example is AudioWorklet, where it's not available
on window
dale: can't find any APIs that are only exposed on Worker
chcunningham: on the Q about deferring, i understand hober's
point; our position is that we're comfortable going forward
because of the data & feedback make us comfortable doing so
chcunningham: clarification, speaking on behalf of Google
chcunningham: Google supports exposing this API on window now,
without deferral
bernard: Q: WebRTC doesn't have great history of threading; is
there experience about problems on the main thread.
… is there data about other APIs having problems on the main
thread
jan-ivar: web apis that duplicate what you can do with native
APIs, yes, no work is done on the main thread; but we're
allowing this API so that apps can do expensive operations
about pre- and post-processing
jan-ivar: we're having the same discussion about insertable
streams, about whether to expose to the window
chcunningham: the point of this being the first API only
exposed to worker, is just to reply to hober's point about
conservative default choices
chcunningham: process Q: how will the chairs decide the results
of the CfC
cpn: if the chairs decide that we don't have consensus, the
chairs have options available: we could decide on a vote, we
could move ahead and receive FOs which will be escalated to the
director
cpn: the AB and the TAG would convene
hober: there's an open question about the future without FOs
going to the director to decide, but realistically the W3C team
will handle FOs
hober: the experiment we had to send escalations to the AB and
the TAG was just that, an experiment
hober: there's no particular reason to believe that the
experiment would continue
hober: having been on the other side, as someone who has FO'd,
it takes a lot of time for a lot of very busy people. the Q is
whether this rises to a FO issue
jan-ivar: is it the case that shipping is blocked on resolving
whether to expose on window?
chcunningham: the debating part is the concern; Chrome thinks
there's nothing new to be found in debating
cpn: my assessment is that the WG does not have consensus
bernard: among the developers i've talked to, those developers
have consensus that it should be exposed on worker, largely
because worker support has been bad
… they feel that getting everything they need to make things
work on workers will take a long time
… they feel blocked by other W3C-related worker APIs; e.g.W3C
transport on workers; their existing work was based on MSE,
which was not available on workers; WebNN worker support
… MediaStream track support in workers
cpn: for certain classes of devices, just using workers is a
problem
… if we go down the route to deferral, what is the criteria for
re-evaluating that decision
hober: w.r.t. chcunningham's point of circle of debate, what if
this issue was just tabled entirely for a certain amount of
time?
… that would give time for implementation and adoption
experience
chcunningham: i think that ignores my point of having high
confidence about moving forward
cpn: my feeling is to side with the developers in the origin
trial; but if we decided on deferral we would need clear
mechanisms for changing our minds
chcunningham: would anyone object to a vote on this issue?
jan-ivar: i would object
Received on Thursday, 15 July 2021 13:10:13 UTC