- From: Francois Daoust <fd@w3.org>
- Date: Wed, 08 Mar 2023 19:03:09 +0000
- To: "public-media-wg@w3.org" <public-media-wg@w3.org>
Hi Media Working Group participants,
The minutes of this week's Media WG call, focused on WebCodecs, are
available at:
https://www.w3.org/2023/03/07-mediawg-minutes.html
... and copied as raw text below.
Thanks,
Francois.
-----
Media WG
07 March 2023
[2]Agenda. [3]IRC log.
[2]
https://github.com/w3c/media-wg/blob/main/meetings/2023-03-07-Media_Working_Group_Teleconference-agenda.md
[3] https://www.w3.org/2023/03/07-mediawg-irc
Attendees
Present
Bernard_Aboba, Chris_Needham, Eric_Carlson,
Eugene_Zemtsov, Francois_Daoust, Harald_Alvestrand,
Jer_Noble, jernoble
Regrets
-
Chair
-
Scribe
cpn, tidoust
Contents
1. [4]WebCodecs - Information on encoder/decoder performance
2. [5]Where to put per-frame QP in VideoEncoderEncodeOption
3. [6]TPAC 2023
Meeting minutes
WebCodecs - Information on encoder/decoder performance
[7]Issue 604
[7] https://github.com/w3c/webcodecs/issues/604
aboba: Some feedback from developers getting different results
when using "prefer-hardware" and "no-preference". With hardware
acceleration, you may get an error. With "no-preference", the
implementation may fallback to software, hiding the error.
… Developers were confused.
… One option is to give them the error from the ground up,
instead of having a fallback mechanism in place that may hide
it.
… For a high level API, we would keep things under the cover.
For a low-level API, it may make more sense to expose the error
directly.
jernoble: So, no "no-preference"? If developers don't care, why
would they care about getting the error?
aboba: An alternative is to use an event. But it seems
preferable to surface the error. Also, the spec does not say
anything about fallback. Question is: if you select
"no-preference", do you get the error?
… Unusual to hide things in an encoding API.
jernoble: Is there any guarantee in the spec that
"prefer-hardware" will give you hardware?
aboba: You may get an error back. If you don't get an error
back, in Chrome you will not have a software fallback from
under you, although that's not explicit in the spec.
jernoble: We don't have an API that requires hardware, so I'm
confused.
aboba: In practice, that's not a hint in Chromium. If you don't
get hardware, you get an error.
jernoble: We went through this in the previous round of
discussions on these hints.
… Now, we're coming back saying that Chromium implemented the
hint as requirement and thus we need an event or error. That
seems backwards.
… In this particular use case, they should be calling
"prefer-hardware".
aboba: Treating this setting as a hint does not work well
enough.
… If it's not a hint, you get the error immediately. That's
more simple.
jernoble: A lot of things are easier if you don't try to
protect privacy.
aboba: No privacy either way, just look at performances.
… FPS will drop significantly with software encoding.
jernoble: We have to go through the Privacy Interest Group when
we develop the spec.
aboba: What we're talking about here is the meaning of
"no-preference", and about fallback.
… This is not about performance information
… Because of the fallback, hardware errors are not being
surfaced.
cpn: The PR [8]#645 has been overtaken by more recent
discussions.
[8] https://github.com/w3c/webcodecs/issues/645
aboba: Right.
eugene: From an implementation point of view, do you plan to
change the spec to prevent browsers from switching when
"no-preference" is set?
aboba: No, it would be clearer that "no-preference" is not the
right setting if you're willing to be notified about hardware
errors.
… The question becomes when should the error be fired.
… That's a separate question. I can work with the developers to
understand what they really want.
… If you provided the error, they can probably call configure()
again. If that works for them, event may be the solution.
… If hardware does get taken away, there should be a
notification.
cpn: Proposed resolution is to add a note to the spec about
"no-preference", and surfacing the error as event.
hta: In WebRTC, this is useful to tell peer that another
encoding/decoding config needs to be used. In the WebCodecs, is
it really easy to switch between codecs?
aboba: That's what I'm trying to figure out. If it's a serious
GPU issue, then the error may not be recoverable. I don't
really know.
… Otherwise configure() may work.
… We may not want to expose the fact that you crashed your GPU
for instance.
cpn: Has that come up in WebGPU for instance?
aboba: That's a good question.
… Resolution is to add a note that "no-preference" means no
preference, and create an issue to check whether and how to
surface the error.
cpn: Implication is that "no-preference" would not trigger an
error.
Where to put per-frame QP in VideoEncoderEncodeOption
Issue [9]#270
[9] https://github.com/w3c/webcodecs/issues/270
eugene: Advanced feature that allows constant quality encoding,
and a few advanced use cases. Two ways to do this.
… First is to have these QP parameters inside a generic
quantizer field to VideoEncoderEncodeOptions. Second one is to
have codec specific quantizer field to
VideoEncoderEncodeOptions through sub dictionaries.
… The second option is more complicated.
… Different parameters per codec.
… There may be codecs in the future that don't have these
parameters at all.
… If we put the parameters in a generic field, we still need to
explain the ranges per codec.
tess: Why not do both?
eugene: The argument against having both is that this may be
misleading for users who might not know where to put these
parameters in the end.
cpn: In the discussion, it appeared that these QP parameters
could have different values per channel, e.g. Y, U, V.
eugene: Yes, codec-specific would be more flexible to
accommodate such issues in the future.
… We only need to provide supported ranges in the registry.
They're not going to scale. WebCodecs is low-level API, so we
don't want to introduce more complexity.
cpn: The decision is that there will be something per codec
that defines the ranges no matter what, and then the question
is whether to put the parameters in a generic field of codec
specific field.
jernoble: With my engineering hat on, we should reserve generic
settings to settings that apply across all codecs. It's not
obvious to me that these settings are generic enough, e.g. for
old codecs?
eugene: I'm more concerned about future codecs than about old
ones.
… Codecs like this would reject QP parameters if they don't
support them.
hta: If the application knows the difference between codecs, it
knows where to put codec-specific info. If it does not know, it
won't know what value ranges to use.
eugene: That's a good argument. Based on this feedback, codec
specific subdictionary seems a preferred option.
cpn: I think we have a consensus on this one.
eugene: Thanks, I'll write down the position of the Media WG
and merge the relevant PR.
hta: On a different topic, I'm wondering about the privacy
implications of revealing the use of a hardware codec.
Discussed in WebRTC as well.
cpn: That may be worth doing as a joint discussion.
<hta> [10]w3c/webrtc-stats#725
[10] https://github.com/w3c/webrtc-stats/pull/725
hta: We've been asked to provide a sensible algorithm to reveal
that hardware codec is used. The feeling of WebRTC stat folks
is that WebCodecs would be the right place for it.
TPAC 2023
cpn: W3C Team is starting to look into TPAC, and asking groups
to reserve slots. Do we want to meet? Do we want to have joint
meetings?
… Deadline around May, we still have a bit of time.
Received on Wednesday, 8 March 2023 19:03:13 UTC