- From: Alan Stearns <stearns@adobe.com>
- Date: Mon, 20 Jul 2020 16:13:44 +0000
- To: W3C style mailing list <www-style@w3.org>
Cross-posting for those who access CSS minutes from this list.
On 7/20/20, 5:25 AM, "Francois Daoust" <fd@w3.org> wrote:
Hi all,
The minutes of last week's joint meeting between the Media WG and the
CSS Media WG are available at:
https://www.w3.org/2020/07/14-mediawg-minutes.html
The groups discussed solutions and problems to expose bi-plane
(video/graphics) capabilities to applications. Minutes are copied as raw
text below.
Thanks,
Francois.
-----
Joint meeting between CSS WG and Media WG
14 July 2020
[2]Agenda. [3]IRC log.
[2] https://github.com/w3c/media-wg/issues/18
[3] https://www.w3.org/2020/07/14-mediawg-irc
Attendees
Present
Alan Stearns, Barbara Hochgesang, Becca Hughes, Chris
Needham, Daniel Holbert, Florian Rivoal, Gary Katsevman,
Greg Freedman, Jer Noble, Jimmy Huang, Kevin Babbitt,
Matt Wolenetz, Mounir Lamouri, Nigel Megitt, Peng Liu,
Rossen Atanassov, Simon Fraser, Simon Thompson, Theresa
O'Connor, Vi Nguyen
Regrets
-
Chair
Jer, Mounir
Scribe
Jer, Mounir, Vi
Contents
1. [4]Introductions
2. [5]Video Media Queries
Meeting minutes
Introductions
* Florian Rivoal, Invited Expert
* Daniel Holbert, Mozilla
* Alan Stearns, Adobe
* Matt Wolenetz, MSE spec co-editor, Google
* Jer Noble, Apple
* Greg Freedman, Netflix
* Simon Fraser, Apple
* Rossen Atanassov, Microsoft
* Jimmy Huang, Intel
* Mounir Lamouri, Google
* Kevin Babbitt, Microsoft
* Nigel Megitt, BBC, Chair TTWG
* Simon Thompson, BBC
* Theresa O'Connor, Apple
* Vi Nguyen, Microsoft, Media Capabilities
* Peng Liu, Apple
* Chris Needham, BBC, Media & Entertainment IG co-chair
* Gary Katsevman, Brightcove, Chair TTWG
* Chris Cunningham, Google
Video Media Queries
Related issues:
* [6]Media WG issue #135 — hdrSupported - Screen.video or
simply Screen?
https://github.com/w3c/media-capabilities/issues/135
* [7]CSS WG issue #4471 —
[cssom-1][mediaqueries-5][css-color] Dealing with bi-plane
(video/graphics) when reporting values
[7] https://github.com/w3c/csswg-drafts/issues/4471
florian: introduction of topic: some devices have separate
planes for video and graphics contents
florian: these planes have separate properties including width,
height, and resolution. The existing queries target only the
graphics plane
florian: video sites care mostly about the graphics plane, and
would like api to choose an appropriate variant
florian: originally the CSS WG accepted this proposal, but
found issues and wondered if these were the correct way of
solving this problem
florian: Media Queries need to come with a unit; CSS pixels?
CSS Pixels can mean an angular measurement that changes size
based on the distance from the screen. Physical pixels? anchor
a CSS measurement to a physical measurement.
florian: for the video plane, you may want actual literal
pixels, which would include a third definition.
florian: One possibility would be to make these properties
unitless.
florian: this issue has an analogue for images, and the general
path has been not to do this through Media Queries.
florian: images must also be sensitive to changes in resolution
as windows move between windows, but also to available
bandwidth
florian: for images, the UA will choose the appropriate image
variant based on not only resolution, but other factors.
florian: this may not be the same solution available for video,
but there could be a .js API in which the page provides the
available resolutions and properties, then the UA chooses the
correct variant
florian: were there other arguments pro or con for Media
Queries?
chcunningham: question: does it not solve the problem by
mapping the pixel ratio to a fixed value?
florian: we could make an allowance, but it's not currently
allowed.
florian: what does that then mean when people query for the
video plane in "em" units?
florian: There's not a number that is based on font sizes that
is correct.
chcunningham: if you use the existing mechanism that ratio, you
could discover the ratio.
florian: Media Queries don't allow you to query for the size of
the screen, it only allows you to query for whether the screen
is within a provided value
florian: maybe this is the wrong tool, since you can't query
the value directly.
chcunningham: I don't think that's a dealbreaker, since sites
will have a list of formats or variants with specific
resolutions, so that works with the way Media Queries work
chcunningham: w.r.t. the Picture API; this would be a profound
change from how media APIs are used on the web.
chcunningham: in MSE and WebCodecs, the page has low level
access to variant selection
chcunningham: clients prefer to make these decisions themselves
dholbert: there are also "inch' and "centimeter" units
dholbert: that would be something difficult to define; argument
for unitless value
florian: if we wanted to do this, we don't want to break all of
CSS units, so it would still be a fixed ratio between this
value and a CSS inch
cpn: There was a proposal to expose these values on the screen
object; is there a value to re-visiting that decision?
florian: the query example is only one possibility; we could
also just expose those numbers explicitly.
cpn: a proposal which inverts the current model that gives all
the variant selection to the client would be a big change
wolenetz: is there already a way for JS apps to determine the
resolution where the video will be composited?
florian: I think the Media Queries was proposed as a way to
expose that value.
wolenetz: the current way media works on the web is to expose
to clients underlying capabilities and allow the site to choose
variants.
Simon_T_BBC: set top boxes should be thought about.
Simon_T_BBC: HDMI capabilities must also be taken into account;
some may only be capable of a certain frame rate
Simon_T_BBC: There are some non-screen parameters that are
important for deciding which variant to choose.
florian: you may want to make a different tradeoff than just
available resolution
jernoble: as the person who proposed MediaQueries as a possible
solution. Media Queries already provides event mechanism for
when that ratio changes when moving windows to different
screens. If we did expose something to Screens API, we have to
figure out events
chcunningham: I understand CSS WG is concerned that this is not
the correct solution. But we ran this by the WG before it was
proposed. Are we back to the drawing board, or is this a minor
issue?
florian: I don't think the CSS WG has concluded that this was
impossible; merely uneasy. On further reflection, the way it is
spec'd cannot work. It doesn't work for print media. We would
need to fix the unit issue to make this possible.
florian: we could fix this, but the questions this raises makes
this solution feel "odd".
florian: there's also a desire in the CSS WG that for devices
that do not have a separate video plane, these properties
should be the same for the graphics plane.
florian: screen properties were deprecated, because leaking
non-important information, and those properties have privacy
implications
florian: CSS pixels can and will be used incorrectly
florian: we think this should return to the drawing board.
florian: the current approach is possible, but not ideal.
ving: the Media WG did discuss exposing this to the Screen
object; could we get a summary of why Screen is not the ideal
proposal?
ving: also, there was a concern about interfaces and cables,
but that should be handled through MediaCapabilities API. We're
trying to solve the problem of exposing the exact resolution of
the video plane on the actual display.
chcunningham: where are you seeing these concerns jernoble
listed?
ving: in the notes.
chcunningham: I don't oppose putting this on screen; florian
what do you think of having device pixels on the Screen?
florian: my personal feeling is that's ok; we'd need events if
that changes
florian: is this a concern for those devices that have a
separate video plane?
mounir: I don't think an event is really an issue. The Screen
object doesn't have an event. This is an existing problem in
that the Screen's properties can change without an event.
chcunningham: if we expose device pixels on the screen object,
we're hitting the same reasons why the existing properties are
deprecated
florian: if we expose the video device pixel, you'd just get a
number that is in device pixels. You wouldn't have a way to ask
for the video plane in "em" unit.s
florian: yes, we'd still have the privacy concern.
GregFreedman: if you put device pixels on the Screen object,
you'd still have the problem of the difference between the
video and graphics plane. Would you have device pixels for both
planes?
GregFreedman: I kind of liked where we landed with the Media
Queries. what we want to do is say: can you support this
resolution on the video plane; and that seems like this works
with Media Queries well.
GregFreedman: is the problem that CSS doesn't like the unitless
query?
florian: yes, CSS as a language could support it, but if we're
on a device that does not have a separate plane, we probably
want width: to do the same thing as video-width:
florian: additionally, that definition also affects print media
florian: so what should you do on print media?
<Zakim> nigel, you wanted to ask for a reminder of the use
cases that remain if we use MediaCapabilities to choose which
video to get
nigel: the use case is originally to detect what variant to
display. If MediaCapabilities is the best way to query for
that, what's left in CSS ?
GregFreedman: the device may be capable of doing 4k UHD, but if
the display is only 1080p SD, there's no reason to pick that
variant. so we split it into two APIs: one to query decode, one
to query display.
ving: MediaCapabilies also is not screen aware.
nigel: could Media Queries handle that scenario, where there's
multiple displays?
hober: we already handle that with the 50%+1 to determine which
monitor's properties to return.
chcunningham: that's the implementation in chrome.
cpn: can the UA play this particular content is answered by the
combination of screen and decode capabilities.
cpn: asking these two questions separately doesn't give a
complete picture of whether a variant can be successfully
played
GregFreedman: Media Capabilities (MC) asks "can we decode this"
and Media Queries (MQ) asks "should we decode this".
nigel: is the set of things we need to know just the video
plane resolution, and the frame rate?
nigel: are there other properties that we need to include?
florian: those do not yet exist?
<Zakim> wolenetz, you wanted to say that decoder capabilities
exceeding display capabilities shouldn't limit what is
decodable
nigel: just wondering about design and what info needs to be
included.
wolenetz: Media Capabilities already gives you the answer of
whether you can decode a given piece of media. Do we try to
move some of the properties in Media Capabilities into Media
Queries?
mounir: oginially there was information in the Screen object.
wolenetz: since Media Capabilities is already being asked about
candidates, maybe Media Capabilities is the best place to ask
more questions
wolenetz: since I'm already using Media Capabilities, doing
that again against a different API is difficult.
chcunningham: I have the exact opposite opinion, respectfully.
the Screen API and the Media Queries API already exists.
chcunningham: we never considered putting Media Capabilities
queries into the Screen API
mounir: websites already will react to window size changes,
e.g., when going into fullscreen.
jernoble: we want to answer the question whether the video
should be upscaled or downscaled
jernoble: so maybe we should design an API for that?
smfr: I would be reluctant to add new APIs to the Screen api
due to fingerprinting concerns.
<Zakim> florian, you wanted to suggest that the JS API could
take the frame rate as input more easily than Media Queries
florian: if the frame rate matters, it would be more amenable
to a Screen API, since you can pass in parameters to that API,
but that style of query isn't possible with Media Queries.
chcunningham: no option seems to be the clear winner; IMO the
least of several evils was the CSS Media Queries options.
<nigel> CSS Media Queries don't remove the fingerprinting
vector
chcunningham: I wonder if there is a way to fix this problem
florian: I'd like to ask Simon to clarify this answer about
fingerprinting
smfr: it depends on whether it allows you to enumerate all
screens. and depends on whether the API allows you to detect
whether users have changed their screen resolution or are using
non-standard third party screens
smfr: it's still doable with Media Queries, but you have to
binary search, so it's not as easy.
mounir: what's the next step here? shall we move this to GitHub
issues?
chcunningham: we haven't discussed any options beyond "fix
Media Queries" or "move to Screen".
<astearns> lost my webex connection - would there be a way of
making a video-playback-specific API that would expose less
fingerprinting surface but still be able to answer coarse
questions on what can be displayed? How fine-grained do we
really need to be for playback?
chcunningham: maybe now that we've had this meeting the CSS WG
will come up with an alternate solution that better solves all
the use cases
mounir: lets follow up on GitHub; lets try to discuss between
MediaWG and CSS WG people there
florian: it's unfortunate that there were fewer voices in the
CSS WG than I anticipated; there are conflicting desires in the
CSS WG.
florian: I'm unable to represent all those various POV.
florian: but yes, we can explore; either alternative ways of
speccing the Media Queries or alternative API entirely.
chcunningham: thank you florian! and thank you for representing
the CSS WG.
Received on Monday, 20 July 2020 16:14:00 UTC