W3C home > Mailing lists > Public > public-css-archive@w3.org > July 2021

Re: [csswg-drafts] [css-contain-2]: Pause media playback in iframes hidden by content-visibility:hidden (#6468)

From: CSS Meeting Bot via GitHub <sysbot+gh@w3.org>
Date: Tue, 27 Jul 2021 15:59:07 +0000
To: public-css-archive@w3.org
Message-ID: <issue_comment.created-887634200-1627401545-sysbot+gh@w3.org>
The CSS Working Group just discussed `Pause media playback in iframes hidden by content-visibility:hidden`, and agreed to the following:

* `RESOLVED: conceptually play (invisibly and muted) any media playback for things that are skipped due to content-visibility, skipping any events unless queried (as for animations)`

<details><summary>The full IRC log of that discussion</summary>
&lt;astearns> topic: Pause media playback in iframes hidden by content-visibility:hidden<br>
&lt;astearns> github: https://github.com/w3c/csswg-drafts/issues/6468<br>
&lt;fantasai> scribenick: fantasai<br>
&lt;JakeA> Thanks again for inviting me!<br>
&lt;fantasai> chrishtr: Web developer raised use case of wanting to automatically pause videos, media<br>
&lt;fantasai> chrishtr: in situation where media happens to be in iframe, page can't really access it, because usually cross-origin<br>
&lt;fantasai> chrishtr: atm can only do that by coordinating with each provider of video<br>
&lt;fantasai> chrishtr: Perhaps platform can make that possible<br>
&lt;fantasai> chrishtr: Suggestion was content-visibility to control that<br>
&lt;fantasai> chrishtr: content-visibility is meant as a perf tool, and this is about reducing unnecessary use of CPU<br>
&lt;fantasai> chrishtr: so ...<br>
&lt;fantasai> chrishtr: so seems that media under content-visibility be automatically paused<br>
&lt;fantasai> chrishtr: final point is, as I understand, for web-compat a lot of sites expect video, esp with audio, when not visibible for other reasons<br>
&lt;fantasai> chrishtr: e.g. off screen or detached from the DOM<br>
&lt;fantasai> chrishtr: But since content-visibility is new, probably not compat worry<br>
&lt;fantasai> TabAtkins: I'm surprised at how many things still allow video to play<br>
&lt;fantasai> TabAtkins: This seems like a good idea<br>
&lt;emilio> q+<br>
&lt;emilio> q+<br>
&lt;fantasai> astearns: I'm not sure I'd look for content-visibility as a way to make cross-origin video stop playing<br>
&lt;fantasai> florian: In that direction not obvious<br>
&lt;fantasai> florian: but in other direction, would you expect a video with content-visibility to play?<br>
&lt;fantasai> flackr: display:none might have similar effect<br>
&lt;fantasai> flackr: might also unload frame...<br>
&lt;astearns> ack emilio<br>
&lt;fantasai> Rossen_: Display:none is a big hammer<br>
&lt;TabAtkins> It does unload the frame<br>
&lt;fantasai> emilio: Idea is to also do this for ??  stuff?<br>
&lt;fantasai> emilio: ...<br>
&lt;astearns> s/??/same-origin/<br>
&lt;fantasai> emilio: Is there a ? on if you remove element from the DOM, then it doesn't stop playing, then you add content-visibility and then remove from DOM it's paused, that's weird<br>
&lt;fantasai> emilio: Doesn't pausing media change styles?<br>
&lt;fantasai> emilio: causes :pause to apply ...<br>
&lt;fantasai> emilio: maybe not a huge deal if under subtree, but also weird<br>
&lt;fantasai> emilio: overall, I think removing video from DOM should pause<br>
&lt;fantasai> emilio: seems weird that content-visibility is the only thing that does this<br>
&lt;fantasai> chrishtr: Yes, it would be all media<br>
&lt;fantasai> chrishtr: issue says iframes, but would also apply to same-origin video within the document<br>
&lt;fantasai> chrishtr: wrt relative timing, seems similar to detaching video after user presses pause button<br>
&lt;flackr> q+<br>
&lt;Rossen_> q<br>
&lt;fantasai> chrishtr: same thing, rendered as visibible under content-visibility<br>
&lt;fantasai> chrishtr: thrid point was about how styling changes when it's paused<br>
&lt;fantasai> chrishtr: I think yes, those styles would conceptually apply, just like all boxes within content-visibility subtree<br>
&lt;fantasai> chrishtr: but UA is allowed to not render within that, for perf reasons<br>
&lt;fantasai> emilio: Does content-visibility currently change styles under its sub-tree?<br>
&lt;fantasai> emilio: I guess it pauses animations, which is something<br>
&lt;fantasai> flackr: It pauses animations in a way that is not very developer-visibible<br>
&lt;fantasai> flackr: the timer keeps going<br>
&lt;fantasai> flackr: For this video, could maybe not change state to pause, but would not continue playing<br>
&lt;fantasai> Rossen_: Is that how we have it specced?<br>
&lt;fantasai> Rossen_: content-visibility is currently defined as behaving like 'display: none' on contents<br>
&lt;fantasai> Rossen_: [quotes spec]<br>
&lt;fantasai> Rossen_: display:none does not exhibit the behavior you just described for animations<br>
&lt;fantasai> flackr: That's a misleading statement, interaction should not continue ..<br>
&lt;fantasai> astearns: That note is about accessibility, but it's not clear what that note is hooked to<br>
&lt;Rossen_> https://drafts.csswg.org/css-contain/#content-visibility<br>
&lt;dholbert> https://drafts.csswg.org/css-contain-2/#valdef-content-visibility-hidden<br>
&lt;fantasai> emilio: But this isn't what the OP filed the issue about<br>
&lt;fantasai> Rossen_: you get to some part of the video, or would you start from the beginning?<br>
&lt;fantasai> astearns: The idea is it will pause, and then resume after visible<br>
&lt;fantasai> astearns: detaching would restart the video which is not good<br>
&lt;fantasai> chrishtr: Pausing video seems simpler to implement<br>
&lt;flackr> content-visibility animation pausing was spec'd by https://github.com/w3c/csswg-drafts/commit/82fc7ddb1abf9ce0c3f616cbd91a0f1d2069a215<br>
&lt;fantasai> chrishtr: already way to pause videos in UAs<br>
&lt;fantasai> chrishtr: Wrt styles, can leave styles uncomputed<br>
&lt;fantasai> emilio: Why not for animations?<br>
&lt;fantasai> Rossen_: display:none will stop animations entirely<br>
&lt;fantasai> Rossen_: but wrt animations, it's different than what I'm hearing about video that's described by ??<br>
&lt;fantasai> Rossen_: in case of animations, supposed to pretend they're running, but not actually run and use up CPU<br>
&lt;fantasai> Rossen_: when you return, get to more advanced stage<br>
&lt;fantasai> Rossen_: video ...<br>
&lt;fantasai> Rossen_: saying 2 are the same is not right<br>
&lt;flackr> q-<br>
&lt;fantasai> Rossen_: If we want this behavior for video or media<br>
&lt;fantasai> Rossen_: differentiate clearly so people would know what to expect<br>
&lt;fantasai> chrishtr: Leaving video playing but not using CPU to do texture stuff is problem, because still have audio<br>
&lt;fantasai> chrishtr: otherwise UA would have to keep audio stream running, but not the video stream<br>
&lt;fantasai> Rossen_: you can pause, and then take them forward, by the delta<br>
&lt;fantasai> Rossen_: as if the video was playing<br>
&lt;emilio> q+<br>
&lt;astearns> ack fantasai<br>
&lt;smfr> q+<br>
&lt;TabAtkins> fantasai: What Rossen said - if you wanted animations and videos to be consistent, you'd need to advacne the timer on it even when it's not playing<br>
&lt;TabAtkins> fantasai: I think it would be good to treat them the same<br>
&lt;fantasai> astearns: and if you did that, then you would not set the pause style on the video<br>
&lt;TabAtkins> astearns: And if you did that, you would *not* set pause on the video<br>
&lt;fantasai> fantasai: right<br>
&lt;fantasai> flackr: because content-visibility:auto hides things as scrolled out of view<br>
&lt;fantasai> flackr: ...<br>
&lt;smfr> q-<br>
&lt;astearns> ack emilio<br>
&lt;fantasai> flackr: this is why we had this behavior for animations<br>
&lt;fantasai> emilio: pausing a video not only changes styles, but also triggers JS events<br>
&lt;fantasai> emilio: really a problem when style change triggers a bunch of JS handlers<br>
&lt;fantasai> emilio: ...<br>
&lt;florian> q?<br>
&lt;fantasai> chrishtr: for animations, we did decide that although video is conceptually advancing in time, it doesn't fire events<br>
&lt;florian> q+<br>
&lt;fantasai> chrishtr: since that goes against perf goals<br>
&lt;fantasai> chrishtr: could potentially just not fire the events<br>
&lt;fantasai> flackr: concern was audio, could also conceptually mute the audio<br>
&lt;fantasai> astearns: that would meet the perf goal, but I wonder<br>
&lt;Rossen_> q?<br>
&lt;fantasai> astearns: I get annoyed when I scroll Twitter just slightly enough, but want to keep hearing it<br>
&lt;fantasai> TabAtkins: I have exactly opposite problem with Youtube videos when I scroll away<br>
&lt;fantasai> chrishtr: Could conceptually allow the video to continue, but mute it, and get all the perf advantages<br>
&lt;fantasai> chrishtr: solve use case of pausing the videos some other way<br>
&lt;TabAtkins> s/Youtube videos open in a tweet where they *don't* stop playing/<br>
&lt;astearns> ack florian<br>
&lt;Rossen_> q+<br>
&lt;fantasai> florian: conceptually playing doesn't mean actually playing, can just move the timer and not do work<br>
&lt;fantasai> florian: but for use cases where authors do want to pause<br>
&lt;fantasai> florian: could send an event<br>
&lt;fantasai> florian: and then video can react to that state<br>
&lt;fantasai> florian: same thing could apply to animations, things applying to canvas, etc.<br>
&lt;fantasai> florian: all things trying to be on the same timer<br>
&lt;fantasai> chrishtr: I think you're referring to an event where you can identify when you are skipped?<br>
&lt;fantasai> chrishtr: Discussed on and off. There are use cases for it, like avoiding work in scripts<br>
&lt;fantasai> chrishtr: A video, for example, inside cross-origin iframe could pause when skipped<br>
&lt;fantasai> chrishtr: Similar use case from gmail, cross-origin iframes, would like to be able to stop themselves when they are noticed they are skipped<br>
&lt;fantasai> chrishtr: don't want to poll server to ask for more when not visible<br>
&lt;astearns> ack Rossen_<br>
&lt;fantasai> Rossen_: appears we're drifting off the main task of pausing the video<br>
&lt;fantasai> Rossen_: rather than aligning behavior with scrolling and animations<br>
&lt;fantasai> Rossen_: user request we have as well, people want to pause video when scrolling away<br>
&lt;fantasai> Rossen_: At least the current described behavior for content-visibility:hidden doesn't quite align with that<br>
&lt;fantasai> Rossen_: so behavior that advances animations and advances stuff, are we talking about a different behavior here<br>
&lt;fantasai> Rossen_: a pause-media? could be helpful<br>
&lt;fantasai> Rossen_: allows two behaviors co-exist<br>
&lt;fantasai> Rossen_: or have users control one or the other<br>
&lt;fantasai> Rossen_: also what is the event model going to look like?<br>
&lt;fantasai> Rossen_: are you going to get all the events on the DOM side<br>
&lt;fantasai> Rossen_: if trying to synchronize animations with your video<br>
&lt;fantasai> Rossen_: one moved ahead and other paused, etc.<br>
&lt;fantasai> Rossen_: 2 different behaviors, let's discuss them separately<br>
&lt;fantasai> Rossen_: define what pausing looks like and then figure out how they can co-existy<br>
&lt;fantasai> chrishtr: I think 2 options are pause the video, or conceptually continue video but muted<br>
&lt;florian> ?<br>
&lt;fantasai> Rossen_: They weren't asking for that<br>
&lt;florian> q+<br>
&lt;dholbert> q+<br>
&lt;fantasai> chrishtr: that's what person was asking for, however ...<br>
&lt;fantasai> astearns: Fits more with existing behavior of content-visibility, to conceptually run it<br>
&lt;fantasai> astearns: So should do that<br>
&lt;fantasai> astearns: and then discuss use case of pausing<br>
&lt;fantasai> chrishtr: Talked to developer<br>
&lt;fantasai> chrishtr: didn't want to keep playing video, audio is annoying<br>
&lt;fantasai> chrishtr: and also didn't want to take up bandwidth<br>
&lt;fantasai> chrishtr: easiest way was to kill it off<br>
&lt;fantasai> chrishtr: I do think that perf and muting things won't solve everything but will help<br>
&lt;Rossen_> q?<br>
&lt;fantasai> emilio: Other question is pausing, does it fire events, does it not fire events<br>
&lt;fantasai> emilio: Media has a bunch of crazy things going on<br>
&lt;astearns> ack fantasai<br>
&lt;fantasai> fantasai: Should we resolve on the timer continuing while it's actually paused?<br>
&lt;dholbert> q-<br>
&lt;fantasai> astearns: Proposed to treat video similar to animations in content-visibility, is conceptually muted, will return with intervening time skipped<br>
&lt;fantasai> astearns: any objections?<br>
&lt;fantasai> flackr: don't object, but that's not identical to animations<br>
&lt;Rossen_> q+<br>
&lt;Rossen_> ack florian<br>
&lt;fantasai> flackr: not ticking animations is what user would have seen if animation continued to tick<br>
&lt;dholbert> q+<br>
&lt;fantasai> flackr: but muting audio is a bit different from ...<br>
&lt;smfr> q+<br>
&lt;fantasai> flackr: but given that content-visibility is a perf optimization I"m not objecting<br>
&lt;fantasai> Rossen_: with this behavior, are there any events?<br>
&lt;fantasai> chrishtr: no events will fire, it's not observable<br>
&lt;fantasai> chrishtr: querying the element might trigger events or something, kinda like animations<br>
&lt;fantasai> Rossen_: So observability will be aligned with that of CSS animations as well<br>
&lt;fantasai> chrishtr: yes<br>
&lt;dholbert> my question (in case my audio won't work) is: does this mean Twitter couldn't use content-visibility &amp; also match Alan's expectations of letting videos continuing to play even when they're scrolled offscreen?<br>
&lt;fantasai> chrishtr: observability to script same, observability to user a bit different because muted<br>
&lt;fantasai> astearns: note Twitter right now stops the video when scrolling out of view<br>
&lt;fantasai> dholbert: Twitter version that matched Alan's expectation mean can't use content-visibility?<br>
&lt;fantasai> dholbert: if want to still have audio play while off-screen<br>
&lt;Rossen_> q-<br>
&lt;fantasai> astearns: You just don't use content-visibility in that case<br>
&lt;fantasai> dholbert: yes, but Twitter might want to use content-visibility ...<br>
&lt;fantasai> TabAtkins:...<br>
&lt;Rossen_> conten-visibility: hidden pause;<br>
&lt;astearns> q?<br>
&lt;fantasai> dholbert: I feel like I"ve heard from our media team that we already stop streaming video in background tabs, there's optimizations that can be made there<br>
&lt;astearns> zakim, close queue<br>
&lt;Zakim> ok, astearns, the speaker queue is closed<br>
&lt;fantasai> dholbert: maybe can do that here too<br>
&lt;dholbert> q-<br>
&lt;astearns> ack dholbert<br>
&lt;astearns> ack smfr<br>
&lt;fantasai> smfr: websites like youtube and ??<br>
&lt;fantasai> smfr: manipulating through script<br>
&lt;TabAtkins> s/.../the benefits of c-v are precisely that the browser can stop committing resources to things, rather than keeping videos around multiple times/<br>
&lt;fantasai> smfr: if no notification to video that it's being paused, will still run all that stuff<br>
&lt;fantasai> smfr: which is a waste<br>
&lt;fantasai> smfr: are we resolving for skipped content (also affects content-visibility:auto) or only content-visibility:hidden<br>
&lt;fantasai> chrishtr: proposal is to apply to all skipped content<br>
&lt;fantasai> smfr: are we trying to avoid detectability of when content becomes skipped or unskipped?<br>
&lt;fantasai> chrishtr: the events that determine whether skipped, can also consider<br>
&lt;fantasai> chrishtr: raised issue of ? video<br>
&lt;fantasai> chrishtr: or youtube video automatically pause themselves when they notice that they are skipped<br>
&lt;fantasai> chrishtr: in order to raise the other use case in the discussion<br>
&lt;astearns> ack fantasai<br>
&lt;Zakim> fantasai, you wanted to respond to flackr<br>
&lt;TabAtkins> fantasai: wrt the idea that muting the audio isn't entirely consistent<br>
&lt;astearns> zakim, open queue<br>
&lt;Zakim> ok, astearns, the speaker queue is open<br>
&lt;TabAtkins> fantasai: it kinda is if the audio is part of the visibility of the element<br>
&lt;TabAtkins> fantasai: which is true in an audio enviro<br>
&lt;TabAtkins> fantasai: so don't think it's entirely inconsistent<br>
&lt;TabAtkins> fantasai: not sure if this is the mechanism we want to use to allow videos to be paused<br>
&lt;TabAtkins> fantasai: might want to do it more explicitly, rather than building it into this<br>
&lt;smfr> +1 for more explicit control. you don’t want Command-F to start videos making noise<br>
&lt;TabAtkins> fantasai: bc depending on the use-case you might want it to still play<br>
&lt;fantasai> florian: idea reasonable, especially with event that allows you to customize behavior<br>
&lt;fantasai> florian: maybe can then combine with API to pause an entire subtree<br>
&lt;fantasai> florian: but comment from smfr got me wondering, if you just think about things being skipped or not being off-screen, whole thing we discussed is reasonable<br>
&lt;fantasai> florian: but if things are unskipped<br>
&lt;fantasai> florian: suppose you search for word 'play', then they all start playing with video<br>
&lt;fantasai> florian: not a great user experience<br>
&lt;fantasai> florian: can no longer be skipped due to things other than being off-screen, maybe isn't great<br>
&lt;fantasai> flackr: if you scrolled something into view, would start playing<br>
&lt;fantasai> flackr: but because most search results off-screen<br>
&lt;fantasai> flackr: we'd resolve their style to figure out whether they match the search, but they're still off-screen and will continue to be skipped<br>
&lt;fantasai> chrishtr: would be able to see the track time, but not playing<br>
&lt;fantasai> smfr: what if UA makes all content unskipped when finding in content-visibility:auto<br>
&lt;fantasai> chrishtr: spec says that [...]<br>
&lt;fantasai> chrishtr: it will only become unskipped when scrolled into view<br>
&lt;fantasai> smfr: per-element granularity?<br>
&lt;fantasai> chrishtr: every element that becomes unskipped<br>
&lt;fantasai> smfr:  confused about the granularity of skipping<br>
&lt;fantasai> smfr: suppose container with auto<br>
&lt;fantasai> smfr: inside it each element becomes skipped or not individually?<br>
&lt;fantasai> chrishtr: granularity is content-visibility:auto element<br>
&lt;fantasai> astearns: This seems to be more generally, could you read through spec and open an issue if it's a problem?<br>
&lt;fantasai> smfr: ok<br>
&lt;fantasai> astearns: back to media playback, any other comments before we try for a resolution?<br>
&lt;fantasai> [silence]<br>
&lt;TabAtkins> (There's text in &lt;https://drafts.csswg.org/css-contain-2/#using-cv-auto> with rough guidance about this exact issue.)<br>
&lt;fantasai> RESOLVED: conceptually play (invisibly and muted) any media playback for things that are skipped due to content-visibility, skipping any events unless queried (as for animations)<br>
&lt;fantasai> &lt;br duration=10m><br>
</details>


-- 
GitHub Notification of comment by css-meeting-bot
Please view or discuss this issue at https://github.com/w3c/csswg-drafts/issues/6468#issuecomment-887634200 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Tuesday, 27 July 2021 15:59:10 UTC

This archive was generated by hypermail 2.4.0 : Tuesday, 19 October 2021 01:31:47 UTC