- From: CSS Meeting Bot via GitHub <sysbot+gh@w3.org>
- Date: Tue, 27 Jul 2021 15:59:07 +0000
- To: public-css-archive@w3.org
The CSS Working Group just discussed `Pause media playback in iframes hidden by content-visibility:hidden`, and agreed to the following: * `RESOLVED: conceptually play (invisibly and muted) any media playback for things that are skipped due to content-visibility, skipping any events unless queried (as for animations)` <details><summary>The full IRC log of that discussion</summary> <astearns> topic: Pause media playback in iframes hidden by content-visibility:hidden<br> <astearns> github: https://github.com/w3c/csswg-drafts/issues/6468<br> <fantasai> scribenick: fantasai<br> <JakeA> Thanks again for inviting me!<br> <fantasai> chrishtr: Web developer raised use case of wanting to automatically pause videos, media<br> <fantasai> chrishtr: in situation where media happens to be in iframe, page can't really access it, because usually cross-origin<br> <fantasai> chrishtr: atm can only do that by coordinating with each provider of video<br> <fantasai> chrishtr: Perhaps platform can make that possible<br> <fantasai> chrishtr: Suggestion was content-visibility to control that<br> <fantasai> chrishtr: content-visibility is meant as a perf tool, and this is about reducing unnecessary use of CPU<br> <fantasai> chrishtr: so ...<br> <fantasai> chrishtr: so seems that media under content-visibility be automatically paused<br> <fantasai> chrishtr: final point is, as I understand, for web-compat a lot of sites expect video, esp with audio, when not visibible for other reasons<br> <fantasai> chrishtr: e.g. off screen or detached from the DOM<br> <fantasai> chrishtr: But since content-visibility is new, probably not compat worry<br> <fantasai> TabAtkins: I'm surprised at how many things still allow video to play<br> <fantasai> TabAtkins: This seems like a good idea<br> <emilio> q+<br> <emilio> q+<br> <fantasai> astearns: I'm not sure I'd look for content-visibility as a way to make cross-origin video stop playing<br> <fantasai> florian: In that direction not obvious<br> <fantasai> florian: but in other direction, would you expect a video with content-visibility to play?<br> <fantasai> flackr: display:none might have similar effect<br> <fantasai> flackr: might also unload frame...<br> <astearns> ack emilio<br> <fantasai> Rossen_: Display:none is a big hammer<br> <TabAtkins> It does unload the frame<br> <fantasai> emilio: Idea is to also do this for ?? stuff?<br> <fantasai> emilio: ...<br> <astearns> s/??/same-origin/<br> <fantasai> emilio: Is there a ? on if you remove element from the DOM, then it doesn't stop playing, then you add content-visibility and then remove from DOM it's paused, that's weird<br> <fantasai> emilio: Doesn't pausing media change styles?<br> <fantasai> emilio: causes :pause to apply ...<br> <fantasai> emilio: maybe not a huge deal if under subtree, but also weird<br> <fantasai> emilio: overall, I think removing video from DOM should pause<br> <fantasai> emilio: seems weird that content-visibility is the only thing that does this<br> <fantasai> chrishtr: Yes, it would be all media<br> <fantasai> chrishtr: issue says iframes, but would also apply to same-origin video within the document<br> <fantasai> chrishtr: wrt relative timing, seems similar to detaching video after user presses pause button<br> <flackr> q+<br> <Rossen_> q<br> <fantasai> chrishtr: same thing, rendered as visibible under content-visibility<br> <fantasai> chrishtr: thrid point was about how styling changes when it's paused<br> <fantasai> chrishtr: I think yes, those styles would conceptually apply, just like all boxes within content-visibility subtree<br> <fantasai> chrishtr: but UA is allowed to not render within that, for perf reasons<br> <fantasai> emilio: Does content-visibility currently change styles under its sub-tree?<br> <fantasai> emilio: I guess it pauses animations, which is something<br> <fantasai> flackr: It pauses animations in a way that is not very developer-visibible<br> <fantasai> flackr: the timer keeps going<br> <fantasai> flackr: For this video, could maybe not change state to pause, but would not continue playing<br> <fantasai> Rossen_: Is that how we have it specced?<br> <fantasai> Rossen_: content-visibility is currently defined as behaving like 'display: none' on contents<br> <fantasai> Rossen_: [quotes spec]<br> <fantasai> Rossen_: display:none does not exhibit the behavior you just described for animations<br> <fantasai> flackr: That's a misleading statement, interaction should not continue ..<br> <fantasai> astearns: That note is about accessibility, but it's not clear what that note is hooked to<br> <Rossen_> https://drafts.csswg.org/css-contain/#content-visibility<br> <dholbert> https://drafts.csswg.org/css-contain-2/#valdef-content-visibility-hidden<br> <fantasai> emilio: But this isn't what the OP filed the issue about<br> <fantasai> Rossen_: you get to some part of the video, or would you start from the beginning?<br> <fantasai> astearns: The idea is it will pause, and then resume after visible<br> <fantasai> astearns: detaching would restart the video which is not good<br> <fantasai> chrishtr: Pausing video seems simpler to implement<br> <flackr> content-visibility animation pausing was spec'd by https://github.com/w3c/csswg-drafts/commit/82fc7ddb1abf9ce0c3f616cbd91a0f1d2069a215<br> <fantasai> chrishtr: already way to pause videos in UAs<br> <fantasai> chrishtr: Wrt styles, can leave styles uncomputed<br> <fantasai> emilio: Why not for animations?<br> <fantasai> Rossen_: display:none will stop animations entirely<br> <fantasai> Rossen_: but wrt animations, it's different than what I'm hearing about video that's described by ??<br> <fantasai> Rossen_: in case of animations, supposed to pretend they're running, but not actually run and use up CPU<br> <fantasai> Rossen_: when you return, get to more advanced stage<br> <fantasai> Rossen_: video ...<br> <fantasai> Rossen_: saying 2 are the same is not right<br> <flackr> q-<br> <fantasai> Rossen_: If we want this behavior for video or media<br> <fantasai> Rossen_: differentiate clearly so people would know what to expect<br> <fantasai> chrishtr: Leaving video playing but not using CPU to do texture stuff is problem, because still have audio<br> <fantasai> chrishtr: otherwise UA would have to keep audio stream running, but not the video stream<br> <fantasai> Rossen_: you can pause, and then take them forward, by the delta<br> <fantasai> Rossen_: as if the video was playing<br> <emilio> q+<br> <astearns> ack fantasai<br> <smfr> q+<br> <TabAtkins> fantasai: What Rossen said - if you wanted animations and videos to be consistent, you'd need to advacne the timer on it even when it's not playing<br> <TabAtkins> fantasai: I think it would be good to treat them the same<br> <fantasai> astearns: and if you did that, then you would not set the pause style on the video<br> <TabAtkins> astearns: And if you did that, you would *not* set pause on the video<br> <fantasai> fantasai: right<br> <fantasai> flackr: because content-visibility:auto hides things as scrolled out of view<br> <fantasai> flackr: ...<br> <smfr> q-<br> <astearns> ack emilio<br> <fantasai> flackr: this is why we had this behavior for animations<br> <fantasai> emilio: pausing a video not only changes styles, but also triggers JS events<br> <fantasai> emilio: really a problem when style change triggers a bunch of JS handlers<br> <fantasai> emilio: ...<br> <florian> q?<br> <fantasai> chrishtr: for animations, we did decide that although video is conceptually advancing in time, it doesn't fire events<br> <florian> q+<br> <fantasai> chrishtr: since that goes against perf goals<br> <fantasai> chrishtr: could potentially just not fire the events<br> <fantasai> flackr: concern was audio, could also conceptually mute the audio<br> <fantasai> astearns: that would meet the perf goal, but I wonder<br> <Rossen_> q?<br> <fantasai> astearns: I get annoyed when I scroll Twitter just slightly enough, but want to keep hearing it<br> <fantasai> TabAtkins: I have exactly opposite problem with Youtube videos when I scroll away<br> <fantasai> chrishtr: Could conceptually allow the video to continue, but mute it, and get all the perf advantages<br> <fantasai> chrishtr: solve use case of pausing the videos some other way<br> <TabAtkins> s/Youtube videos open in a tweet where they *don't* stop playing/<br> <astearns> ack florian<br> <Rossen_> q+<br> <fantasai> florian: conceptually playing doesn't mean actually playing, can just move the timer and not do work<br> <fantasai> florian: but for use cases where authors do want to pause<br> <fantasai> florian: could send an event<br> <fantasai> florian: and then video can react to that state<br> <fantasai> florian: same thing could apply to animations, things applying to canvas, etc.<br> <fantasai> florian: all things trying to be on the same timer<br> <fantasai> chrishtr: I think you're referring to an event where you can identify when you are skipped?<br> <fantasai> chrishtr: Discussed on and off. There are use cases for it, like avoiding work in scripts<br> <fantasai> chrishtr: A video, for example, inside cross-origin iframe could pause when skipped<br> <fantasai> chrishtr: Similar use case from gmail, cross-origin iframes, would like to be able to stop themselves when they are noticed they are skipped<br> <fantasai> chrishtr: don't want to poll server to ask for more when not visible<br> <astearns> ack Rossen_<br> <fantasai> Rossen_: appears we're drifting off the main task of pausing the video<br> <fantasai> Rossen_: rather than aligning behavior with scrolling and animations<br> <fantasai> Rossen_: user request we have as well, people want to pause video when scrolling away<br> <fantasai> Rossen_: At least the current described behavior for content-visibility:hidden doesn't quite align with that<br> <fantasai> Rossen_: so behavior that advances animations and advances stuff, are we talking about a different behavior here<br> <fantasai> Rossen_: a pause-media? could be helpful<br> <fantasai> Rossen_: allows two behaviors co-exist<br> <fantasai> Rossen_: or have users control one or the other<br> <fantasai> Rossen_: also what is the event model going to look like?<br> <fantasai> Rossen_: are you going to get all the events on the DOM side<br> <fantasai> Rossen_: if trying to synchronize animations with your video<br> <fantasai> Rossen_: one moved ahead and other paused, etc.<br> <fantasai> Rossen_: 2 different behaviors, let's discuss them separately<br> <fantasai> Rossen_: define what pausing looks like and then figure out how they can co-existy<br> <fantasai> chrishtr: I think 2 options are pause the video, or conceptually continue video but muted<br> <florian> ?<br> <fantasai> Rossen_: They weren't asking for that<br> <florian> q+<br> <dholbert> q+<br> <fantasai> chrishtr: that's what person was asking for, however ...<br> <fantasai> astearns: Fits more with existing behavior of content-visibility, to conceptually run it<br> <fantasai> astearns: So should do that<br> <fantasai> astearns: and then discuss use case of pausing<br> <fantasai> chrishtr: Talked to developer<br> <fantasai> chrishtr: didn't want to keep playing video, audio is annoying<br> <fantasai> chrishtr: and also didn't want to take up bandwidth<br> <fantasai> chrishtr: easiest way was to kill it off<br> <fantasai> chrishtr: I do think that perf and muting things won't solve everything but will help<br> <Rossen_> q?<br> <fantasai> emilio: Other question is pausing, does it fire events, does it not fire events<br> <fantasai> emilio: Media has a bunch of crazy things going on<br> <astearns> ack fantasai<br> <fantasai> fantasai: Should we resolve on the timer continuing while it's actually paused?<br> <dholbert> q-<br> <fantasai> astearns: Proposed to treat video similar to animations in content-visibility, is conceptually muted, will return with intervening time skipped<br> <fantasai> astearns: any objections?<br> <fantasai> flackr: don't object, but that's not identical to animations<br> <Rossen_> q+<br> <Rossen_> ack florian<br> <fantasai> flackr: not ticking animations is what user would have seen if animation continued to tick<br> <dholbert> q+<br> <fantasai> flackr: but muting audio is a bit different from ...<br> <smfr> q+<br> <fantasai> flackr: but given that content-visibility is a perf optimization I"m not objecting<br> <fantasai> Rossen_: with this behavior, are there any events?<br> <fantasai> chrishtr: no events will fire, it's not observable<br> <fantasai> chrishtr: querying the element might trigger events or something, kinda like animations<br> <fantasai> Rossen_: So observability will be aligned with that of CSS animations as well<br> <fantasai> chrishtr: yes<br> <dholbert> my question (in case my audio won't work) is: does this mean Twitter couldn't use content-visibility & also match Alan's expectations of letting videos continuing to play even when they're scrolled offscreen?<br> <fantasai> chrishtr: observability to script same, observability to user a bit different because muted<br> <fantasai> astearns: note Twitter right now stops the video when scrolling out of view<br> <fantasai> dholbert: Twitter version that matched Alan's expectation mean can't use content-visibility?<br> <fantasai> dholbert: if want to still have audio play while off-screen<br> <Rossen_> q-<br> <fantasai> astearns: You just don't use content-visibility in that case<br> <fantasai> dholbert: yes, but Twitter might want to use content-visibility ...<br> <fantasai> TabAtkins:...<br> <Rossen_> conten-visibility: hidden pause;<br> <astearns> q?<br> <fantasai> dholbert: I feel like I"ve heard from our media team that we already stop streaming video in background tabs, there's optimizations that can be made there<br> <astearns> zakim, close queue<br> <Zakim> ok, astearns, the speaker queue is closed<br> <fantasai> dholbert: maybe can do that here too<br> <dholbert> q-<br> <astearns> ack dholbert<br> <astearns> ack smfr<br> <fantasai> smfr: websites like youtube and ??<br> <fantasai> smfr: manipulating through script<br> <TabAtkins> s/.../the benefits of c-v are precisely that the browser can stop committing resources to things, rather than keeping videos around multiple times/<br> <fantasai> smfr: if no notification to video that it's being paused, will still run all that stuff<br> <fantasai> smfr: which is a waste<br> <fantasai> smfr: are we resolving for skipped content (also affects content-visibility:auto) or only content-visibility:hidden<br> <fantasai> chrishtr: proposal is to apply to all skipped content<br> <fantasai> smfr: are we trying to avoid detectability of when content becomes skipped or unskipped?<br> <fantasai> chrishtr: the events that determine whether skipped, can also consider<br> <fantasai> chrishtr: raised issue of ? video<br> <fantasai> chrishtr: or youtube video automatically pause themselves when they notice that they are skipped<br> <fantasai> chrishtr: in order to raise the other use case in the discussion<br> <astearns> ack fantasai<br> <Zakim> fantasai, you wanted to respond to flackr<br> <TabAtkins> fantasai: wrt the idea that muting the audio isn't entirely consistent<br> <astearns> zakim, open queue<br> <Zakim> ok, astearns, the speaker queue is open<br> <TabAtkins> fantasai: it kinda is if the audio is part of the visibility of the element<br> <TabAtkins> fantasai: which is true in an audio enviro<br> <TabAtkins> fantasai: so don't think it's entirely inconsistent<br> <TabAtkins> fantasai: not sure if this is the mechanism we want to use to allow videos to be paused<br> <TabAtkins> fantasai: might want to do it more explicitly, rather than building it into this<br> <smfr> +1 for more explicit control. you don’t want Command-F to start videos making noise<br> <TabAtkins> fantasai: bc depending on the use-case you might want it to still play<br> <fantasai> florian: idea reasonable, especially with event that allows you to customize behavior<br> <fantasai> florian: maybe can then combine with API to pause an entire subtree<br> <fantasai> florian: but comment from smfr got me wondering, if you just think about things being skipped or not being off-screen, whole thing we discussed is reasonable<br> <fantasai> florian: but if things are unskipped<br> <fantasai> florian: suppose you search for word 'play', then they all start playing with video<br> <fantasai> florian: not a great user experience<br> <fantasai> florian: can no longer be skipped due to things other than being off-screen, maybe isn't great<br> <fantasai> flackr: if you scrolled something into view, would start playing<br> <fantasai> flackr: but because most search results off-screen<br> <fantasai> flackr: we'd resolve their style to figure out whether they match the search, but they're still off-screen and will continue to be skipped<br> <fantasai> chrishtr: would be able to see the track time, but not playing<br> <fantasai> smfr: what if UA makes all content unskipped when finding in content-visibility:auto<br> <fantasai> chrishtr: spec says that [...]<br> <fantasai> chrishtr: it will only become unskipped when scrolled into view<br> <fantasai> smfr: per-element granularity?<br> <fantasai> chrishtr: every element that becomes unskipped<br> <fantasai> smfr: confused about the granularity of skipping<br> <fantasai> smfr: suppose container with auto<br> <fantasai> smfr: inside it each element becomes skipped or not individually?<br> <fantasai> chrishtr: granularity is content-visibility:auto element<br> <fantasai> astearns: This seems to be more generally, could you read through spec and open an issue if it's a problem?<br> <fantasai> smfr: ok<br> <fantasai> astearns: back to media playback, any other comments before we try for a resolution?<br> <fantasai> [silence]<br> <TabAtkins> (There's text in <https://drafts.csswg.org/css-contain-2/#using-cv-auto> with rough guidance about this exact issue.)<br> <fantasai> RESOLVED: conceptually play (invisibly and muted) any media playback for things that are skipped due to content-visibility, skipping any events unless queried (as for animations)<br> <fantasai> <br duration=10m><br> </details> -- GitHub Notification of comment by css-meeting-bot Please view or discuss this issue at https://github.com/w3c/csswg-drafts/issues/6468#issuecomment-887634200 using your GitHub account -- Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Tuesday, 27 July 2021 15:59:10 UTC