Re: [csswg-drafts] Proposal: shared element transitions (#6464)

<details>
<summary>Minutes of 2021-09-02 discussion</summary>
<pre>

07:07 <TabAtkins> khush: At previous session we introduct SET
07:07 <TabAtkins> khush: Goal of getting seamless transitions like native apps
07:08 <vmpstr> https://docs.google.com/document/d/1r22ITZhL--X_KiPodv5Ju5hWaaoQSXO2xajvW7N33bQ/edit#heading=h.cl4cs2jqxgff
07:08 <TabAtkins> khush: Now we're discussing design problems
07:08 <vmpstr> present+
07:08 <TabAtkins> khush: Hope this doc gives us starting points for discussion
07:09 <TabAtkins> khush: basic model here is that you cache some state from the old dom, and when th enew page loads you animate from that cache. "cache" is a pixel snapshot and a little metadata about sizing/positioning
07:10 <TabAtkins> khush: So is this the right idea? we want it to work for same-document transitions also
07:10 <TabAtkins> Rossen_: So what's your dfn of pixel snapshot?
07:10 <emilio> q+
07:10 — Zakim sees emilio on the speaker queue
07:11 <TabAtkins> khush: We keep a copy of the texture the subtree was rastered into
07:11 <TabAtkins> JakeA: so at this point the old dom is gone - if this is cross-page we've already done 'unload' on the previous document
07:11 <TabAtkins> JakeA: it's "donated" parts of itself for the transition
07:11 <TabAtkins> JakeA: Same for same-page
07:11 <TabAtkins> JakeA: Can't lend the DOM itself across documents, that's a hard limit, so what we've landed on so far is that it's just a pixel repr
07:12 <TabAtkins> Rossen_: I can see how that can work in simple scenarios... is this pre-blend, post-blend...?
07:12 <TabAtkins> Rossen_: what exactly do you mean by "save the texture"?
07:12 <TabAtkins> khush: exactly what we need to define here
07:12 <TabAtkins> khush: "stacking context output" is probably the closest concept we have
07:12 <TabAtkins> khush: Before you apply post-effects - filter or ancestor opacity, etc
07:13 <fantasai> Shared Element Transitions Doc (archived): https://lists.w3.org/Archives/Public/www-archive/2021Sep/att-0001/shared-element-transitions-concepts.pdf
07:13 <Rossen_> ack emilio
07:13 — Zakim sees no one on the speaker queue
07:13 <TabAtkins> khush: if you've just done a transition to an element with changes of filter/opacity, we try to animate those nicely
07:13 <TabAtkins> emilio: How does this work with window resizes?
07:13 <TabAtkins> JakeA: window resizes during transition?
07:13 <TabAtkins> emilio: Any time after you've captured the pixels
07:14 <TabAtkins> JakeA: so we take the pixel snapshots at a phase, then some time after that the resize happens
07:14 <TabAtkins> JakeA: My assumption so far is that we'd abandon the transition
07:14 <TabAtkins> emilio: That's an option
07:14 <TabAtkins> khush: It becomes so difficult - the layout can change, but we've captured the layout in the previous layout
07:14 <Rossen_> q?
07:14 — Zakim sees no one on the speaker queue
07:14 <TabAtkins> JakeA: To be clear "abandoned" just means skip to the end, not leave the page in a broken state
07:14 <TabAtkins> flackr: Layout can change due to non-resize events too, tho, right?
07:15 <TabAtkins> khush: Right, but a resize isn't under the page's control, that's author. Other things, the page can control whether layout changes or not.
07:16 <TabAtkins> Rossen_: The new page will be a new layout anyway, this should be a base case to be successful
07:16 <TabAtkins> khush: Going from one layout to another is the base thing we're trying to transition to
07:16 <TabAtkins> khush: Issue is that you figure out what your animation will be based on the final layout. If the user then changes things while the transition is going, what do you do?
07:17 <TabAtkins> flackr: Treating as a transition, you could retarget what the transition is going to if things shift
07:17 <TabAtkins> khush: That's actually what the current impl does, to try to make things happen smoothly
07:17 <TabAtkins> khush: If something shifts quite a lot right near the end, it'll look abrupt, but it'll look abrupt no matter what
07:18 <TabAtkins> JakeA: Related open question - to what degree do we freeze the incoming page during the transition? If we do, this problem goes away.
07:18 <TabAtkins> JakeA: Another benefit is that during a page load things shift around, it tends to cause jank
07:18 <TabAtkins> JakeA: Downside is that you might get a big shift at the end of the transition when everything moves to its final state
07:18 <TabAtkins> JakeA: And a jump for animations or gifs
07:19 <TabAtkins> Rossen_: Want to make sure we stay on the first topic about pixel snapshots
07:19 <TabAtkins> Rossen_: my understandign now is that you ahve the render list, post-layout construct, from the previous doc
07:19 <TabAtkins> Rossen_: i'm assuming this is pre-blending/etc, because you don't have the other context
07:19 <TabAtkins> Rossen_: So you're prone to "popping" in terms of color, etc
07:20 <TabAtkins> Rossen_: What's the handling there? Easing, don't worry about it, leave it popping by default, etc?
07:20 <TabAtkins> khush: Let me rephrase to understand
07:20 <TabAtkins> khush: You have a dom element being blended into its background. when the transition starts it'll blend into a different background. how do we make sure it's not distracting?
07:21 <TabAtkins> Rossen_: yes. I also think it should be a base req that you're not bleeding pixels from unrealted elements
07:21 <chrishtr> q+
07:21 — Zakim sees chrishtr on the speaker queue
07:21 <TabAtkins> khush: You're right that when we capture, it's just the content of the subtree, before it's blended
07:22 <TabAtkins> khush: And for security reasons, you can't access any of the captured pixels
07:22 <TabAtkins> emilio: I assume the trnasition happens in the compositor - the pixels shouldn't end up in the content process at all
07:22 <TabAtkins> Rossen_: Unless the element lands in a canvas context somehow
07:23 <TabAtkins> emilio: How would that happen?
07:23 <Rossen_> q?
07:23 — Zakim sees chrishtr on the speaker queue
07:23 <TabAtkins> khush: from an impl perspective, we're treating these with the same security persepctive as we do with iframes
07:23 <TabAtkins> emilio: with Firefox that happens in the GPU process or the parent process
07:23 <emilio> q+
07:23 — Zakim sees chrishtr, emilio on the speaker queue
07:24 <TabAtkins> Rossen_: So the intended behavior is to have "color popping"?
07:24 <TabAtkins> khush: what do you mean?
07:24 <TabAtkins> flackr: If you have opacity on a parent you'll see stuff behind it, but as soon as the transition starts you'll lose that blended background
07:24 <TabAtkins> khush: Isn't that what you'd always get?
07:25 <TabAtkins> flackr: Imagine you go from black background to white, and your element is gray and partially transparent
07:25 <TabAtkins> flackr: It'll look like it immediately switches
07:25 <vmpstr> q?
07:25 — Zakim sees chrishtr, emilio on the speaker queue
07:25 <vollick> q+
07:25 — Zakim sees chrishtr, emilio, vollick on the speaker queue
07:25 <TabAtkins> JakeA: There will indeed be some "popping", like if an element is partially clipped by a parent - it'll become unclipped because its parent no longer exists
07:26 <TabAtkins> Rossen_: Question isn't just with the background, but with things on top too, like if you have a screening element
07:26 <khush> q+
07:26 — Zakim sees chrishtr, emilio, vollick, khush on the speaker queue
07:26 <TabAtkins> Rossen_: In the transition it'll pop because that screener isn't there anymore
07:26 <TabAtkins> flackr: Yes, that's intentional
07:26 <TabAtkins> Rossen_: So that's popping by design?
07:26 <TabAtkins> [nods]
07:26 <Rossen_> q?
07:26 — Zakim sees chrishtr, emilio, vollick, khush on the speaker queue
07:26 <TabAtkins> Rossen_: Okay as long as that's explicit
07:26 <Rossen_> ack chrishtr
07:26 — Zakim sees emilio, vollick, khush on the speaker queue
07:27 — TabAtkins you can qq+ to jump yourself to the front of the queue
07:27 <TabAtkins> chrishtr: Also clips of ancestors don't apply for the same reasons
07:27 <lea> present+
07:27 <TabAtkins> chrishtr: It seems that transforms do need to apply
07:27 <TabAtkins> chrishtr: To keep the element in the right position
07:27 <TabAtkins> chrishtr: So you dont' want to apply some visual changes but not others
07:27 <TabAtkins> chrishtr: So this does have some wrinkles with other features
07:27 <Rossen_> q+
07:27 — Zakim sees emilio, vollick, khush, Rossen_ on the speaker queue
07:28 <TabAtkins> emilio: I was gonna ask about effects on parents and how they're handled, like clips
07:28 <Rossen_> ack emilio
07:28 — Zakim sees vollick, khush, Rossen_ on the speaker queue
07:28 <TabAtkins> emilio: You mentioend stacking contexts - is that sufficient to say "here's this element as a texture"?
07:28 <TabAtkins> emilio: Stuff can skip stackign contexts, like positioned children
07:28 <TabAtkins> chrishtr: The element will require something that contains its children. Right now we use contain:paint as the req
07:28 <TabAtkins> emilio: that's good
07:29 <vmpstr> 'sharing the element' is also a compositing trigger for us
07:29 <TabAtkins> emilio: It's not just contain:paint, you need info in the compositor - do you forcefully layerize contain:pain elements?
07:29 <TabAtkins> chrishtr: Not generally, but when you signal you want to transition an element, it signals the compositor to layerize and save off the texture.
07:29 <TabAtkins> emilio: So you need an API call on the page you leave as well as the page you go to
07:29 <TabAtkins> chrishtr: yes
07:30 <TabAtkins> vollick: Follow-on to Rossen's popping question
07:30 <TabAtkins> vollick: If I understand, those things that aren't included in the capture that would change, in a traditional case we coudl smoothly transition those
07:30 <TabAtkins> vollick: Wouldn't it be possible to have a smooth transition in this API as well if we capture that data?
07:30 <TabAtkins> vollick: Is this impossible?
07:31 <TabAtkins> khush: That's what was going thru my mind - I assumed you could set up the background so that you'll have a smooth transition
07:31 <TabAtkins> flackr: It's not just background - it's everything behind *and* in front of the element
07:31 <chrishtr> q+
07:31 — Zakim sees vollick, khush, Rossen_, chrishtr on the speaker queue
07:31 <Rossen_> ack vollick
07:31 — Zakim sees khush, Rossen_, chrishtr on the speaker queue
07:31 <vollick> q-
07:31 — Zakim sees khush, Rossen_, chrishtr on the speaker queue
07:32 <TabAtkins> khush: Occlusion is important, es. Keeping paint order of something on top is hard. Hoping that these are edge cases devs wont' run into often, so we can just do the simple thing.
07:32 <TabAtkins> ack khush 
07:32 — Zakim sees Rossen_, chrishtr on the speaker queue
07:33 <TabAtkins> Rossen_: Discussion so far was about one element being transitioned. When you have a collection beign transitioned, what's the thinking there
07:33 <TabAtkins> Rossen_: Same limitations to all individually?
07:33 <TabAtkins> Rossen_: Blend and clip between them?
07:33 <TabAtkins> Rossen_: Say I've got three elements stacked on each other, not ancestors, that blend into each other's pixels, and I transition all of them
07:34 <Rossen_> ack Rossen
07:34 — Zakim sees chrishtr on the speaker queue
07:34 <TabAtkins> khush: So we capture independent snapshots of them
07:34 <TabAtkins> khush: The initial scene blends as you expect
07:34 <TabAtkins> khush: We ahve a "paired transition" concept
07:34 <TabAtkins> khush: Where you're going from A on the old to B on the new
07:34 <TabAtkins> khush: But in the new page the endpoints might be stacked differently
07:35 <TabAtkins> khush: It's not clear to me how these things should transition
07:35 <TabAtkins> khush: I was hoping for these questions to lean on the expertise of those here, I'm not sure
07:35 <TabAtkins> Rossen_: Okay, let's move on for now if we don't have ideas set yet
07:35 <TabAtkins> chrishtr: To sizing, if a resize occurs, this might hcange the size and position of the endpoint element
07:36 <TabAtkins> chrishtr: I think we should just adjust the curve the same way we do with CSS animations today
07:36 <TabAtkins> chrishtr: In addition, there's a global page transition that can occur as well, like a crossfade
07:36 <Rossen_> q?
07:36 — Zakim sees chrishtr on the speaker queue
07:36 <Rossen_> ack chrishtr
07:36 — Zakim sees no one on the speaker queue
07:36 <TabAtkins> chrishtr: That doesn't depend on layout, so even if the SET adjusts or aborts, the other transition still runs
07:38 <TabAtkins> khush: One takeaway question - is using snapshots of elements the right primitive to build on?
07:39 <TabAtkins> Rossen_: A followup for this case - in a resize transition, do you just rasterize that texture and then scale to the target size?
07:40 <TabAtkins> chrishtr: Yes, but similar to transform animation that scales - we don't specify precisely what's up with the pixels
07:40 <TabAtkins> chrishtr: We won't rerender text tho, just do some scaling on the gpu
07:40 <TabAtkins> khush: What you mentioned might be good to bring up now
07:40 <TabAtkins> khush: Seen some cases where you transition between rounded corners
07:41 <TabAtkins> khush: Weren't sure if we should rasterize those corners into the texture
07:41 <JakeA> q?
07:41 — Zakim sees no one on the speaker queue
07:41 <fantasai> TabAtkins: Like if it has 5px rounded corner, but end-point is twice as wide?
07:41 <JakeA> q+
07:41 — Zakim sees JakeA on the speaker queue
07:41 <vmpstr> q+
07:41 — Zakim sees JakeA, vmpstr on the speaker queue
07:41 <TabAtkins> flackr: I think it would be nice to rasterize without the border clip and apply that post, like we would with filters
07:42 <TabAtkins> JakeA: I think that would be hard with rounded corners, those have a compound effect on borders and shadows
07:42 <TabAtkins> JakeA: Model I've been coming to is when we write to a texture, the dev can opt into properties that are excluded from the flattening, and become properties that can be animated in a CSS way
07:42 <Rossen_> ack JakeA
07:42 — Zakim sees vmpstr on the speaker queue
07:42 <TabAtkins> JakeA: We'd limit these to ones we agree can be done on a texture, like transform, opacity, filter
07:43 <TabAtkins> JakeA: If devs want to animate border-radius so they work at differetn aspect ratios, we do have clip-path; not the same, but can function similarly sometimes
07:43 <TabAtkins> JakeA: So question is if that's enough
07:43 <TabAtkins> flackr: Border and border-radius are also things you can apply on top of image-like content, right?
07:43 <TabAtkins> JakeA: It's also a layout, since border size changes content size
07:43 <TabAtkins> chrishtr: I think all the props that apply to self should still apply
07:44 <TabAtkins> chrishtr: Like an opacity on the element itself should apply
07:44 <TabAtkins> chrishtr: So if the shared element had "opacity:.5", the texture shoudl be partially transparent
07:44 <TabAtkins> Rossen_: My understanding, yes. It'll just blend against something different
07:45 <TabAtkins> chrishtr: Right, blend mode or backgrounf-filter implicitly take the background as input - they just don't work in a SET
07:45 <TabAtkins> JakeA: My issue is that right now no browser I know of can animate clip-path on teh compositor
07:45 <TabAtkins> JakeA: So we either make a set list and freeze it, or we make it opt-in and potentiallye xtensible
07:46 <Rossen_> q?
07:46 — Zakim sees vmpstr on the speaker queue
07:46 <TabAtkins> JakeA: [example of starting element using opacity, end elemetn using transparent color, and wanting these to act identically]
07:46 <Rossen_> ack vmpstr
07:46 — Zakim sees no one on the speaker queue
07:46 <TabAtkins> vmpstr: For the contents of the element we do want to cross-fade
07:46 <TabAtkins> vmpstr: Note this'll be a fairly quick transition, cross-fade looks nice
07:46 <khush> q+
07:46 — Zakim sees khush on the speaker queue
07:47 <TabAtkins> vmpstr: Thinking about border-radius and other content clips, to me woudl be weird to have clip path that animates to another while the contents cross-fade in the middle
07:47 <TabAtkins> vmpstr: I'd rather see the shape of the element be the contents of th eelement, and that crossfades
07:47 <TabAtkins> vmpstr: So that does mean that border-radiuses/etc stretch if the destination is significantly different
07:47 <fantasai> TabAtkins: Have seen exact effect, of border-radius stretching, in video games. Looks reasonable.
07:48 <fantasai> TabAtkins: if slow, weird, but if fast not a problem
07:48 <Rossen_> q?
07:48 — Zakim sees khush on the speaker queue
07:48 <TabAtkins> khush: We've had partners trying feature so far, and we're doing this exact thing (border radius baked into the texture) and this isn't a complaint
07:48 <Rossen_> ack khush
07:48 — Zakim sees no one on the speaker queue
07:48 <TabAtkins> khush: So next question is about allowing existing CSS animations to work in sync
07:48 <TabAtkins> khush: So here you declartively create animations
07:49 <TabAtkins> khush: People have tried to do CSS animations on the new dom and they want it to be in sync with this feature
07:49 <fantasai> TabAtkins: We have that timeline concept, right?
07:49 <TabAtkins> Rossen_: Which timeline?
07:49 <TabAtkins> flackr: The animation runs on the new document timeline
07:50 <TabAtkins> emilio: Seems problematic if the new page can be affected by 3rd party content
07:50 <TabAtkins> flackr: Yeah, if there's animations on the old content, it runs on the old timeline, but the SET runs on the new document timeline
07:50 <TabAtkins> chrishtr: I think the old-document animations just have to stop
07:51 <TabAtkins> chrishtr: And the transition just happens on the new timeline as if it's an element inserted
07:51 <TabAtkins> flackr: We could allow the old page to define a transition that happens during this SET, and that would run on the new timeline. Properties that are animatable would be very limited
07:51 <Rossen_> q?
07:51 — Zakim sees no one on the speaker queue
07:51 <TabAtkins> flackr: A little worried about splitting control between two pages
07:52 <TabAtkins> s/flackr/JakeA/
07:52 <TabAtkins> JakeA: LIke if you click an link and old page was v10, new page is v11
07:52 <khush> q+
07:52 — Zakim sees khush on the speaker queue
07:52 <TabAtkins> fantasai: I don't think we shoudl constrain what's possible bc we're concerned about deploying a new page in the middle of a transition
07:53 <TabAtkins> JakeA: Not necessarily during the trans, if it's deployed anytime after the old page loads
07:53 <TabAtkins> fantasai: Sure, but that's still a rare case, and it won't break the transition just maybe look weird. If that's what limits our API tho, that's a rpoblem
07:53 <TabAtkins> khush: Assuming the old animations are stopped and new content is on the new-doc timeline...
07:53 <fantasai> s/the transition/the page/
07:53 <fantasai> s/weird/weird during the transition/
07:53 <TabAtkins> khush: If you have a SET, and you do an animation that changes the new element's layout
07:54 <TabAtkins> khush: We're assumign some thing that stay the same for the target
07:54 <TabAtkins> khush: If you ahve a gif, we assume we can use the new gif frames as it's moving
07:54 <chrishtr> q+
07:54 — Zakim sees khush, chrishtr on the speaker queue
07:54 <TabAtkins> khush: But if it changes its position, those become difficult to reason about
07:54 <Rossen_> ack khush
07:54 — Zakim sees chrishtr on the speaker queue
07:54 <TabAtkins> chrishtr: You're talkinga bout the old element changing?
07:54 <TabAtkins> khush: New element also
07:54 <Rossen_> q+
07:54 — Zakim sees chrishtr, Rossen_ on the speaker queue
07:55 <Rossen_> pack chrishtr
07:55 <TabAtkins> khush: So if you're doing CSS animations on the new page, which animations would run OK is what we need to define
07:55 <TabAtkins> chrishtr: I think it would work the same as if you simulated the same thing with CSS animations
07:56 <TabAtkins> Rossen_: Related: if your starting element is a 2x2 grid, your ending element is a 3x3 grid, this visual change is...?
07:56 <TabAtkins> TabAtkins: It's a crossfade, just pixels changing
07:57 <TabAtkins> JakeA: You could also transition the individual grid items if you wanted, then it can get more complex
07:57 <TabAtkins> Rossen_: Good point, this scenario has been requested; if this feature helps with these kinds of layout changes, it would be powerful
07:57 — fantasai q+ to ask about SPA transitions where the author wants the elements to be live
07:57 — Zakim sees chrishtr, Rossen_, fantasai on the speaker queue
07:58 <TabAtkins> khush: In such a scenario we do a scale and crossfade, but there are patterns where people want to do a clip-reveal; we hope this can be made customizable
07:58 <TabAtkins> khush: so stuff like that, we want to start with crossfade default but make it customizable
07:59 <Rossen_> q?
07:59 — Zakim sees chrishtr, Rossen_, fantasai on the speaker queue
07:59 <Rossen_> ack chrishtr
07:59 — Zakim sees Rossen_, fantasai on the speaker queue
07:59 <Rossen_> ack RRSAgent
07:59 — Zakim sees Rossen_, fantasai on the speaker queue
07:59 <Rossen_> ack Rossen
07:59 — Zakim sees fantasai on the speaker queue
07:59 <fantasai> TabAtkins: If you want to do where it reorganizes, you do shared element transition on the grid items individually, not just the grid container as a whole
07:59 <Rossen_> ack fantasai
07:59 <Zakim> fantasai, you wanted to ask about SPA transitions where the author wants the elements to be live
07:59 — Zakim sees no one on the speaker queue
07:59 <TabAtkins> chrishtr: I think Rossen wants the ability to make this done easily
07:59 <TabAtkins> fantasai: I think authors doing SPA want to do live element transitions, like when a video is playing
08:00 <TabAtkins> fantasai: and also to have more control over what exactly gets animated
08:00 <TabAtkins> fantasai: Is that something we want to consider allowing?
08:00 <TabAtkins> fantasai: If so, do we want to design the API to accommodate?
08:00 <TabAtkins> fantasai: And if so, do we want to leave open the possibility to do MPA transitions live, if perf isn't a concern?
08:00 <vollick> q+
08:00 — Zakim sees vollick on the speaker queue
08:01 <TabAtkins> JakeA: I think that might be one of the things we're giving up to actuall deliver something
08:01 <TabAtkins> JakeA: Having both DOMs alive changes things pretty fundamentally
08:01 <fantasai> s/concern/concern in the future/
08:01 <TabAtkins> JakeA: We've got things like prerender where both doms are alive at the same time
08:01 <TabAtkins> JakeA: But if both pages are live with JS, say, that's complicated
08:01 <TabAtkins> JakeA: Also creates signif memory pressure
08:01 <TabAtkins> fantasai: Right, not expecting this today, but in the future we might not have the same constraints
08:02 <TabAtkins> fantasai: So animating props that are not on the set list might be possible in the future
08:02 <TabAtkins> JakeA: Right, I mentioned an opt-in for props animated in the texture vs outside
08:02 — Rossen_ I'm not seeing this explicitly stated but my hope is that this feature will allow a strict subset of transition/anim capabilities compared to in-page... This is important so we don't induce weird dev patterns where page transitions are used to achieve an effect
08:02 <TabAtkins> JakeA: And theoretically we could scale that up to "everything please"
08:03 <TabAtkins> khush: We've thought about how if you deal with this, how drastically it changes the feature
08:03 <TabAtkins> khush: And in the cross-origin scenario, the security gets so complex
08:03 <vollick> q-
08:03 — Zakim sees no one on the speaker queue
08:03 <TabAtkins> fantasai: I think freezing script on the old doc could be quite reasonable
08:03 <TabAtkins> fantasai: But just animating CSS properties in a lot of ways might be something people might want, so leaving that open for the future
08:04 <TabAtkins> emilio: A lot of interactions with sessions history/bfcache - if you do something remotely observable, that's sketchy
08:04 <fantasai> s/the security gets/running JS on the old page and the security/
</pre>
</details>

-- 
GitHub Notification of comment by tabatkins
Please view or discuss this issue at https://github.com/w3c/csswg-drafts/issues/6464#issuecomment-911788061 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Thursday, 2 September 2021 15:07:51 UTC