Re: [w3c/selection-api] Need spec changes to Range and StaticRange to support nodes in different tree? (Issue #169)

From TPAC 2024 minutes:
 
> [w3c/selection-api#169](https://github.com/w3c/selection-api/pull/169)
> 
> Di: this was opened by Sean and has had multiple conversations, there's a comment at the end that summarizes
> … three big questions
> … we are currently prototyping for getComposedRanges, but there are some changes we need to make to allow selections to be across trees
> … getComposedRanges gets the current selection StaticRange
> … first: output of getComposedRanges
> … currently it returns a list of one StaticRange
> … in spec, a StaticRange is valid if the following are true: start and end are in the same node tree (and other things)
> … that is no longer true – potentially in different shadow trees
> … first option: change StaticRange to allow start/end in different trees
> … second option: new StaticComposedRange in different trees in same doc
> … third option: Add new “shadow-including valid” definition.
> … fourth option: change getComposedRanges to return multiple ranges, one per tree
> 
> Ryosuke: returning multiple ranges is my least favorite solution
> … have to wrestle with dozens of different ranges, each on a different tree
> … have to have discontinuous ranges, too, if a selection starts in one shadow root and ends in another shadow root in slotted content, for example, you can have part of shadow host children selected but not the whole thing
> 
> Anne: you don't get multiple ranges, but you do need to do some of that with the masking, when you have a selection that goes inside the shadow tree and call window.getSelection()
> … getSelectionRange can only return portion that is outside of the shadow tree
> 
> Ryosuke: range which represents the selection which crosses the shadow tree
> … the point of getComposedRange is to get the information inside the shadow tree
> 
> Anne: you do, though, still need to do some of the partial computation
> 
> Mason: agree that #4 is bad
> … re. #1, this sounds best and what it should have been in the first place
> … is there compat risk?
> 
> Sanket: the intent was about disconnected trees
> … highlight API doesn't make sense for disconnected trees
> 
> Anne: should highlights work across shadow boundary?
> 
> Sanket: should
> 
> Anne: valid is only used for highlights
> … option 3 seems interesting but we should consider that once there is a caller
> … "option 5" is don't do anything, leave defn of valid as-is because the only caller (highlight) doesn't span the boundary
> … whatever we return from getComposedRanges isn't "invalid" aside from that
> 
> Ryosuke: we could rename "valid"
> 
> Anne: could call it something opposite of "shadow-including"
> 
> Simon: what is the compat risk with option 1?
> 
> Anne: would have to change highlight
> 
> Simon: valid wasn't exposed?
> 
> various: no, just a spec concept
> 
> Anne: it is observable via highlight
> 
> Daniel: can change highlight so it continues to do the same thing, file a separate issue to evaluate changing its behavior
> 
> Anne: I don't see the need for a change, except maybe valid is a confusing word
> … keep it as-is until they decide they want to change it
> 
> smaug: in our implementation, we return invalid StaticRanges
> 
> Sean: it can have nodes in different trees
> 
> Ryosuke: keep spec dfn as-is, and just let getComposedRanges return "invalid" ranges
> … might be misleading to authors, if we don't rename it something clearer
> 
> Anne: we probably want to look into changing highlight (or not)
> 
> Daniel: maybe the dfn of valid should move into highlight
> 
> Anne: should ranges be able to cross the boundary?
> 
> Simon: should file a spec issue against highlight to consider this
> 
> 
> smaug: this is complicated and highlight API doesn't have the issue we have here
> … you can reorder the visual representation using slots
> … Ranges, otoh, look at the DOM trees
> … the start/endpoint might not correspond to the visual
> 
> Ryosuke: wouldn't that problem already exist today?
> 
> smaug: it exists in getComposedRanges
> … when to use flat tree traversal?
> 
> Mason: fundamental problem: any new API should use flat tree, because that's what the user sees and how the selection is painted
> 
> Ryosuke: I think the problem of StaticRange not covering whole tree, and only covering partial tree, already exists because you can start a selection from a sibling of a shadow host, and end inside slotted content
> … depending where the slotted content appears in the shadow tree, two discontiguous ranges can be highlighted
> 
> smaug: browsers are inconsistent about how they behave in that case
> … toString gives DOM stringification, not visual stringification
> … this is an existing issue, which we should fix
> 
> Mason: toString should return the thing that is painted
> … there is no such thing as a discontinuous endpoint
> 
> Ryosuke: do we need to update the definition of before/after to support the shadow tree?
> 
> Di: yes; that's point #2
> … existing spec uses before/after for start/end
> 
> Ryosuke: those dfns need to be updated to shadow-including before etc
> … do we need to change that in the DOM spec?
> 
> Di: in both Selection API spec and DOM spec
> 
> Anne: we also wanted to make the relationship between selection and the range it owns more explicit
> … when you have a selection and it owns a range, mutating the range mutates the selection
> 
> Ryosuke: that is wonky right now because selection needs to internally have a live range
> 
> Anne: the other algorithm needs to be updated
> 
> Ryosuke: we don't want to change the behavior of Range objects, seems like a web compat issue
> 
> anne: maybe there should be an opt-in way to have a shadow-including range, or composed range
> 
> Ryosuke: what are the use cases for web devs?
> 
> Mason: I thought that was StaticRange, expand it to handle across shadow roots
> 
> Ryosuke: now talking about extending regular range behavior to optionally cross shadow boundary, e.g. with ctor argument
> 
> Anne: keep live composed ranges as an internal concept, only spread StaticRange to new APIs
> … not too happy with live ranges, even though we have to continue using them for this purpose
> … that's a lot of typing
> 
> smaug: what do we do if you mutate a cross-shadow-DOM selection?
> 
> Sean: clear it, pretty much
> … there is a use case to expose live composed range, so that pointInRange method can use flat tree order
> 
> Anne: I could imagine if you have use cases for ranges today, you also have use cases for composed ranges
> … the problem with live composed range is we have to do these updates, but we have to figure them out anyway
> … it seems weird if once you mutate a shadow tree, it works differently to mutating the normal tree
> … we don't necessarily have to expose that API to web developers; we could just give them static ranges and then give bounding rect APIs for those, including across the boundary
> 
> Ryosuke: what use cases require live ranges which cross the boundary, beyond static ranges?
> 
> Anne: we have one – selection
> 
> Ryosuke: is there any point in exposing that as a DOM API?
> 
> Mason: no use case should require a live range; you can do anything with getComposedRanges and set base/extent
> … which updates a live composed range, but one that's not exposed
> 
> Ryosuke: it seems tricky to implement the exact same behavior
> … even if you use a MutationObserver, your selection API using static ranges might potentially see state before the ranges are updated, between when the endpoints are updated due to the mutation and when the observer fires
> … I don't think there is anything you should be able to do with live composed ranges that you cannot do with static composed range
> 
> Sanket: that's the direction we've been trying to go, nudge away from live range
> 
> Anne: cost is high, but there is an ergonomic issue, and potentially the issue Ryosuke mentioned
> … though editors usually tightly control their own node trees
> … if that's not the case, we'll hear about it
> 
> Sanket: if we expose it to Range, we can't take it away; if we start with StaticRange then expanding is always an option
> 
> Sean: flattened tree order: what if end boundary is an unslotted node?
> 
> Mason: should behave as though it's out of the document
> 
> Sean: but it doesn't have an order
> 
> Mason: so it's as if the two endpoints are in different documents; it's invalid
> 
> Ryosuke: with shadow trees, you can have endpoints that are not in the flat tree
> 
> Anne: you're saying a child of the shadow host that doesn't end up anywhere, and when stuff does get slotted, the actual contents of the <slot> element
> 
> Ryosuke: should behave the same as if display:none were applied
> … today if you apply display:none we don't treat such selection as invalid
> … it just happens to start after that element
> … we should do the same thing here
> 
> Anne: selection uses the layout tree, or layout info, to compute what actual contents are
> 
> Ryosuke: for painting, you have to have some logic to determine where the selection starts
> 
> smaug: contents is coming from the range
> 
> Anne: if you select around a display:none thing which contains the word "test" and toString, does it contain "test"?
> 
> Sanket: selection.getRangeAt(0).toString() would
> 
> (https://mozilla.pettay.fi/moztests/reorder.html something to try. Select something and check ^ toString() in web console)
> 
> Ryosuke: selection does skip display:none
> 
> Mason: when the endpoint is not in the flat tree (e.g. unslotted light DOM content), proposal is to act as if the selection contains the entire shadow host?
> 
> Ryosuke: walk the tree in the direction you're intending to walk, find the first node that is in the shadow tree
> 
> Anne: we only have to care about Selection toString, because the live range isn't crossing the boundary
> … static ranges don't have toString
> … apparently Selection toString is already somewhat magic
> 
> smaug: in live DOM, all the nodes you select are in the live DOM and you can reorder them with slots
> 
> Anne: then it just does the normal tree walk? Range toString just does a tree-order traversal
> 
> smaug: Range toString returns DOM representation even though visual representation is different
> 
> Anne: already possible
> 
> Ryosuke: just regular tree walk, concatenate whatever is visible
> 
> Anne: we wouldn't have to touch toString
> 
> Sanket: first thing was: whether we update the dfn of valid
> … agreed on not changing dfn for now, I will file CSS highlight issue
> … second issue: Di, do you want to offer a resolution?
> 
> Di: proposed resolution: new concepts of before/after which are composed-aware, and use that internally but not expose it?
> 
> Anne: need to have live composed ranges, and adjust tree mutation algorithms to account for those
> … for now they would be only internal
> … why do we need to update before?
> 
> Di: before/after are used to compare boundary points, used by setStart, setEnd, …
> 
> Ryosuke: whether start comes before end, etc
> … selection API needs to do before/after in composed trees
> 
> Anne: in order to mutate live composed range? okay
> … live composed range concept, plus algorithms, and Selection API needs to be updated to use those while not messing up and exposing more info than it should through its public Range object
> … traversal is the Selection toString case, not sure we have a resolution
> 
> Di: also relates to comparing positions
> 
> Anne: not sure; that one primarily looks at layout
> 
> Di: say start is in document as normal, end is unslotted, how can we compare those?
> 
> Ryosuke: just compare in the composed tree
> 
> Di: it doesn't exist in the flat tree; it's unslotted
> 
> Ryosuke: composed tree and flat tree are different – everything in the DOM exists in the composed tree
> … you have to do a conversion from composed tree to flat tree, which doesn't exist today
> … that's where display:none, display:contents, etc apply
> 
> Anne: how do we not have a conversion from composed to flat? isn't that what flat tree computation is?
> 
> Ryosuke: we can convert the composed tree to flat tree, but can't convert point in composed tree to point in flat tree
> 
> Anne: so you need some adjustment for the endpoints, okay
> 
> Ryosuke: someone will have to write a spec for that when we spec toString behavior
> 
> Sanket: want to introduce live composed range concept; update selection API to use live composed range concept
> 
> Ryosuke: define new before/after for composed range
> … as for selections where endpoint are not in flat tree, do traversal to find first node in the flat tree and use that – must be after the start for the end, etc
> 
> Sanket: next: Selection::containsNode() mismatch
> 
> Di: right now, if you do a selection.getRangeAt(0), you can isPointInRange, StaticRange doesn't have that
> … would we want to add isPointInRange?
> 
> Daniel: should it be a composed tree traversal?
> 
> Di: probably, yes
> 
> Ryosuke: in addition to regular range?
> … why not also consider comparePoint and intersectsNode?
> … if you are adding isPointInRange, those are equally useful
> 
> Sanket: proposal is to move them up to AbstractRange
> 
> smaug: behavior is different
> … we want something that uses flat tree
> 
> various: composed tree
> 
> Mason: (1) can we change existing APIs, (2) should we have a new API which definitely uses the flat tree, if we can't
> 
> Ryosuke: doesn't seem web-compatible to change to using the flat tree
> … for consistency reasons, isPointInRange etc should continue to use composed tree, because that's what these APIs currently do
> 
> Mason: should we add new APIs that use the flat tree, since that's more useful?
> 
> Anne: if we offer that synchronously, it requires layout
> … we've tried not to expose flat tree for that reason
> 
> Ryosuke: definitely requires updating style for selection
> … what are the use cases where this would be handy?
> … the only difference is if an element is unslotted shadow host child, or fallback slot content
> 
> Mason: or when you rearrange things so the last DOM element gets pulled to the first via a slot
> … a node could be outside selection in composed tree, inside in flat tree
> 
> Ryosuke: in all those cases, because the selection will follow the composed tree order, wouldn't it be more useful to use the composed tree to get this answer?
> … if nodes are swapped due to slotting, what's selected is what's in the middle in the composed tree, not the flat tree
> 
> smaug: no, what's selected is what's in the flat tree – what the user sees
> 
> Mason: agreed with smaug; if a user sees a selection which contains "a", then it is in the selection
> 
> smaug: yes, browsers are broken today
> 
> Ryosuke: it would mean that we have to support discontiguous ranges, because that's the only way you can select a visually contiguous region
> 
> Anne: not necessarily – if you have this magic function that computes from points in the composed tree to the flat tree, then you have a range in the flat tree
> 
> Ryosuke: that's not what the selection reflects, as argued – only when you can select a visually contiguous region when the slots rearrange the order, … [didn't catch this entire statement]
> … consider a concrete example: nodes A, B, C
> … A and C are swapped in slots: C, B, A
> … if you select from C to B, in the DOM sense that selects B and C
> 
> Sanket: still can be contiguous within the range
> 
> Ryosuke: A, B, C, swap to B, A, C
> … select B to C, select will look discontiguous to the user
> … in order to select from end of B to end of C, you have to select A and node C, but that requires that you have discontiguous selection regions
> … we could entertain that route, but that would have massive implications for the rest of the API
> 
> smaug: same argument we've had at Mozilla; I've been arguing for the flat tree (better for the user, even though composed is easier)
> 
> Megan: multi is specced, but nobody correctly or fully implements it
> … it would be very powerful or useful, but huge rat's nest
> 
> Ryosuke: don't want to change the entire model of selection for this particular case
> 
> Sanket: for that particular case, let the API behave as it would today?
> 
> Ryosuke: but then we go back to the original question: isPointInRange, etc., I don't see when you want to have a function which checks the intersection in the flat tree because that's not what is selected
> … WK and Blink don't have that capability today
> 
> smaug: Blink seems to select using flat tree, but uses composed tree for stringification
> … don't know what WebKit does
> 
> Megan: not that, just DOM order only
> 
> Anne: for painting purposes and for computing final text, does it have to be discontinuous?
> … can't we, once we have those points in the flat tree, paint that bit?
> 
> Ryosuke: but then, what does it mean to copy that text?
> … you have to walk in the flat tree to copy the text?
> 
> Sanket: selection would be hard to do
> … we'd have to do something much more difficult to make that possible
> 
> Ryosuke: consider A, B, C; swap A, B
> … if you try to select from A to C, in logical DOM order, that is two discontiguous nodes
> … they are visually next to each other, so if you select in flat tree, they're selectable
> … but in DOM, you need two different ranges pointing at A and C
> 
> Mason: or you add the intervening node, and just accept more stuff may become part of the selection
> 
> Sean: then you don't need to have multiple ranges
> 
> Megan: but then you need a flat tree range
> 
> Sean: that's the argument – using flat tree provides better user experience
> 
> Ryosuke: but the flat tree version of the range cannot represent the DOM state, because DOM is in the composed tree, not the flat tree
> 
> Sanket: not too different from positioned stuff, where it's hard to do such a selection
> … similar concept, just in shadow DOM
> 
> Ryosuke: bi-di and flexbox can also change order
> 
> Megan: with bi-di, if you want a bi-di selection, that would require multiple selections, unless we have another concept of flat tree selection, which is a different concept
> 
> Ryosuke: definitely not easy to implement
> 
> Sanket: within this concept of one contiguous range in the composed tree, is there a solution that would be meaningful for isPointInRange?
> 
> Ryosuke: I don't see a use case for it. All the copy operations etc are done in DOM order, not flat tree order.
> 
> Megan: can you give a concrete example, or is this abstract?
> 
> Sean: I asked because I assumed we were going to change to flat tree order. If we continue to use composed tree order, there's no need to change.
> 
> Megan: maybe flat tree selection is something we should think about because it would be a better user experience? but it's a giant pot of spaghetti. I dream of fixing this.
> 
> Ryosuke: using flat tree for selection endpoints is an interesting idea – it avoids the problem of multiple range objects to manage, but looks visually contiguous to the user
> 
> Anne: except you have to translate it back to composed tree
> 
> Megan: have to figure out what the interface is
> 
> Ryosuke: selection is really on the flat tree, then if you are trying to toString, you just walk the render tree
> 
> smaug: looks like Chromium is using flat tree for copy/paste, but stringifying uses composed tree
> … the browsers are inconsistent
> 
> Megan: using flat tree for selection and copy?
> 
> smaug: seems that way
> 
> Anne: doesn't that mean Selection toString leaks the shadow tree?
> … toString also needs to be amended to accept shadow root parameters?
> 
> Mason: [showing example with slot reordering]
> 
> Megan: would love for that to work consistently, but everything to this point has been using DOM ranges
> … I guess Blink has already switched? Can talk to them about it. Want selection on the web to be more intuitive.
> 
> Mason: getComposedRanges, if we do that, the point is to allow shadow roots to work correctly
> … I'm hoping we don't go halfway, and do it right
> 
> Sanket: one point was to put isPointInRange on StaticRange, no specific use case, correct?
> … proposed resolution: don't need to do that?
> 
> Anne: need to file an issue on Selection toString, because it might leak the shadow tree
> 
> Ryosuke: toString in general needs to be specced more
> 
> Mason: maybe there is opportunity to make it work correctly?
> 
> Megan: +1
> 
> Anne: what are the next steps here? are we going to investigate flat tree selection?
> 
> Megan: would like to do homework, interesting to know that Blink is doing that
> … need to understand more about what's going on with shadow roots
> 
> Ryosuke: if we're solving this problem for shadow roots, might as well solve it for bidi
> 
> Megan: there are so many things, where if you could just have visual selection work on the web, it would be good
> 
> Johannes: column selection in tables?  would that also be affected?
> 
> Ryosuke: wouldn't work, because start and end are part of separate sections, even in the flat tree
> 
> Simon: would need multiple ranges for that
> 
> Mason: same is true of flexbox reordering
> 
> Ryosuke: could start from one point, and what's visually in between them
> 
> Anne: related Range object ends up stringifying to something nonsensical
> 
> Ryosuke: sure, but that's the least of the issues
> 
> Johannes: sounds good, but if we overhaul the whole thing and still don't cover column selection or flexbox, that's something to be aware of
> 
> Ryosuke: for column selection, have a way to select a <col> element?
> 
> Sanket: custom element tables, things which look like tables but aren't?
> 
> Ryosuke: then what you need is geometry-based selection
> 
> Megan: don't know if you can do that without multiple selections
> 
> Anne: also with columns, you don't know if someone is trying to select all the rows, …
> … need some kind of different input
> 
> Megan: need to keep in mind all of the problems we need to solve
> … but if we can advance it without precluding solutions to the more advanced questions, we have to start somewhere
> 
> Anne: flat tree thing, follows the model we have today and solves a problem
> 
> Megan: want to spend more time thinking about it, understand what is going on with Blink
> 
> Ryosuke: when you do editing operations, that's where the problems lie
> 
> smaug: do we need to modify getComposedRanges to deal with flat tree ranges?
> 
> Ryosuke: would need something dramatic like disabling contenteditable in such places
> 
> Simon: there's a proposal for CSS reading flow; have we discussed how selection works when you reorder with reading flow
> 
> Anne: only influences focus direction
> 
> Mason: and a11y tree, but same spirit as selecting across flexbox
> 
> Simon: shouldn't affect selection?
> 
> Mason: agreed we shouldn't boil the ocean; the flat tree thing is useful; the rest can wait
> 
> [adjourned]

-- 
Reply to this email directly or view it on GitHub:
https://github.com/w3c/selection-api/issues/169#issuecomment-2397160208
You are receiving this because you are subscribed to this thread.

Message ID: <w3c/selection-api/issues/169/2397160208@github.com>