W3C home > Mailing lists > Public > www-style@w3.org > August 2022

[CSSWG] Minutes New York F2F 2022-08-01 Part III: CSS Highlight API [css-highlight-api]

From: Dael Jackson <daelcss@gmail.com>
Date: Mon, 29 Aug 2022 19:59:42 -0400
Message-ID: <CADhPm3vT3iN7c+q3fiT5EOTRqAneVnuvVB31vSA5XEYqW5KLOg@mail.gmail.com>
To: www-style@w3.org
  These are the official CSSWG minutes.
  Unless you're correcting the minutes,
 Please respond by starting a new thread
   with an appropriate subject line.

CSS Highlight API

  - dandclark explained the current 3 proposals for issue #7513
      (Approaches for dispatching highlight pointer events) and
      indicated a preference for option C which is having two events,
      one for highlight and one for DOM.
      - emilio introduced a fourth option to add an API to answer if a
          certain point intersects a highlight. There was significant
          discussion around the feasibility of the approach and
          interest in investigating more to see if some concerns could
          be mitigated. emilio added more details to github after the
      - TabAtkins raised the possibility of changing option A where
          each highlight gets its own event to require registration
          before dispatch.
      - Discussion will continue on github to determine the feasibility
          of these new options.
  - Having an API to return what highlights intersected broadly made
      sense as a solution to issue #7512 (Is a HighlightPointerEvent
      type needed?) however the final discussion will need to wait on a
      decision for issue #7513.


Agenda: https://github.com/w3c/csswg-drafts/projects/30

Scribe: TabAtkins

CSS Highlight API

Approaches for dispatching highlight pointer events
  github: https://github.com/w3c/csswg-drafts/issues/7513

  dandclark: So pointer events with highlights, this is about making
             highlight pseudos interactive like for a spellchecker

  dandclark: First question is how do we dispatch these events, what's
             the event order
  dandclark: I assume the highlight object is the right thing to
             receive the event, rather than the individual ranges
  dandclark: If I have a bunch of ranges on a single highlight, I
             probably want to do the same thing for each
  dandclark: seems silly to have to set an event listener on each
             range, especially as the highlight moves
  dandclark: and when considering dispatches, need to consider there
             can be multiple types of highlights, and possibly
  dandclark: like a find-on-page that emphasizes and scrolls, and also
             a spellcheck that underlines misspelled words and shows
             spelling suggestions
  dandclark: might want one of those to eat the click, so spelling
             error can win over find-on-page

  dandclark: so I can talk about a few approaches
  dandclark: In the issue, version A is each highlight gets its own
             event, in descending priority order
  dandclark: They can be cancelable, so the default is to advance to
             the next highlight, and you can preventDefault()
  dandclark: Suspicious to fire an arbitrary number of events for a
             single action though
  dandclark: Approach B is a single event for the top priority
             highlight, define event path so it goes through other
             highlights, then bubbles to dom tree
  dandclark: this seems like pretty normal approach. But there's a web
             compat problem
  dandclark: Currently pointer events can only ever be an element
  dandclark: but now suddenly they can have non-element targets
  dandclark: so might break assumptions
  dandclark: The Edge explainer suggests lying about the event target
             to get around this
  dandclark: Thinking about it more, think it's not the right approach
  dandclark: Approach C is a combo. We dispatch two events, one for
             highlights, one for dom tree. Highlight one propagates
             through overlapping highlights. If it's not
             preventDefault'd by the end, we fire a new one on the dom
  dandclark: That dual-event approach is where I lean now

  emilio: Rather than messing with the dom events and re-targeting, can
          we add an API to highlight that tells "is point X
          intersecting the highlight?"? seems like that would let you
          do what you want with regular dom events
  emilio: you handle the pointer event and ask the highlights if the
          point overlaps, and just deal with normal dom event
  emilio: Seems simpler but still gives all the use-cases
  emilio: If you have two objects, handling pointer event on one or the
  emilio: the normal element pointer event should still work even if
          you click on the highlight
  emilio: Seems simpler and less risky, and doesn't introduce new event
  emilio: and will work with touch events, etc
  emilio: Curious if it's been considered?
  TabAtkins: So assumption is you'd iterate through all your highlights
             and check?
  emilio: Yes. and they can check if it's been defaulted, you can deal
          with the order yourself, etc.

  dandclark: This seems useful for- given I have a click with an
             offset, show me all the highlights under this location? or
             go through each highlight in the registry and check?
  dandclark: Can def see if there's a pointer event handler on the
             body, and I get an event, could ask platform for all the
             highlights at this point
  <emilio> boolean intersects(long clientX, long clientY)
  <bkardell> emilio: that is on highlight?
  emilio: Was thinking about method on the highlight object
  emilio: so you iterate over the highlights in the order you want
  emilio: but that should provide all the flexibility you'll ever need

  emilio: If we want something more convenient, could put one on the
          highlight registry that gives back all of them
  emilio: would just be sugar over a for loop
  emilio: so could use if you want a central place to handle all the
  emilio: but might also want to deal with each highlight separately

  flackr: My point was partially overlapped with emilio, concerned
          about a special event before the dom event
  flackr: Could consider the highlight to be the normal action, so if
          the normal event wasn't defaulted you dispatch to highlight
  flackr: Longer term, we want to support css pseudo targets, wondering
          if that helps so we can just target those
  dandclark: The highlight pseudo is a pseudo object, it's a little
  flackr: I think my main concern is whether those should be given the
          event first
  flackr: I think visually the highlight is behind the text, so from
          dev pov the element should get first crack, and pseudo gets
          opportunity after

  dbaron: I think it's worth trying to get event.stopPropagation vs
          preventDefault to behave as closely as possible to normal
  dbaron: I think what Dan was originally defining was to make
          preventDefault act like stopPropagation
  dbaron: Hope we can do something better even if the dom event spec
          needs to expose some extra concepts

  dbaron: Thoughts on emilio's proposal
  dbaron: first is I'm a little concerned about an approach that forces
          you to add event listeners to the doc, for cases where impls
          want to make optimizations for passive listeners
  dbaron: If it's an event type where registering a non-passive
          listener, and you need to register non-passive for the
          highlight, you have to take the perf penalty document-wide
  dbaron: That seems undesirable
  dbaron: so I think that's a reason you might want to have
          eventListener registration be on more specific targets
  dandclark: I like the simplicity of emilio's idea
  dandclark: if we can find a way around the perf hit

  emilio: The initial proposal was about clicks
  emilio: Don't think we can get much opt about passive vs non-passive
          event listeners
  emilio: don't see how having highlights doesn't prevent you from
          losing optimizations
  dbaron: What you said about the way to use the proposal is put an
          Event Listener on the document, and look up the point
  dbaron: And that way of using it can be problematic
  emilio: Right, if you care about scroll and things that benefit from
  emilio: But API doesn't let you use it for other event types
  dbaron: I think when it's like "this is only for clicks *right now*",
          unsure we want to design around that
  emilio: So API right now takes an x/y
  emilio: Have clear text for mouse events to resolve that
  emilio: Could let the method take an event object instead, so we can
          limit it to certain event types
  dbaron: Ah yeah, other point I'd forgot. I think taking x/y doesn't
          handle [missed]
  dbaron: you might click a piece of text with highlights that obscures
          another piece with highlights
  dbaron: you're probably interested in the top element's highlights,
          not the obscured one
  emilio: Could define the api similar to getelementfrompoint where you
          do account for hit-testing
  emilio: Could define to only care about rects, like union the ranges
  emilio: or like elementFromPoint which cares about hit-testing and
  emilio: Seems you could get that with just an x/y if you define it to
          do hit-testing
  dbaron: yeah that sounds like it works
  emilio: So define it in terms of elementsFromPoint
  emilio: If any of the elements intersect the highlight... something
          like that
  emilio: specific semantics for intersecting highlights could be left
  emilio: if both intersect the consumer can choose

  flackr: I think x/y works if you can get highlights on the targeted
  flackr: that'll account for hit-testing
  flackr: but this is def going to be a performance issue, even if
          limited to mouse events
  flackr: people might want to drag highlights, so they'll have
          pointerstart, and that means we have to delay scrolling
  flackr: but if we fire on the highlight we can recognize it won't
          stop scrolling
  emilio: But that would penalize other scenarios
  emilio: not sure what the best way to do it is
  emilio: needs real thought especially for pointerdown
  emilio: [missed]
  emilio: There are use-cases for pointerdown
  emilio: If we extend proposal, we'd dispatch to highlights, then to
          web content... we'd do it for every event type anyone would
          care about
  flackr: Highlight needs to be in the event target tree, but yeah
          currently tree is strictly ancestor tree so that would be a

  astearns: Question about emilio's proposal
  astearns: You get coords, ask document for highlights.
  astearns: Are we still able to have top priority highlight prevent
            other highlights from responding?
  emilio: Yes, just check if the event was preventDefault'd and return
          rather than moving on to the next highlight
  <dbaron> (do you mean stopPropagation or preventDefault for that
  <TabAtkins> (don't think it matters for the discussion)
  astearns: So it's manual?
  TabAtkins: Yes, all fully manually handled, they're not fired on the
             highlights by the browser stuff

  fremy: So if you have multiple types of code doing highlights they
         don't know about each other
  fremy: How can they work together?
  emilio: Initial proposal was a bool on the highlight
  emilio: If you need to be aware of one vs the other you need a
          central place to do it
  TabAtkins: Right, given dan's initial use-cases (scroll and
             spellcheck) you *need* them to coordinate with each other
  emilio: With my proposal there's no change to event prop. You fire
          the same events as with no highlights
  emilio: Up to the website to either centralize their highlight
          handling, or have different listeners that handle them
  fremy: But you can't decide which acts first
  emilio: With a bool and two coords approach, that's right because you
          have to handle the highlights
  emilio: But if we have the method on the highlight registry that
          returns highlights in order, you can check yourself if the
          one you're in charge of is first
  fremy: So if you're not, do you try again?
  emilio: If you don't have event listeners in the right order you can
          have weirdness
  fremy: But order isn't one order
  fremy: Not sure there's one correct order
  dbaron: I think emilio's proposal is that page author has to write
          their own event registration system if they want to deal with
          multiple highlights
  dbaron: We provide an api in the platform that says "here's the
          ordered list of highlights affected by this" and the author
          can process the list with their own reg system however they
  fremy: Yeah, just weird we don't provide that
  fremy: You assume libraries will cooperate, not clear to me they will
         if they're different sources
  fremy: strange to assume that
  <fantasai> +1 fremy
  <GameMaker> +1 fremy as well
  emilio: my proposal is minimum that lets you build this
  emilio: but solving general case firing event listeners in an order
          that solves every use case, yeah it's a much more complicated
  emilio: have to define which events do this, we start with click but
          have a million events

  fremy: Another option is we provide it. For each highlight, define
         which events you want to receive, and we have an api that
         dispatches to them in a pseudo-event manner
  fremy: So does preventDefault for you, etc
  emilio: I'm not even sure there's a canonical order you want to define
          events on
  emilio: spellcheck error with a link inside
  emilio: Might seem reasonable to fire spellcheck first and link if
          not handled, but might be reasonable other way around
  emilio: I think easiest is to make website choose
  dandclark: Advantage of having platform handle this, where we just
             propagate the event, is we already had an order of
  dandclark: so think there is an advantage to use the existing order
             since things are built in already
  emilio: There is reasons to do before, but also after
  emilio: Not clear to me why we should do them before or after in
  fremy: I think this makes sense, let authors to do this
  emilio: I think having an api that gives you ranges in order would be
          low level
  emilio: if libraries say we can't use this because there's no good
          way to coordinate, then we can decide whether to fire events
          before or after
  emilio: seems like an elegant way to handle the use-cases without
          making a call

  scribe: fantasai

  TabAtkins: I think one detail of fremy's proposal is being missed,
             which is getting the list ordered is reasonable and
             authors handling on his own
  TabAtkins: The problem is that you have to re-invent event dispatch,
             and hope other libraries can coordinate with you
  TabAtkins: Given a list of highlights to dispatch on, it fires events
             using normal event listener mechanics
  TabAtkins: handles preventDefault etc, as if part of the normal event
             dispatch order
  TabAtkins: so don't have to re-invent
  TabAtkins: doesn't require cross-library coordination of the events
  TabAtkins: e.g. fire highlight event, take a list of highlights,
             cranks through them in order as if part of event chain
  emilio: So would be equivalent to making highlight event listener
  emilio: so basically it would be a matter of writing that loop with
          the current event and dealing the pD/etc in that order
  emilio: with api from before, give me a list of highlights, would be
          a matter of writing that loop with current event and dealing
          with event propagation and stuff manually
  emilio: that seems reasonable
  emilio: page would be responsible at some point, to propagate the
          highlight events
  TabAtkins: on the highlight registry, yeah
  emilio: Seems like either extension or replacement
  emilio: Still requires page's coordination

  astearns: Tab described your first option from the issue?
  dandclark: Sounds similar, except instead of dispatching, need to
             register to dispatch
  TabAtkins: Yes, page author still has to handle ordering and do what
             they need to do
  TabAtkins: but otherwise platform handles events for you
  dandclark: So if I have highlights on my page
  TabAtkins: Ask the highlight registry are at this x,y, then call this
             with all of those
  TabAtkins: in priority order
  TabAtkins: if a highlight preventsDefault, then later things don't
             get the event
  TabAtkins: which is what you want
  TabAtkins: [missed]

  dandclark: How do we know it's not happening twice?
  dandclark: If my final page library says, okay, I need to trigger
             this ? and propagate the highlights
  dandclark: [too fast]
  dandclark: I have two different libraries
  TabAtkins: If you have 2 libraries both installing event handler on
             the element
  TabAtkins: If not preventDefault, each one would run through list,
             and you'd get repetitious behavior
  florian: This approach both allows but also requires you to coordinate
  florian: simple scenarios can coordinate by accident, but overall, if
           doing with multiple independent libraries
  florian: need some kind of plan for interaction
  florian: you need to think through it

  florian: Another thing I wanted to ask, unsure if decides,
  florian: all this discussion here, we're going to have a very similar
           discussion for every other type of pseudo
  florian: e.g. ellipsis
  florian: We shouldn't have 3-4 different solutions for each type of
  florian: I think this approach works better, you're in charge of
           coordinating, coordinate to your heart's delight
  florian: but it is a low-level approach, you *have to* coordinate
  florian: For those who have thought about this, how do you generalize
           to other pseudos?

  TabAtkins: Only concern I have is the actual invocation of your
             behavior, already have API for that called event listeners
  TabAtkins: not using that sucks
  TabAtkins: but if we recommend to authors to call dispatchEvent on
             your things, that's probably sufficient
  TabAtkins: we have communication API ... but at least not inventing a
             new callback system
  florian: can we do both? By default have a dispatcher that that walks
           up the highlight priority list
  florian: and if you don't like that, do your own?
  florian: If you don't say anything, we dispatch like this. If you
           don't like it, roll your own
  fremy: If you want [missed]
  fremy: we already have mechanism for that
  emilio: But initial proposal fires at DOM
  fremy: If you decide that you want to dispatch something, if it's
         important, you can say it's capture phase. We already have
         mechanism for it, this coordination happens every day
  fremy: We have bubble phase, capturing, etc.
  fremy: we already have these
  fremy: it works

  emilio: If the page hasn't called dispatchEvent on the registry
  emilio: if we do ourselves, prevents page from doing it later
  emilio: so only default we could choose is after DOM dispatch
  emilio: but might want to dispatch earlier than that
  emilio: so I think making highlight inherit from EventTarget makes
  emilio: and propagating to highlight whenever page wants is reasonable
  emilio: but how would that look like?
  emilio: undefined ?? x,?
  emilio: do for mouse events normally?
  emilio: do the list thing?
  TabAtkins: Do the list thing manually
  TabAtkins: You can check preventDefault by defaultPrevented
  TabAtkins: We can put example code in the spec
  emilio: So you don't want an API that does it for you?
  TabAtkins: Probably not
  *fantasai would like a summary

  dandclark: If we have 2 libraries that are not coordinating, which
             one writes the for loop?
  TabAtkins: You should check to see if default has been prevented
  TabAtkins: if you're first, you can preventDefault() if you've
             handled things
  TabAtkins: or you can just stop propagation immediately
  TabAtkins: Event system can handle that
  fremy: You may say that you don't want to prevent, but still click on
         the button
  TabAtkins: Events should check themselves to see if prevented
  fremy: if you preventDefault, the button won't click

  emilio: That's the key of the issue, there's no good answer on
          whatever is the good answer
  emilio: if you have a spelling error on a button, you might want a
          popup to suggest a fix
  emilio: or you might want to click the button
  emilio: It's not known which you want
  fremy: I agree, but if every library makes its own for loop because
         it's necessary, then ??
  emilio: We're not suggesting that
  emilio: The library doesn't need to write the for loop. They add e.g.
          click event listener on its object
  emilio: something on the page writes the for loop once, and
          dispatches to all highlights
  emilio: So no library needs to write the for loop
  emilio: That's done once whenever page wants to handle that

  fremy: If I use grammarly, grammarly needs to write the for loop or
         else it doesn't work
  fremy: So they will write it because they need it
  emilio: Then they would only handle ranges that they register
  fremy: But it still does not solve issue of order
  fremy: run the grammarly thing and no consideration that some other
         things on top of them
  emilio: If preventDefaulted then ???
  fremy: Still the same issue, it's not preventDefaulted if there's
         another highlight before it
  fremy: You have multiple highlights. Grammarly wants to work. So they
         will need to add an event to handle fix. If they want to make
         for loop, they can only include grammarly stuff
  fremy: but they might want to run grammarly code only if no highlight
         on top of them
  emilio: If the page has dispatched the same event to the highlight
  emilio: like you would dispatch the mouse event from the platform
  emilio: If another highlight provider did something, registered dom
          event in capturephase, and have preventDefaulted it, then
          grammarly can check that and bail out
  emilio: This event has been handled before, we may not even need our
          for loop
  fremy: I don't quite understand, it's not clear what are we doing

  TabAtkins: There are various ways for event handler to check if I've
             been called with this before
  TabAtkins: In most cases not required, all first-party content, and
             don't care much
  TabAtkins: should be generally OK, and complex cases need complexity
  TabAtkins: You can say I only want to be dispatched once, and
             guarantee that on your own, by checking and immediately
             returning if you get called a second time
  fremy: ...
  TabAtkins: Normal event dispatch would not call the same event on the
             same element more than once
  TabAtkins: but we're not using normal event dispatch
  <TabAtkins> (store the last event you saw dispatched to yourself, and
              check if the new one is the same as the last; if so
              return immediately)
  <TabAtkins> so you can avoid the "double-dispatch" entirely on your
              own without any 3rd party coordination

  astearns: Any other lingering questions, or take back to the issue to
            discuss the current state of this fourth way
  * fantasai doesn't quite understand what the proposal is
  dandclark: I agree with taking back to the issue, I'm also a bit lost
  astearns: Emilio, can I ask you to write up the proposal with all the
            details so that Dan has something to digest more easily
            than the minutes?

  dandclark: Does this still have the perf issue dbaron raised?
  dandclark: [missed]
  emilio: For clicks sure there's no perf issue
  emilio: For things like pointerDown, if you want notPassive handling
          of pointerDown, it feels like a global event listener is
          maybe not ideal
  emilio: This may be fixable another way
  emilio: like, maybe we need another member of listenerOptions
  emilio: I don't have an answer for that right now

  flackr: Maybe the highlight could have a different touchAction?
  flackr: Normally we encourage people to use passive events and use
          touch Action to specify whether to scroll or not
  flackr: if we had a way to specify the touch-action on the highlights
          to not depend
  emilio: Yes, you might want to allow touch-action on highlight
          pseudos, make that behave properly
  emilio: it's a very good idea

  astearns: Anything more on this issue?
  dandclark: Yes, I think we're good
  astearns: Do we discuss next issue? or wait on the details of this
  dandclark: I think we wait
  dandclark: Next one is about answering question of given dispatch,
             how to we determine which range was targetted. I suggested
             adding a new event type
  astearns: so closing off current issue and going to next issue

Is a HighlightPointerEvent type needed?
  github: https://github.com/w3c/csswg-drafts/issues/7512

  dandclark: If the solution we go with here involves highlight being
             an event target
  dandclark: which either way I think will be the case
  dandclark: how do I know which range was targeted?
  dandclark: e.g. I'm a spell checker, but got a click, but which range
             was clicked?
  dandclark: Multiple ways to fix
  dandclark: Original solution maybe doesn't make sense, but if we do
             have some kind of intersects API then maybe that answers
             the question
  dandclark: If we're re-using the existing pointer events, then
  dandclark: Call some kind of API, what were my intersection ranges?
  dandclark: I need to know which ranges and which highlight
  dandclark: perhaps API could be just onHighlight? similar to what ??
             was proposing
  dandclark: I add intersects() function to Highlight, it takes x,y and
             it returns a list of ranges in the highlight
  dandclark: maybe that's the simplest
  dandclark: highlight could potentially have several overlapping on
             the same text
  dandclark: Does that sound reasonable?

  emilio: yes...
  emilio: given we already have a hit testing function that gives you
          list of highlights, maybe also takes ranges
  emilio: I think this is related to previous issue
  emilio: Either way, if we go through event listener path or dom event
          thing, or the thing we were discussing
  emilio: I'd rather resolve that first, even if it's not the final
  emilio: maybe instead of sequence of highlights we need to return
          sequence of highlights and ranges
  dandclark: Sounds good, just want to make sure the problem is on the

  flackr: I don't think we should have a specific event type, because I
          think we will want to target multiple types of events at
  flackr: prefer to have event that bubbles up. You dispatch to
          specific range, and then on the highlight you can check for
          which range
  emilio: but target would be the DOM element
  flackr: right, but ...
  emilio: But unsure about event target
  emilio: [missed]
  emilio: We may need to figure out a way... obviously, if we provide
          the API so that if we dispatch both highlights and ranges,
          the page could put the ranges property and expand all of the
  emilio: It's worth thinking about, but depends on other issues

  GameMaker: I just want to make sure that when we're talking about
             ranges that highlights are done in abstract ranges, so
             keep that in mind
  GameMaker: make sure we remember they're AbstractRanges
  emilio: You can build highlights with StaticRange and Range
  GameMaker: Yeah, AbstractRange can be either
  emilio: Right

  astearns: Anything else on this issue?
  astearns: OK, we'll come back to both of these relatively soon
Received on Tuesday, 30 August 2022 00:00:23 UTC

This archive was generated by hypermail 2.4.0 : Monday, 23 January 2023 02:15:20 UTC