- From: Nic Jansma <nic@nicj.net>
- Date: Mon, 6 Nov 2023 09:09:27 -0500
- To: Yoav Weiss <yoavweiss@google.com>
- Cc: public-web-perf <public-web-perf@w3.org>, "Jansma, Nic" <njansma@akamai.com>, Carine Bournez <carine@w3.org>
- Message-ID: <CAALun4vCTLc5C3w-tp2R7eaL_CUDq8oq=n8D4mY9q-Z4W6ik1A@mail.gmail.com>
(Apologies for the delay in posting these)
Minutes are now available:
Linked to from our WebPerf WG Agenda document
<https://docs.google.com/document/d/10dz_7QM5XCNsGeI63R864lF9gFqlqQD37B4q8Q46LMM/edit#>
Published to the web-performance Github meetings page
<https://w3c.github.io/web-performance/meetings/>
... and copied below:
Participants
Dan Shappir, Tariq Raffique, Ian Clelland, Andrew Comminos, Sia
Karamalegos, Nic Jansma, Alex N Jose, Benjamin de Kosnik, Michal Mocny,
Noam Helfman, Carine Bournez, Abishek Ghosh, Giacomo Zecchini, Katie
Sylor-Miller, Noam Rosenthal, Paul Irish, Sean Feng, Timo Tijhof
Admin
- Post-TPAC edition!
- Yoav: Looking for feedback on remote attendance
- Charter in AC review
<https://www.google.com/url?q=https://www.w3.org/2002/09/wbs/33280/webperf-charter-2023/&sa=D&source=editors&ust=1699283057039489&usg=AOvVaw3iVtWOhFlRdXjD0VmM2EtT>
!
- Yoav: Ping your AC rep (or if you are one), to review and vote
- Tim: How does current AC rep vote?
- Yoav: AI to follow-up with link
- Next call on October 12, earlier slot
MinutesNavigation ID updates - Ian
Recording
<https://www.google.com/url?q=https://youtu.be/6hL_X5G_F7g&sa=D&source=editors&ust=1699283057040803&usg=AOvVaw1_OVDPJUvLDEqMV70SCx4R>
- Ian: Identifying user-perceived navigation during page load
-
- ... Could be expanded in the future
- ... Segment timeline by navigations
- ... Two outstanding questions
-
- ... (1) For the shape, initially the proposal was to have a counter
-
- ... Start at 0, increment every time
- ... Downsides is developers may depend on these IDs
- ... Other proposal is to use a UUID, makes them unpredictable
- ... No semantic information in the ID at all
- ... Currently in Chrome if you turn this on experimentally
- ... Another option is to use the type of navigation and start time
- ... That's currently what's in spec'd PR
- ... Under review, proposal was to use an object that contains the same
information
- ... Then the developer doesn't have to break down the information
themselves
- ... (2) Do we need to have an API to filter by these?
-
- [discussion]
- Dan: Regarding counter, I understand if it's officially a counter it
could be problematic. If it's more like setTimeout(), you can't rely on it
- ... Regarding UUID, might be a bit of an overkill here
- ... Regarding object, one problem we have is people might rely on
referential identifiers for equality checking
- ... Checking equality might be an issue if they check an object
- ... Regarding filtering, we have JavaScript but that's also true for
getEntriesByType() and byName()
- ... They exist because they're perceived to be more efficient in some
cases
- ... If there's merit in case for those two, there's merit in
getEntriesByNavigationId()
- ... Not fan of using a generic filter, as it tries to replace what can
be done in JavaScript
- Yoav: Regarding a counter, the API was "counter-shaped"
- ... The very first code Barry took to demonstrate Soft Navigation, and
how one can play with them, he treated the non-counter as a counter
- ... If it looks like a counter and behaves like a counter, people will
treat it like one
- ... Regarding Object, people could toString() it to test equality, but
(inefficient)
- ... Regarding filters, I think having filter ahead of time rather than
after giving all entries to JavaScript, in Chromium we have separate
buffers and accumulate into a single buffer and give to the developer
- ... Doing all that work for the developer to trim that list to what
they actually need, seems sub-optimal
- ... We could optimize implementation, filtering ahead of time is better
- ... No opinion on specific API vs. general API
- Katie: Curious how you're envisioning users using this navigation ID
- ... Missing piece is the semantics of what a new navigation ID means?
- ... How do you tie that back to RUM monitoring
- ... Soft Navigation in a React app, I want to segment to an /about
route to a /product/xyz route
- ... Piece that's missing for me when thinking about navigation ID
- ... Great because it groups everything, but how do I turn that into a
user action?
- Ian: I think there may be a few examples of how you could use this
- Michal: Doesn't the navigation ID point to the NavigationTiming that
has a URL?
- Ian: Something like that
- Yoav: Maybe we need some easy way to find "the navigation" entry for
this ID, not just all the entries
- ... navigation ID on performance entry, relevant ResourceTiming
entries, whatnot
- ... But maybe we need a canonical way to get a specific Soft Nav of
BFCache entry
- Michal: I was talking to a developer this morning, and that was the
request they had
- ... For PerformanceEntries like EventTiming that can come late, you
have to go "back" to lookup the NavigationTiming entry that initiated it
- ... Developer has to go "backwards" to look it up
- ... As we begin to slice navigation entires on varius types, the the
most recent navigation is the one that's your identifier
- ... Developer may want to know the most recent of a specific type
- ... is startTime the only one that's necessary?
- ... getEntriesByStartTime() or filter
- ... Could counter be startTime, and do we need the type?
- Ian: You'd have navigationTime and not navigation ID
- Michal: You could look back to any navigation entry of any type that
was the most recent one
- Noam: About Object, it feels like ECMAScript records, compare by
semantics
- ... Going by Michal, we already have what is unique in the timestamp
- ... In this case we know it's a navigation ID, in other cases in the
future maybe it's another group
- ... Semantic across the whole timeline could be useful
- Timo: About counter, is it a number that only counts in one direction?
- ... Or goes up, but not by one each time? There could be holes in it
- ... Allowing it to go down would allow flexibility in some cases
- ... For the retrieval, I see a similarity with other platform use cases
- ... For example, FCP, it doesn't have a dedicated type, you narrow down
- ... Not clear to me how large those entries if you narrow by type
- ... Want to be able to get the specific entry in a way that's not too
fragile
- Ian: We talked about having a supertype, but maybe we need
getNavigationById()
- Yoav: We have a ResourceTiming initiator effort, and we need to relate
one entry to another
- ... For that case counter doesn't cut it, we'd need a general counter
for entries
- ... If we count this by milliseconds, which some browsers do, there
can be collisions on single resources
- ... If we go with something more random, like type+name, that can add
randomness (but strings will be larger)
- Noam: For filter, we can get entries and collections in Iterable form
rather than Array form. JavaScript filter that doesn't have the overhead.
- Yoav: Filter would have to go over all entries to pick the one or two
it wants.
- Dan: I love iterators, but realistically, they're a dud. Maybe if the
iterator helpers happen.
- ... I can't think of a single API that uses them
- Timo: Maybe a race-condition that may or may not matter, and Iterator
that you hold onto for a while
- Alex: What is wrong with Counter on its own?
- Ian: Using it as a Counter, it's problematic. Expect things to show up
in a certain order at certain indexes.
- ... Whether we can just make it a general number might be harder
- Yoav: Example is
getEntriesByType("soft-navigation")[softNavigationId==2]
- ... What if implementation changes? Code can break
- Alex: Implies poor coding practices
- Yoav: People saw what looks to be an iterator, and used it as one. We
didn't provide any guarantees it'd remain the index
- Michal: If you can't use it as an offset, the "12th" entry in a list,
it might as well just be some number. startTime is a number that has an
alternative value.
- Yoav: Yes, but it's not enough on its own because we want an
equivalence with Resources which can share a startTime
- Timo: Assuming multiple navigation types, would there be an array
where it'd be an off-by-one index?
- ... Seems like there'd be a number of ways that wouldn't work
- Yoav: That works if you only have hard navs and soft navs, but breaks
with BFCache navigations
- Timo: How do we do that with setTimeout() today, do we start at 1000?
- ... Returns an integer that is unique, doesn't conflict with any other
timer
- Ian: Though there's no way to get a list of setTimouts outstanding
- ... We've identified problems with a lot
- ... Soft support and not strong objections with string identifier
- ... For filtering options, no strong opinion we absolutely need one
- ... But we may need a way to get the canonical navigation by ID if you
get it from another one
- Yoav: Once we have issues, we'll add to the minutes
JS profiling dual mode - Andrew
- Overhead from `Document-Policy: js-profiling`
<https://www.google.com/url?q=https://github.com/WICG/js-self-profiling/issues/79&sa=D&source=editors&ust=1699283057053271&usg=AOvVaw3Ngj7_70emy2xXXHoWR_8Z>
- Andrew: Context on API
- ... Sampling API that lets you collect samples from page execution
- ... Not unlike devtools profiler
- ... Had some concerns from vendors, excessive work require to setup UA
- ... V8 had to build mappings
- ... Spidermonkey needed some de-optimizations
- ... Suggested a document-policy header, when present, page would like
to request profiling
- ... Before this change, on prototype impl on Chrome, it'd take a lot
of time to build this metadata on critical path
- ... Core use-cases of API is to instrument interactions that were slow
and janky for real users
- ... Would have contributed to the problem
- ... On the flip-side, shipping of header itself incurs a cost
- ... For Chrome, you have to maintain this overhead
- ... Additionally, it's not as ergonomic to set as being able to
enable/disable profilter at runtime
- ... Desire for a dual-mode
- ... If you want to profile early in lifecycle, you could do a header.
If not, you could turn on at runtime (e.g. by page state)
- ... Major questions:
- ... (1) Does the use-case seem valid? Seems valid to me. Adoption on
some products hampered that you need to decide on a per-load basis
- ... (2) Curious from other UAs how they feel about the overhead
- ... One of the other concerns we had was that data would be built on
critical thread, potentially blocking interactions
- ... Maybe we could do this on another async thread, with tradeoffs on
accuracy for interactability
- Yoav: One advantage of document policy, is you can start profiling at
the start of the lifetime of the document
- ... Also gives first-party control over access of profiling
- ... In some ways, I don't think we should do away with policy, but
maybe another option?
- ... "lazy" profiling enabled, vs. "eager" profiling enabled
- ... "lazy" use-case of User clicking "get extra debugging info" where
a spinner starts profiling, then they can provide additional data about
their session
- ... Top-level origin needs to be in control, not have random widgets
on the page turn it on for their own purposes
- Andrew: With exposing to 3P scripts to do warmup opens a can of worms
- ... We could do an origin check
- ... Union of header options
- Noam H: Use-case was random idle profiling over duration of a page,
can you expand?
- Andrew: Satisfied with document policy, but could be optimized with
dual-mode. One thing we've seen is that it's useful to get a overall
portrait of JS execution time, on e.g. specific endpoint
- ... Useful for finding dead code, battery draining, etc
- ... Provide a heatmap
- Noam H: Set it, profiles at random periods?
- Andrew: Yes. Similarly used technique in server-side implementations,
even one sample is good enough to build aggregate.
- Noam H: For Excel we use a profiler in a similar approach, but we
write code that distributes sampling at random. Would be nice if that was
part of the API
- ... Configuration option?
- Andrew: Don't serve document-policy header, just ask with API to
record a span at some point?
- Noam: We start(), collect(), start(), collect(), use a random function
for determining how long we collect
- ... Over a large session base, that's probably enough for aggregate
statistics
- ... If that was part of API and I could start it once, provide samples
without me needing to randomize process
- Andrew: If you're going to treat as one span, you benefit from over
the wire wins from using same data structures
- ... Run for a while then pause() functionality
- Noam H: Then let me collect results periodically
- Andrew: Can we lock this to origin?
- ... Other options besides adding a new document-policy like variant
- Yoav: Theoretically we could have the same document-policy, but a JS
API that turns off profiling/overhead, then turns it back on w/ warmup cost
later on
- Andrew: You want a context w/ own lifecycle
- ... Scares me when always keeping it enabled on page load is those are
often the most important interactions
- Paul: I was only able to piece together things in history of the header
- ... Good amount of time when this API was available before the header
was there
- ... Concern about document owner allowing permissions, was that a
discussion point beforehand?
- Andrew: I think it came up in TPAC 2020?
- ... document-policy pitched as an ancillary benefit, for turning
header on
- ... In format you're describing it was never a shipping config, was an
Origin Trial
- ... We got negative feedback it was causing jank at the time
- ... Risk expensive operation on lower end devices
- Paul: Concern about 3Ps being able to do a warmup on their own, is the
side-channel attack?
- Yoav: My concern before side-channel attack, something to address, is
performance impact. 3Ps may not be aligned on incentives in user
experience. Can be incentivized to turn on this API if it gives them more
data while ignoring the cost on users.
- Dan: I agree with what you said. I see 3Ps kill BFCache just because
it makes it easier for them in some context.
- Andrew: Any precedent of eliminating APIs, requiring scripts of the
top-level origin
- Yoav: Only thing that looks slightly like this, is 3P origin trials
(Chromium)
- ... meta tag injection needs to happen from an execution task of a
script that comes from that registered 3P origin
- ... Don't know how we'd specify that
- ... New grounds for web-exposed API
- Andrew: Order of operations for least to most pain: make profiler fast
across all UAs, then next step being able to signal that yes profiling
allowed but not doing warming (in header)
- ... Need to get some concrete proposals together
- Dan: If I understand correctly, particularly problematic on some
low-end devices
- ... Agent might be aware of that, it's running on a constrained
environment
- ... e.g. don't enable that functionality if it's likely to adversely
impact
- Andrew: From perspective of publisher or UA?
- Dan: From UA. Preventing request reduce animation(?).
- Andrew: UA could control the ability of this API based on coin flip or
deterministic source?
- ... Strong hint from fingerprinting POV
- ... Problem where eliminating low-end devices have worst performance.
Maybe not disable entirely
- Dan: Lazy-loading where slowest devices are most conservative about
lazy loading content, where they could benefit the most
- Andrew: Remediations that are possible on low-end devices, find
issues, change sampling accordingly
- ... Lazy-loading is a terminal state
- Paul: Guidance to folks leveraging this API on what sort of overhead
to expect, and how to measure it for your own case
- ... Helpful to understand what the cost is to the user
- ... Then they can make their own decisions
- ... Is the warmup method that you pitched, seems attractive
- ... Keep header, add warmup method to have control over when that cost
happens
- Andrew: Big drawback is latency, you didn't profile before and you're
on a hot interaction you want to profile
- ... Definitely accomplishes a lot of the goals in the use-cases you
pointed out
- ... Control API at runtime
- ... Also breakage concerns of moving from prior API shape to this new
one
- ... We could revisit significance of LongTask now that some
implementations are more optimized
- ... document policy, enabling usage of API, warmup, invocation of API
itself could be blocking or instant if you did warmup
- Katie: Sounds like a good compromise
- ... Brings additional developer overhead, when enabling random
sampling of 1% of all traffic
- ... Need to ship right JavaScript that calls warmup() that ships the
document policy header
- ... Often-times those are two systems
- ... Way in JavaScript to check before you call it that logging is
enabled in this request
- ... Split adds additional overhead it feels
- Andrew: Agree
- ... Naive usage of API is decoupled
- Yoav: Either we can figure out a way to feature detect, or without
document policy does nothing, the rate is determined by the header
- Paul: Current feature usage for this API isn't so hot
- Dan: ServerTiming mechanism that you can see stuff in headers, maybe
that could be used in this context
- Andrew: Warmup primitive is interesting, ergonomics tough in some cases
- ... I'll write up the ideas that were discussed, and maybe we can
revisit later with some concrete ideas
- Nichttp://nicj.net/
@NicJ
On Wed, Sep 27, 2023 at 3:11 AM Yoav Weiss <yoavweiss@google.com> wrote:
> Hey folks,
>
> Hope you managed to recover from all the TPAC excitement!
> This week we have on the agenda
> <https://docs.google.com/document/d/10dz_7QM5XCNsGeI63R864lF9gFqlqQD37B4q8Q46LMM/edit?pli=1#heading=h.46y6uxx59b6u>
> an update on NavigationID, as well as a discussion on JS profiling dual
> mode, to enable some sort of a Just-In-Time profiling without incurring
> overhead in typical cases.
>
> See <https://meet.google.com/agz-fbji-spp?authuser=0> y'all there! :)
>
> Yoav & Nic
>
> P.S. The TPAC recordings and minutes are on their way. Apologies for the
> delay...
>
Received on Monday, 6 November 2023 14:09:48 UTC