- From: Dael Jackson <daelcss@gmail.com>
- Date: Fri, 16 Oct 2015 19:44:45 -0400
- To: public-houdini@w3.org
Async Style
-----------
- The spec has been re-thought in a manner to more closely
resemble how other APIs are being approached. In this new
approach you would have a registered compositor hook, let you
specify a bunch of code that had your hook, and we're writing
output and outputting myCustomState.
- Exposing information about the viewports will be added to the
spec.
- There's an outstanding concern about the data structure.
- There wasn't a conclusion about where in the cascade this
should go- the suggestions were with with or after Animations.
- Some people felt a flat list and a way to pass through the
hierarchy would be better then the tree structure in the
spec.
- Another large issue was what to do with a script that's running
long, either because it's poorly written or because there's
limited space on the compositor. There was a general thought
that whatever happens there should be a notification so
developers can fix janky code. Options for the script that
were:
- Just kill it (no one really liked this idea).
- Have the browser ask the user if they want to kill it.
- Move it to the main thread automatically.
- Give the developers a method to write fallback behavior on
the main thread for if their original script doesn't run
in time.
- Lower the frame rate/leverage the statistical behavior so
users don't see too many dropped frames
- A motivation/use case needs to be added for why one would want
to have a post-commit hook that could get sent the output
after you collect information to send to the main thread
- There is a question as to if this should be done with JS which
gives you more power but more ability to get into trouble. It
seemed as though the group leaned toward this approach, but
there was no firm conclusion.
- The name async style was agreed to be misleading and/or
confusing. There was no resolution on name, so the options
from the group plus a create your own name field will be
pulled together into a poll. Options put forward are:
- Custom Composite
- Threaded Effects
- UI Workers
- Async UI
- Custom Animation
- Threaded Animation
- UI Animation
- RESOLVED: Change the name of Async Style to something else,
pending the results of a poll to be made by TabAtkins
===== FULL MINUTES BELOW ======
scribe: dael
Async Style
-----------
<vollick> https://docs.google.com/document/d/1xhuwlwdoqox3Fo3e4zFltgSGnbtMb-ptTYNx1loqPXU/edit#heading=h.f6va9m4yz8xn
vollick: Async style...it needs a better name. I was suggesting
'custom composite'. Does everyone agree?
dbaron: You should tell us what it is before we name it.
vollick: That's an excellent point.
vollick: It's allowing authors to write their own threaded effects
in JS so you wouldn't be limited to the accelerated
animations we have today.
vollick: We're doing custom work on the compositor thread.
esprehn: Let's start with use cases.
vollick: So a physics-based animation. You have to send something
to the thread, run some script on the thread, and muck
with some set of properties.
vollick: When I presented before I had requestAnimationFrame on a
compositor worker and users could run it on whatever
thread. It didn't look like the other APIs. Previously
you would create on the main thread a proxy, post that to
the compositor worker, and you would schedule a raft
callback
vollick: To make this more like the other custom APIs you would
have a registered compositor hook, let you specify a
bunch of code that had your hook, and we're writing
output and outputting myCustomState.
vollick: Is this clear?
dino: Can you expand where it's outputting?
vollick: In this example...this is less than fully baked. We're
not outputting myCustomState. If you're doing a physics
simulator and want to update your velocity you could do
that. If you want to do something like raf you could put
a ticker.
smfr: So this is different from the other hooks where you can
store state?
vollick: You have to stuff it on the other items you pass in. I
did image; they're custom properties.
zcorpan: I don't follow how the string that sets ancestor scroller.
esprehn: This is the thing I mentioned earlier where you have a
fictional syntax for custom dependencies if you're not
CSS.
vollick: So say you want to implement sticky. You care a lot about
your ancestor and your scroller. We've just provided
those as things available on the nodes passed in.
smfr: So sticky would require the border-box and containing-block.
So there could be any number of magic things depending on
what authors want, so what's that list?
shane: Ancestor-scroll and containing-block were what we thought
of as meaningful in current layout. If we do want more
inputs and the list is too big we need another way.
flackr: We had determined you could have an apply hook on the
overflow that would descend down.
shane: It's hard for scroller and containing-block because there's
multiple types.
shane: You only do a * recalc when you change style.
smfr: You can send down the scroll-top as a custom property.
flackr: No, the scroller element.
heycam: This looks different from other things where you're
running the page's script where as the apply hook has a
special script element that you evaluate on the other
thread.
vollick: This would be installed in a CSS worker. It's in a
separate execution context.
rbyers: On a thread in sync with the compositor.
Florian: Is this a rehash of UI workers or it works with them?
vollick: UI worker was a different API to this. It's a rephrasing
of that concept using an API that looks more similar.
We're not meant to have both, this replaces UI worker.
smfr: So you transform the style map, are you limited to transform
opacity?
vollick: It would be great to add more to the list. In the
previous proposal you would fail to create a proxy. If
you add to the outputs as something illegal you can put
something on that. We're looking at things that are the
intersection of what browsers can do quickly, but I don't
see that as the list for all time.
zcorpan: Do you expose information about the viewports?
vollick: I don't but that's a great thing to add in. I have to
think about that and come back. It needs to be added to
the spec.
vollick: Something else to sort out is how do you figure out your
position in relation to another element
Florian: Out of all the custom APIs I think this one needs to have
the lowest latency.
vollick: That's true. My hope here was just to give a tour. I have
some big open questions on this. Do you mind if we come
back, it is important.
smfr: When you set properties on the *-map, things never go back,
right?
vollick: We're implementing this in a non-web exposed way and we
do pass them back. We will mutate them here.
smfr: They'll pass back asynchronously?
ojan: The best example is like how we do accelerated animations
smfr: You get it back with a bit of fuzz immediately, though.
Also, if I'm running an animation on transform, does this
win?
vollick: Yes. You would get a stale value back, but it's an
approximation of what's on the thread.
vollick: The first issue is where to run these things.
vollick: Should we run in paired down execution or dedicated
global scope?
vollick: Would folks object to doing this in a similar way to
layout and paint in a paired down worker?
smfr: Does that preclude the animation frame?
vollick: No. They can both exist.
smfr: This seems like a case where you want to store state or have
an object based formulation where you can have a complex
calculation up front and store.
vollick: So you could reuse results between implications. Yeah, I
would need to put more thought into how to do that. We
had in mind you would store your persistent state on the
node objects that are passed in.
ojan: I was picturing...this is part of the problem yesterday that
all the custom properties need some steady state and we
don't have an set answer.
smfr: So the composite hook takes a tree?
vollick: You get a sparse tree that matches the topology of the
DOM.
vollick: This is not a complete example.
vollick: The myHook bit, I would style with compositeCoin.myHook.
you would just get the node itself.
smfr: So it's a DOM order tree so it doesn't tell you stacking
context or z-order. Can you get that yourself from the
styles? Like position: fixed inside a transform doesn't
behave like position: fixed.
vollick: It could be complex. This comes back to having only
access to ancestor containing block.
smfr: I worry that this is adding complexity and it would be hard
to use the tree for authors.
vollick: Would having a list be better?
smfr: If you only had your current node you couldn't coordinate.
vollick: We wanted to do a physics based API so people moved
together and that would be hard.
flackr: You need to know about the elements that contribute to
snap on the ancestor that's doing scrolling.
dbaron: How does the output produce scroll position so you can
implement snap points?
vollick: We take the output from this call. We would apply that
before we produce the visuals and asynchronously send it
back. I do have some magic outputs.
esprehn: We think the set is fixed, it's what we do for animation
and scroll.
shane: It's reducing the amount of magic because we're describing
the things.
dbaron: And the magic scrolling outputs go back to the main thread
and the CSS does not?
vollick: They do. You can query these, but they'll be stale.
esprehn: Same way animations work.
TabAtkins: Like if we animate opacity it will work, but you might
not get the right value if you query.
smfr: When you call get.style we do the interpolation.
shane: We maintain one frame out of sync thats what you get when
you call back.
dbaron: Animations have a place in the cascade so this would need
a place as well.
smfr: Which is right at the end.
TabAtkins: I would hope so.
esprehn: I could be the same as animations.
Florian: This would override animations, but not !important?
esprehn: This has no more power than animations.
TabAtkins: I think we don't do that but it's difficult to observe.
UA has a bunch of things in there, but not ones that
you animate.
dbaron: I believe Gecko does that, but I forget exactly how. I can
take a look.
TabAtkins: We could hack that in if it's necessary.
esprehn: UA !important overrides animation? We should fix that
anyway.
<Florian> https://drafts.csswg.org/css-cascade/#cascade-origin
Florian: I just pasted what overrides what.
<zcorpan> example of !important in UA stylesheet:
https://html.spec.whatwg.org/multipage/rendering.html#hidden-elements
esprehn: Back to the tree question, it's useful in a lot of cases
where you want to do complex animations. Like where cards
are bouncing all around. This would like you own the card
and take all the pieces of the tree and inside here you
could animate in a coordinated way
smfr: The only way you could change the z ordering is 3D
transforms?
esprehn: Nothing in there contains your paint order. Once you have
one of these your paint order is set.
vollick: If they exist in the same 3D context you should be able
to swap.
smfr: My concern about the tree is the special magic thing that
authors don't understand. It seems adding more complexity,
but I understand why.
vollick: We have to make a justification for why it's useful.
ojan: It's the author that's creating the tree. it's where they
put the hook.
smfr: I'm not convinced that they would think about the tree-ness.
I wonder if a flat list and a way to pass through the
hierarchy would be better.
esprehn: We should add an issue to think about the data structure.
vollick: I'm going to switch to what to do when things run long.
vollick: We're sensitive to time on the compositor, so what do we
do when there's too much work to do or a hook that's
badly behaved?
vollick: There's two types of failure that will need different
mitigation. One is too much work to do, we're overloaded,
and one is a badly behaved hook
vollick: When things run long it seems like it would be beneficial
if we let the hook run to completion and not add it again
to the schedule. We're trying to avoid killing a context.
ojan: I think that stands on the question how we do the cache API.
Presumably we want something to persist when we kill the
workers.
vollick: So we shouldn't talk about that if it's not decided. Do
we kill it or let it run to completion?
Florian: So it depends on if you're on a low end device and the
browser can't do it or if you're on a high end computer
and it's running too slow. The alternative doing it in a
JS way would be slower, so I think it's beneficial to let
them run.
smfr: If you kill a worker you get something janky so it will look
bad.
smfr: I think the differentiation between crazy and not crazy
isn't good because if it's run for 40ms you don't know if it
will run for 400ms more.
vollick: My hope is when we do take one of these actions we
communicate to the main thread that you failed and you
can have something to do in that case. You try and get in
and you get rejected and you try on the main thread.
vollick: I think you would say we couldn't accommodate you and you
have to cope on your own and have have to take action.
rbyers: That can mean not having some effects.
vollick: That can be up to the author.
dino: I feel like that provides for something that doesn't run to
the budget they can issue a new page, I'm trying to think as
a dev that if I'm going to have to listen to if this works I
have to write more code.
Florian: At least you get informed you didn't run.
dino: If you didn't run and you throw and exception you're
instant-broken. If you didn't run because you're slow...
there's plenty of ways to write slow script, why is this
different?
ojan: Yesterday both of you were wondering if you want to run more
script, this is trying to respond to that. I'd like to stick
with if you write crappy code, you write crappy code. This
responds to that concern. Would you ship script on the
scrolling thread?
smfr: Scrolling is supposed to be fast.
ojan: My hope is there's so little you can do there's little you
can do for slow.
smfr: There's ways.
rbyers: If we build this so there's culpability and reporting,
this is our chance to have a new place from the beginning
where you can reason about performance better.
smfr: Developers don't test on all devices.
gregwhitworth: That was my main thought because all your use cases
are paramount to not fail.
gregwhitworth: I don't know of a good solution, but I don't know
if I want it fails so it fails.
vollick: If I was writing a sticky library and it failed, I would
write a backup on the main thread. You want to minimize
the number of footguns and if you do still hit them we
have way of rejecting.
Florian: If we have multiple hooks and one is too slow, you might
want to bail on all the hooks.
smfr: So you register as groups.
vollick: I think the groups was presuming it would be automatic,
but Florian was suggesting you could turn them off.
smfr: Is there a reason the UA couldn't run this on the main
thread if it was too slow?
esprehn: It might be desirable. If your page is too slow you lose
the privilege to run in lock step with the compositor.
vollick: Yeah, I need to think about that.
dino: But you don't have to write the fallback for that. So you
have to toggle something in the developer tools to do the
fallback.
vollick: That is true of any time you're writing a time sensitive
program.
rbyers: What if the fallback is what you do anyway for feature
detection? We want to make this distinction up front. We
have to decide with load if this page has access to the
full screen. When we find sites that have crappy
performance we want to say this isn't available.
vollick: Perhaps the next thing to do is try an experimental
implementation to see how they do.
vollick: We could also lower the frame rate. That wouldn't require
you to dump back to the main thread.
zcorpan: Deciding up front if the API is available, it seems to me
the performance can vary over time depending on what else
is happening.
rbyers: We're worried about a statistical thing. We won't prevent
every drop frame, but we want to make sure most people
aren't seeing a lot of drop frames all the time. Maybe we
can leverage the statistical behavior. We're trying to
solve the problem in large.
smfr: So do you move things between threads dynamically?
rbyers: If one user doesn't get the feature that's not that big of
a deal, but you don't want the developer to reason about
it being different every time it loads.
Florian: When you're writing code for soft real time where it's
good to be fast but if you blow the budget it's fine or
hard real time where you have to fit in the frame budget
or you're killed, you'll build different ways for those
two approaches. The coding approach people will take is
different so we have to decide on which.
esprehn: A hard system we can't do with JS.
vollick: We're soft at best.
Florian: When you say we'll kill you that's a hard system.
vollick: I didn't intend to suggest we would know if you run
forever, but if you exceed a certain amount of time we
kill you.
zcorpan: If you do while true on the main thread, browsers can do
that but the time is much longer. Does it need to be
shorter here?
vollick: It would be a strange case, but it could be the same.
zcorpan: I think most browsers ask the user if they want to kill.
vollick: Doesn't seem crazy to me. That's seems reasonable. I'll
add that as an option.
vollick: There's the when to run. I think this is a moot point and
not interesting to discuss. We'll be trigger if your
input parameters change or you returned true to signal
you want to run next frame.
vollick: Communicating changes into and out of is interesting.
What if you collect information to send to the main
thread. You could have a post-commit hook that could get
sent the output. Would anybody have object to that?
dino: Can you give an example of what you would do?
vollick: You may want to say where something was. Here's all the
places the compositor was.
dino: And what would make a decision on what the output was.
vollick: Yes.
dino: It would be interesting to have an example. Like here's an
animation I want to do, I want the compositor to run as fast
as you can, and once in a while I want to know what's going
on.
shane: That's happening right now. They want to know that for
things like scroll-top.
shane: Do you have a specific example, vollick?
dino: I don't object, I just want an example added.
vollick: I'll add a motivation for this.
vollick: So I think the last thing I was going to touch on is the
difference between the first draft and the future plans.
Initially I was hoping to provide the ability to
transform opacity and respond to custom property changes
from main thread and be able to do raf-like animations. I
wasn't going to tackle getting inputs set or driving
scroll customization. I'm hoping to do those in the
future. If we can get a hook of the main shape adding
those will be easy.
vollick: I was wondering if folks thought this was important
enough to stick in or keep the limited version?
dbaron: What is the distinction for features?
vollick: If we had the ability on one of those proxies to register
a touch handler you could implement a drag to refresh
quite easily.
ojan: Basically touch input.
rbyers: And scroll.
dbaron: How does the scroll differ from what it has?
rbyers: It can read and write to scroll top which is good enough
for these scroll events, but the more sophisticated things
like snap points add a level of complexity that is best
postponed.
ojan: Next thing on the agenda is scroll customizations
gregwhitworth: Can you speak to the existential thing?
vollick: So it's do we want to use JS for this. If gives you a lot
of power but opens you to some footguns. If we had a
domain specific language we could both be guaranteed
performance and not deal with other crud. Though this
would meet a lot of use cases, it doesn't catch
everything so I would err toward JS thing because it gave
us more power.
Florian: It also depends on the kind of DSL, but the more we run
toward hard run time the more we should go toward DSL.
gregwhitworth: My issue with DSL is that's what C++ is. For here
snap points isn't possible.
rbyers: Coming up with a simple declarative language isn't rich
enough for snap points.
Florian: I think you could have a imperative language that could
express some things, but you could run forever. When
you're in it you can predict how long it would run.
vollick: You would admit it's a huge can of worms.
Florian: Yes.
vollick: If you couldn't service your hook in time you could
reevaluate with the new value of scroll top. That's
something in between where you run your JS callback and
your DSL.
rbyers: We have this DSL, it's CSS, and authors should get used to
the idea that ...for example we're standardizing snap
points and people should use that API. Our implementation
is bounded and we know it works well. Once we've promoted
this, you don't have to be a performance wizard to know it
works.
smfr: That reminds me, the user many have their own hooks so we
need to define who runs first.
rbyers: It's more complicated. I might have several different
scripts and who runs first.
dino: No two hooks can run for the same element.
rbyers: I wonder if that's a problem.
rbyers: We're going to implement snap points with these hooks and
are we saying you can't have these hooks?
esprehn: This is a general problem with CSS.
shane: We need to fix the accumulation problem in CSS.
ojan: So to be clear, the CSS problem with where you set transform
here and there's one that needs to override.
TabAtkins: We need to fix it for real.
dbaron: We've discussed this a number of times
<TabAtkins> In particular, fixing this *properly* gives us an easy
solution to the "can't have two apply hooks outputting
to the same property", at least in the case of list-
valued properties.
<TabAtkins> You'd just output in whatever fashion invokes
"additive" (if you're okay with that), and the
combination is handled automatically.
<TabAtkins> Note: the fundamental "additive" problem is ordering.
*That's* the thing we need to solve, dbaron. The rest
of the details are trivial.
<dbaron> TabAtkins, we could have a "passthrough" (better name
needed) keyword that's a special keyword that goes within
lists in list-valued properties
<TabAtkins> dbaron: That's relying on specificity to handle
ordering. That's not great in the first place, and it
doesn't help in places without specificity, like apply
hooks.
<TabAtkins> dbaron: It also doesn't let you *override* a component
when you want; it's *purely* additive, which is imo
too weak.
<dbaron> TabAtkins, named *and* numbered components seem kinda
heavyweight...
<TabAtkins> dbaron: Agree. :/ Tho I hate it, maybe François' idea
of auto-ordering by lexico order, perhaps with some
way to supply a numeric order to the names when
desired (with ties continuing to tiebreak with lexico
order).
heycam: Going back in time to the card with elements flying off,
it sounds like you have one of these hooks that's set with
composite on the parent and all the children, or for you
have separate hooks for children and parent?
vollick: You'd have two input properties of the parent and the
adult and style both with myCrazyExplosion hook.
vollick: So if I've got this card and the children will explode
around and I have to specify composite:explosion I can't
do it with another thing. Oh, so I couldn't be the child
and the root exposer.
shane: It's probably two problems. If you want multiple composite
effects you need to be able to inherit from two different
CSS rules. The other is solvable, I think, with a more
sophisticated library where you specify which are parents
and which are children.
heycam: If the composite property takes a list and gives it
myEffect twice you need to indicate in the custom property
that you'll have the parent and the child and work out
which call is which.
Flackr: If you flatten to a list it's hard to figure out which
child is to which parent.
heycam: You've got some parents and some children using the same
effect keyword, would this get called once for each and
for the middle level you have to manage multiple?
vollick: Yes.
vollick: This is hard cases of assigning behaviors to elements.
This is a straight forward way. I think the issues are
solvable and something we have to outline if we do the
draft with this style
heycam: It makes me think composite is the wrong way to put this
up.
rbyers: The main issue is consistency with the rest of the Houdini
APIs and does that matter?
ojan: I want to clarify, what sorts of things is the UA doing for
hooks?
smfr: The UA might use a hook for scroll snap or sticky.
smfr: One high level comment, it feels different than the other
hooks. It has this tree and you talked about pre- and post-
commit and I feel like these Houdini hooks should have the
same API surface.
vollick: And we wouldn't be able to have a tree through that.
smfr: Maybe there's a way to ask tree-like questions.
esprehn: Custom layout has a list.
Rossen: It's pretty different
shane: I agree with the principle. We'll inevitably find different
requirements, but it's an important principle.
Rossen: Anything else on this topic?
vollick: Nothing from me.
Rossen: We had one more topic before lunch.
dino: What are the next steps on this?
vollick: I can get an ED ready before TPAC.
shane: We have a resolution from Sydney to work on something like
this. The name is horrible.
smfr: I don't like composite.
heycam: Async style describes it better.
smfr: It's about scrolling an animation.
bruce: The original name I had no idea, but custom composite I got
the idea
ojan: Threaded Effects.
Florian: UI workers
ojan: Threaded effect is too general.
smfr: I don't think authors know what compositor means.
rbyers: Web authors care about performance. What does Safari show
in the time line? We have painting and composite separate.
smfr: We added composite time but it's not really accurate.
rbyers: We've been training them that transforms is fast. Maybe we
should be teaching them a term?
Rossen: leaverou do you have an opinion?
Rossen: The Async Style, we think it's a terrible name. One
proposal was custom composite, but smfr is concerned it
won't mean anything to devs.
leaverou: Won't it be confused with compositor spec?
TabAtkins: Why don't we put up a poll with a choose your own name
box.
Florian: It shouldn't be something-something-style-something
because the style stage isn't this one.
shane: I agree.
ojan: Async UI
smfr: It's sync with scrolling.
flackr: I was thinking Custom Animate.
rbyers: Threaded Animation
smfr: UI Animation
<dbaron> I'm happy with custom animation.
TabAtkins: How about we resolve to change the name, I throw up a
poll?
RESOLVED: Change the name of async style to something else,
pending the results of a poll to be made by TabAtkins
dbaron: This document looks like it's world comment-able, is that
okay?
<dbaron> ok for sharing widely (and I thought it was
world-editable).
rbyers: Yeah, it generally works out okay. Once in a while I get
some spam.
<break=lunch> return 1:30 Paris time
<franremy> @TabAtkins, @dbaron: I made a better proposal later on
than just using lexical order, based on "cascade"
keyword @dbaron was talking about
<franremy> (but I can't find it back)
<franremy> (i'll do some more research in the archives and come
back to this; maybe ping me if I forget)
<franremy> @TabAtkins @dbaron: ok, found the mail I was thinking
about:
https://lists.w3.org/Archives/Public/www-style/2014Apr/0013.html
<gregwhitworth> franremy: that seems interesting, it also models
how you do collections in html forms and of course
js for that matter
<TabAtkins> franremy: Nothing based on cascade specificity will
help with resolving the apply-hook ordering/collision
problem. ;_;
<franremy> @TabAtkins: that's why there's also the "priority"
factor in there, in addition to cascade specificity; I
propose cascadeOrder+name+value+priority tuples
<franremy> @TabAtkins: (with priority taking over cascadeOrder if
defined, of course)
<TabAtkins> Ah, I missed the priority part. I'll think on this.
<TabAtkins> franremy: Okay, reviewed it more. Still problematic -
its ultimate tiebreaker is still cascade-based.
<TabAtkins> franremy: We need a final tiebreaker that is inherent
to the items.
<gregwhitworth> franremy: open up an issue on GitHub about your
idea
franremy @gregwithworth: Yeah, I will do. I think advancing
towards a possible cooperation between the main-thread
and the css-thread would ease some of my unease with a
few specs by allowing them to provide more "immediate"
value at the cost of less accuracy or more latency when
used in this cooperative way, though way less than what
only-the-main-thread faces currently.
TabAtkins franremy: I think I'm landing back over on "just use
numbers", but use hierarchical numbers so you don't have
to do stupid tricks like starting out with 100, 200, 300
to "reserve some space".
TabAtkins franremy: That's super simple, and doesn't invoke
specificity at all, and is the most similar to just
defining longhands for any map-valued property.
franremy @TabAtkins: something like "0" < "1" < "1-2" < "1-2-1" <
"1-2-1.5" ? that's less semantic, but I can see the point
of the tradeoff; let's kick some discussion on www-style
about this, maybe?
Received on Friday, 16 October 2015 23:45:44 UTC