- From: Dael Jackson <daelcss@gmail.com>
- Date: Sun, 5 Mar 2017 10:59:56 -0500
- To: "public-houdini@w3.org" <public-houdini@w3.org>
- Cc: www-style@w3.org
=================================================
These are the official Houdini Task Force
minutes. Unless you're correcting the
minutes, please respond by starting
a new thread with an appropriate subject line.
=================================================
CSS Painting API Level 1
------------------------
- RESOLVED: Move all Houdini APIs previously on window to
window.CSS
- iank introduced his solution to addressing when multiple worklet
global scopes are registered with the same name.
- Text around memory constraints will be added to the spec.
- There were some lingering concerns, but iank will add the
edits into the spec for people to review actual language.
- iank will add links to examples and details from the explainer
into the spec.
- Using JS to registerPaint for foo and calling it in one global
scope but not another will be added to the spec as an error
case.
- RESOLVED: New WD of Paint after edits
===== FULL MINUTES BELOW ======
Agenda: https://github.com/w3c/css-houdini-drafts/wiki/Seattle-F2F-Jan-10-2017
Present:
Rossen Atanassov, Microsoft
Tab Atkins, Google
David Baron, Mozilla
Bert Bos, W3C
Bogdan Brinza, Microsoft
Emil A Eklund, Google
Elika Etemad, Invited Expert
Koji Ishii, Google
Ian Kilpatrick, Google
Peter Linss, Invited Expert
Jack Moffitt, Mozilla
Simon Pieters, Opera Software
Naina Raisinghani, Google
Hiroshi Sakakibara, Beyond Perspective Solutions
Elliott Sprehn, Google
Alan Stearns, Adobe
Shane Stephens, Google
Surma, Google
Jet Villegas, Mozilla
Philip Walton, Google
Greg Whitworth, Microsoft
Chris Wilson, Google
Scribe: fantasai
CSS Painting API Level 1
========================
Issue #330: https://github.com/w3c/css-houdini-drafts/issues/330
----------------------------------------------------------------
iank: From TAG review:
iank: We added attribute paintWorklet[?], TAG feedback was should
this be on window.CSS instead of Window.
iank: There was previously some hesitation to add there, because
was for utility functions like supports.
Rossen: What swayed TAG?
dbaron: General concern of adding things to Window, because also
pollutes global scope.
dbaron: It certainly doesn't have to be .CSS
dbaron: Suggestion was, can this be something other than Window.
TabAtkins: .CSS sounds find.
esprehn: If ppl use worklet, won't collide.
plinss: Maybe something layout related rather than just CSs, to
future-proof?
fantasai: window.layout?
plinss: Keep .CSS for css-specific functions.
iank: Also question of moving registerProperty from properties and
values spec.
iank: If we move one, should probably move all of them.
iank: Not hearing any strong objections to moving all of the
worklet stuff to .CSS
fantasai: Just have one point about whether to move to
.somethingElse rather than .CSS
zcorpan: These are properties?
zcorpan: There was a proposal to make .CSS into a webIDL namespace
zcorpan: Right now only support methods, but could change webIDL
spec to allow properties.
TabAtkins: Seems like an easy fix to me.
iank: Do we move things like registerProperty to CSS namespace as
well.
fremy: Makes sense to me.
iank: Any objections?
Rossen: No
RESOLVED: Move all Houdini APIs previously on window to window.CSS
<zcorpan> https://github.com/heycam/webidl/pull/121
Threading issues
---------------
iank: Next issue was new threading behavior for multiple worklet
global scopes on different threads, potentially.
iank: Spec is registerPaint and draw??
iank: Summary of what happens how...
iank: Every time you register paint function against global scope,
iank: can happen in multiple global scopes.
iank: Each time you do that, potential for two different global
scopes to register subtly different classes under same name
iank: e.g. possible for someone to register a paint class under
the name foo, possible that it can have two different class
structures in the different global scopes, e.g. one
invalidates on property one, and one invalidates on property
two.
iank: Behavior here is that once inside of the global scope, we've
registered, created a new definition. We queue a task
function on the document, which will then insert into the
map.
iank: This checks if the input property exists on that map. If it
does, checks if they are exactly the same.
iank: If they don't match, sets that entry to invalid.
iank: If you try to use that name now, it gets treated as invalid
because you tried to register two different things under the
same name.
fremy: Why allow this at all?
fremy: Why not restart the worker?
iank: Can't do that, b/c on different threads, so racy.
iank: Need to have an essential truth, which is the document's
version.
iank: If you wanted to do that, you'd have to acquire a lock.
fremy: Could write a promise.
fremy: When you register, you get a promise. If it registers, it
resolves.
iank: It's still racy.
iank: I think this is pretty simple.
iank: Might get a case where valid where previously invalid.
iank: This mainly impacts Firefox and ?
jack: We have some concerns about worklet multiple scopes thing.
jack: Simon filed a bug asking for normative language asking for
this to be ?
jack: As someone ships an impl that allows a single scope ...
iank: We have a bug I haven't fixed, we need to add to spec threat
there needs to be at least two global scopes per
implementation unless there's memory pressure.
<astearns> Issue iank mentioned:
https://github.com/w3c/css-houdini-drafts/issues/327
jack: Concern about that solution is that it's a hacky way around
the problem, which is that you don't want the worklets to be
able to pollute the scope.
jack: Is there some other way to achieve without doing this round
robinning?
jack: So ppl don't depend on ?
iank: Other way to do this is do something like freezing the
globals inside of the worklet global scope.
iank: I don't think that's a good situation, doesn't really solve
? problem
iank: ...
jack: Can't ensure that each worklet, when it starts executing,
has a fresh scope.
esprehn: No way to do that. JS is the wrong language for that.
esprehn: Also hostile to polyfilling.
jack: ...
esprehn: Oh, so you start/stop the whole system each time?
jack: If some worklet polyfills in a scope, then another worklet
runs in that same scope, has a bunch of polyfills it didn't
want.
esprehn: Start/stop cost and memory overhead is to much for that.
jack: If this becomes a web compat issue, that they have to run in
the same scope, will prevent any parallelism possible in
Servo in the future.
esprehn: Shouldn't be as bad as it seems.
iank: Added many things to mitigate that.
iank: All run as modern scripts instead of classic scripts, so
don't have access to the self attribute, similar to workers
iank: which allows you to access the globals.
iank: It's possible to stash stuff on the Array prototype
iank: but I think that having ? normally applying, at least two
global scopes is the best we can do.
jack: Except for memory pressure...is that just gonna be wording
in spec?
jack: Is it expected that Android version will always have one?
iank: Something with 512MB of memory, e.g.
iank: Does that address concerns?
jack: I'd say we're still concerned, but will wait to review spec.
iank: [gives sample wording]
jack: Simon's bug also mentioned, need a way to pass state to ?
jack: The issue we want to prevent is stashing state in global
scope, then depending that that's there the next time you run
jack: But e.g. animation worklet needs to be able to stash state,
will there be a special API?
iank: Killing an instance of paint class and restoring it should
be okay, don't expect much state.
<dbaron> what's being shown on the screen is the end of
https://drafts.css-houdini.org/css-paint-api/#registering-custom-paint
iank: Expect that people in constructor will build up a canvas ?
iank: Inside the animation worklet, want state from frame to
frame, which is not true for paint or layout.
iank: Many ways to solve, maybe callback "we're going to kill
global scope, save state into this object, when we construct
again will bas this object in"
iank: That is a possible mechanism for this.
iank: Custom properties was mentioned as a mechanism, I don't
think that would work.
jack: Don't have a particular preference, as long as there is a
way to do it so that people don't do crazy things.
iank: We'll see once we ship if ppl stash state.
iank: I don't think they'll do for paint or layout.
iank: Might do some caching inside structure, that's about it.
iank: Does that match your intuition of what devs might do?
dbaron: One question I have about the thing you're inserting:
dbaron: What normally happens when you register two different
things under the same name, and why is one piece of it
different?
iank: Previously, the spec language was, at the very top it would
check entering the properties map ...
iank: Was wrong, because if two were going one would fail.
iank: This registers everything on the global scope, does a
cross-thread jump to the document.
iank: And basically does a check. Gonna try to add its set of
painting input properties to the map
iank: If they're different, will nullify that class name.
iank: You tried to register "foo" with two different input.
dbaron: Doesn't there need to be some other global thing that says
how to paint a particular image?
iank: That's per-worklet scope.
dbaron: So if you have, e.g. background: paint(something), you go
ask multiple worklet global scopes?
iank: What you do is, you do a bunch of work on the document to
see if that's a valid ?
iank: Grab the input properties map
iank: If it doesn't exist, it's invalid.
iank: If it has the invalid flag, treat it as invalid image.
iank: At that stage select a WorkletGlobalScope.
iank: Then do a cross-thread jump, and then painting on that
thread.
dbaron: Partly I haven't really seen examples with the whole thing
together.
dbaron: Last time I looked in spec, didn't see those.
dbaron: Wasn't clear, what makes that.
dbaron: I just don't understand how this fits together and it
still doesn't make sense.
dbaron: If you're saying the identifier that you're registering is
registered from paintWorklet, what makes that paintWorklet
exist in the first place.
iank: Up to the UA.
iank: UA must have at least two running.
iank: When you import this function it, makes those local scopes
exist.
iank: When you call import on the script, it will import that
script into the two worklets.
dbaron: And then that script will call register.
iank: Yes.
iank: Language to ensure global scopes exist, and then run.
dbaron: Feels weird to me that registration happens inside worklet
rather than outside.
iank: I don't think you can do it a different way, since it has to
be local to ?
iank: You can't register class in one thread and use it in another.
iank: If you could it would be amazing!
<dbaron> Yeah, I agree it doesn't make sense... but it still seems
weird.
smfr: Custom paint is running on separate thread.
smfr: How does author synch custom paint with other things in the
thread?
iank: On import, once the promise returned from import has
resolved, the spec language is that ? and everything should
be set up, and in next frame if you use custom paint with
that name it should work.
smfr: Wondering if you can request animation frame, change content
in the document in the normal way, and also have the paint
worklet, and have the results show up.
iank: Since you don't know if there's going to be a network round
trip when you import, we have to wait until promise resolve.
smfr: First time, subsequent times can be synchronized.
iank: Right.
smfr: So subsequent paints are still happening on the thread,
though?
smfr: So UA has to waiting until custom paint ?
iank: Promise will return once all scripts have run.
iank: That includes doing this potentially post-message, hey
document, I have registered paint function.
smfr: you get some small amount of parallelism running the script
iank: All run synch immediately, but UA can choose ...
jack: Circle can run each iframe already its own separate thing,
not in the same context.
jack: This means you have multiple things that might be
generating, painting...
jack: already happens in Servo.
iank: We're doing similar thing with out-of-process iframe.
jack: Notice that this paint in particular, doesn't have much in
the way of motivating examples. Some small examples in the
end, like painting single conical gradient. Is there going
to be more?
iank: Explainer in github
<astearns> https://github.com/w3c/css-houdini-drafts/blob/master/css-paint-api/EXPLAINER.md
fantasai: Any reason not to have that in the spec?
iank: Could pull it into spec.
jack: Get a link to spec, read it, seems okay but what is it for?
astearns: Explainer doesn't seem to have that much more in terms
of motivating examples.
jack: Might be interesting to know what is the planned use for
this?
jack: What feature were you trying to add to gmail that lead you
down this path.
esprehn: Ink drop effect in material design,
esprehn: The puddle effect,
esprehn: people do crazy things to make that work.
esprehn: You want to just specify center of ripple.
esprehn: Currently do crazy things with expanding divs
<astearns> codepen for inkdrop-ish effect:
http://codepen.io/440design/pen/iEztk/
jack: So you'd register paint worklet on the button, but then
don't have mouse coords
esprehn: JS has that info,
esprehn: Because input of painting takes variable.
esprehn: You can do background-image: paint(ripple) and then use
vars to pass things into painting.
TabAtkins: We also added arguments.
iank: I can add it to the spec.
fantasai: Just put it all into the spec directory, even if not
inlined into main spec HTML file, can be linked from it
and all published together as a package that explains
itself.
iank: We were seeing ppl dumping huge amounts of DOM to get these
effects.
iank: I'll add that to explainer
iank: Final part of thread-hopping thing
iank: is once you select a worklet global scope to invoke paint
function on, that workeletGlobalScope may never have seen
that registered class, in which case it does ? back to
document, saying this thing is now invalid.
iank: Might have a racy thing where it works once but not
subsequently.
iank: I think it's fine, something author shouldn't be doing
anyway.
dbaron: What is it author did wrong there?
iank: Author did is in JS that registerPaint for foo didn't get
called in one global scope, but did in another.
iank: Worklet global scope that was used for painting may not have
seen foo.
dbaron: E.g. if they imported multiple scripts and they loaded in
different orders in different worklets?
iank: No, worklets spec always loads in same order. Did something
nasty, they imported one script, a paint function got called
that set a global variable on Array, and a subsequent script
checked var on that array, but then didn't register foo.
iank: Will work on one frame potentially, but not subsequently.
jack: What is the error behavior here?
jack: Seems like something to prevent in general.
jack: Does the spec say UA should issue a warning?
iank: Yeah, that's a good point.
ACTION: iank make this case an error
<RRSAgent> records action 1
Namespaces
----------
iank: ... namespaces
iank: A few minor edits.
iank: Other major addition to spec is input arguments.
TabAtkins: Paint function can pass arguments, instead of having to
rely on variables. Can use more than one instance of
the paint function.
plinss: From CSS?
TabAtkins: Yeah.
iank: We resolved basically this.
<iank> https://github.com/w3c/css-houdini-drafts/issues/219
iank: There's a new static getter which describes exactly the
syntax in the properties and value spec.
iank: can have <length>, <color>, etc.
iank: Additional input arguments.
Moving toward publication
-------------------------
Rossen: That's all on paint?
iank: Yep.
Rossen: So republish once edits are in?
iank: Yep
TabAtkins: Because arguing about it in PR.
TabAtkins: Way you wrote several arguments does very explicit type
checking.
TabAtkins: I suggested doing more webIDL like thing where it
coerces more.
TabAtkins: e.g. can coerce into a string instead of type-checking
and throwing out.
iank: I converted e.g. alphaValue
iank: This basically copies what custom elements is now doing.
iank: I converted to infra maps.
[?]
jack: We spend last year in Servo building web render
jack: This is absolute wost case per for us, using canvas2D with
custom paint functions.
jack: Understand it's very pragmatic, but hoping that in future
there's a version of API that's easier to optimize to GPUs
alex: Would be nice to have that as a change request to canvas,
would affect all other apis.
jack: Is there a draft spec?
iank: Can you write one? :)
dbaron: This need to be as bad as you think.
dbaron: This is using a subset of canvas API that doesn't have any
readback mechanisms.
dbaron: You don't need to implement in terms of a bitmap.
jack: That's one issue
jack: But when you do things like [...]
jack: Batch operations where 1st & 3rd require a draw ? and 2nd
requires another draw ?
jack: modified version of canvas API, same calls, but not order
dependent.
jack: Able to optimize that, regardless of pixel readback.
dbaron: Don't see how you can't do that.
jack: Canvas is order-dependent.
esprehn: Can still build a graph underneath that.
dbaron: Don't see how it's worse than adding objects to a scene
graph.
jack: Ok, haven't done any investigation of that.
dbaron: Have to think about whether we want to ship a Gecko
version of this using bitmap canvas, or to wait until we
have something better than bitmap canvas to ship
iank: ...
iank: we already have all that machinery for canvas anyway.
jack: Is the ink drop effect reproducible with this version of
canvas2D?
iank: Yes.
iank: There's 2-3 custom properties, one which is for xy mouse
position
iank: one is for radius of ink drop effect.
iank: Set CSS animation on radius to go from 50 to 100 or whatever
iank: and then custom paint just draws circles relative to xy and
radius.
iank: Actual material design spec is more complicated.
iank: If got multiple button clicks, could have multiple ripples.
iank: Also if you move your finger around while held down, will
move around ripples.
iank: Another example is in gmail-
iank: E.g. this icon here is 20-30 elements.
iank: You could imagine this is just one custom paint, it's a
string of images to paint.
iank: Suddenly this swings down to O(N) insted of O(30*N).
iank: This point you raised here, wrt referencing canvas api:
(you = Alex)
alex: process issue, not really sure TAG cares.
alex: Common with things that have an HTML hook.
alex: How far along is HTMl5.1, 5.2, etc.
iank: Other thing to discuss is canvas filters and canvasText.
iank: Wanted to think more about how it fits into typed OM, so
will come in a later WD once we've sorted that out.
dbaron: Text might also have threading issues, depending whether
system's text APIs can run in multiple threads.
iank: canvasText has a font attribute, which when you set it
instantly does a cross-thread jump to check if font is
loaded or not, and can potentially fail.
iank: Instead what we want to do is have a custom properties which
is a resolved font face.
iank: That's a resource in terms of typed OM, can be loaded, can
be ready.
iank: That means if font face changes state changes at any point,
know it's ready to use.
iank: Instead of accepting a string, will accept fontFace values.
iank: Similarly for filters, cuz can reference SVG.
iank: Need to disable that, because that has same loading issue.
iank: So if we want to support SVG portion of canvas filters, need
to add that through typed OM.
<dbaron> there was something there about the resource state
changing meaning that there's a change in the typed OM
value which means stuff gets called again.
iank: Should we resolve to publish another WD after edits?
RESOLVED: New WD of Paint after edits
<br duration=15m>
<div id="DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br />
<table style="border-top: 1px solid #D3D4DE;">
<tr>
<td style="width: 55px; padding-top: 13px;"><a
href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon"
target="_blank"><img
src="https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif"
width="46" height="29" style="width: 46px; height: 29px;" /></a></td>
<td style="width: 470px; padding-top: 12px; color: #41424e;
font-size: 13px; font-family: Arial, Helvetica, sans-serif;
line-height: 18px;">Virus-free. <a
href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link"
target="_blank" style="color: #4453ea;">www.avast.com</a>
</td>
</tr>
</table><a href="#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1"
height="1"></a></div>
Received on Sunday, 5 March 2017 16:07:23 UTC