Re: [css-transforms] CSS3D breaks with opacity flattening

On Mon, Sep 19, 2016 at 1:51 AM, /#!/JoePea <trusktr@gmail.com> wrote:

> Hey all,
>
> Sorry if my previous replies were rash, but I've collected myself and
> prepared the following response which can hopefully paint a picture of the
> problem as well as propose a solution.
>
> Let's step outside of the web box for a second, and just imagine: we have
> a 3D engine, and with that engine we have game characters or objects that
> are made of arbitrary number of descendant children (subtrees) in an
> overall scene graph. Imagine these objects are in a game where things fade
> in and out of view and therefore involve transparency.
>
> I believe we all want the API for doing this to be as easy as possible.
> And, since this is the case, then it makes sense that applying opacity less
> than one to the root node of an object in a scene graph will make the whole
> object transparent without flattening it.
>
> Take for example, this basic car I just made:
>
> https://jsfiddle.net/trusktr/ymonmo70/1
>
=== Challenge ===
>
> Here's a challenge for you all: make the whole car transparent without
> modifying the markup (it is important not to modify the markup because that
> is what CSS is about). But, if you want to modify the markup, I am open to
> seeing that solution as well (and I already know the non-nested solution
> will work, but will be tedious to migrate to). Keep in mind, this is very
> very easy with "legacy" behavior, and we can make the whole car transparent
> by simply adding the following CSS into the CSS text box:
>
> ```css
> .car {
>   opacity: 0.5;
> }
> ```
>
>
> which results in the following (works in Safari 9 and Firefox 47, broken
> in Chrome 53):
>
> https://jsfiddle.net/trusktr/ymonmo70/2
>

Isn't the rendering result of the car wrong? I can see the tires that are
supposed to be occluded by the frame of the car.


>
> As far as I know, I don't think any of you will have a solution that is as
> simple as simply applying an opacity to the root node of the car. The
> simplest solution I can imagine is applying opacity to all the leaf nodes,
> and that will involve much more work than the addition of a single opacity
> property to a single selector or element.
>

There is. Apply opacity on the scene.


> With the new behavior, it isn't so simple any more -- *the web's 3D API
> is now **more difficult to use*, which isn't what we want as API
> developers.
>
> We should aim to make 3D programming more easy, not more difficult!
> Opacity flattening makes 3D programming more difficult because the 3D
> library programmer now has to go and keep track of opacities virtually, for
> every node in the scene (DOM nodes in our case) and then has to traverse
> the scene graph (DOM tree) manually and multiply all opacities manually in
> order to finally apply actual opacities to the leaf-most items (elements)
> in the scene graph.
>
> Note, even with "legacy" behavior or the new flattening behavior, there is
> currently no way to make rendered non-leaf nodes transparent. In the
> following example, try and make *only* the red box transparent but not
> anything else (i.e. not it's children):
>
> https://jsfiddle.net/trusktr/ymonmo70/4
>

you can just apply transparency to the color of the red box.


> With the new behavior, we are limited to making only leaf-nodes of an
> existing scene transparent, and keep in mind we'd like to apply CSS *without
> modifying markup*.
>
> One of the great things about preserve-3d and nested elements is that
> transformation matrices can be cached, and don't need to be re-calculated
> all the time on the JavaScript side because the HTML engine has them cached
> in the DOM hierarchy. This prevents a TON of string manipulation in
> JavaScript due to converting numbers into strings to pass to the CSS engine
> via style tags (that will be fixed with Typed CSSOM, but we're not sure
> when that's making it into the wild).
>
> In legacy behavior, the same applies to opacity: the HTML/CSS engine
> caches values and does multiplication automatically, which makes the end
> API easier to use for 3D programming, and *more performant.*
>
> In order to solve the problem that the new opacity behavior entails, the
> best solution is (unfortunately) to abandon preserve-3d and nested DOM (*losing
> the performance benefits*), then keeping track of the transformation and
> opacity hierarchy manually, multiplying down the tree manually, and finally
> applying *all the values via number-to-string conversion in the style
> attributes, which may involve string parsing, splitting, joining, etc,
> before being actually passed to the style attribute (yikes!)*.
>
> Reverting to the non-nested approach means that we are forfeiting the
> performance advantages of the nested DOM approach just to achieve what we
> want.
>
> Note, using the non-nested approach, virtual hierarchies, and manual
> handling of the math allows us to easily make a parent in the hierarchy
> transparent but not its children (which is impossible in the nested
> approach using the `opacity` property).
>
> === Impact ===
>
> One argument for the go-ahead to implement this change is that "not many
> people are impacted".
>
> I'd like to argue that 3D in the web (as far as DOM goes) is already
> fairly difficult compared to code-based APIs (i.e. imperative APIs rather
> than declarative APIs, and not just in the web), and relatively few people
> actually use CSS3D. The vast majority of websites still use only the
> decades-old 2-dimensional technology of the web, and the 3D aspects of
> HTML/CSS are relatively new and difficult to mix with the 2-dimensional
> features. It may be easy to think that many people aren't affected by this,
> but let's take into consideration the ratio of the number of people who use
> opacity in 3D scenes to the number of people who program CSS3D scenes. If
> you take these numbers into consideration, I am willing to bet that the
> number will be a lot higher than the 0.006% mentioned in the blink-dev
> thread
> <https://groups.google.com/a/chromium.org/d/msg/blink-dev/eBIp90_il1o/9q3M0ww2BgAJ>
> .
>
> Famous was making a an open-source library that depended on the legacy
> behavior, and this change breaks that library. They've gone proprietary,
> and their proprietary stuff probably still depends on the legacy behavior.
> They will now face the chore of re-writing parts of their library to
> convert from nested dom to non-nested dom if they wish for opacity to work
> the same as before.
>
> I'd like to show you how these changes impact my own library at
> http://infamous.io (which is being renamed and moved to a new domain
> soon).
>
> My library gives us both a JavaScript API (imperative) and a
> Custom-Element-based HTML API (declarative) for defining 3D scenes. When
> using the JavaScript API, it generates the same elements as when using the
> HTML API. For our purposes, I will show only the HTML API.
>
> Here is the same car example as above (or, at least similar), made with my
> custom elements:
>
> https://jsfiddle.net/trusktr/ymonmo70/5
>
> What is great about this example is that those custom elements are *the
> elements themselves* in the final rendering (look in the element
> inspector and notice that they have `transform: matrix3d()` applied to
> them). This means we can place anything into those elements. For example,
> let's place some text onto the surfaces of the car just for fun:
>
> https://jsfiddle.net/trusktr/ymonmo70/6
>
> Now, if we apply opacity, the whole car will be flattened in Chrome 53
> (compare it to Safari 9) because we've applied opacity onto the `.car`
> element element:
>
> https://jsfiddle.net/trusktr/ymonmo70/7
>
> I believe I have something really nice in the works, and plan to add WebGL
> soon, but now it is fundamentally broken: in order to fix the rendering,
> but keep my same nested markup, I will have to render new elements into a
> flat structure that sits next to my <motor-scene> elements. This means that
> for each motor-node element I will create a reciprocal element in the
> actual rendering context, while my custom elements will not actually be
> rendered (they will be display:none). In other words, although I'd like for
> my API to be nested and using preserve-3d, I will have to render *completely
> separate elements* next to my custom elements in order to solve the
> opacity problem that is on my hands now.
>
> But, there are some problems with this! Remember that we were able to
> place content like text inside the custom elements, and it worked fine
> because the custom elements are part of the actual rendered scene?
>
> Well, now that won't be the case any more. Now I have the following
> problems to deal with:
>
>    1. I have to convert my code so it renders non-nested elements next to
>    my custom elements (or maybe inside a ShadowDOM root).
>    2. I will have to implement transformation caching just for DOM
>    rendering, and can no longer take advantage of the HTML engine's transform
>    caching.
>    3. Same for opacity caching down the scene graph, I will need to
>    implement it rather than let the HTML engine handle it.
>    4. Lastly, but most problematically, I have to find a way to clone or
>    copy the user's content from inside my custom elements into the new
>    non-nested elements. This will have detrimental effects on the user's
>    ability to target and select those elements for styling, as they will no
>    longer appear in their original places. Users will need to use IDs all over
>    the place when they probably shouldn't, and will likely mess up a bunch of
>    times before discovering the ID workaround. This may also have an impact on
>    SEO, but I haven't looked into that much yet.
>
> The first three points aren't as bad as the fourth, as I'll have to
> implement the first three points when I add WebGL anyways. But, the fourth
> point is really bad! The new behavior is an extreme pain because of point
> number four.
>
> I hope you can see how, from my perspective as a library author writing on
> top of HTML/CSS3D, this is a very unwelcoming change if opacity is to have
> any meaning in my 3D scenes.
>
> There aren't many people doing what I'm doing. There are only three
> libraries out there that are doing what I am doing (that I know of):
>
>    - Three.js <http://threejs.org> - Three.js uses the non-nested
>    approach, so does not face the opacity problem. People who use Three.js'
>    CSS3DRenderer know what they are working with from the start, and they
>    won't possibly get confused in targeting their elements for styling,
>    because their elements will not be transplanted like with my library. But
>    note, Three.js will not be able to update to the nested-approach due to
>    this problem, and therefore won't be able to have the performance benefits
>    of the nested approach.
>    - Famo.us <http://deprecated.famous.org> (0.3 and below) - Like
>    Three.js, uses non-nested approach, and is worry-free like Three.js.
>    - Famo.us Engine <http://github.com/famous/engine> (0.5 and above) -
>    Uses the nested approach, and is now broken just like my library.
>    - Samsara <http://samsarajs.org> - Uses the non-nested approach, so no
>    problems there.
>    - Infamous <http://infamous.io> - My library, which uses the nested
>    approach. Problem!
>
> In sincerity, opacity flattening is a breaking change that shouldn't have
> been implemented by any browsers without a full solution for "legacy" code
> with 3D in mind.
>
> === What We Need ===
>
> What we need are possibly at least three forms of opacity:
>
>    1. 3-dimensional, like the legacy behavior where things don't get
>    flattened and opacities are multiplied down the hierarchy.
>    2. Parent-only (opacity on a single element), where nothing is
>    flattened, and opacity is applied only to the target element and not its
>    children. Note, there is currently no way to do this in the nested
>    approach, only in the non-nested approach with virtual hierarchies. In
>    "legacy" behavior, we can only apply opacity to an entire subtree of a 3D
>    context, not just to a parent element, so that's either all or nothing.
>    3. Flattening, which is the new behavior, though I fail to see why
>    anyone applying opacity to a 3D object desires for the object to be
>    suddenly flat, as that doesn't make much sense and is unintuitive, and
>    means the API is more difficult to use that what we'd like.
>
> Note, with "legacy" behavior, *it is already possible to create a new 3D
> context and apply opacity to it*, so this new behavior isn't actually
> needed (from a functional perspective), because we can currently achieve
> the same behavior with the "legacy" implementation. The new behavior does
> two things:
>
>    1. it eliminates the desired 3D behavior in nested scenes that use
>    preserve-3d
>    2. it adds a second method of opacifying an entire 3D scene which *we
>    can already do*.
>
> Here is an example of just that using "legacy" techniques, where I simply
> add the style `motor-scene {opacity: 0.5}` to the CSS, which opacifies the
> 3D context:
>
> https://jsfiddle.net/trusktr/ymonmo70/8
>
> We all agree that the spec (whatever it says) should be well defined so
> that it is clear what browsers should implement. I also believe that the
> legacy behavior is much more desirable for 3D programmers than is the
> flattening of objects (legacy-like behavior just needs to be clearly
> spec'd).
>
> === Possible Solution ===
>
> Maybe we can improve the spec (before making breaking changes, and browser
> developers, you all roll back to "legacy" implementation ASAP) so that the
> spec can appease all three opacity styles.
>
> This could be achieved with something like a new CSS property called
> `opacity-style`, f.e.
>
>    - `opacity-style: 3d` which is similar to the legacy behavior and is
>    great for 3D scenes, and does not flatten anything. Opacity is multiplied
>    all the way down the tree with other same-style opacities and stops at
>    elements that have a different opacity-style (`flat` or `single`).
>    - `opacity-style: single` which does not flatten anything and only
>    applies opacity to the target element but not its children
>    - `opacity-style: flat` which is like the new behavior that Chrome 53
>    just introduced, things are flattened and a new 3D context is made. I still
>    fail to see how this is desirable. *Assuming we can apply an opacity
>    to a 3D object to make it transparent and then having it become flat like
>    paper is simply not intuitive.*
>
> The default value can be `opacity-style: single`, which I think is the
> least impacting of the three. We can then see what 3D programmers end up
> using more often (I'd like to bet that `opacity-style: flat` will be the
> least used). Or, perhaps the default for a 3D object can be `single`, and
> `flat` for non-3D objects.
>
> What are your thoughts on something like this, `opacity-style`? Because,
> as it stands, opacity in the 3D web isn't very workable now in nested
> scenes (it is much more workable with the legacy behavior).
>
> */#!/*JoePea
>
> On Thu, Sep 15, 2016 at 11:16 AM, Tab Atkins Jr. <jackalmage@gmail.com>
> wrote:
>
>> On Thu, Sep 15, 2016 at 10:02 AM, Rik Cabanier <cabanier@gmail.com>
>> wrote:
>> > On Wed, Sep 14, 2016 at 8:44 PM, /#!/JoePea <trusktr@gmail.com> wrote:
>> >> Here's an example using Famous Engine (http://github.com/famous/engine
>> ):
>> >>
>> >> First, with opacity at the default value of 1.0, the "Famous Code" logo
>> >> moves back and forth and rotates:
>> >> http://jsfiddle.net/trusktr/spauv8fs/5
>> >>
>> >> With opacity reduced, it breaks in Chrome 53:
>> >> http://jsfiddle.net/trusktr/spauv8fs/6
>> >>
>> >> Famous Engine has the ability to mix WebGL with DOM, and this includes
>> >> opacity. This new behavior causes the DOM elements not to move in 3D
>> space
>> >> the WebGL meshes.
>> >>
>> >> First, here's a demo, with opacity at 1.0:
>> >> http://jsfiddle.net/trusktr/spauv8fs/7
>> >>
>> >> The, with opacity at 0.7:
>> >> http://jsfiddle.net/trusktr/spauv8fs/8
>> >>
>> >> What you are supposed to see is a DOM element and a WebGL Mesh that
>> both
>> >> seem to intersect with the pink DOM-based background (the
>> implementation is
>> >> not perfect yet...). In the second fiddle (spauv8fs/8) the "Famous
>> Code"
>> >> logo appears not to move any more while the WebGL mesh continues to
>> move.
>> >>
>> >> There is a bug in the WebGL renderer which I believe may be due to
>> changes
>> >> in WebGL, but the WebGL part of those examples is supposed to be
>> transparent
>> >> as well.
>> >>
>> >> I myself am working on a new implementation of DOM + WebGL, and it
>> allows
>> >> application of opacity. This change in Chrome 53 completely breaks how
>> that
>> >> is supposed to work. I don't have an online demo yet...
>> >>
>> >> I would say let's definitely consider undoing the changes to the spec
>> so
>> >> that flattening does not happen when applying an opacity to 3D DOM
>> elements.
>> >> The reasoning is not just to prevent breaking apps, but because the new
>> >> behavior simply doesn't make sense as far as 3D goes.
>> >
>> >
>> > I don't understand how you come to that conclusion.
>> > The new behavior seems more logical since it applies the opacity to
>> element
>> > that has the property applied. The old implementation distributed the
>> value
>> > to its children which is counter to any other CSS value.
>> > Are you proposing that we also do this for filters, blending, backdrop
>> > blurring and other effects, or is opacity special in some way,
>> >
>> > As Simon stated, if you want the old behavior, just add a selector to
>> your
>> > opacity parameter so it's applied to the children.
>>
>> Yes. This is not a spec bug, it's a natural and unavoidable
>> consequence of doing a "group effect", which opacity, filters, and a
>> few other effects are.  These types of effects require the group to be
>> rendered as a unit, then have the effect applied; in 3d space, this
>> has the effect of flattening them. (If you didn't flatten, then other
>> items could get between the individual pieces in 3d order, and there's
>> no consistent or sensible way to render that. This is identical to how
>> z-index is "flattened" by opacity and other group effects, so you
>> can't sandwich elements elsewhere in the page between the elements in
>> the group.)
>>
>> If you want to sandwich things, you need to push the effect further
>> down into the leaves, so it doesn't group as many things together.
>> This lets you do more re-ordering, but has a different visual effect -
>> if one item occludes another in a group, when they're made transparent
>> they still occlude; if they're made individually transparent, you'll
>> be able to see the second item behind the first.  Similar differences
>> exist for other group effects - if you're doing a gaussian blur,
>> blurring two boxes as a group can look quite different than blurring
>> them individually.
>>
>> As Simon said, Chrome 52 and earlier Safari are simply buggy and not
>> "grouping" things correctly; they're instead automatically pushing the
>> opacity down further toward the leaves when 3d is involved.  If you
>> want the same effect, just do so manually, as Rik recommends.
>>
>> (We had an almost identical request in the SVG Working Group a few
>> days ago. I posted
>> https://github.com/w3c/svgwg/issues/264#issuecomment-246750601 as an
>> extended explanation of what's happening.)
>>
>> ~TJ
>>
>
>