Re: [css-transforms] CSS3D breaks with opacity flattening

Rik,

What is the exact effect you are trying to achieve?

If I break the rule of markup changes I can achieve this, which is to wrap
it into a sub-scene to target the composited render-to-texture layer:
https://jsfiddle.net/acp4vk5u/

Thus, the wheels are properly occluded etc.

Or is there a demo of more sophisticated 3D scene graph and how it
interacts with other 3D surfaces that are problematic?

--

My own proposal would be to have preserve-3d manipulate sub-scene
perspective layers to be in the same coordinate system, as I would expect
it to be behave as a web developer. (E.g. no need to layer with extra div,
but proper implementation of the Impostor technique as I mentioned before)

Additional feature could be being able to enable depth-testing against the
impostors depth scene.


Regards,

Martin Pitt

On Mon, 19 Sep 2016 at 09:34 Rik Cabanier <cabanier@gmail.com> wrote:

> On Mon, Sep 19, 2016 at 1:51 AM, /#!/JoePea <trusktr@gmail.com> wrote:
>
>> Hey all,
>>
>> Sorry if my previous replies were rash, but I've collected myself and
>> prepared the following response which can hopefully paint a picture of the
>> problem as well as propose a solution.
>>
>> Let's step outside of the web box for a second, and just imagine: we have
>> a 3D engine, and with that engine we have game characters or objects that
>> are made of arbitrary number of descendant children (subtrees) in an
>> overall scene graph. Imagine these objects are in a game where things fade
>> in and out of view and therefore involve transparency.
>>
>> I believe we all want the API for doing this to be as easy as possible.
>> And, since this is the case, then it makes sense that applying opacity less
>> than one to the root node of an object in a scene graph will make the whole
>> object transparent without flattening it.
>>
>> Take for example, this basic car I just made:
>>
>> https://jsfiddle.net/trusktr/ymonmo70/1
>>
> === Challenge ===
>>
>> Here's a challenge for you all: make the whole car transparent without
>> modifying the markup (it is important not to modify the markup because that
>> is what CSS is about). But, if you want to modify the markup, I am open to
>> seeing that solution as well (and I already know the non-nested solution
>> will work, but will be tedious to migrate to). Keep in mind, this is very
>> very easy with "legacy" behavior, and we can make the whole car transparent
>> by simply adding the following CSS into the CSS text box:
>>
>> ```css
>> .car {
>>   opacity: 0.5;
>> }
>> ```
>>
>>
>> which results in the following (works in Safari 9 and Firefox 47, broken
>> in Chrome 53):
>>
>> https://jsfiddle.net/trusktr/ymonmo70/2
>>
>
> Isn't the rendering result of the car wrong? I can see the tires that are
> supposed to be occluded by the frame of the car.
>
>
>>
>> As far as I know, I don't think any of you will have a solution that is
>> as simple as simply applying an opacity to the root node of the car. The
>> simplest solution I can imagine is applying opacity to all the leaf nodes,
>> and that will involve much more work than the addition of a single opacity
>> property to a single selector or element.
>>
>
> There is. Apply opacity on the scene.
>
>
>> With the new behavior, it isn't so simple any more -- *the web's 3D API
>> is now **more difficult to use*, which isn't what we want as API
>> developers.
>>
>> We should aim to make 3D programming more easy, not more difficult!
>> Opacity flattening makes 3D programming more difficult because the 3D
>> library programmer now has to go and keep track of opacities virtually, for
>> every node in the scene (DOM nodes in our case) and then has to traverse
>> the scene graph (DOM tree) manually and multiply all opacities manually in
>> order to finally apply actual opacities to the leaf-most items (elements)
>> in the scene graph.
>>
>> Note, even with "legacy" behavior or the new flattening behavior, there
>> is currently no way to make rendered non-leaf nodes transparent. In the
>> following example, try and make *only* the red box transparent but not
>> anything else (i.e. not it's children):
>>
>> https://jsfiddle.net/trusktr/ymonmo70/4
>>
>
> you can just apply transparency to the color of the red box.
>
>
>> With the new behavior, we are limited to making only leaf-nodes of an
>> existing scene transparent, and keep in mind we'd like to apply CSS *without
>> modifying markup*.
>>
>> One of the great things about preserve-3d and nested elements is that
>> transformation matrices can be cached, and don't need to be re-calculated
>> all the time on the JavaScript side because the HTML engine has them cached
>> in the DOM hierarchy. This prevents a TON of string manipulation in
>> JavaScript due to converting numbers into strings to pass to the CSS engine
>> via style tags (that will be fixed with Typed CSSOM, but we're not sure
>> when that's making it into the wild).
>>
>> In legacy behavior, the same applies to opacity: the HTML/CSS engine
>> caches values and does multiplication automatically, which makes the end
>> API easier to use for 3D programming, and *more performant.*
>>
>> In order to solve the problem that the new opacity behavior entails, the
>> best solution is (unfortunately) to abandon preserve-3d and nested DOM (*losing
>> the performance benefits*), then keeping track of the transformation and
>> opacity hierarchy manually, multiplying down the tree manually, and finally
>> applying *all the values via number-to-string conversion in the style
>> attributes, which may involve string parsing, splitting, joining, etc,
>> before being actually passed to the style attribute (yikes!)*.
>>
>> Reverting to the non-nested approach means that we are forfeiting the
>> performance advantages of the nested DOM approach just to achieve what we
>> want.
>>
>> Note, using the non-nested approach, virtual hierarchies, and manual
>> handling of the math allows us to easily make a parent in the hierarchy
>> transparent but not its children (which is impossible in the nested
>> approach using the `opacity` property).
>>
>> === Impact ===
>>
>> One argument for the go-ahead to implement this change is that "not many
>> people are impacted".
>>
>> I'd like to argue that 3D in the web (as far as DOM goes) is already
>> fairly difficult compared to code-based APIs (i.e. imperative APIs rather
>> than declarative APIs, and not just in the web), and relatively few people
>> actually use CSS3D. The vast majority of websites still use only the
>> decades-old 2-dimensional technology of the web, and the 3D aspects of
>> HTML/CSS are relatively new and difficult to mix with the 2-dimensional
>> features. It may be easy to think that many people aren't affected by this,
>> but let's take into consideration the ratio of the number of people who use
>> opacity in 3D scenes to the number of people who program CSS3D scenes. If
>> you take these numbers into consideration, I am willing to bet that the
>> number will be a lot higher than the 0.006% mentioned in the blink-dev
>> thread
>> <https://groups.google.com/a/chromium.org/d/msg/blink-dev/eBIp90_il1o/9q3M0ww2BgAJ>
>> .
>>
>> Famous was making a an open-source library that depended on the legacy
>> behavior, and this change breaks that library. They've gone proprietary,
>> and their proprietary stuff probably still depends on the legacy behavior.
>> They will now face the chore of re-writing parts of their library to
>> convert from nested dom to non-nested dom if they wish for opacity to work
>> the same as before.
>>
>> I'd like to show you how these changes impact my own library at
>> http://infamous.io (which is being renamed and moved to a new domain
>> soon).
>>
>> My library gives us both a JavaScript API (imperative) and a
>> Custom-Element-based HTML API (declarative) for defining 3D scenes. When
>> using the JavaScript API, it generates the same elements as when using the
>> HTML API. For our purposes, I will show only the HTML API.
>>
>> Here is the same car example as above (or, at least similar), made with
>> my custom elements:
>>
>> https://jsfiddle.net/trusktr/ymonmo70/5
>>
>> What is great about this example is that those custom elements are *the
>> elements themselves* in the final rendering (look in the element
>> inspector and notice that they have `transform: matrix3d()` applied to
>> them). This means we can place anything into those elements. For example,
>> let's place some text onto the surfaces of the car just for fun:
>>
>> https://jsfiddle.net/trusktr/ymonmo70/6
>>
>> Now, if we apply opacity, the whole car will be flattened in Chrome 53
>> (compare it to Safari 9) because we've applied opacity onto the `.car`
>> element element:
>>
>> https://jsfiddle.net/trusktr/ymonmo70/7
>>
>> I believe I have something really nice in the works, and plan to add
>> WebGL soon, but now it is fundamentally broken: in order to fix the
>> rendering, but keep my same nested markup, I will have to render new
>> elements into a flat structure that sits next to my <motor-scene> elements.
>> This means that for each motor-node element I will create a reciprocal
>> element in the actual rendering context, while my custom elements will not
>> actually be rendered (they will be display:none). In other words, although
>> I'd like for my API to be nested and using preserve-3d, I will have to
>> render *completely separate elements* next to my custom elements in
>> order to solve the opacity problem that is on my hands now.
>>
>> But, there are some problems with this! Remember that we were able to
>> place content like text inside the custom elements, and it worked fine
>> because the custom elements are part of the actual rendered scene?
>>
>> Well, now that won't be the case any more. Now I have the following
>> problems to deal with:
>>
>>    1. I have to convert my code so it renders non-nested elements next
>>    to my custom elements (or maybe inside a ShadowDOM root).
>>    2. I will have to implement transformation caching just for DOM
>>    rendering, and can no longer take advantage of the HTML engine's transform
>>    caching.
>>    3. Same for opacity caching down the scene graph, I will need to
>>    implement it rather than let the HTML engine handle it.
>>    4. Lastly, but most problematically, I have to find a way to clone or
>>    copy the user's content from inside my custom elements into the new
>>    non-nested elements. This will have detrimental effects on the user's
>>    ability to target and select those elements for styling, as they will no
>>    longer appear in their original places. Users will need to use IDs all over
>>    the place when they probably shouldn't, and will likely mess up a bunch of
>>    times before discovering the ID workaround. This may also have an impact on
>>    SEO, but I haven't looked into that much yet.
>>
>> The first three points aren't as bad as the fourth, as I'll have to
>> implement the first three points when I add WebGL anyways. But, the fourth
>> point is really bad! The new behavior is an extreme pain because of point
>> number four.
>>
>> I hope you can see how, from my perspective as a library author writing
>> on top of HTML/CSS3D, this is a very unwelcoming change if opacity is to
>> have any meaning in my 3D scenes.
>>
>> There aren't many people doing what I'm doing. There are only three
>> libraries out there that are doing what I am doing (that I know of):
>>
>>    - Three.js <http://threejs.org> - Three.js uses the non-nested
>>    approach, so does not face the opacity problem. People who use Three.js'
>>    CSS3DRenderer know what they are working with from the start, and they
>>    won't possibly get confused in targeting their elements for styling,
>>    because their elements will not be transplanted like with my library. But
>>    note, Three.js will not be able to update to the nested-approach due to
>>    this problem, and therefore won't be able to have the performance benefits
>>    of the nested approach.
>>    - Famo.us <http://deprecated.famous.org> (0.3 and below) - Like
>>    Three.js, uses non-nested approach, and is worry-free like Three.js.
>>    - Famo.us Engine <http://github.com/famous/engine> (0.5 and above) -
>>    Uses the nested approach, and is now broken just like my library.
>>    - Samsara <http://samsarajs.org> - Uses the non-nested approach, so
>>    no problems there.
>>    - Infamous <http://infamous.io> - My library, which uses the nested
>>    approach. Problem!
>>
>> In sincerity, opacity flattening is a breaking change that shouldn't have
>> been implemented by any browsers without a full solution for "legacy" code
>> with 3D in mind.
>>
>> === What We Need ===
>>
>> What we need are possibly at least three forms of opacity:
>>
>>    1. 3-dimensional, like the legacy behavior where things don't get
>>    flattened and opacities are multiplied down the hierarchy.
>>    2. Parent-only (opacity on a single element), where nothing is
>>    flattened, and opacity is applied only to the target element and not its
>>    children. Note, there is currently no way to do this in the nested
>>    approach, only in the non-nested approach with virtual hierarchies. In
>>    "legacy" behavior, we can only apply opacity to an entire subtree of a 3D
>>    context, not just to a parent element, so that's either all or nothing.
>>    3. Flattening, which is the new behavior, though I fail to see why
>>    anyone applying opacity to a 3D object desires for the object to be
>>    suddenly flat, as that doesn't make much sense and is unintuitive, and
>>    means the API is more difficult to use that what we'd like.
>>
>> Note, with "legacy" behavior, *it is already possible to create a new 3D
>> context and apply opacity to it*, so this new behavior isn't actually
>> needed (from a functional perspective), because we can currently achieve
>> the same behavior with the "legacy" implementation. The new behavior does
>> two things:
>>
>>    1. it eliminates the desired 3D behavior in nested scenes that use
>>    preserve-3d
>>    2. it adds a second method of opacifying an entire 3D scene which *we
>>    can already do*.
>>
>> Here is an example of just that using "legacy" techniques, where I simply
>> add the style `motor-scene {opacity: 0.5}` to the CSS, which opacifies the
>> 3D context:
>>
>> https://jsfiddle.net/trusktr/ymonmo70/8
>>
>> We all agree that the spec (whatever it says) should be well defined so
>> that it is clear what browsers should implement. I also believe that the
>> legacy behavior is much more desirable for 3D programmers than is the
>> flattening of objects (legacy-like behavior just needs to be clearly
>> spec'd).
>>
>> === Possible Solution ===
>>
>> Maybe we can improve the spec (before making breaking changes, and
>> browser developers, you all roll back to "legacy" implementation ASAP) so
>> that the spec can appease all three opacity styles.
>>
>> This could be achieved with something like a new CSS property called
>> `opacity-style`, f.e.
>>
>>    - `opacity-style: 3d` which is similar to the legacy behavior and is
>>    great for 3D scenes, and does not flatten anything. Opacity is multiplied
>>    all the way down the tree with other same-style opacities and stops at
>>    elements that have a different opacity-style (`flat` or `single`).
>>    - `opacity-style: single` which does not flatten anything and only
>>    applies opacity to the target element but not its children
>>    - `opacity-style: flat` which is like the new behavior that Chrome 53
>>    just introduced, things are flattened and a new 3D context is made. I still
>>    fail to see how this is desirable. *Assuming we can apply an opacity
>>    to a 3D object to make it transparent and then having it become flat like
>>    paper is simply not intuitive.*
>>
>> The default value can be `opacity-style: single`, which I think is the
>> least impacting of the three. We can then see what 3D programmers end up
>> using more often (I'd like to bet that `opacity-style: flat` will be the
>> least used). Or, perhaps the default for a 3D object can be `single`, and
>> `flat` for non-3D objects.
>>
>> What are your thoughts on something like this, `opacity-style`? Because,
>> as it stands, opacity in the 3D web isn't very workable now in nested
>> scenes (it is much more workable with the legacy behavior).
>>
>> */#!/*JoePea
>>
>> On Thu, Sep 15, 2016 at 11:16 AM, Tab Atkins Jr. <jackalmage@gmail.com>
>> wrote:
>>
>>> On Thu, Sep 15, 2016 at 10:02 AM, Rik Cabanier <cabanier@gmail.com>
>>> wrote:
>>> > On Wed, Sep 14, 2016 at 8:44 PM, /#!/JoePea <trusktr@gmail.com> wrote:
>>> >> Here's an example using Famous Engine (
>>> http://github.com/famous/engine):
>>> >>
>>> >> First, with opacity at the default value of 1.0, the "Famous Code"
>>> logo
>>> >> moves back and forth and rotates:
>>> >> http://jsfiddle.net/trusktr/spauv8fs/5
>>> >>
>>> >> With opacity reduced, it breaks in Chrome 53:
>>> >> http://jsfiddle.net/trusktr/spauv8fs/6
>>> >>
>>> >> Famous Engine has the ability to mix WebGL with DOM, and this includes
>>> >> opacity. This new behavior causes the DOM elements not to move in 3D
>>> space
>>> >> the WebGL meshes.
>>> >>
>>> >> First, here's a demo, with opacity at 1.0:
>>> >> http://jsfiddle.net/trusktr/spauv8fs/7
>>> >>
>>> >> The, with opacity at 0.7:
>>> >> http://jsfiddle.net/trusktr/spauv8fs/8
>>> >>
>>> >> What you are supposed to see is a DOM element and a WebGL Mesh that
>>> both
>>> >> seem to intersect with the pink DOM-based background (the
>>> implementation is
>>> >> not perfect yet...). In the second fiddle (spauv8fs/8) the "Famous
>>> Code"
>>> >> logo appears not to move any more while the WebGL mesh continues to
>>> move.
>>> >>
>>> >> There is a bug in the WebGL renderer which I believe may be due to
>>> changes
>>> >> in WebGL, but the WebGL part of those examples is supposed to be
>>> transparent
>>> >> as well.
>>> >>
>>> >> I myself am working on a new implementation of DOM + WebGL, and it
>>> allows
>>> >> application of opacity. This change in Chrome 53 completely breaks
>>> how that
>>> >> is supposed to work. I don't have an online demo yet...
>>> >>
>>> >> I would say let's definitely consider undoing the changes to the spec
>>> so
>>> >> that flattening does not happen when applying an opacity to 3D DOM
>>> elements.
>>> >> The reasoning is not just to prevent breaking apps, but because the
>>> new
>>> >> behavior simply doesn't make sense as far as 3D goes.
>>> >
>>> >
>>> > I don't understand how you come to that conclusion.
>>> > The new behavior seems more logical since it applies the opacity to
>>> element
>>> > that has the property applied. The old implementation distributed the
>>> value
>>> > to its children which is counter to any other CSS value.
>>> > Are you proposing that we also do this for filters, blending, backdrop
>>> > blurring and other effects, or is opacity special in some way,
>>> >
>>> > As Simon stated, if you want the old behavior, just add a selector to
>>> your
>>> > opacity parameter so it's applied to the children.
>>>
>>> Yes. This is not a spec bug, it's a natural and unavoidable
>>> consequence of doing a "group effect", which opacity, filters, and a
>>> few other effects are.  These types of effects require the group to be
>>> rendered as a unit, then have the effect applied; in 3d space, this
>>> has the effect of flattening them. (If you didn't flatten, then other
>>> items could get between the individual pieces in 3d order, and there's
>>> no consistent or sensible way to render that. This is identical to how
>>> z-index is "flattened" by opacity and other group effects, so you
>>> can't sandwich elements elsewhere in the page between the elements in
>>> the group.)
>>>
>>> If you want to sandwich things, you need to push the effect further
>>> down into the leaves, so it doesn't group as many things together.
>>> This lets you do more re-ordering, but has a different visual effect -
>>> if one item occludes another in a group, when they're made transparent
>>> they still occlude; if they're made individually transparent, you'll
>>> be able to see the second item behind the first.  Similar differences
>>> exist for other group effects - if you're doing a gaussian blur,
>>> blurring two boxes as a group can look quite different than blurring
>>> them individually.
>>>
>>> As Simon said, Chrome 52 and earlier Safari are simply buggy and not
>>> "grouping" things correctly; they're instead automatically pushing the
>>> opacity down further toward the leaves when 3d is involved.  If you
>>> want the same effect, just do so manually, as Rik recommends.
>>>
>>> (We had an almost identical request in the SVG Working Group a few
>>> days ago. I posted
>>> https://github.com/w3c/svgwg/issues/264#issuecomment-246750601 as an
>>> extended explanation of what's happening.)
>>>
>>> ~TJ
>>>
>>
>>