[fxtf-drafts] [compositing] Question: How to properly blend and composite isolated groups (#440)

straker has just created a new issue for https://github.com/w3c/fxtf-drafts:

== [compositing] Question: How to properly blend and composite isolated groups ==
https://drafts.fxtf.org/compositing-1/#groups

**TL;DR**: how should isolated groups be blended and composited together? would it be possible to add explicit algorithms and diagrams showing how to composite and blend multiple isolated groups?

I work on [axe-core](https://github.com/dequelabs/axe-core) and we are trying to implement support for `mix-blend-mode` when calculating an elements background color. We've been using the spec to add all the blending algorithms needed to run this calculation, but we've run into a problem trying to calculate compositing of the background colors. 

From what I understand from the spec, without `mix-blend-mode` (i.e. `mix-blend-mode: normal`), background colors are composited together from bottom-up order. That is, using the following HTML we would blend element A and B together, then the result of that blend would be blended with element C, giving the result of `rgb(223, 112, 96)` (following the formulas of simple alpha compositing with blending substituted from https://drafts.fxtf.org/compositing-1/#blending). A visual inspection of this result compared to the browser shows that they are the same.

```html
<div id="A" style="background-color: rgba(255, 255, 255, 1.0);">
  <div id="B" style="background-color: rgba(0, 128, 0, 0.25);">
    <div id="C" style="background-color: rgba(255, 0, 0, 0.5);">Test Element</div>
  </div>
</div>
```

However, once `mix-blend-mode` is applied to an element, things are different. From the spec I understand that this should create an [isolated group](https://drafts.fxtf.org/compositing-1/#isolatedgroups) since `mix-blend-mode` [creates a stacking context](https://drafts.fxtf.org/compositing-1/#csscompositingrules_CSS), however what I'm not clear on from the spec is how to then composite an isolate group with other groups. The [section on groups](https://drafts.fxtf.org/compositing-1/#groups) has a bit of explanation, but it is unclear to me what that algorithm looks like.

> A compositing group is rendered by first compositing the elements of the group onto the initial backdrop. The result of this is a single element containing color and alpha information. This element is then composited onto the group backdrop. Steps shall be taken to ensure the group backdrop makes only a single contribution to the final composite.

If we take the same HTML as before and add `mix-blend-mode: difference` to element C, what it sounds like should happen is that there are now two groups: an isolated group containing element C, and the [root group](https://drafts.fxtf.org/compositing-1/#rootgroup) containing element A, B, and the isolated group.

```html
<div id="A" style="background-color: rgba(255, 255, 255, 1.0);">
  <div id="B" style="background-color: rgba(0, 128, 0, 0.25);">
    <div id="C" style="background-color: rgba(255, 0, 0, 0.5); mix-blend-mode: difference;">Test Element</div>
  </div>
</div>
```

```
Root Group
  - #A
  - #B
  - isolated group
    - #C 
```

From the paragraph, it sounds like the groups are first composited separately onto their initial backdrop, then again onto the group backdrop. 

So we start with the root group (since it is the group backdrop of the isolated group) and composite element A and B onto the root groups initial backdrop of white (so blending white and element A, then the result of that with element B). Since `mix-blend-mode: difference` is a [separable blend mode](https://drafts.fxtf.org/compositing-1/#blendingseparable), we use `mix-blend-mode: normal` for these blendings. The result would be `rgb(191, 233, 191)`. As the root group has no group backdrop, we stop there.

Next we go to the isolated group. It's initial backdrop is transparent black, so we composite element C onto that using `mix-blend-mode: difference`. That results in `rgb(128, 0, 0, 0.5)`. Then we blend this color with the group backdrop, which is the composite of the root group, so `rgb(191, 233, 191)` using `mix-blend-mode: normal` (from what I gather from [effect of group isolation on blending](https://drafts.fxtf.org/compositing-1/#isolationblending)). The final result of the color is `rgb(160, 117, 96)`. However a visual inspection of the browser shows that this is not the case.

I've tried a few different combinations of trying to blend the two different groups, but nothing seems to work out correct. Further confusing to me is when there is just a single element with `mix-blend-mode: difference`, nothing changes about the color, which to me means that transparent black isn't being applied to the isolated group at all. It's not until I add white as the background of the `body` or `html` element does the `mix-blend-mode` take affect.

```html
<div style="background-color: rgba(204, 204, 204, 0.43); mix-blend-mode: difference;">Test element</div>
```

So how should these groups blend and composite together? Would it be possible to add explicit algorithms and diagrams showing how to composite and blend multiple isolated groups? That to me would be the most helpful in understanding what should happen. Multiple diagrams showing where the transparent black should be inserted into a stack of colors, how groups blend together, etc.

Thanks for your help.

PS. You can [see our code for blending here](https://github.com/dequelabs/axe-core/blob/bdf806823b3a84070553940a5e492c1f281acf8c/lib/commons/color/flatten-colors.js).

Please view or discuss this issue at https://github.com/w3c/fxtf-drafts/issues/440 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Monday, 25 October 2021 16:50:30 UTC