Re: Compositing and Blending

A RESEND TO THE GROUP WITH NOTES INSERTED...

On 8/6/12, Rik Cabanier <cabanier@gmail.com> wrote:
> On Mon, Aug 6, 2012 at 2:50 AM, Bob Holmes <rangsynth@gmail.com> wrote:
>
>> If it helps...
>>
>> 1) In the previous composite spec it gave concise formulas for all
>> modes and the modes like lighten and screen(now the blending modes in
>> the new spec), were also given in full. Although now I see that the
>> formulas that include blending in the old spec is only for "src-over".
>> I completely did not understand that. If you look at the code for "agg
>> 2.4" you will see they misread that in the same way. Seems maybe to be
>> the case for the Skia library as well. I completely did not understand
>> that the "src-over" mode was being incorporated with the blending
>> function in the old spec.
>>
>
> Yes, this was not clearly spelled out in that spec.You can infer it by
> looking at the formulas but that's it.
>
>
It will help a lot of people implementing if there can be a section on
this concept of how the old mode composites combined the blending with
src-over. That threw me off a lot.
>>
>> 2) Section 5.1. in the new spec...
>> The text starts describing Porter Duff operators and then show some
>> examples in red. I think that prior to this section or as part of the
>> section must be some text that places much more importance on the fact
>> that "composite" can be applied directly with "blending". There is one
>> long paragraph and then it jumps directly to the definition for clear,
>> but there is no mention of blending! What would have made that part
>> really simple to understand is if it had two examples, say for
>> src-over and then for xor that completely define the formulas
>> "combined" with the blending function.(To show that compositing is
>> able to work with blending but that it can be separated as an
>> operation on Cs and Cb beforehand to produce a new Cs).
>>
>> 3) Section 6. Blending
>> This section also starts with some text about blending and then goes
>> on to describe an example that uses simple alpha composite together
>> with the blend mode. However there is not really any simple alpha
>> composite in section on Porter Duff Operators.
>
>
> See section 1.2. In the PD section this is called source-over.
>
>
>> If the example was
>> labelled "Example of blending function with Src-Over" that might make
>> more sense, and it would make even more sense if it was like two
>> versions of the example with the one showing how the unpremultiplied
>> result of the blending function is then re-premultiplied before going
>> through the compositing logic to produce the premultipled result.
>>
>
> Well, we try to keep them as separate operations.
> Blending simply creates a new source color that is then fed into
> compositing. I'm not sure if we need to mention it in the compositing
> section.
> The only reason that the complete formula is there, is only as an example.
>
I just fully understand that now. In the place where it says about it
forming a new graphic element for the blending and then plugging that
into the composite function it will help if it can say that the alpha
is unaffected and that the blending only works on RGB.
>
>>
>> What would make it even better if there were more examples and each
>> one was clearly labelled with a heading. For example "Example of
>> blending function with Xor Composite" and "Example of blending
>> function with Src-In" composite. Putting such examples would make it
>> immediately clear that implementations can choose to break down the
>> thing into one combined part and also that there are differences in
>> how the blending is combined with the composite.
>>
>
> I agree. We need more examples.
>
>
>>
>> The number of implementations I have seen all doing the composite and
>> blending as if all blending was done with "src-over" is many. Like for
>> agg the functions are even labelled using the notation of the old svg
>> composite spec so as a result the lighten mode and the screen mode in
>> agg are using src-over composite! There is no way to then get all the
>> different composites combined. The apple quartz library seems to have
>> the same singular blend mode as well.
>>
>
> True. This is also the case in PDF and Flash.
>
>
>>
>> I was not even planning on using premultiplied values before so that
>> is why it was so confusing. I come from a background of drawing with
>> 24 bit bitmaps in windows. The learning of keeping a saved alpha with
>> premul is definitely confusing from that perspective.
>>
>
> In the spec, premultiplied alpha is only there to make the formulas easier.
> You are free to implement them however you feel. If you stay in
> non-premultiplied spec, you will have better quality in 8 bit mode since
> you don't lose precision bits.
>
I am only putting in premul now. I have this PixelFmt which describes
the pixels of the input to each successive drawing object. The
pixelFmt has bits like pxfPremul and pxfAlpha. If the premul bit is
set then it uses premultipled versions otherwise non-premul versions
for example. Some sources cannot be premultiplied and some sources do
not have the concept, like 24 bit source pixels coming through in RGBA
words can be composited much faster since the input pixelFmt at the
blending point says there is no alpha or premul(if there were no
filters to make it otherwise).


>
>> ----------------------
>>
>> 4) Porter Duff Ops
>> For each porter duff operator like "5.1.10. Source Atop" it would be
>> very useful if there could be extra formulas.
>>
>> This describes the basic Source Atop
>>  Fa = áb; Fb = 1 - ás
>>     co = ás x Cs x áb + áb x Cb x (1 - ás)
>>     áo = ás x áb + áb x (1 - ás)
>>
>> What would help a lot is if there can be for each operator the
>> definition of where/how the blend mode is incorporated as well as or
>> in addition to the basic formula of the operator.
>>
>> And to complement that, for each blend mode, an example using one of
>> the composite operators would help a lot.
>>
>> ===========================================
>> I am going to be implementing it as split now because it is definitely
>> better...
>> 1) Much more effects
>> 2) Because the blender is defined to use unpremul colors additional
>> custom blending functions can be ad libbed to create a lot of effects,
>> and the premul before actually doing the composite.
>> 3) Because the blender op produces an unpremul result, if effects like
>> a "global blend" need to be implemented the code can blend for example
>> half the original background back into the output of the blender to
>> create even more effects or to do "POST BLENDING" blending per
>> channel. So after the blender if I want to keep the red channel as was
>> from the source I can do so. but leave the green and blue as was
>> produced by the blend function.
>>
>> HOWEVER...
>> a) If the composite function is "src-over"
>> b) and there are no custom color effects or post blending blending.
>> --> Then use the optimized path as described in the old version of SVG
>> composite spec.
>> --> And also try to use optimized paths for any other common
>> blending/composite steps etc...
>>
>
> Yes, that sounds right.
> In what software are you planning on implementing this?
>
>
I am currently using delphi 5/object pascal. I had planned to map it
out in pascal and then go to C++ but C++ just does not have the
features, so it will stay as pascal code because it has procedures of
objects and try/finallys and so on that are excellent for this type of
modular work.
>>
>> MISSING
>> =======
>> So from the spec I think missing is features that are common in some
>> ways...
>> 1) Global Alpha...
>>   A global constant applied to each src pixels alpha before entering
>> the blending function. Not per channel, just modifies the alpha of the
>> source pixel.
>>
>
> that already exists as alpha on a group. Also, it is defined in different
> specs as 'opacity'.
> Do you think we need to call it out?
Yes. Opacity is a confusing term. A name like "GlobalAlpha" is more
meaningful since it refers directly to the alpha. Actually I implement
this using two values. AlphaMin and AlphaMax. If AlphaMin is zero then
it is like normal. Otherwise it quantizes the alpha between alphaMin
and AlphaMax. Otherwise if AlphaMin is greater than AlphaMax it
inverse quantizes the alpha into the inverted range. Just to give it
full effect capability.
>
>
>>
>> 2) Custom blend function - Theoretically, since the blending function
>> is defined to work with unpremul it can be mentioned that a large
>> variety of different functions can be implemented as the blending
>> function.
>>
>
> There have been a couple of request for this.
> Unfortunately, getting this implemented securely is not possible today (or
> at least, we haven't found a way)
>
> A future spec will have support for them.
>
>
>>
>> 3) "POST BLENDING" functions. The result of blending is defined as
>> unpremultipled and therefore additional things can be done after the
>> blending, such as keeping unblended source or dest channels.
>>
>> 4) Composite/Blender chains.
>> An implementation is not tied to having just one blend/composite
>> object. A stack of blenders can be created and then called bottom to
>> top where each blenders output becomes the input to the next and
>> finally onto the canvas pixels. Like "LIGHTEN/XOR" into
>> "SCREEN/SRC-OVER" for example. This might be for lower level canvas
>> work. I implement the extra blenders as a list called "XBlenders" that
>> is not there by default but can be added to to create chains of
>> blenders. Then normally I just have the "blender".
>>
>
> maybe... I'm not sure if this is a strong use case. You could implement
> this with grouping with the current spec...
>
>
>>
>> =======
>> So please, any more examples would be appreciated. I think if you look
>> at Skia, Agg and other libraries you will note there is huge confusion
>> all over the place about the blending vs. composite step. Therefore it
>> should definitely be split.
>>
>> THE SOLUTION FOR CHANGING EXISTING INCORRECT IMPLEMENTATIONS
>> Current imps use modes like "screen" as a "SCREEN/SRC-OVER".
>> The way to correct these imps is to call the current version of screen
>> the Screen_src_over operator as a compositing operator, and then it
>> can be safely fused into a system that uses the split blending and
>> composite from the new spec. It would result in about ten or more new
>> compositing operators for such implementations, which are then just
>> variations that have been incorrectly implemented.? I think I will put
>> this so if someone wants legacy colorDodge as implemented everywhere
>> they can just say "canvas.compositeMode :=
>> blendModeColorDodge_SrcOver" and leave the "canvas.blendMode" alone.
>>
>> But new code can say "canvas.compositeMode = srcOver" and
>> "canvas.blendMode = colorDodge"
>>
>> I hope any of this helps. Thanks.
>>
>> On 8/6/12, Bob Holmes <rangsynth@gmail.com> wrote:
>> > Two stage blend then composite is how I originally implemented it.
>> > This was a while back and I just implemented from descriptions of
>> > raster modes in windows help files and from the modes in apple quartz
>> > help files.
>> >
>> > So my first implementation used unpremultiplied for what was called
>> > the blend, and then I had an option of either doing a source over or a
>> > direct write after the blend.
>> >
>> > I was reading your spec, libpixman and the apple quartz files and pdf
>> > and it seemed like it was all one stage, so I changed my
>> > implementation to support a one stage approach.
>> >
>> > With one operator only if you want to do color effects after the
>> > composite then you need to unpremultiply the result, and also possibly
>> > the input dca and sca if you need those in post composite effects.
>> >
>> > I think the split is a good idea, but there seem confusing statements
>> > floating around in SVG spec itself that says things like this for
>> > "feBlend" filter effect...
>> >
>> > "NORMAL MODE IS THE SAME AS SRC-OVER ON feCOMPOSITITE and the SAME AS
>> > DEFAULT BEHAVIOUR ON feMerge"
>> >
>> > But how can the NORMAL BLEND MODE be the same as SRC-OVER. Surely that
>> > terminology means that there is NO BLEND MODE and ONLY SRC-OVER
>> > COMPOSITE.
>> > Your new spec does not do this and just returns "Cs" for NORMAL MODE.
>> >
>> > Does this mean that feBlend will get a composite mode as well?
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On 8/6/12, Rik Cabanier <cabanier@gmail.com> wrote:
>> >> On Sun, Aug 5, 2012 at 3:10 AM, Bob Holmes <rangsynth@gmail.com>
>> >> wrote:
>> >>
>> >>> THESE DOCUMENTS...
>> >>>
>> >>> 1)
>> >>>
>> https://dvcs.w3.org/hg/FXTF/rawfile/tip/compositing/index.html#backdropexamples
>> >>> 2) http://www.w3.org/TR/SVGCompositing/
>> >>>
>> >>> States that ...
>> >>>
>> >>> Blending is the aspect of compositing that calculates the mixing of
>> >>> colors where the source element and backdrop overlap. Blending takes
>> >>> the colors of the source element and mixes them with the backdrop in
>> >>> areas where the source element and backdrop overlap. Conceptually,
>> >>> the
>> >>> colors in the source element are blended in place with the backdrop.
>> >>> After blending, the modified source element is composited with the
>> >>> backdrop. In practice, this is usually all performed in one step.
>> >>>
>> >>> I THINK WHAT IT MEANS TO SAY IS THAT "ALL THE BLENDING AND
>> >>> COMPOSITING" IS ACTUALLY REDUCED TO WELL DEFINED FUNCTIONS AS
>> >>> DESCRIBED IN THE SECOND DOCUMENT.
>> >>>
>> >>> In other words the first document is making a distinction between the
>> >>> blending and the compositing, but in theory the functions are defined
>> >>> as singular functions already in the second? So why the distinction
>> >>> between the two in the first document?
>> >>>
>> >>> Is this correct?
>> >>>
>> >>
>> >> Not quite. Only blending + src-over is described in the SVG
>> >> compositing
>> >> spec.
>> >> In practice, this is what will happen most of the time so I expect
>> >> that
>> >> browsers (that do software rendering) will optimize that particular
>> path.
>> >>
>> >> The new document allows any composting operation to happen after
>> >> blending.
>> >> This will allow effects that were not very hard or impossible to
>> >> achieve
>> >> with the old spec.
>> >> For instance, the new spec allows you to blend with the backdrop, but
>> >> then
>> >> composite with 'source'. This will remove the backdrop but will still
>> >> show
>> >> the interaction of the backdrop with the source.
>> >> Other hard to achieve effects allow masking of the blended source with
>> >> the
>> >> backdrop,
>> >>
>> >> We've had a conversation on the splitting of the properties on this
>> list.
>> >> Just search for [css-compositing].
>> >> Let me know if you have question or if you feel that the split should
>> not
>> >> happen.
>> >>
>> >> Rik
>> >>
>> >
>>
>

Received on Monday, 6 August 2012 21:48:14 UTC