Re: [Fwd: Pre-multiplication of samples in Filter Effects]

On Wed, 2 Jun 1999, Peter Graffagnino wrote:

> These are good comments. The general rule is that for things that  
> manipulate color independent of coverage, premultiplication is not  
> convenient.  But for things that manipulate the image as a whole,  
> premultiplication is more natural.  

So basically we have two general cases which have different requirements.
Hm.

> For example, I disagree with the  
> analysis of the GaussianBlur case.  To correctly evaluate a  
> convolution through a mask or alpha channel, I would argue that it is  
> more natural to weight the contribution of each pixel by its mask  
> component, and then renormalize the result by the sum of the weights.  
>  This is much more complex in the non-premultiplied case.  

I personally don't think that the weighting / renormalization is 'much
more complex' - it's exactly the same amount of work that is added to the
ColorMatrix filter (first un-pre-multiply, then re-pre-multiply
afterwards), except in reverse; and that extra work was obviously not seen
to be prohibitively much.

> The  
> reason for this is that the alpha channel says how much of the pixel  
> is "there", so any contribution to a filter or resampling kernel  
> should be appropriately weighted.

In some cases - in some of the filters which are defined in the SVG draft 
- the alpha channel is not used as 'coverage' information at all, but for
instance as a 'bump map'. If I want to gaussian blur my image as well as
the bump map (which happens to be stored in the alpha channel), I most
certainly do not want the pixels to be premultiplied, because in that case
the alpha channel is used not as coverage information, but as a convenient
place to store a bump map. And this is a use of the alpha channel that is
already defined in the SVG draft, this is not just a random invention of
mine.

> Consider this example.  I have a large non-premultiplied RGBA image  
> where alpha is zero everywhere except for a smaller rectangle in the  
> middle of the image where alpha is one.  The color pixels are white  
> in the rectangle and pink outside.  Normally when I "use" this image  
> by compositing it over a background I get a white rectangle -- the  
> pink pixels are not seen since alpha is zero.  In other words the  
> pink pixels "don't contribute" to the image in normal useage.  Now  
> suppose I blur the image.  If I don't "weight" the contribution of  
> each pixel by its alpha channel, I will get pink in the resulting  
> image, which arguably quite unexpected.  If I "weight" the  
> contributions by the alpha channel, no pink color will contribute to  
> the result 

Complete agreement so far.

> which is more natural.  

... if you want to achieve that effect; however, if I have carefully
constructed the image to have those colors and then blur it exactly in
order to geth the  blending between the white and the pink at the edge,
while at the same time the coverage decreases, then non-premultiplied
alpha would be 'more natural'. Or, at least, the _desired_ result.

> If the original image were  
> premultiplied, I can just blur the 4 channels independently and  
> everything works as expected.  These considerations also apply when  
> resampling an RGBA image while scaling - if I scale up the above  
> image of a white rectangle with a bicubic resampling, I certainly  
> would not expect to see pink pixels in the result.

Again, I would argue that while in most cases you probably would not
expect pink to show up, I can also see how someone might indeed expect
and desire such a result.

> Hope this helps.


Greatly :)


But this doesn't restrain me from making comments and suggestions; please,
bear with me. 

We have two different possible results of the operation, depending on
wether the filter works on premultiplied or non-premultiplied pixels.
Now, regardless of which is in fact more natural, wouldn't it be nice to
allow the user to decide, by allowing the user to choose which sort of
image to send to the filter? 

Including a pair of 'PreMultiply' and 'UnPreMultiply' filters would allow
explicit control over this; each other filter could just assume that it
can work with whatever pixels it gets. That way, people who want to access
_weird_ behaviour (such as applying a ColorMatrix to premultiplied
pixels, or a GaussianBlur to non-premultiplied ones) could do so. 

Another thing which would be possible would be to 'tag' the RGBA matrix
to indicate wether it had premultiplied alpha or not, and adding a
parameter to each filter to indicate wether it should first convert its
input data to the most 'natural' format, or should accept them as they
are, where the default would obiously be to convert the data to the most
'natural' format. This would preserve the current semantics completely,
while making conversions 'lazy' and at the same time offering SVG authors
the chance to use the 'unnatural' variations of the filters in order to
achieve whatever effects they desired. This, coupled with 'PreMultiply'
and 'UnPreMultiply' filters (which would explicitly convert the input
image to the desired format, if it was not already in that format) would
offer maximum flexibility, maximum fidelity (for instance, a 'ColorMatrix'
followed by a 'ComponentTransfer' would not lose any accuracy due to pre-
and then un-pre-multiplication), and maximum performance (unnecessary
conversions don't take place and thus don't take time).

> For a good reference see the article on "Compositing theory and  
> practice" in  Jim Blinn's book, "Dirty Pixels".

I will, thank you for the reference.

> Peter

Best regards

// Christian Brunschen

Received on Thursday, 3 June 1999 06:30:34 UTC