- From: Peter Graffagnino <pgraff@apple.com>
- Date: Wed, 2 Jun 1999 11:09:29 -0700
- To: www-svg@w3.org
These are good comments. The general rule is that for things that manipulate color independent of coverage, premultiplication is not convenient. But for things that manipulate the image as a whole, premultiplication is more natural. For example, I disagree with the analysis of the GaussianBlur case. To correctly evaluate a convolution through a mask or alpha channel, I would argue that it is more natural to weight the contribution of each pixel by its mask component, and then renormalize the result by the sum of the weights. This is much more complex in the non-premultiplied case. The reason for this is that the alpha channel says how much of the pixel is "there", so any contribution to a filter or resampling kernel should be appropriately weighted. Consider this example. I have a large non-premultiplied RGBA image where alpha is zero everywhere except for a smaller rectangle in the middle of the image where alpha is one. The color pixels are white in the rectangle and pink outside. Normally when I "use" this image by compositing it over a background I get a white rectangle -- the pink pixels are not seen since alpha is zero. In other words the pink pixels "don't contribute" to the image in normal useage. Now suppose I blur the image. If I don't "weight" the contribution of each pixel by its alpha channel, I will get pink in the resulting image, which arguably quite unexpected. If I "weight" the contributions by the alpha channel, no pink color will contribute to the result which is more natural. If the original image were premultiplied, I can just blur the 4 channels independently and everything works as expected. These considerations also apply when resampling an RGBA image while scaling - if I scale up the above image of a white rectangle with a bicubic resampling, I certainly would not expect to see pink pixels in the result. Hope this helps. For a good reference see the article on "Compositing theory and practice" in Jim Blinn's book, "Dirty Pixels". Peter Begin forwarded message: > From: Chris Lilley <chris@w3.org> > Date: 1999-06-02 18:29:26 +0200 > To: w3c-svg-wg@w3.org > Subject: [Fwd: Pre-multiplication of samples in Filter Effects] > X-Loop: w3c-svg-wg@w3.org > Resent-sender: w3c-svg-wg-request@w3.org > X-Mailer: Mozilla 4.6 [en] (WinNT; I) > X-Mailing-List: <w3c-svg-wg@w3.org> archive/latest/732 > Resent-date: Wed, 02 Jun 1999 12:38:13 -0400 (EDT) > Resent-from: w3c-svg-wg@w3.org > X-Accept-Language: en > > This just turned up on the public (www-svg) list. I'd like to add a > discussion of the issues to the Thursday telcon. > > -- > ChrisFrom: Christian Brunschen <cb@df.lth.se> > Date: Wed, 2 Jun 1999 16:01:25 +0200 (CEST) > To: www-svg@w3.org > Subject: Pre-multiplication of samples in Filter Effects > Resent-Sender: www-svg-request@w3.org > Resent-Date: Wed, 2 Jun 1999 10:01:31 -0400 (EDT) > X-Mozilla-Status2: 00000000 > X-Mailing-List: <www-svg@w3.org> archive/latest/136 > X-Loop: www-svg@w3.org > Resent-From: www-svg@w3.org > > > Greetings. > > Having just read the latest draft SVG spec, I cannot help but wonder one > thing. > > In section 15.5.4, the built-in Filter Effects are listed, with this > little paragraph as a lead-in: > > ``The following is a catalog of the individual processing nodes proposed. > All filters operate on linear premultiplied RGBA samples. Filters which > work more naturally on non premultiplied data (ColorMatrix and > ComponentTransfer) will temporarily undo and redo premultiplication as > specified.'' > > Since RGBA rasters can not store sample values with infinite resolution, > the combination of premultipluying and the un-premultiplying pixels is > almost guaranteed to introduce inaccuracies at some places. Assuming an > 8-bit-per-sample representation, a simple example is one where the > original pixel is painted with an RGBA value of 3,5,7,127 - a very dark > shade, with a 50% coverage. The resulting pre-multiplied pixel would > contain values something like 1,2,3,127 - and un-pre-multiplying would > result in 2,4,6,127, with each of the RGB samples being markedly different > from the original. Basically, pre-multiplication guarantees a loss of > information. > > I would further like to suggest that pre-multiplied alpha is not the best > choice for the default alpha handling for, for instance, the > 'DisplacementMap' Filter: > > -- begin quote -- > > NodeType DisplacementMap > > Image Inputs 2 > > Attributes scale > x-channel-selector one of R,G,B or A. > y-channel-selector one of R,G,B or A. > > Description Uses Input2 to spatially displace Input1, > similar to the Photoshop displacement filter. This is > the transformation to be performed: > > P'(x,y) <- P( x + scale * XC(x,y), y + scale * YC(x,y) ) > > where P(x,y) is the source image, Input1, and P'(x,y) is > the destination. XC(x,y) and YC(x,y) are the component > values of the designated by the x-channel-selector and > y-channel-selector. For example, to use the R component > of Image2 to control displacement in x and the G component > of Image2 to control displacement in y, set > x-channel-selector to "R" and y-channel-selector to "G". > > Comments The displacement map defines the inverse of the mapping > performed. > > Implementation > issues This filter can have arbitrary non-localized > effect on the input which might require substantial > buffering in the processing pipeline. However with this > formulation, any intermediate buffering needs can be > determined by scale which represents the maximum > displacement in either x or y. > > -- end quote -- > > If I want to displace something according to the 'R' and 'G' channels of > an image I have constructed, it is not necessarily obvious that I want the > pre-multiplied-alpha Red and Green channels. > > Furthermore, if I definitely want to use the _non_premultiplied-alpha Red > and Green channels from my image, I cannot get at them easily. > > > The lead-in paragraph states that ``Filters which > work more naturally on non premultiplied data (ColorMatrix and > ComponentTransfer) will temporarily undo and redo premultiplication as > specified'' which appears based in the assumption that for _all other > filters_, pre-multiplied-alpha is the `more natural' state of affairs. So > let's take a look at the different filters defined (these are in my > personal opinion, of course): > > Filter 'Most Natural' Comment > > ColorMatrix non-premultiplied explicitly stated in the > description > > Color N/A has no inputs > > ComponentTransfer non-premultiplied explicitly stated in the > description > > Composite premultiplied? could easily be written > to handle non-premultiplied > while maintaining fidelity > of output; or could be stated > to require premultiplied > alpha > > DiffuseLighting N/A uses only the Alpha channel, > does not care about wether RGB > are premultiplied or not > > DisplacementMap I'm not sure As I discussed above, I don't > but it feels like see how premultiplied could be > non-premultiplied seen as 'more natural' here > would be better anyway > > GaussianBlur non-premultiplied for maximum fidelity in the > resulting blurred image > > Image N/A has no inputs > > Merge premultiplied? premultiplied preferred for > performance reasons, but could > use non-premultiplied with the > same fidelity (like Composite) > > Morphology non-premultiplied to ensure fidelity in the > resulting image > > Offset N/A doesn't care about the colors > of the pixels it shifts > > SpecularLighting N/A only uses the Alpha channel > > Tile N/A doesn't care > > Turbulence non-premultiplied to ensure fidelity, or else > the values to be added also > have to be premultiplied, lest > non-opaque pixels receive an > unproportionally large change > > > So, in summary, of the 14 filters in the spec, 2 _definitely_ need > non-premultiplied Alpha (already in the spec), 4 more seem to me to > probably be better off with non-premultiplied, 6 simply don't care, and > only 2 can really make a demand on premultiplied - Composite and Merge. So > I would interpret this to mean that non-premultiplied would be overall > 'more natural' than premultiplied. > > Again, this is all in my own opinion. > > > Anyway, my main argument for _not_ premultiplying RGN with A is that of > maintaining fidelity, of not losing resolution in the RGB channels just > because they happen to have a not-fully-opaque A value. > > The main argument for premultiplied Alpha would be performance, unless I > am mistaken: Compositing an image with pre-multiplied Alpha onto another > saves one multiplication per pixel over Compositing the same image > non-premultiplied. > > However, consider the case of a graphic that gets offset, a ColorMatrix > filter applied, then offset again (stupid example, but bear with me): > > Premultiplied Alpha > > * Graphic renders to RGBA matrix : premultiplications take place here > * Offset: shifts pixels around > * ColorMatrix: _Un_premultiply, perform the filter, then _Re_premultiply > * Offset: shift pixels around > * Merge: composite to destination > > Multiplications take place in three places: Once when the graphic is > rendered to the RGBA matrix, then when Un-premultiplying before the > ColorMatrix filter can do its job, then re-premultiplying afterwards. > > > Now, with non-premultiplied Alpha: > > * Graphic renders to RGBA matrix > * Offset: shifts pixels around > * ColorMatrix: perform the filter > * Offset: shift pixels around > * Merge: multiply RGB with A, then composite to destination > > There's only one place where RGB samples are multiplied with A, and that's > at the very end when the resulting image is to be composited onto the > background. > > > Note also that in the non-premultiplied case, the ColorMatrix filter can > access the actual RGB values rendered by the Graphic, rather than the > 'pre-multiplied, then un-pre-multiplied' values offered with the current > scheme. > > > For those cases where a certain image will be used pre-multiplied more > than once, it would be easy to define a filter which premultiplies RGB > with A on an image: > > -- begin suggestion -- > > NodeType PreMultiply > > Image Inputs 1 > > Attributes none > > Description multiplies each of the R, G and B channels with the A > channel > > Comments a very common operation > > Implementation > issues none, really > > -- end suggestion -- > > which could be used whenever needed, for whatever reason. > > > So, in summary, I am questioning the use of premultiplied alpha as the > default representation for RGBA images in filter effects. > > > Best regards > > // Christian Brunschen >
Received on Wednesday, 2 June 1999 14:09:50 UTC