SVG Filters 2

Apologies if this is a duplicate, but I sent the original mail to this
list yesterday and it still doesn't show on the archive page, so I'm
trying again:

In my opinion there are still many scenarios when SVG filters are
superior to Canvas based filtering. Yes, you can include a VIDEO
element, hide it, render it to canvas on timechange and then finally
apply the effects but it's just much more natural to apply the effect
directly to the video and let the browser figure out how and when to
update it.

And I think there's room for both approaches in one future SVG filters
spec: The filter-primitive-for-a-specific-purpose approach of the
current spec (which is well suited for graphics) and a more general
system that allows you to implement basically anything.

The goal would be to streamline filters by getting rid of certain
properties, mainly:
a) Strange implicit behavior (like implicit, non-skipable color
pre-multiplication)
b) A tendency of primitives to be tailored to one very specific purpose.

You may notice a lot of what I'm writing below mirrors behavior you'll
typically find in programs like Blender's compositing mode or
AviSynth. That's no coincidence: In my opinion filtering dynamic
content easily is possibly the strongest point of SVG Filters vs.
Canvas, so why ignore what we can learn from applications that have
done this for a very long time?

===Value attributes===
There are currently quite a few filter primitives that perform the
same function, but one time with constant values and one time with
input from a second channel... wouldn't it nice to unify them?

For example:
feBlend[type=screen] and feComponentTransfer.feFunc*[type=linear] are
both used to adjust the brightness of an image: One time using a
second input and one time a constant value.

Instead, we could have just one primitve: Let's call it feBrighten.
This primitive could be used with different kinds of arguments for "in" and in2:
 in2="0.5" would add 0.5 to every channel
 in2="someResult" would add each channel from someResult to the
corresponding one from "in"
 in2="0.1 0.2 0.3 someResult[0]" would add 0.1 to red, 0.2 to green,
0.3 to blue and the 0th channel (red) of someResult to channel 3
(alpha).
(The precise notation would probably have to be different in order to
not break SMIL and a lot of other stuff, but you get the idea).

===Stop interpreting input data===
You may have noticed that I wrote [0] above and not .red:
The point is that limiting channel 0 to Redis an unnecessary
restriction. Why should channel 0 have to be red? Couldn't it contain
intensity or alpha? Don't think of [0] as red, [1] as green and so on.
Just think of them as data.
The reason this is important is that in the current SVG filters spec
we limit many operations to certain channels, which often means that
the developer will do a lot of unnecessary colorMatrix operations just
to remix channels (or give up when he realizes that he cannot work
freely with the alpha component due to colorMatrix'
un-pre-multiplication), when there is no actual reason why he
shouldn't be able to apply any operation to alpha the same way as he
does to red.

Doing this would also mean getting rid of pre-multiplied colors (or at
least the implicit conversions and the default setting of using
pre-multiplied RGBA input), since these link RGB to A and make
independent working impossible.

Of course, there are some operations that only make sense in a certain
colorspace, but I think with good documentation there is little danger
of this confusing people. And even those primitives should allow for
mapping the input data differently than standard RGBA.

===Colorspace converters===
If you stop linking channel data to a particular meaning, you're
actually free to allow easy usage of different colorspaces. just
provide a converter like
<feConvert source-format="RGBA" dest-format="HSI" />
or
<feConvert source-format="RGBA" dest-format="RGBApre" />
and just let the developer figure out what to do with it. After all,
they should know best.

===Allow filter primitves to only operate on some channels===
Sometimes you want an effect to only run on say, channel 0,1,2 (RGB
for RGBA). Other times you may want it to run only on channel 3 (A on
RGBA): No need to provide different filter primitives for that, just
add a common attribute to all elements target="channel list", for
example target="0 1 2". The important thing is that you are able to
actually leave channels untouched.

===Inline SVG===
Right now, you can link in external resources via feImage, but often
you need only something very simple, like a rotated or a scaled image.
Using an external reference for this or adding the necessary elements
to the local DOM in order to reference them means that your filter is
no longer self-contained. Data URLs work, but that's at best an ugly
workaround. Additionally, you can't feed any results that your filter
has generated so far into it.

If a feViewport element could include full SVG directly that would
make filters that much cleaner.

This also would keep SVG filters from duplicating parts of SVG: Often
in a filter you have to move something around, rotate it or scale it.
Or you need a gradient as input data. You could either do that by
adding special transformation, gradient, ... filter primitives (much
like feOffset) or allow inline SVG. If you give inline SVG the power
to reference the results that the filter has created so far, you get
very powerful functionality without duplicating work that has already
been done on SVG.

A transformation of a blurred image may then look something like this:
<feGaussianBlur result="foo" />
<feViewport result="bar">
   <image xlink:href="result(foo)" transform="..." />
</feViewport>

(It would also be very nice if 3D transformations were available, but
that's something that should be handled in SVG directly, not SVG
filters)

===Add basic math===
No filter system can ever capture all eventualities, but it's possible
to add a fallback: mathematical filters. Many of these are needed
anyway (like + for blending and brightness adjustment or * for
contrast), but something like division, modulo and so on is not
available in the current SVG filters spec. Adding them would be
trivial and the benefit would be enormous, especially if coupled with
good feature-testing (so that a SVG file could use the fallback only
if a newer, faster extension is not available). Basically, you could
provide endless backwards compatibility for your filters by simply
specifying a fallback made of mathematical filters.

===Precise mode===
Usually the range 0-255 is fine, but especially if you allow
mathematical filters, then it will sometimes be useful to keep the
results with maximum precision and unclamped as floats. It shouldn't
be enabled by default (since memory usage would explode), but its
availability would add tremendous value to the math filter primitives.
For math filter chains you'll usually end up with a direct chain where
only one or two buffers get reused over and over again, so memory
usage should not increase dramatically if this option is used
properly.



Of course these are all just half-baked ideas. For example changing
all operations from premultiplied to non-premultiplied by default
would completely break backwards-compatibility, but better to talk
about it now than to wait until the damage is done :) . What do you
think? Is this a direction that would be possible for a SVG filters
update? Or is it too radical?

--
Hans Schmucker
Mannheim
Germany
hansschmucker@gmail.com
http://www.tapper-ware.net

Received on Sunday, 30 May 2010 23:44:55 UTC