[whatwg] Support filters in Canvas

Hi,

I'd like to revive this discussion.

On Sat, Mar 15, 2014 at 12:03 AM, Dirk Schulze <dschulze@adobe.com> wrote:

> I would suggest a filter attribute that takes a list of filter operations
> similar to the CSS Image filter function[1]. Similar to shadows[2], each
> drawing operation would be filtered. The API looks like this:
>
> partial interface CanvasRenderingContext2D {
>     attribute DOMString filter;
> }
>
> A filter DOMString could looks like: “contrast(50%) blur(3px)”

This approach sounds good to me, and it's what I've implemented for
Firefox in bug 927892 [1]. The Firefox implementation is behind the
preference canvas.filters.enabled which is currently off by default.

> Filter functions include a reference to a <filter> element and a
> specification of SVG filters[4]. I am unsure if a reference do an element
> within a document can cause problems. If it does, we would just not support
> SVG filter references.

I've included support for SVG filters in the Firefox implementation.
It's a bit of work and it increases the number of edge cases we need
to specify, but I think it's worth it.

Here's a more fleshed-out proposal that attempts to define the edge
cases I've encountered during development.

The ctx.filter property should behave like the ctx.font property in some senses:
 - It's part of the state of the context and honors ctx.save() and
ctx.restore().
 - Setting an invalid value is ignored silently.
 - Both "inherit" and "initial" are invalid values, as with font.
 - Setting a valid value sets the current state's filter to that
value, and the getter will now return this value, possibly
reserialized.
Question: Do we want the getter to return the serialized form of the
filter? I don't really mind either way, and I'm not sure in what cases
the results would differ. I guess extraneous whitespace between
individual filter functions would be cleaned up, and "0" length values
would get set to 0px. Anything else?
- Resetting the state to "no filtering" is done using ctx.filter =
"none". Values such as "", null, or undefined are invalid and will be
ignored and not unset the filter.
Question: Is this what we want?

Filter rendering should work similarly to shadow rendering:
 - It happens on every drawing operation, with the input to the filter
being what that operation would have rendered regularly.
 - The transform of the context is applied during rendering of the
input. The actual filtering is not be subject to the transform and
happens in device space. This means that e.g. a drop-shadow(0px 10px
black) filter always offsets the shadow towards the bottom, regardless
of the transform.
 - The results in the canvas pixel buffer will be the same regardless
of the CSS size of the canvas in the page, and regardless of whether
the canvas element is in the page at all or just a detached DOM node.
 - The global composite operation is respected when compositing the
filtered results into the canvas contents. The filter input drawing
operation is always rendered with "over" into a conceptual transparent
temporary surface.
 - The same applies for global alpha.

Interaction with shadow:
 - If both a filter and a shadow are set on the canvas, filtering will
happen first, with the shadow being applied to the filtered results.
In that case the global composite operation will be respected when
compositing the result with shadow into the canvas.
 - As a consequence of the other statements, this is true: If a valid
filter is used, appending " drop-shadow(<shadowOffsetX>px
<shadowOffsetY>px <shadowBlur>px <shadowColor>)" to the filter will
have the same results as using the shadow properties, even if there is
a transform on the context.

Units:
 - The CSS px unit refers to one canvas pixel, independent of the CSS
size of the canvas on the page. That is, a drop-shadow(0 10px black)
filter will have the same results in the canvas-internal pixel buffer,
regardless of whether that canvas is specified using <canvas
width="100" height="100" style="width: 100px; height: 100px;"> or
<canvas width="100" height="100" style="width: 20px; height: 20px;">.
 - Lengths in non-px units refer to the number of canvas pixels you
get if you convert the length to CSS px and interpret that number as
canvas pixels.

Font size relative units:
 - Lengths in em are relative to the font size of the canvas context
as specified by ctx.font.
 - The same applies for lengths in ex; and those use the x-height of
the font that's specified in ctx.font.

SVG filter specific considerations:
 - Relative URLs in SVG filter reference url() functions are relative
to the canvas element (i.e. the base URL of the owner document of the
canvas element (?)).
 - The "user space" for SVG filtering is (0, 0, canvas.width,
canvas.height), with one user space unit equal to one canvas pixel
(equal to one CSS pixel).
 - The "bounding box of the filtered element" is also (0, 0,
canvas.width, canvas.height) and independent of what the filtered
drawing operation renders.
 - Filter regions and subregions are respected as usual.
 - SourceGraphic and SourceAlpha inputs are supported in the expected way.
 - FillPaint and StrokePaint refer to a filter-region-sized input that
is filled with the current fillStyle / strokeStyle of the context.
 - BackgroundImage and BackgroundAlpha refer to the contents of the
canvas before the drawing operation.
Question: Do we want this? I haven't included it in the Firefox
implementation, because Firefox doesn't yet support BackgroundImage /
BackgroundAlpha anywhere.

Liveness considerations:
 - If SVG filter references are used ("url(#someFilter)"), changes to
the referenced filter are respected during the next drawing operation.
The user does not need to assign ctx.filter another time for that to
happen.
 - The same applies when changing the canvas context font size with
ctx.font if font size relative units are used in the filter.

Async-related considerations:
 - If the filter refers to an SVG filter in an external resource
document, and that document hasn't finished loading when the filtered
drawing operation is invoked, nothing gets drawn.
Question: This is slightly suboptimal. Do we want to ignore the filter
instead? Do we want to add a DOM event that indicates that all
resources needed for filtering are now available?
 - For a <feImage> primitive, if the required image hasn't finished
loading at the time of drawing, this <feImage> primitive renders
transparent black.

Random other things:
 - The "font color" of the context is always "black". This is used for
the CSS drop-shadow() filter function if no shadow color is specified.
Since the color of normal canvas text drawing is determined by the
fillStyle of the context, there's not really anything we can reuse
here.

Please discuss. :-)

Greetings,
Markus

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=927892

Received on Monday, 29 September 2014 17:20:39 UTC