Re: [whatwg] Support filters in Canvas

Hi Markus,

Thanks for coming back with you implementation details. I think most of the suggestions make sense overall but I’ll look at them in more detail, especially regarding filter regions. Just a question to SVG filter references inline for now.


On Sep 29, 2014, at 7:20 PM, Markus Stange <mstange@themasta.com> wrote:

> Hi,
> 
> I'd like to revive this discussion.
> 
> On Sat, Mar 15, 2014 at 12:03 AM, Dirk Schulze <dschulze@adobe.com> wrote:
> 
>> I would suggest a filter attribute that takes a list of filter operations
>> similar to the CSS Image filter function[1]. Similar to shadows[2], each
>> drawing operation would be filtered. The API looks like this:
>> 
>> partial interface CanvasRenderingContext2D {
>>    attribute DOMString filter;
>> }
>> 
>> A filter DOMString could looks like: “contrast(50%) blur(3px)”
> 
> This approach sounds good to me, and it's what I've implemented for
> Firefox in bug 927892 [1]. The Firefox implementation is behind the
> preference canvas.filters.enabled which is currently off by default.
> 
>> Filter functions include a reference to a <filter> element and a
>> specification of SVG filters[4]. I am unsure if a reference do an element
>> within a document can cause problems. If it does, we would just not support
>> SVG filter references.
> 
> I've included support for SVG filters in the Firefox implementation.
> It's a bit of work and it increases the number of edge cases we need
> to specify, but I think it's worth it.
> 
> Here's a more fleshed-out proposal that attempts to define the edge
> cases I've encountered during development.
> 
> The ctx.filter property should behave like the ctx.font property in some senses:
> - It's part of the state of the context and honors ctx.save() and
> ctx.restore().
> - Setting an invalid value is ignored silently.
> - Both "inherit" and "initial" are invalid values, as with font.
> - Setting a valid value sets the current state's filter to that
> value, and the getter will now return this value, possibly
> reserialized.
> Question: Do we want the getter to return the serialized form of the
> filter? I don't really mind either way, and I'm not sure in what cases
> the results would differ. I guess extraneous whitespace between
> individual filter functions would be cleaned up, and "0" length values
> would get set to 0px. Anything else?
> - Resetting the state to "no filtering" is done using ctx.filter =
> "none". Values such as "", null, or undefined are invalid and will be
> ignored and not unset the filter.
> Question: Is this what we want?
> 
> Filter rendering should work similarly to shadow rendering:
> - It happens on every drawing operation, with the input to the filter
> being what that operation would have rendered regularly.
> - The transform of the context is applied during rendering of the
> input. The actual filtering is not be subject to the transform and
> happens in device space. This means that e.g. a drop-shadow(0px 10px
> black) filter always offsets the shadow towards the bottom, regardless
> of the transform.
> - The results in the canvas pixel buffer will be the same regardless
> of the CSS size of the canvas in the page, and regardless of whether
> the canvas element is in the page at all or just a detached DOM node.
> - The global composite operation is respected when compositing the
> filtered results into the canvas contents. The filter input drawing
> operation is always rendered with "over" into a conceptual transparent
> temporary surface.
> - The same applies for global alpha.
> 
> Interaction with shadow:
> - If both a filter and a shadow are set on the canvas, filtering will
> happen first, with the shadow being applied to the filtered results.
> In that case the global composite operation will be respected when
> compositing the result with shadow into the canvas.
> - As a consequence of the other statements, this is true: If a valid
> filter is used, appending " drop-shadow(<shadowOffsetX>px
> <shadowOffsetY>px <shadowBlur>px <shadowColor>)" to the filter will
> have the same results as using the shadow properties, even if there is
> a transform on the context.
> 
> Units:
> - The CSS px unit refers to one canvas pixel, independent of the CSS
> size of the canvas on the page. That is, a drop-shadow(0 10px black)
> filter will have the same results in the canvas-internal pixel buffer,
> regardless of whether that canvas is specified using <canvas
> width="100" height="100" style="width: 100px; height: 100px;"> or
> <canvas width="100" height="100" style="width: 20px; height: 20px;">.
> - Lengths in non-px units refer to the number of canvas pixels you
> get if you convert the length to CSS px and interpret that number as
> canvas pixels.
> 
> Font size relative units:
> - Lengths in em are relative to the font size of the canvas context
> as specified by ctx.font.
> - The same applies for lengths in ex; and those use the x-height of
> the font that's specified in ctx.font.
> 
> SVG filter specific considerations:
> - Relative URLs in SVG filter reference url() functions are relative
> to the canvas element (i.e. the base URL of the owner document of the
> canvas element (?)).
> - The "user space" for SVG filtering is (0, 0, canvas.width,
> canvas.height), with one user space unit equal to one canvas pixel
> (equal to one CSS pixel).
> - The "bounding box of the filtered element" is also (0, 0,
> canvas.width, canvas.height) and independent of what the filtered
> drawing operation renders.
> - Filter regions and subregions are respected as usual.
> - SourceGraphic and SourceAlpha inputs are supported in the expected way.
> - FillPaint and StrokePaint refer to a filter-region-sized input that
> is filled with the current fillStyle / strokeStyle of the context.
> - BackgroundImage and BackgroundAlpha refer to the contents of the
> canvas before the drawing operation.
> Question: Do we want this? I haven't included it in the Firefox
> implementation, because Firefox doesn't yet support BackgroundImage /
> BackgroundAlpha anywhere.
> 
> Liveness considerations:
> - If SVG filter references are used ("url(#someFilter)"), changes to
> the referenced filter are respected during the next drawing operation.
> The user does not need to assign ctx.filter another time for that to
> happen.
> - The same applies when changing the canvas context font size with
> ctx.font if font size relative units are used in the filter.
> 
> Async-related considerations:
> - If the filter refers to an SVG filter in an external resource
> document, and that document hasn't finished loading when the filtered
> drawing operation is invoked, nothing gets drawn.
> Question: This is slightly suboptimal. Do we want to ignore the filter
> instead? Do we want to add a DOM event that indicates that all
> resources needed for filtering are now available?
> - For a <feImage> primitive, if the required image hasn't finished
> loading at the time of drawing, this <feImage> primitive renders
> transparent black.

I think there is more than the asynch consideration. CSS does not have setting for cross origin content. While it is planned, it simply isn’t there yet. That means SVG filters can be loaded from pretty much any origin. I wonder if this should taint the canvas. Have you though about this?

Greetings,
Dirk

> 
> Random other things:
> - The "font color" of the context is always "black". This is used for
> the CSS drop-shadow() filter function if no shadow color is specified.
> Since the color of normal canvas text drawing is determined by the
> fillStyle of the context, there's not really anything we can reuse
> here.
> 
> Please discuss. :-)
> 
> Greetings,
> Markus
> 
> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=927892

Received on Monday, 29 September 2014 18:10:27 UTC