Re: [whatwg] I believe source rectangles for HTML5 Canvas drawImage are specified incorrectly

On Mon, 20 Aug 2012, Kevin Gadd wrote:
>
> Hi, I've been digging into an inconsistency between various browsers' 
> Canvas implementations and I think the spec might be allowing 
> undesirable behavior here.
> 
> The current version of the spec says 
> (http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-drawimage):
> 
> If the original image data is a bitmap image, the value painted at a 
> point in the destination rectangle is computed by filtering the original 
> image data. The user agent may use any filtering algorithm (for example 
> bilinear interpolation or nearest-neighbor). When the filtering 
> algorithm requires a pixel value from outside the original image data, 
> it must instead use the value from the nearest edge pixel. (That is, the 
> filter uses 'clamp-to-edge' behavior.)
> 
> While clamp-to-edge is desirable, the way this is specified means that 
> it only ever clamps to the edges of the source bitmap, not to the source 
> rectangle. That means that attempting to do the equivalent of css 
> sprites or video game style 'tile sets' - where a single source image 
> contains many smaller images - is not possible, because the spec allows 
> implementations to read pixels from outside the source rectangle.

There's two ways to do scaled sprites with drawImage(): have a border of 
transparent black around each sprite, or copy the data out of the sprite 
sheet and into a temporary canvas at its original size, then scaling from 
that.


> Unfortunately, at present Internet Explorer and Firefox both read pixels 
> from outside the source rectangle, as demonstrated by this test case:
>   https://dl.dropbox.com/u/1643240/canvas_artifacts.html
> Worse still, in implementations with imageSmoothingEnabled available, 
> turning off image smoothing is not sufficient to eliminate the 
> artifacts.

Disabling image smoothing will increase artefacts, that's kind of the 
point. :-) Having said that, I don't really see what that test case is 
demonstrating. Can you elaborate?


> Google Chrome appears to implement this the way you would probably want 
> it to work - by clamping to the edges of the source rectangle, instead 
> of the source image. I can't think of a good reason to prefer the 
> current behavior over what Chrome does, and I haven't been able to find 
> a reliable way to compensate for the current behavior.

The reason to prefer the current behaviour is if you want to just update a 
small part of an image. For example, if you draw a bit photo, then draw 
text over it, then want to remove the text by just drawing the photo over 
where the text was but not redrawing the whole thing. If we clamped to 
source rectangle, we'd get artefacts in this case that couldn't be worked 
around (unlike the problems with scaling sprites, which can be worked 
around, albeit in a suboptimal fashion).


On Mon, 20 Aug 2012, Justin Novosad wrote:
> 
> The same artifact use to be present in Chrome not that long ago. When we 
> fixed it, we chose to interpret "original image data" as meaning the 
> part of the image data that is within the bounds of the of the source 
> rectangle.

I don't think this reading is consistent with the usage of the term 
"original image data" in the spec. It's pretty consistently used to mean 
the whole image.

I've clarified the spec to make this even less ambiguous.


> Also, it makes more sense to do it that way.

I think arguments could be made for both ways, but the problem is there's 
only a way to work around one of the ways.


On Mon, 10 Sep 2012, Vladimir Vukicevic wrote:
>
> This is pretty tricky to get right -- there's just a general graphics 
> problem in this case.  There are valid use cases for both sampling 
> outside and not sampling outside the source rectangle, as well as 
> implementation issues for being able to do source rectangle clamping.  
> For example, should you be able to take a source image and draw it 
> scaled up using 4 rectangles (one for each quadrant) and have the result 
> be equal to just doing it in one draw?  Or take any random subimage (for 
> example, for efficient updates of some destination) and draw it in.

Right.


> At best, I think a new mode toggle or flag would be needed to allow you 
> to select.

I'd be happy to add a flag if it's something browser implementors want to 
support.


> Additionally, I think there's a related bug filed from a while ago about 
> defining how to sample pixels that are outside of the source bounds -- 
> do you clamp to edge, do you sample transparent black, etc.

Fixing that bug is how the current text came to be, IIRC.


On Mon, 10 Sep 2012, Justin Novosad wrote:
>
> You are referring to the fact that clamping modes only apply to texture 
> borders in OpenGL and DirectX?  That is not that big of a big deal, you 
> can implement the clamp in a shader.
>
> I think Vladimir makes a good point that there are valid use cases for 
> both ways. To summarize the issues:
>
> * clamping to source rect can cause filtering artefacts at boundaries, 
> especially visible when tiling.
>
> * clamping to source image bounds can cause color bleeding artefacts 
> when rendering a sprite from a spritemap.

Right. The second is easy enough to work around by having some empty 
pixels (not ideal, but then neither is scaling sprites, really...). The 
first has no work around that I'm aware of, so if we don't have a flag, it 
should be the one we support.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Received on Monday, 10 December 2012 18:24:47 UTC