Re: [whatwg] Canvas and color colorspaces (was: WebGL and ImageBitmaps)

Great to hear!
Are there minutes posted?

On Sunday, May 1, 2016, Justin Novosad <junov@google.com> wrote:

> There is currently an ongoing discussion with the Khronos Web3D group to
> develop a proposal that solves these problems in canvas, over the past few
> weeks we have converged on a solution that I think is pretty solid. I am in
> the process of writing-up the HTML (non-WebGL) part of the proposal and I
> intend to post it to the WICG shortly so that we can incubate it further,
> with a broader audience.  When that happens, I will update this thread.
>
> On Sat, Apr 30, 2016 at 2:07 PM, Rik Cabanier <cabanier@gmail.com
> <javascript:_e(%7B%7D,'cvml','cabanier@gmail.com');>> wrote:
>
>> [Sorry to revive this old thread]
>> All,
>>
>> with the advent of DCI-P3 compliant monitors and Apple's Safari doing
>> color managing to the device, we're seeing some issues in this area.
>>
>> - Currently, WebKit sets the profile of the canvas backing store to sRGB
>> regardless of the output device. Because of this, high gamut images are
>> always clipped to sRGB. [1]
>> It would be ideal if we can specify that the canvas backing store is in
>> the device profile.
>> Alternatively, we could add an API to attach a color profile to the
>> canvas.
>> - The spec currently states that toDataURL should not include a profile.
>> However, if the backing store is in device pixels, the generated image
>> should include the correct profile. Otherwise if you draw the bitmap in a
>> compliant browser (ie Safari), the colors will look too saturated.
>>
>> If we agree that canvas is in the device space, I'd like to see the spec
>> [2] clarified to state that compositing on the canvas should match
>> compositing on the HTML surface.
>> Specifically:
>>
>> The canvas
>> <https://html.spec.whatwg.org/multipage/scripting.html#the-canvas-element> APIs
>> must perform colour correction at only two points: when rendering images
>> with their own gamma correction and colour space information onto a bitmap,
>> to convert the image to the colour space used by the bitmaps (e.g. using
>> the 2D Context's drawImage()
>> <https://html.spec.whatwg.org/multipage/scripting.html#dom-context-2d-drawimage> method
>> with an HTMLOrSVGImageElement
>> <https://html.spec.whatwg.org/multipage/scripting.html#htmlorsvgimageelement> object),
>> and when rendering the actual canvas bitmap to the output device.
>>
>> Becomes:
>>
>> The canvas
>> <https://html.spec.whatwg.org/multipage/scripting.html#the-canvas-element> APIs
>> must perform colour correction at only one point: when rendering content
>> with its own gamma correction and colour space information onto a bitmap to
>> the colour space used by the bitmaps (e.g. using the 2D Context's
>> drawImage()
>> <https://html.spec.whatwg.org/multipage/scripting.html#dom-context-2d-drawimage> method
>> with an HTMLOrSVGImageElement
>> <https://html.spec.whatwg.org/multipage/scripting.html#htmlorsvgimageelement>
>>  object).
>>
>>
>> ToDataURL and ToBlob [3] should also be enhanced so they include the
>> device profile if it is different from sRGB.
>>
>> It would also be great if the browser could let us know what profile (if
>> any) it was using.
>>
>> 1:
>> https://github.com/WebKit/webkit/blob/112c663463807e8676765cb7a006d415c372f447/Source/WebCore/platform/graphics/ImageBuffer.h#L73
>> 2:
>> https://html.spec.whatwg.org/multipage/scripting.html#colour-spaces-and-colour-correction
>> 3:
>> https://html.spec.whatwg.org/multipage/scripting.html#dom-canvas-todataurl
>>
>>
>>
>> On Thu, May 22, 2014 at 12:21 PM, Justin Novosad <junov@google.com
>> <javascript:_e(%7B%7D,'cvml','junov@google.com');>> wrote:
>>
>>> tl;dr: The color space of canvas backing stores is undefined, which
>>> causes problems for many web devs, but also has non-negligible advantages.
>>> So be careful what you wish for.
>>>
>>> I saw some confusion and questions needing answers in the "WebGL and
>>> ImageBitmaps" thread regarding color management. I will attempt to clarify
>>> to the best of my abilities. Though I am knowledgeable on the subject, I am
>>> not an absolute authority, so others are welcome to correct me if I am
>>> wrong about anything.
>>>
>>> Color management... To make a long story short, there are two types of
>>> color profiles : input profiles and output profiles for characterizing
>>> input devices (cameras, scanners) and output devices (displays, printers)
>>> respectively.
>>> Image files will usually encode their color information in a standard
>>> color space or in a an input device dependent space. If colors are encoded
>>> in a color space that is different from the format's default, then a color
>>> profile or a color space identifier must be encoded into the image
>>> resource's metadata.
>>>
>>> To present color-managed image content on screen, the image needs to be
>>> converted from whatever color space the image was encoded into into a
>>> standard "connection space" using the color profile or color space metadata
>>> from the image resource. Then the colors need to be converted from the
>>> profile connection space to the output space, which is provided by the
>>> OS/display driver. Depending on the OS and hardware configuration, the
>>> output space may be a standard color space (like sRGB), or a
>>> device-specific color profile.
>>>
>>> Currently, many color-managed software applications rely on the codec to
>>> take care of the entire color-management process for image and video
>>> content, meaning that the decoded image data is in output-referred color
>>> space (i.e. the display's profile was applied).  There are practical
>>> reasons for this, the most important ones being color fidelity and memory
>>> consumption.  Let me explain. The profile connection space is typically CIE
>>> XYZ or CIE L*a*b. I wont get into the technical details of how these work
>>> except to say that they are device independent and allow for an accurate
>>> representation of the whole spectrum of human-visible colors. This makes it
>>> possible to map colors from a wide gamut camera to a wide gamut display
>>> with high color fidelity for all the colors that are located in the
>>> intersection of the color gamuts of both the input and output devices. If
>>> we were forced to convert the image to an intermediate sRGB representation,
>>> the colors in the image would be clamped to the sRGB gamut (which is
>>> narrower than the gamuts of many devices). Currently, most browsers avoid
>>> doing that for <img>, and therefore provide (more or less) optimal image
>>> and video color fidelity for users of wide gamut devices. Also, an
>>> intermediate representation in 8-bit sRGB means loss of precision due to
>>> rounding errors, as opposed to the profile connection space which uses
>>> higher precision registers for intermediate color values to avoid precision
>>> issues caused by rounding.  To avoid perceptible precision issues in an
>>> intermediate sRGB representation, we'd have to increase the bit depth and
>>> therefore use more RAM for storing decoded image data.
>>>
>>> All of this is to say that there are good reasons for the current
>>> situation where we deal with decoded images that have the output device's
>>> color profile pre-applied: color fidelity and memory consumption.
>>>
>>> In the case of 2D canvas, the color space for the backing store is
>>> unspecified, and many implementations have chosen to use the output
>>> device's color space, which has many advantages:
>>> * images and videos are already decoded directly into that space
>>> * no color conversion is necessary when presenting the canvas on screen
>>> (good for performance)
>>> * there is no loss of precision due the use of a limited-precision
>>> intermediate color space.
>>> * the color gamut is not constrained by an intermediate color space
>>> (like sRGB).
>>> And disadvantages:
>>> * Compositing operations produce incorrect results because most of them
>>> (including source-over) are affected by the color space.
>>> * direct pixel manipulation using put/getImageData exposes data in a
>>> color space that is undefined, making it extremely challenging to perform
>>> many types of image processing and image generation tasks in a
>>> device-independent way.
>>> * The device-dependent behavior of a drawImage/getImageData round trip
>>> is a known fingerprinting vector.
>>>
>>> Right now, I am hearing a lot of complaints regarding the lack of a
>>> standardized color space for canvases, and in particular the impact this
>>> has on applications that try to do cool things with put/getImageData, or
>>> generate images procedurally.  I want to make sure everyone understands
>>> there is a trade-off to fixing this, so be careful what you wish for.
>>>
>>> I am especially concerned about the issue of color gamut clamping, which
>>> will increase in relevance as wide gamut devices become more widespread.
>>> If we were to decide that canvas backing stores had to be in sRGB, that
>>> would mean that a wide gamut image viewed on a wide gamut display would
>>> look best when displayed in an <img> and would be duller when drawn through
>>> a <canvas>, whether it be 2D or WebGL.  Is that something we are willing to
>>> live with in the name of standardizing the color space of ImageData?  I
>>> must admit I was in favor of moving canvases to sRGB until I reviewed some
>>> of Noel Gordon's recent work which brought the gamut and precision issues
>>> to my attention.
>>>
>>> Rik: to answer your question about your experiment: there is no issue
>>> with a put/getImageData round trip. You will get back the same color you
>>> put in (at least for opaque colors, but that is another story).  The issue
>>> is with a drawImage/getImageData round trip.  For the same source image,
>>> getImageData will return different values depending on the display profile
>>> of the system you are running on.
>>>
>>> Ken: You mentioned to me off thread that built-in support for sRGB
>>> render buffers in OpenGL ES 3 may make it easier to move rendering to sRGB
>>> on next gen devices. I tend to agree, and I think it will also mitigate the
>>> loss of precision issue, but it still implies clamping wide gammut media to
>>> the sRGB range.
>>>
>>>     -Justin
>>>
>>>
>>>
>>
>

Received on Sunday, 1 May 2016 17:08:57 UTC