Re: [csswg-drafts] [css-color-4] Linear transformation matrices are very slightly inaccurate (#7675)

Related to this: the floating-point errors in the sample code are substantial enough that they're actually visible in the test cases. For example, the [xyz-d50-001 test case](https://github.com/web-platform-tests/wpt/blob/master/css/css-color/xyz-d50-001.html) lists the conversion for #008000 as `color(xyz-d50 0.08312 0.154746 0.020961)`. Running the transformation with lossless rationals yields `color(xyz-d50 0.08313899747975508 0.15474758917173437 0.02095602320031179)`, which would round to `color(xyz-d50 0.08314 0.154748 0.020956)`. The rounding errors will accumulate even more dramatically for transformations that require more computations—I _think_ the lms-001 test is incorrect to the second decimal digit in one channel.

Obviously it's unrealistic to expect implementations to do these transformations losslessly, but it seems like the test cases maybe should reflect the true mathematical values with the expectation that individual implementations will be less accurate than that given the realities of floating-point arithmetic.

-- 
GitHub Notification of comment by nex3
Please view or discuss this issue at https://github.com/w3c/csswg-drafts/issues/7675#issuecomment-1261560260 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 28 September 2022 23:14:52 UTC