Re: Numeric precision in CSS numbers

On 1/13/10 1:57 PM, Simon Fraser wrote:
>     Many properties that allow an integer or real number as a value
>     actually restrict the value to some range. If the restriction is to
>     a non-negative value, then a non-negative number type is used. A
>     user agent may clamp very large positive or negative values, and may
>     not maintain the full precision of numbers in decimal notation.

Note that the last time I touched the relevant Gecko code I discovered 
that while not maintaining the full precision is ok one has to be pretty 
careful about how one does it.  In particular, some transforms-related 
testcases we have started failing when I accidentally changed some 
rounding code so that it could round a value <pi/2 to a value >pi/2; 
turns out that taking the tangent leads to VERY different results in 
those cases.... ;)

I'm not really sure what, if anything, the spec should say on the matter 
of such rounding.

-Boris

Received on Wednesday, 13 January 2010 19:21:26 UTC