W3C home > Mailing lists > Public > www-style@w3.org > January 2010

Re: Numeric precision in CSS numbers

From: Boris Zbarsky <bzbarsky@MIT.EDU>
Date: Wed, 13 Jan 2010 14:05:03 -0500
Message-ID: <4B4E195F.70704@mit.edu>
To: Simon Fraser <smfr@me.com>
CC: Anne van Kesteren <annevk@opera.com>, www-style list <www-style@w3.org>
On 1/13/10 1:57 PM, Simon Fraser wrote:
>     Many properties that allow an integer or real number as a value
>     actually restrict the value to some range. If the restriction is to
>     a non-negative value, then a non-negative number type is used. A
>     user agent may clamp very large positive or negative values, and may
>     not maintain the full precision of numbers in decimal notation.

Note that the last time I touched the relevant Gecko code I discovered 
that while not maintaining the full precision is ok one has to be pretty 
careful about how one does it.  In particular, some transforms-related 
testcases we have started failing when I accidentally changed some 
rounding code so that it could round a value <pi/2 to a value >pi/2; 
turns out that taking the tangent leads to VERY different results in 
those cases.... ;)

I'm not really sure what, if anything, the spec should say on the matter 
of such rounding.

-Boris
Received on Wednesday, 13 January 2010 19:21:26 GMT

This archive was generated by hypermail 2.3.1 : Tuesday, 26 March 2013 17:20:23 GMT