Re: Numeric precision in CSS numbers

On Tuesday 22 December 2009, Simon Fraser wrote:
> The section on Numbers in CSS3 Values and Units at
> <http://www.w3.org/TR/css3-values/#numbers> says nothing about
> numeric precision, other than some wording about values being
> restricted to a range for some properties.
>
> For integers, it isn't stated how large values can get before they
> fail to parse, or fail to round-trip; for floating point values, it
> isn't specified how many significant digits are retained by parsing,
> and round-tripping. (By round-tripping, I mean setting the value via
> the CSS OM, and then reading it back.)
>
> For both large integers and floating point values with many decimal
> places, implementations behave differently. Both WebKit and Gecko
> fall into scientific notation in some cases (e.g. set 'left' to
> '12345678px', and reading it back will give you '1.23457e+7px' which
> fails to parse if you in turn try to use it to set the value); Opera
> does not. WebKit and Gecko also retain 6 significant digits for
> floating point values; Opera, on the other hand, rounds to two
> decimal places.
>
> This is a real-world issue. When we changed our number formatting in
> WebKit, we broke a site
> <https://bugs.webkit.org/show_bug.cgi?id=32078>. Also, we need to be
> able to round-trip floating point numbers with high precision for CSS
> transforms.
>
> There is a test page here (but note that it doesn't work in Opera
> because of value limiting):
> <https://bug-30341-attachments.webkit.org/attachment.cgi?id=45276>
>
> I think there are two things that we need to do:
> 1. Be more explicit about the behavior of round-tripping large
> integer values, and floating point values with lots of significant
> digits. 2. Ensure that the CSSValue replacement in CSS OM allows
> authors to modify style without going through number/string
> conversions, and therefore losing precision.

It's a tricky problem, especially because, as Anne said, the underlying 
technology changes. The change is typically towards higher precision 
and larger limits, but that can still cause unexpected differences.

I suspect, however, that other standards have had to deal with this 
already. Is there no ISO or IEEE standard we can borrow?

TeX specifies that all lengths are fixed-point, 31-bit numbers where the 
smallest length is 1/65536th of a pt and thus the largest expressible 
length is about 576cm. That's plenty small enough (1/100th of the 
wavelength of light) and large enough for paper, although not enough 
for a large billboard. 31 bits is about 9 decimal digits, i.e., quite 
precise enough. Using fixed-point arithmetic (carefully, reordering 
calculations in order to round as late as possible) ensures that all 
implementations work exactly the same.

I don't think we have to specify that much detail. If implementers apply 
some basic rules from numerical computing, the results should be 
reasonable without the spec requiring anything special.

Maybe a more interestring question is how to round. Because if the 
output device can only draw text at pixel boundaries, then the renderer 
will have to round one way or another and one box of 399px will not be 
as wide as four boxes of 25%. (Firefox and IE apply some, different, 
heuristic to round up or down based on context, other browsers seem to 
always round down.)



Bert
-- 
  Bert Bos                                ( W 3 C ) http://www.w3.org/
  http://www.w3.org/people/bos                               W3C/ERCIM
  bert@w3.org                             2004 Rt des Lucioles / BP 93
  +33 (0)4 92 38 76 92            06902 Sophia Antipolis Cedex, France

Received on Wednesday, 20 January 2010 16:54:25 UTC