[Bug 3759] [F+O] round-half-to-even and float precision

http://www.w3.org/Bugs/Public/show_bug.cgi?id=3759





------- Comment #2 from mike@saxonica.com  2006-09-27 22:55 -------
I'm not sure what do about it either. I'd like to be consistent with other
parts of the spec, but we are very strict about numeric precision in some
places, and very lax in other places. My own implementation of
round-half-to-even(xs:float) does the arithmetic in double precision, which I
think is reasonable but it's hard to argue that it's the only reasonable
approach - you still get rounding errors, but they are smaller than with float.

I think at this stage it's probably best to add a note saying that in the case
of xs:float and xs:double, the arithmetic used to implement round-half-to-even
may use floating point arithmetic, and that we don't prescribe the exact
algorithm which means that rounding errors may occur and are
implementation-dependent; the effect is that a number that is close to half-way
between two powers of ten may round either up or down.

Received on Wednesday, 27 September 2006 22:55:33 UTC