[Bug 6998] Test orderBy35 / values larger than 1.0E6

http://www.w3.org/Bugs/Public/show_bug.cgi?id=6998





--- Comment #2 from Bogdan Butnaru <bogdanb+w3c+bugzilla@gmail.com>  2009-06-06 01:10:57 ---
(In reply to comment #1)
> Actually, there's nothing special about float values above 1E6. The problem
> applies to all floating-point values. For example, the query float('1.0') can
> legitimately produce the output 1.0000000000000000001.

I'm not sure about this one. Of course, “xs:float(REPR)” _can_ produce
other representations than REPR. However, unless I'm misreading something,
xs:string(xs:float('1.0')) can only produce the string "1.0E0":

a) First the string '1.0' is converted to a value in the float value space.
Neither [1] nor [2] explicitly say how to do that, but presumably [3] applies,
which means that the xs:float value is exactly 1 × 2^0 (i.e., 1). 

b) Converting this value to a string is done according to [4]; the constraint
under the first sub-bullet applies, since the absolute value is between one
millionth (inclusive) and one million (exclusive). Thus, the value is converted
to an xs:decimal and the resulting xs:decimal is converted to an xs:string
according to the rules above, “as though using an implementation of
xs:decimal that imposes no limits on the totalDigits or fractionDigits
facets”.

c) According to [5], “[i]f ST is xs:float or xs:double, then TV is the
xs:decimal value, within the set of xs:decimal values that the implementation
is capable of representing, that is numerically closest to SV. If two values
are equally close, then the one that is closest to zero is chosen.” In this
case, the numerically closest xs:decimal value is “1 × 10^0”.
Incidentally, this value must be exactly represented by all minimally
conforming processors (totalDigits is 1); however, even if that were not true,
[4] says to do it as though there were no limit on totalDigits or
fractionDigits.

d) According to [1], this (unlimited-precision) decimal is transformed to a
string according to a canonical representation (xs:integer if applicable,
xs:decimal otherwise), which is by definition unique.

[1]
http://www.w3.org/TR/2007/REC-xpath-functions-20070123/#casting-from-strings
[2] http://www.w3.org/TR/2007/REC-xpath-functions-20070123/#casting-to-float
[3] http://www.w3.org/TR/xmlschema-2/#float
[4] http://www.w3.org/TR/2007/REC-xpath-functions-20070123/#casting-to-string
[5] http://www.w3.org/TR/2007/REC-xpath-functions-20070123/#casting-to-decimal

In fact, unless there's something wrong in my analysis above, all float and
double values between 1.0E-6 and 1.0E6 have exactly one possible representation
when converted to an xs:string.

* * *

However, all the above—and especially the “unlimited precision”
part—would imply that, e.g., xs:float('0.1') can only be represented as
"0.100000001490116119384765625". 

Which doesn't seem what the specs want to say (for double, the number of digits
can be 55). Also, a lot of the tests would be wrong.*

What am I missing?

(Until now I've been doing the float-to-string conversion via Java's functions
until now, which seems to make most tests happy. However, using the strict
procedure above I get different results on most tests.)


-- 
Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

Received on Saturday, 6 June 2009 01:11:03 UTC