W3C home > Mailing lists > Public > public-qt-comments@w3.org > February 2004

ORA-FO-175-B: things you can say about precision of the result

From: Stephen Buxton <Stephen.Buxton@oracle.com>
Date: 16 Feb 04 14:38:45
Message-Id: <200402162238.i1GMcjb17692@rgmgw4.us.oracle.com>
To: public-qt-comments@w3.org
Cc:

SECTION 6.2: functions and operators on numerics

last sentence before 6.2.1: "The number of digits of precision 
returned...is implementation-dependent."  I think this statement
is only necessary when the return type is xs:decimal.
If the return type is xs:integer, then there are obviously
0 places precision after the decimal point, and the precision 
above is limited by the implementation-defined maximum and 
minimum values for xs:integer.  If the return type is 
xs:float or xs:double, you can say that the operation,
and hence the precision of the result, is dictated by the 
IEEE floating point standard.  As for xs:decimal, the precision
ought to be implementation-defined rather than 
implementation-dependent, which leaves the user completely in the 
dark about the behavior.

- Steve B.
Received on Monday, 16 February 2004 17:38:48 UTC

This archive was generated by hypermail 2.3.1 : Wednesday, 5 February 2014 07:13:57 UTC