Re: Question about decimal arithmetic

In message

http://lists.w3.org/Archives/Public/public-qt-comments/2003Nov/0238.html

Jeff Kenton wrote:

<quote>
Divison of decimal numbers is not well specified.  There are two aspects of
the problem:

   1. How many fraction digits should be kept? For example, what is the
result of 1.0 div 3.0?

   2. Should the answer be rounded or truncated?

I have been told previously that the answer is "implementation defined", but
it's hard for me to accept arithmetic that is "implementation defined".
Since I am chairman of the OASIS XSLT Conformance committee, I would like to
be able to produce a test suite that expects known answers to numerical
operations.

I have a suggestion:

For XSLT processors which are schema-aware, it is possible to specify the
number of fraction digits in the result from any operation.  In this case,
my first question has a clear answer.  I propose that some mechanism be
added to XSLT so that, even without schemas, the number of fraction digits
can be specified.  I believe that this would solve my problem, and that,
along with a statement in the spec about rounding, it would make decimal
arithmetic precise, as it should be.
</quote>

Because this is an F&O comment but was raised against XSLT, we haven't
really got round to discussing it properly yet. We clearly need to do so.

Section 6.2.4 (op:numeric-divide) currently says nothing about decimal
division, not even that the results are implementation defined.

I would like to offer the following suggestion for a more interoperable
definition of decimal division:

If the types of $arg1 and $arg2 are xs:integer or xs:decimal, then the
result is of type xs:decimal. The precision of the result is
implementation-defined, but it must be not be less than min((18, max((p1,
p2)), R)) digits, where p1 is the precision of $arg1, p2 is the precision of
$arg2, and R is the number of digits (possibly infinite) required to
represent the exact mathematical result. "Precision" here means the total
number of significant decimal digits in the value; all digits are considered
significant other than leading zeros before the decimal point and trailing
zeros after the decimal point. If rounding is necessary, the value must be
rounded towards zero. Handling of overflow and underflow is defined in
section 6.2.

Michael Kay
(personal contribution)

Received on Monday, 5 April 2004 04:33:08 UTC