RE: Question about decimal arithmetic

The latest IEEE - 854 spec has wisdom to offer in this area.
Let me do some homework and I'll get back to you.

All the best, Ashok

-----Original Message-----
From: public-qt-comments-request@w3.org
[mailto:public-qt-comments-request@w3.org] On Behalf Of Michael Kay
Sent: Monday, April 05, 2004 1:33 AM
To: public-qt-comments@w3.org
Cc: jkenton@datapower.com
Subject: Re: Question about decimal arithmetic


In message

http://lists.w3.org/Archives/Public/public-qt-comments/2003Nov/0238.html

Jeff Kenton wrote:

<quote>
Divison of decimal numbers is not well specified.  There are two aspects
of
the problem:

   1. How many fraction digits should be kept? For example, what is the
result of 1.0 div 3.0?

   2. Should the answer be rounded or truncated?

I have been told previously that the answer is "implementation defined",
but
it's hard for me to accept arithmetic that is "implementation defined".
Since I am chairman of the OASIS XSLT Conformance committee, I would
like to
be able to produce a test suite that expects known answers to numerical
operations.

I have a suggestion:

For XSLT processors which are schema-aware, it is possible to specify
the
number of fraction digits in the result from any operation.  In this
case,
my first question has a clear answer.  I propose that some mechanism be
added to XSLT so that, even without schemas, the number of fraction
digits
can be specified.  I believe that this would solve my problem, and that,
along with a statement in the spec about rounding, it would make decimal
arithmetic precise, as it should be.
</quote>

Because this is an F&O comment but was raised against XSLT, we haven't
really got round to discussing it properly yet. We clearly need to do
so.

Section 6.2.4 (op:numeric-divide) currently says nothing about decimal
division, not even that the results are implementation defined.

I would like to offer the following suggestion for a more interoperable
definition of decimal division:

If the types of $arg1 and $arg2 are xs:integer or xs:decimal, then the
result is of type xs:decimal. The precision of the result is
implementation-defined, but it must be not be less than min((18,
max((p1,
p2)), R)) digits, where p1 is the precision of $arg1, p2 is the
precision of
$arg2, and R is the number of digits (possibly infinite) required to
represent the exact mathematical result. "Precision" here means the
total
number of significant decimal digits in the value; all digits are
considered
significant other than leading zeros before the decimal point and
trailing
zeros after the decimal point. If rounding is necessary, the value must
be
rounded towards zero. Handling of overflow and underflow is defined in
section 6.2.

Michael Kay
(personal contribution)

Received on Monday, 5 April 2004 12:02:43 UTC