From: Ashok Malhotra <ashokma@microsoft.com>

Date: Mon, 5 Apr 2004 09:02:10 -0700

Message-ID: <EDB607C8AC991F40BE646533A1A673E801BC122F@RED-MSG-42.redmond.corp.microsoft.com>

To: "Michael Kay" <mhk@mhk.me.uk>, <public-qt-comments@w3.org>, <jeni@jenitennison.com>

Cc: <jkenton@datapower.com>

Date: Mon, 5 Apr 2004 09:02:10 -0700

Message-ID: <EDB607C8AC991F40BE646533A1A673E801BC122F@RED-MSG-42.redmond.corp.microsoft.com>

To: "Michael Kay" <mhk@mhk.me.uk>, <public-qt-comments@w3.org>, <jeni@jenitennison.com>

Cc: <jkenton@datapower.com>

The latest IEEE - 854 spec has wisdom to offer in this area. Let me do some homework and I'll get back to you. All the best, Ashok -----Original Message----- From: public-qt-comments-request@w3.org [mailto:public-qt-comments-request@w3.org] On Behalf Of Michael Kay Sent: Monday, April 05, 2004 1:33 AM To: public-qt-comments@w3.org Cc: jkenton@datapower.com Subject: Re: Question about decimal arithmetic In message http://lists.w3.org/Archives/Public/public-qt-comments/2003Nov/0238.html Jeff Kenton wrote: <quote> Divison of decimal numbers is not well specified. There are two aspects of the problem: 1. How many fraction digits should be kept? For example, what is the result of 1.0 div 3.0? 2. Should the answer be rounded or truncated? I have been told previously that the answer is "implementation defined", but it's hard for me to accept arithmetic that is "implementation defined". Since I am chairman of the OASIS XSLT Conformance committee, I would like to be able to produce a test suite that expects known answers to numerical operations. I have a suggestion: For XSLT processors which are schema-aware, it is possible to specify the number of fraction digits in the result from any operation. In this case, my first question has a clear answer. I propose that some mechanism be added to XSLT so that, even without schemas, the number of fraction digits can be specified. I believe that this would solve my problem, and that, along with a statement in the spec about rounding, it would make decimal arithmetic precise, as it should be. </quote> Because this is an F&O comment but was raised against XSLT, we haven't really got round to discussing it properly yet. We clearly need to do so. Section 6.2.4 (op:numeric-divide) currently says nothing about decimal division, not even that the results are implementation defined. I would like to offer the following suggestion for a more interoperable definition of decimal division: If the types of $arg1 and $arg2 are xs:integer or xs:decimal, then the result is of type xs:decimal. The precision of the result is implementation-defined, but it must be not be less than min((18, max((p1, p2)), R)) digits, where p1 is the precision of $arg1, p2 is the precision of $arg2, and R is the number of digits (possibly infinite) required to represent the exact mathematical result. "Precision" here means the total number of significant decimal digits in the value; all digits are considered significant other than leading zeros before the decimal point and trailing zeros after the decimal point. If rounding is necessary, the value must be rounded towards zero. Handling of overflow and underflow is defined in section 6.2. Michael Kay (personal contribution)Received on Monday, 5 April 2004 12:02:43 UTC

*
This archive was generated by hypermail 2.3.1
: Wednesday, 7 January 2015 15:45:19 UTC
*