[Bug 9009] New: oops! in recommended minimum partial implementation of precisionDecimal

http://www.w3.org/Bugs/Public/show_bug.cgi?id=9009

           Summary: oops! in recommended minimum partial implementation of
                    precisionDecimal
           Product: XML Schema
           Version: 1.1 only
          Platform: Macintosh
        OS/Version: Mac System 9.x
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Datatypes: XSD Part 2
        AssignedTo: David_E3@VERIFONE.com
        ReportedBy: davep@iit.edu
         QAContact: www-xml-schema-comments@w3.org
                CC: cmsmcq@blackmesatech.com


In section 5.4 we require:

"All ˇminimally conformingˇ processors must support all precisionDecimal values
in the ˇvalue spaceˇ of the otherwise unconstrained ˇderivedˇ datatype for
which totalDigits is set to sixteen, maxScale to 369, and minScale to
−398."

By comparison, we recommend:

"Note: The conformance limits given in the text correspond to those of the
decimal64 type defined in [IEEE 754-2008], which can be stored in a 64-bit
field. The XML Schema Working Group recommends that implementors support limits
corresponding to those of the decimal128 type. This entails supporting the
values in the value space of the otherwise unconstrained datatype for which
totalDigits is set to 34, maxScale to 6176, and minScale to −6111."

The IEEE spec prescribes a maximum floating-point precision (total significant
digits) and a maximum exponent for each of their datatypes (which they call
"formats").  The total digits limit the significand, a number of absolute value
less than 10; i.e., one that can be represented by a decimal numeral with one
digit preceding the decimal point.  It also points out that if that significand
is to be interpreted as an integer (no decimal point), the corresponding
exponent must be offset by one less than the floating-point precision.  We
interpret maxScale and minScale as limiting with an integer significand, so we
must offset the exponent limits.

The IEEE specifies decimal64 to have a maximum allowed exponent of 384 and
floating-point precision of 16.  Therefore our corresponding maxScale should be
(and is) 384 - 15 = 369.  OK so far.  But for decimal128, the max exponent is
specified as 6144 and the max f-p precision as 34, so our corresponding 
maxScale should be 6144 - 33 = 6111 .  Correspondingly, minScale should be
-6143 - 33 = -6176.  We got the signs right but the digit strings switched in
the Note quoted above.


-- 
Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

Received on Tuesday, 16 February 2010 04:18:22 UTC