[Bug 3666] Wrong number of decimal places in numeric divide tests

http://www.w3.org/Bugs/Public/show_bug.cgi?id=3666

           Summary: Wrong number of decimal places in numeric divide tests
           Product: XML Query Test Suite
           Version: 1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: XML Query Test Suite
        AssignedTo: andrew.eisenberg@us.ibm.com
        ReportedBy: oliver@cbcl.co.uk
         QAContact: public-qt-comments@w3.org


Section 2.3.2 of Xml Schema Part 2: Datatypes Second Edition states:
Note:  All ˇminimally conformingˇ processors ˇmustˇ support decimal numbers
with a minimum of 18 decimal digits (i.e., with a ˇtotalDigitsˇ of 18).
However, ˇminimally conformingˇ processors ˇmayˇ set an application-defined
limit on the maximum number of decimal digits they are prepared to support, in
which case that application-defined maximum number ˇmustˇ be clearly
documented.

However in the test op-numeric-dividesht2args-5, which contains the following
query:
xs:short("-32768") div xs:short("32767")

suggests the result should be "-1.000030518509", which only contains 13 decimal
digits.   Although the number of digits returned from arithmetic operations are
implementation defined, if the results are not quoted as if they were
calculated with the full 18 decimal places of accuracy, the number of decimal
places used in the test suite documentation.

Another similar issue occurs in the following tests:

op-numeric-divideintg2args-2
op-numeric-divideintg2args-4
op-numeric-dividedec2args-2
op-numeric-dividedec2args-4
op-numeric-dividenint2args-2
op-numeric-dividenint2args-4
op-numeric-dividenpi2args-2
op-numeric-dividenpi2args-4

These tests all have the following form:

fn:round-half-to-even(___ div ___),18)

However the results quoted all only have 12 decimal places.  These tests should
probably be changed to the following form:

fn:round-half-to-even(___ div ___),12)

and the tests op-numeric-dividesht2args-3 and op-numeric-dividesht2args-5
should be changed to this form.

Received on Thursday, 7 September 2006 12:53:57 UTC