Fwd: avg testcase issues (was: Re: syntax error in testcase manifest)

(this is just part of the comments message but a part I wanted to check 
up on)


> -------- Original Message --------
> Subject: avg testcase issues (was: Re: syntax error in testcase manifest)
...
> From: Jeen Broekstra <jeen.broekstra@gmail.com>
...
> 1. testcase aggregates/agg-avg-01 ("AVG")
>
> This testcase currently assumes a particular precision in xsd:decimal
> division, which is not mandated by the XPath definition. XPath defines
> that in xsd:decimal operations, rounding is implementation dependent
> (see http://www.w3.org/TR/xpath-functions/#op.numeric, near the end of
> the section).
>
> In the test, five numbers of type xsd:decimal are averaged (1.0, 2.0,
> 3.5, 2.2, 2.2). The average is computed by executing a op:numeric-divide
> on the sum of the five decimals: 11.1/5. The result of this is, as the
> testcase defines, 2.22. However, this result increases precision to 2
> decimals (note that the precision of the input is 1 decimal).
>
> While this is not wrong, it is not the *only* correct answer: an
> implementation that would round or truncate to 1 decimal and return 2.2
> as the answer would IMHO also perform the operation correctly.
>
> To fix this, I would suggest that the testcase numbers are adapted so
> that the result of the division when not rounded is of the same
> precision as the input. Alternatively, change the test case to work on
> floats or doubles, as division behavior is more precisely defined for
> those datatypes.

The issue seems to be that reading "2.0" is implying a precision of one 
decimal place in the decimal.  This isn't true - XSD decimal must 
support at least 18 digits of precision and precision is not a facet of 
the lexical space.

http://www.w3.org/TR/xmlschema-2/#decimal

"""
Note:  All ·minimally conforming· processors ·must· support decimal 
numbers with a minimum of 18 decimal digits (i.e., with a ·totalDigits· 
of 18)
"""

1.0, 2.0 etc are at least 18 digits of precision and the way the decimal 
is written is not relevant.  i.e. In XSD, 2.00 is not promising more 
precision than 2.0 - they are different ways to write the same value down.

"""
Precision is not reflected in this value space; the number 2.0 is not 
distinct from the number 2.00.
"""

which is a little different from Java BigDecimal where scale is a factor 
in BigDecimal.equals or some scientific styles where the number of 
places does imply accuracy.

 Andy

Received on Monday, 28 February 2011 14:41:11 UTC