[Bug 15233] [QT3TS] math-exp10-004, math-tan-006, math-tan-007

https://www.w3.org/Bugs/Public/show_bug.cgi?id=15233

--- Comment #3 from Michael Kay <mike@saxonica.com> 2012-05-23 15:10:27 UTC ---
The first test is exp10(0.5), which means the same as sqrt(10), which Wolfram
Alpha gives as

3.162277660168379331998893544432718533719555139325216826857504852792594438639238221344248108379300295187...

This lies between the two candidate results of 3.162277660168379 and
3.1622776601683795.

Note that for math-tan-006 and math-tan-007 the mathematical answer is infinity
(and this is the value that Wolfram Alpha gives). So the test is close to a
singularity in the function, which means that expecting high precision in the
answer is unreasonable. If we accept pi = 3.141592653589793, we have two
possible value for pi/2 depending on the rounding mode: 1.5707963267948966 and
1.5707963267948967. According to WA, tan(1.5707963267948966) is
5.1998506188720270660194741661226868475811544986... × 10^16, while tan
(1.5707963267948966) is -1.238103706714353220651983005784184862681851980... ×
10^16. So if we allow pi/2 to be rounded either way, the range of permissible
values here is vast.

In the spec we say "the preferred quantum is implementation-defined". I think
this may be a misunderstanding of what the 'quantum' is. I suspect that by
specifying that the operations are done in xs:double floating point, we have
effectively defined the quantum.

More pertinently to this bug, th IEEE spec says (in 4.3) there is a choice of
"rounding direction", with options such as towards zero, towards positive,
towards negative. We haven't said anything about this. We should probably say
"The choice of rounding direction, and any mechanisms for influencing the
choice of rounding direction, are implementation-defined." 

This means that the two results for sqrt(10) given above differ only in the
choice of rounding direction, and both are therefore acceptable.

Recommendation:

(a) for math-tan-006 and math-tan-007, change the tests to avoid evaluating the
function so close to a singularity, where there is a high sensitivity to
legitimate implementation differences.

(b) for math-exp10-004, accept both results

(c) in the spec, change what we say about precision of results, to include the
statement about rounding direction given above.

-- 
Configure bugmail: https://www.w3.org/Bugs/Public/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

Received on Wednesday, 23 May 2012 15:10:35 UTC