[Bug 29408] [QT3] Precision in extvardeclwithouttype-18

https://www.w3.org/Bugs/Public/show_bug.cgi?id=29408

--- Comment #7 from Abel Braaksma <abel.braaksma@xs4all.nl> ---
> may use an otherwise equivalent algorithm that avoids arithmetic overflow
I would understand "equivalent" as not allowing a different outcome...

> For xs:decimal values the number of digits of precision returned by the 
> numeric operators is ·implementation-defined·.
I understood that as the implementation-defined limit we have on xs:decimal
anyway, with a minimum of 18 digits. I think (but not sure) that in IEEE-754
there's a section on decimals that requires additional digits if necessary to
prevent precision-issues, but I don't think XPath leans on IEEE-754 for decimal
arithmetic (only used for double/float).

If what you are say is the de facto rule, then I think there are many tests
that should be more lenient in what answers they expect. I remember fixing
quite a few on my end to get the proper amount of digits in the result to fit
the tests....

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

Received on Tuesday, 5 April 2016 15:08:23 UTC