W3C home > Mailing lists > Public > public-qt-comments@w3.org > April 2016

[Bug 29408] [QT3] Precision in extvardeclwithouttype-18

From: <bugzilla@jessica.w3.org>
Date: Tue, 05 Apr 2016 15:08:20 +0000
To: public-qt-comments@w3.org
Message-ID: <bug-29408-523-1hxH8S8MPV@http.www.w3.org/Bugs/Public/>
https://www.w3.org/Bugs/Public/show_bug.cgi?id=29408

--- Comment #7 from Abel Braaksma <abel.braaksma@xs4all.nl> ---
> may use an otherwise equivalent algorithm that avoids arithmetic overflow
I would understand "equivalent" as not allowing a different outcome...

> For xs:decimal values the number of digits of precision returned by the 
> numeric operators is ·implementation-defined·.
I understood that as the implementation-defined limit we have on xs:decimal
anyway, with a minimum of 18 digits. I think (but not sure) that in IEEE-754
there's a section on decimals that requires additional digits if necessary to
prevent precision-issues, but I don't think XPath leans on IEEE-754 for decimal
arithmetic (only used for double/float).

If what you are say is the de facto rule, then I think there are many tests
that should be more lenient in what answers they expect. I remember fixing
quite a few on my end to get the proper amount of digits in the result to fit
the tests....

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
Received on Tuesday, 5 April 2016 15:08:23 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:57:59 UTC