W3C home > Mailing lists > Public > www-ql@w3.org > October to December 2003

precision on op:numeric-divide

From: Damien Fisher <damien@sodatech.com>
Date: Wed, 10 Dec 2003 13:00:53 +1100
To: <www-ql@w3.org>
Message-Id: <03Dec9.210109-0500_est.215309-1087+43194@ams.ftl.affinity.com>
op:numeric-divide has a special case when the two inputs are of type
xs:integer, and in this case returns an xs:decimal.


What, then, is the answer to op:numeric-divide(1, 3)?  Is it 0.3?  0.33?
0.333?  In other words, to what precision should irrational answers be
returned?  And should rational answers always be returned exactly, even if
that requires an enormous amount of precision?

Received on Tuesday, 9 December 2003 21:40:46 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 22:43:43 UTC