W3C home > Mailing lists > Public > www-ql@w3.org > October to December 2003

precision on op:numeric-divide

From: Damien Fisher <damien@sodatech.com>
Date: Wed, 10 Dec 2003 13:00:53 +1100
To: <www-ql@w3.org>
Message-Id: <03Dec9.210109-0500_est.215309-1087+43194@ams.ftl.affinity.com>
op:numeric-divide has a special case when the two inputs are of type
xs:integer, and in this case returns an xs:decimal.

 

What, then, is the answer to op:numeric-divide(1, 3)?  Is it 0.3?  0.33?
0.333?  In other words, to what precision should irrational answers be
returned?  And should rational answers always be returned exactly, even if
that requires an enormous amount of precision?

 
Received on Tuesday, 9 December 2003 21:40:46 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 20:17:16 UTC