On 20/09/10 04:40, Gregory Williams wrote: > On Sep 18, 2010, at 9:17 AM, Andy Seaborne wrote: > >> :mixed2 is the sum of "2.2"^^xsd:decimal and "2e-1"^^xsd:double. So the decimal is converted to a double and the result added together but ARQ gets 2.4000000000000004e0 - there is one bit of low end noise introduced by the addition being done on doubles of different exponent (this is Java double+double doing that step). xsd:float, not double, works. > > > Will float work consistently? If so, I can update the test to use it. If not, any thoughts on how we might test the execution of sum with datatype promotion in a portable manner? Float might work (if it does, it's by chance). Better to choose a double that has an exact representation (0.2, a fifth, does not, it's bit pattern as a double is 0x3FC999999999999A). Andy > > .greg >Received on Monday, 20 September 2010 07:49:16 GMT
This archive was generated by hypermail 2.3.1 : Tuesday, 26 March 2013 16:15:43 GMT