# [XQuery] IBM-XQ-028 NaN Comparison

From: Andrew Eisenberg <andrew.eisenberg@us.ibm.com>
Date: Fri, 9 Apr 2004 14:59:21 -0400

Message-ID: <OFC8CE6F00.00C6DA31-ON85256E71.00680D47-85256E71.00684F9F@us.ibm.com>
```I believe that "le" and "ge" return the wrong values when NaN values are
provided for either of the two operands.

Consider the following query:

let \$x := 1.0e0
let \$nan := xs:double('NaN')
return
\$x eq \$nan,     (: will return false :)
\$x lt \$nan,     (: will return false :)
\$x gt \$nan,     (: will return false :)
\$x le \$nan,     (: will return true :)
\$x ge \$nan,     (: will return true :)
\$x ne \$nan      (: will return true :)

I believe that "le" and "ge" should return false to be consistent with
IEEE 754. They return true because:

1) The definition of op:numeric-less-than and  op:numeric-greater-than in
F&O both contain the following:

"If \$arg1 or \$arg2 is NaN, the function returns false."

2) The Binary Operator table in XQuery, B.2 Operator Mapping, contains the
following:

A ge B  numeric         numeric fn:not(op:numeric-less-than(A, B))
xs:boolean
A le B  numeric         numeric fn:not(op:numeric-greater-than(A,
B))     xs:boolean

To effect the change I suggest, change the that two rows of the Binary
Operator table to the following:

A ge B  numeric         numeric         op:numeric-greater-than(A,
B) or op:numeric-equal(A,B)     xs:boolean
A le B  numeric         numeric         op:numeric-less-than(A, B)
or op:numeric-equal(A,B)        xs:boolean

-- Andrew

--------------------
Andrew Eisenberg
IBM
4 Technology Park Drive
Westford, MA  01886

andrew.eisenberg@us.ibm.com
Phone: 978-399-5158    Fax: 978-399-7012
```
Received on Friday, 9 April 2004 14:59:55 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:56:56 UTC