# [Bug 3976] floatingPointRound function can be simplified

From: <bugzilla@wiggum.w3.org>
Date: Fri, 17 Nov 2006 16:59:11 +0000
CC:

Message-Id: <E1Gl73X-0002Q5-C9@wiggum.w3.org>
```
http://www.w3.org/Bugs/Public/show_bug.cgi?id=3976

Summary: floatingPointRound function can be simplified
Product: XML Schema
Version: 1.1 only
Platform: All
OS/Version: All
Status: NEW
Severity: normal
Priority: P2
Component: Datatypes: XSD Part 2
AssignedTo: cmsmcq@w3.org
ReportedBy: sandygao@ca.ibm.com

This function currently has
2.  2^cWidth * 2^(e-1) < |nV| <= 2^cWidth * 2^e
3a. (c-1) * 2^e <= |nV| < c * 2^e
3b. 2^(cWidth-1) < c <= 2^cWidth

It's suggested to (correctly) change (2) to the following
2.  2^cWidth * 2^(e-1) <= |nV| < 2^cWidth * 2^e

With the new (2) and (3a), (3b) is now redundant.

Combine the first half of (3a) and the second half of (2)

(c-1) * 2^e <= |nV| < 2^cWidth * 2^e
-> (c-1) < 2^cWidth
-> c < 2^cWidth + 1
-> c <= 2^cWidth (because c is an integer)

Similarly, combine the first half of (2) and the second half of (3a)

2^cWidth * 2^(e-1) <= |nV| < c * 2^e
-> 2^cWidth * 2^(e-1) < c * 2^e
-> 2^cWidth < c * 2
-> 2^(cWidth-1) < c

So both conditions in (3b) are implied by (2) and (3a), and (3b) should be
removed.
```
Received on Friday, 17 November 2006 16:59:14 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Sunday, 6 December 2009 18:13:11 GMT