[Bug 3976] floatingPointRound function can be simplified

http://www.w3.org/Bugs/Public/show_bug.cgi?id=3976

           Summary: floatingPointRound function can be simplified
           Product: XML Schema
           Version: 1.1 only
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Datatypes: XSD Part 2
        AssignedTo: cmsmcq@w3.org
        ReportedBy: sandygao@ca.ibm.com
         QAContact: www-xml-schema-comments@w3.org


This function currently has
2.  2^cWidth * 2^(e-1) < |nV| <= 2^cWidth * 2^e 
3a. (c-1) * 2^e <= |nV| < c * 2^e
3b. 2^(cWidth-1) < c <= 2^cWidth 

It's suggested to (correctly) change (2) to the following
2.  2^cWidth * 2^(e-1) <= |nV| < 2^cWidth * 2^e 

With the new (2) and (3a), (3b) is now redundant.

Combine the first half of (3a) and the second half of (2) 

   (c-1) * 2^e <= |nV| < 2^cWidth * 2^e 
-> (c-1) < 2^cWidth 
-> c < 2^cWidth + 1 
-> c <= 2^cWidth (because c is an integer) 

Similarly, combine the first half of (2) and the second half of (3a) 

   2^cWidth * 2^(e-1) <= |nV| < c * 2^e 
-> 2^cWidth * 2^(e-1) < c * 2^e 
-> 2^cWidth < c * 2 
-> 2^(cWidth-1) < c 

So both conditions in (3b) are implied by (2) and (3a), and (3b) should be
removed.

Received on Friday, 17 November 2006 16:59:14 UTC