[Bug 9574] New: Floating point number tolerance

http://www.w3.org/Bugs/Public/show_bug.cgi?id=9574

           Summary: Floating point number tolerance
           Product: HTML WG
           Version: unspecified
          Platform: PC
        OS/Version: Windows NT
            Status: NEW
          Severity: normal
          Priority: P3
         Component: HTML5 spec bugs
        AssignedTo: dave.null@w3.org
        ReportedBy: jmann@microsoft.com
         QAContact: public-html-bugzilla@w3.org
                CC: ian@hixie.ch, mike@w3.org, public-html@w3.org


http://lists.w3.org/Archives/Public/public-canvas-api/2010AprJun/0005.html

From: public-canvas-api-request@w3.org
[mailto:public-canvas-api-request@w3.org] On Behalf Of Jatinder Mann
Sent: Wednesday, April 14, 2010 12:01 PM
To: public-canvas-api@w3.org
Subject: Floating point number tolerance

I  came across an issue regarding floating point numbers. The specification
doesn't clearly define how close two floating point numbers have to be before
they are considered the same number. 

For example, the following code may result in either a tiny arc or a complete
circle, depending on how the user agent implement floating point tolerance:

0 ctx.lineWidth = 1; 
1 var diff = 0.01;
2 ctx.beginPath(); 
3 ctx.arc(100, 100, 100, 0, 2*Math.PI-diff, true); 
4 ctx.stroke();

In my testing, Chrome, Firefox, Opera and Safari all show a tiny arc with this
code. 

If the diff variable is changed to 0.001, all the above mentioned browsers show
nothing. However, if the diff variable is changed to 0.00000001 Chrome, Firefox
and Safari will show a full circle (treat the number as 0). Opera on the other
hand will show a full circle when the diff variable is 0.0000000000000001.

This same issue of tolerance applies to serialization of color, strokeRect(),
closePath()  and arc() functions. The spec should define how close two numbers
need to be before they are considered the same number.

Thank you,
Jatinder

Jatinder Mann | Microsoft Program Manager | jmann@microsoft.com

-- 
Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

Received on Thursday, 22 April 2010 01:47:12 UTC