W3C home > Mailing lists > Public > public-canvas-api@w3.org > April to June 2010

RE: Floating point number tolerance

From: Jatinder Mann <jmann@microsoft.com>
Date: Thu, 15 Apr 2010 19:27:55 +0000
To: Jatinder Mann <jmann@microsoft.com>, "public-canvas-api@w3.org" <public-canvas-api@w3.org>
Message-ID: <EAACC9DF6817544486F0D0ADA05CA4281A452CB4@TK5EX14MBXC120.redmond.corp.microsoft.com>
My apologies for the repeat email - it appears that my original mail was delayed.


From: public-canvas-api-request@w3.org [mailto:public-canvas-api-request@w3.org] On Behalf Of Jatinder Mann
Sent: Wednesday, April 14, 2010 12:01 PM
To: public-canvas-api@w3.org
Subject: Floating point number tolerance

I  came across an issue regarding floating point numbers. The specification doesn't clearly define how close two floating point numbers have to be before they are considered the same number. 

For example, the following code may result in either a tiny arc or a complete circle, depending on how the user agent implement floating point tolerance:

0 ctx.lineWidth = 1; 
1 var diff = 0.01;
2 ctx.beginPath(); 
3 ctx.arc(100, 100, 100, 0, 2*Math.PI-diff, true); 
4 ctx.stroke();

In my testing, Chrome, Firefox, Opera and Safari all show a tiny arc with this code. 

If the diff variable is changed to 0.001, all the above mentioned browsers show nothing. However, if the diff variable is changed to 0.00000001 Chrome, Firefox and Safari will show a full circle (treat the number as 0). Opera on the other hand will show a full circle when the diff variable is 0.0000000000000001.

This same issue of tolerance applies to serialization of color, strokeRect(), closePath()  and arc() functions. The spec should define how close two numbers need to be before they are considered the same number.

Thank you,

Jatinder Mann | Microsoft Program Manager | jmann@microsoft.com
Received on Thursday, 15 April 2010 19:28:30 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:10:26 UTC