RE: Floating point number tolerance

My apologies for the repeat email - it appears that my original mail was delayed.

Thanks,
Jatinder

From: public-canvas-api-request@w3.org [mailto:public-canvas-api-request@w3.org] On Behalf Of Jatinder Mann
Sent: Wednesday, April 14, 2010 12:01 PM
To: public-canvas-api@w3.org
Subject: Floating point number tolerance

I  came across an issue regarding floating point numbers. The specification doesn't clearly define how close two floating point numbers have to be before they are considered the same number. 

For example, the following code may result in either a tiny arc or a complete circle, depending on how the user agent implement floating point tolerance:

0 ctx.lineWidth = 1; 
1 var diff = 0.01;
2 ctx.beginPath(); 
3 ctx.arc(100, 100, 100, 0, 2*Math.PI-diff, true); 
4 ctx.stroke();

In my testing, Chrome, Firefox, Opera and Safari all show a tiny arc with this code. 

If the diff variable is changed to 0.001, all the above mentioned browsers show nothing. However, if the diff variable is changed to 0.00000001 Chrome, Firefox and Safari will show a full circle (treat the number as 0). Opera on the other hand will show a full circle when the diff variable is 0.0000000000000001.

This same issue of tolerance applies to serialization of color, strokeRect(), closePath()  and arc() functions. The spec should define how close two numbers need to be before they are considered the same number.

Thank you,
Jatinder

Jatinder Mann | Microsoft Program Manager | jmann@microsoft.com

Received on Thursday, 15 April 2010 19:28:30 UTC