Floating point number tolerance

I  came across an issue regarding floating point numbers. The specification doesn't clearly define how close two floating point numbers have to be before they are considered the same number.



For example, the following code may result in either a tiny arc or a complete circle, depending on how the user agent implement floating point tolerance:



0 ctx.lineWidth = 1;

1 var diff = 0.01;

2 ctx.beginPath();

3 ctx.arc(100, 100, 100, 0, 2*Math.PI-diff, true);

4 ctx.stroke();



In my testing, Chrome, Firefox, Opera and Safari all show a tiny arc with this code.



If the diff variable is changed to 0.001, all the above mentioned browsers show nothing. However, if the diff variable is changed to 0.00000001 Chrome, Firefox and Safari will show a full circle (treat the number as 0). Opera on the other hand will show a full circle when the diff variable is 0.0000000000000001.



This same issue of tolerance applies to serialization of color, strokeRect(), closePath()  and arc() functions. The spec should define how close two numbers need to be before they are considered the same number.



Thank you,

Jatinder



Jatinder Mann | Microsoft Program Manager | jmann@microsoft.com<mailto:jmann@microsoft.com>

Received on Wednesday, 14 April 2010 19:01:17 UTC