Monthly Archive for December, 2007

Isn’t this just great? I stumbled upon this problem in actionscript a while ago, but at that time I just worked around it and never bothered again… until today. A colleague came to me with the same problem and I decided to investigate a little more.

Apparently it is some weird bug when it comes to multiplying decimals values by 3. Take this little script for example:

1
2
3
4
5

var j:Number; 
for(var i:int=0; i<10; i++){ 
	j = i/10; 
	trace(j+"*3 =\t"+(j*3)); 
}

It outputs:
0*3 = 0
0.1*3 = 0.30000000000000004
0.2*3 = 0.6000000000000001
0.3*3 = 0.8999999999999999
0.4*3 = 1.2000000000000002
0.5*3 = 1.5
0.6*3 = 1.7999999999999998
0.7*3 = 2.0999999999999996
0.8*3 = 2.4000000000000004
0.9*3 = 2.7

And it doesn’t stop just there. The same problems occur when adding a decimal value to the double of it’s value:

1
2
3
4

trace(.1+.2) //0.30000000000000004 
trace(.2+.4) //0.6000000000000001 
trace(.3+.6) //0.8999999999999999 
trace(.4+.8) //1.2000000000000002

The problem occurres in Javascript as well as in Actionscript (and probably some other languages too) so it’s probably caused by the way decimal values are treated at byte-level. Because it’s not likely there will be a fix for this soon, here is a little workaround:
i = int(i *10)/10; //use int() in favour of Math.round() to speed up the code