The Origin Forum
File Exchange
Try Origin for Free
The Origin Forum
Home | Profile | Register | Active Topics | Members | Search | FAQ | Send File to Tech support
 All Forums
 Origin Forum for Programming
 Forum for Origin C
 Floating number precision

Note: You must be registered in order to post a reply.
To register, click here. Registration is FREE!

Screensize:
UserName:
Password:
Anti-Spam Code:
Format Mode:
Format: BoldItalicizedUnderlineStrikethrough Align LeftCenteredAlign Right Horizontal Rule Insert HyperlinkUpload FileInsert Image Insert CodeInsert QuoteInsert List
   
Message:

* HTML is OFF
* Forum Code is ON
Smilies
Smile [:)] Big Smile [:D] Cool [8D] Blush [:I]
Tongue [:P] Evil [):] Wink [;)] Clown [:o)]
Black Eye [B)] Eight Ball [8] Frown [:(] Shy [8)]
Shocked [:0] Angry [:(!] Dead [xx(] Sleepy [|)]
Kisses [:X] Approve [^] Disapprove [V] Question [?]

 
Check here to subscribe to this topic.
   

T O P I C    R E V I E W
zeron Posted - 07/26/2007 : 05:27:44 AM
Origin Version (Select Help-->About Origin): 7.5885
Operating System: Windows XP

Hello,
I recently discovered a problem with calculating simple differences of floating numbers. For simplification I reduced the code to finally just one line:


void test()
{
float d= 1.2-0.3;
}


which I checked with the debugger and there it says d= 0.8999999761581421. Same code but with double precision: d= 0.8999999999999999.
Why doesn't it calculate the correct number of 0.9?
The problem became apparent when I tried to calculate something like this:

int x= (int)(1.2-0.3)/0.1+1;

where I wanted to use just typecasting to convert into integer, because I always know the result should be some integer. But in the shown case the result is 9 instead of 10, because it calculates to 9.999... and the truncates. So I have to always use round to get the right result. For sure, I can live with that, but I was really wondering about that. Is it really meant this way? Shouldn't it give the exact result? Or is there some compiler switch or something like that? Thanks in advance.
2   L A T E S T    R E P L I E S    (Newest First)
zeron Posted - 07/26/2007 : 08:47:44 AM
Hmm. Interesting. I've never been aware of these inaccuracies. But I already made some workaround. Thanks again.
Mike Buess Posted - 07/26/2007 : 08:38:17 AM
See http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems and other sections of that Floating Point topic.

Mike Buess
Origin WebRing Member

The Origin Forum © 2020 Originlab Corporation Go To Top Of Page
Snitz Forums 2000