01-10-2010, 06:09 PM
#1 (permalink )
Registered Member
Join Date: Aug 2009
Posts: 58
Float rounding incorrectly
Hi All,
I have a float which I get by adding a few things and dividing them -- a weighted average.
If I log the float value, it gives me 8.250000, but if I log the %.1f value, it gives me 8.2! All my other values that are x.5 return the rounded up value.
Code:
float avgRating = totalRating / 4;
NSLog(@"ratings: %f, %f, %.1f", totalRating, avgRating, avgRating);
........
The console output is:
ratings: 32.000000, 8.000000, 8.0
ratings: 38.000000, 9.500000, 9.5
ratings: 40.000000, 10.000000, 10.0
ratings: 39.000000, 9.750000, 9.8
ratings: 34.000000, 8.500000, 8.5
ratings: 30.000000, 7.500000, 7.5
ratings: 35.000000, 8.750000, 8.8
ratings: 35.000000, 8.750000, 8.8
ratings: 33.000000, 8.250000, 8.2
Note the last line...
Any ideas?
Cheerio,
Tim
01-10-2010, 06:18 PM
#2 (permalink )
Programming Wizard
Join Date: Oct 2008
Location: Northampton, MA USA
Age: 45
Posts: 193
Code:
NSLog(@"ratings: %f, %f, %.1f", totalRating, avgRating, (avgRating + 0.05));
__________________
|
| I wrote
ChordCalc ...
A cool fretboard calculator .
|
01-10-2010, 06:24 PM
#3 (permalink )
Registered Member
Join Date: Aug 2009
Posts: 58
Quote:
Originally Posted by
slahteine
Code:
NSLog(@"ratings: %f, %f, %.1f", totalRating, avgRating, (avgRating + 0.05));
This throws other values out:
Code:
float avgRating = totalRating / 4;
NSLog(@"ratings: %f, %f, %.1f", totalRating, avgRating, avgRating+0.05);
...
ratings: 32.000000, 8.000000, 8.1
ratings: 38.000000, 9.500000, 9.6
ratings: 40.000000, 10.000000, 10.1
ratings: 39.000000, 9.750000, 9.8
ratings: 34.000000, 8.500000, 8.6
ratings: 30.000000, 7.500000, 7.5
ratings: 35.000000, 8.750000, 8.8
ratings: 35.000000, 8.750000, 8.8
ratings: 33.000000, 8.250000, 8.3
01-10-2010, 06:25 PM
#4 (permalink )
Registered Member
Join Date: Aug 2009
Posts: 58
Quote:
Originally Posted by
slahteine
Code:
NSLog(@"ratings: %f, %f, %.1f", totalRating, avgRating, (avgRating + 0.05));
although if you meant to say 0.005, bravo!
works a treat.
cheerio
01-10-2010, 09:20 PM
#5 (permalink )
Registered Member
Join Date: Nov 2009
Posts: 580
Are you sure that 8.250000 is the exact value of avgRating? Or is this value itself being rounded to 6 decimal places when it's being logged? If the true value of the variable were 8.24999999 then it would actually be doing the correct thing when rounding to one place.
I'd try putting a breakpoint in your code and inspecting the actual value of avgRating in the debugger, not through a log statement.
Thread Tools
Display Modes
Linear Mode
Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
» Advertisements
» Online Users: 319
12 members and 307 guests
2Apps1Day , akacaj , Domele , Duncan C , GraffitiCircus , michelle , NetGuru , NSString , Paul Slocum , Sloshmonster , soohyun , v1n2e7t
Most users ever online was 1,387, 04-10-2012 at 04:21 AM.
» Stats
Members: 175,650
Threads: 94,114
Posts: 402,885
Top Poster: BrianSlick (7,990)
Welcome to our newest member, soohyun