What type is tipSelected, and what is its value? if it's value is less than one, then tipTotal will be much smaller then one cent, and will get rounded to 0.00 by your %.2f. Your first set of code seems to give the mathematically "correct" answer if I plug in 10 and 50.
Code:
int tipSelected = 10;
float subtotal = 50.00
float tipTotal = tipSelected * 3.8 / subtotal;
NSLog(@"tipTotal: %f", tipTotal);
I don't really understand your code or what you're calculating though; what is the 3.8 for, and why are you dividing by the bill amount?