Garmin's formula for converting degrees to semicircles to degrees is:
d = s * (180.0D / 2^31)
When s = 119304640
d = 9.9999994039536
But 10 was entered on the GPS. Does anyone know of a more accurate
formula for converting semicircles to degrees?
d = s * (180.0D / 2^31)+0.000002
When s = 119304640 and D=1 then d = 10.000001403953552525650000000000
Does that look better?
This reminds me of an operator that was continually writing up a
system that was supposed to maintain 1500 pounds and the digital meter
kept reading 1499. Tweak the meter to read 1501 and all the complaints
stopped. How important is the difference other than it doesn't look
right when there are a string of 9's rather than a string of 0's?
You should never expect more accuracy from an answer to a formula than
the accuracy of the input. Therefore your d should be no more than 9
significant digits or 9.99999940. If we consider the 180.0 to really
be 180 and an infinite number of zeros then the real max accuracy of
the formula would be limited by 2^31 or again 9 digits. 9 digit
accuracy is VERY good. It is the difference between 30 years and 30
years plus 1 second. Do you really need more than that or is it just a
string of 9's isn't as personally satisfying as a string of 0's??
