Page 1 of 1
Maximal decimal number ?
Posted: Wed May 20, 2020 4:00 pm
by prem111
Hi. Can the accuracy of the decimal number be increased to 6 digits ? eg: 50.820257
Re: Maximal decimal number ?
Posted: Thu May 21, 2020 7:03 pm
by dhylands
A 32-bit float only has six significant digits of precision.
Having said that, you can control the number of digits printed by using format:
Code: Select all
>>> print('{:.8f}'.format(50.12025733333333))
50.12025733
(that was using the unix version). On a pyboard you'll get:
Code: Select all
>>> print('{:.8f}'.format(50.12025733333333))
50.12025833
(notice the 833 rather than 733). The answer is still correct to six significant digits (2 before the decimal and 4 after in this case). If you print any more than that they'll only be correct some of the time.
Here's a link to the format specification (the :.8f portion):
https://docs.python.org/3.4/library/str ... formatspec
Re: Maximal decimal number ?
Posted: Fri May 22, 2020 5:50 am
by pythoncoder
The Unix version and the Pyboard D SF6W use double precision floats due to having the capability in hardware.
I think there is a compiler flag to force other platforms to use software double precision at the price of performance.
Re: Maximal decimal number ?
Posted: Fri May 22, 2020 10:28 pm
by patrickw
This Issue has the info on the Float Precision compiler flag
https://github.com/micropython/micropython/issues/4380