Page 1 of 1

Maximal decimal number ?

Posted: Wed May 20, 2020 4:00 pm
by prem111
Hi. Can the accuracy of the decimal number be increased to 6 digits ? eg: 50.820257

Code: Select all

print(50.12025733333333)
50.82025



Re: Maximal decimal number ?

Posted: Thu May 21, 2020 7:03 pm
by dhylands
A 32-bit float only has six significant digits of precision.

Having said that, you can control the number of digits printed by using format:

Code: Select all

>>> print('{:.8f}'.format(50.12025733333333))
50.12025733
(that was using the unix version). On a pyboard you'll get:

Code: Select all

>>> print('{:.8f}'.format(50.12025733333333))
50.12025833
(notice the 833 rather than 733). The answer is still correct to six significant digits (2 before the decimal and 4 after in this case). If you print any more than that they'll only be correct some of the time.

Here's a link to the format specification (the :.8f portion): https://docs.python.org/3.4/library/str ... formatspec

Re: Maximal decimal number ?

Posted: Fri May 22, 2020 5:50 am
by pythoncoder
The Unix version and the Pyboard D SF6W use double precision floats due to having the capability in hardware.

I think there is a compiler flag to force other platforms to use software double precision at the price of performance.

Re: Maximal decimal number ?

Posted: Fri May 22, 2020 10:28 pm
by patrickw
This Issue has the info on the Float Precision compiler flag
https://github.com/micropython/micropython/issues/4380