int() Anomaly

[scruss]

This seemed to work for someone on the Raspberry Pi forum: Re: MicroPython double precision

[TheSilverBullet]

I believe this is correct, but I haven't actually tried it.

Thanks Peter,
Will give it a try as soon as I'm back. (Traveling right now)

[Phisatho]

I built with micropython after applying #define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_DOUBLE) as thesilverbullet and can confirm that the "anomaly" has disappeared.

[TheSilverBullet]

I built with micropython after applying #define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_DOUBLE) as thesilverbullet and can confirm that the "anomaly" has disappeared.

Nice. But rest assured, the ›anomaly‹ hasn't disappeared, it has been shift from the seventh to the seventeenth digit. Not as visible as before.
print(f'{1.0/9.0:.30f}')
0.111111111111111104943205418749

[karfas]

Nice. But rest assured, the ›anomaly‹ hasn't disappeared, it has been shift from the seventh to the seventeenth digit

This isn't an "anomaly". It is quite normal that the precision of floats is limited.
For more precision, you can use the integers (in micropython with arbitrary size).

[TheSilverBullet]

Nice. But rest assured, the ›anomaly‹ hasn't disappeared, it has been shift from the seventh to the seventeenth digit

This isn't an "anomaly". It is quite normal that the precision of floats is limited.
For more precision, you can use the integers (in micropython with arbitrary size).

Trust me, I know precisely what it is. IEEE knows it as well. They have the 754…

[Roberthh]

So we have 16 significant digits at least in this little test:

With CPython:

print(f'{1.0/9.0:.30f}')
0.111111111111111104943205418749

With Micropython at Teensy 4.x, PyBoard D SF6 and rp2, compiled with double precision:

print(f'{1.0/9.0:.30f}')
0.11111111111111111604543566500

[TheSilverBullet]

So we have 16 significant digits at least in this little test:

Exactly!
16 significant decimal digits which translates to a binary mantissa of size:
ln(10) / ln(2) * 16 = 53,…
And those 53 binary digits/bits, combined with 11 binary digits/bits as exponent are what's defines as double precision floats.
The important part is:
It is absolutely impossible to represent certain fractions in binary notation.
Pretty much the same way as it's impossible to represent certain decimal fractions.

But a ›double‹ resolution variable is 2^19 times better/more precise than a regular ›float‹ variable.
And that's a significant improvement, if really needed. Because at the same time it might be a bit slower.

[Roberthh]

Let me add that the binary representation of that number is the same with CPython and MicroPython. Only the printed results differ in the part, which is anyhow not significant.

[Phisatho]

Between single precision and double precision, the underlying representation itself differs - not just the displaying format. In my case, it was significant. I was trying to subtract a fractional offset from NTP_OFFSET. The conversion error caused 88 seconds difference.