[ADAFRUIT_F405_EXPRESS] struct.pack doesn't error on overflow

Discussion and questions about boards that can run MicroPython but don't have a dedicated forum.
Target audience: Everyone interested in running MicroPython on other hardware.
Post Reply
CarlFK
Posts: 7
Joined: Sun Aug 16, 2020 6:18 am

[ADAFRUIT_F405_EXPRESS] struct.pack doesn't error on overflow

Post by CarlFK » Tue Sep 22, 2020 9:24 pm

I am guessing this is to save space, but the result is I have to write more python code, which may take up more space.
Or maybe it is an over site.

Can someone help me track down how CP implements this so i can look at the cosponsoring MP code that is around here:
https://github.com/micropython/micropyt ... uct.c#L215

Code: Select all

Python 3.6.9 (default, Jul 17 2020, 12:50:27) 
>>> import struct
>>> struct.pack("H", 65536)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: ushort format requires 0 <= number <= (0x7fff * 2 + 1)

Adafruit CircuitPython 5.3.1 on 2020-07-13; Adafruit CircuitPlayground Express with samd21g18
>>> import struct
>>> struct.pack("H", 65536)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: value must fit in 2 byte(s)

MicroPython v1.13 on 2020-09-07; Adafruit Feather STM32F405 with STM32F405RG
Type "help()" for more information.
>>> import struct
>>> struct.pack("H", 65536)
b'\x00\x00'

User avatar
jimmo
Posts: 2015
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: [ADAFRUIT_F405_EXPRESS] struct.pack doesn't error on overflow

Post by jimmo » Wed Sep 23, 2020 12:43 am

CarlFK wrote:
Tue Sep 22, 2020 9:24 pm
I am guessing this is to save space, but the result is I have to write more python code, which may take up more space.
Or maybe it is an over site.
Could be both. In general things that decrease firmware code size at the cost of slightly more Python code are preferred because everyone pays the firmware cost, but only people using the feature pay the Python cost. But in this case, it does seem like a bit of a bug that should also be fixed in MicroPython.
CarlFK wrote:
Tue Sep 22, 2020 9:24 pm
Can someone help me track down how CP implements this so i can look at the cosponsoring MP code that is around here:
https://github.com/micropython/micropyt ... uct.c#L215
Two ways to do this:
1. (the lazy but fast way) whatever code you're interested in must contain the string "value must fit" so just search the entire codebase for it.
2. (the hard work way) trace the path from modstruct.c --> mp_binary_set_val in binary.c --> mp_obj_int_buffer_overflow_check in objint.c

Either way, the corresponding CircuitPython commit is https://github.com/adafruit/circuitpyth ... 9cac6a4302

CarlFK
Posts: 7
Joined: Sun Aug 16, 2020 6:18 am

Re: [ADAFRUIT_F405_EXPRESS] struct.pack doesn't error on overflow

Post by CarlFK » Wed Sep 23, 2020 8:21 pm

https://github.com/micropython/micropython/issues/6477

I am trying to port that commit now:

git fetch https://github.com/adafruit/circuitpython
git cherry-pick 095c844004bcd8680a4bf68901adbd9cac6a4302

4 patch fails, trying to resolve them.
Which is tricky as the starting point has subtle changes like

Code: Select all

diff micropython/py/binary.c circuitpython/py/

<             if (mp_obj_is_type(val_in, &mp_type_int)) {
>             if (MP_OBJ_IS_TYPE(val_in, &mp_type_int)) { 

Code: Select all

<         return mp_obj_new_float(fpu.f);
>         return mp_obj_new_float((mp_float_t) fpu.f);

User avatar
jimmo
Posts: 2015
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: [ADAFRUIT_F405_EXPRESS] struct.pack doesn't error on overflow

Post by jimmo » Thu Sep 24, 2020 2:11 am

Yeah you might be better off trying to create the same effective change from scratch.

But at least in those two examples the resolution is straightforward, but in general MicroPython has made many code changes and reorganisations since CircuitPython last rebased.

Post Reply