Improvements to uctypes module/infrastructure

[pfalcon]

I was recently working on uctypes module fixing and improvements. The main improvement is ability to automatically calculate field offsets in structures - something which was envisioned from the beginning, but kept being backlogged. Other changes are described in https://github.com/pfalcon/micropython/issues/14 and in commits in my fork, https://github.com/pfalcon/micropython . I started to spool these to mainline, but patch processing in the mainline goes veeeery slow, so it make take long time until all changes are there.

Beyond work on uctypes modules itself, I also started some helper modules in the micropython-lib project. Those are work in progress and subject to change, but cover some ideas which were in thinking for some time too. Commit message should give info what they're about:

• https://github.com/pfalcon/micropython- ... dfd89d2938
• https://github.com/pfalcon/micropython- ... 312710b866

[stijn]

Looks good! Just needs examples but I see those are on the TODO list already

[marfis]

+1 from me too.
and +2 for examples and hints on using uctypes...

[jickster]

I'm a C, C++ programmer and I JUST realized "offsets" is there to perform the function of casting a void* to a struct in C.

Maybe I'm slow but it'd be very helpful to state the above fact in the documentation.

[pfalcon]

A test-example for uctypeslib was posted: https://github.com/pfalcon/micropython- ... ib/test.py . It was roughly based on https://github.com/pfalcon/micropython/ ... ypes_le.py , so can be used for comparison.

Code: Select all

desc = OrderedDict((
    ("s0", uc.UINT16),
    ("sub", uc.STRUCT(od((
        ("b0", uc.UINT8),
        ("b1", uc.UINT8),
    )))),
    ("arr", uc.ARRAY(uctypes.UINT8, 2)),
    ("arr2", uc.ARRAY(uc.STRUCT(od([("b", uc.UINT8)])), 2)),

    ("bitf0", uc.BITFIELD(0, 8, type=uc.UINT16)),
    ("bitf1", uc.BITFIELD(8, 8)),

    ("bf0", uc.C_BITFIELD(4, type=uc.UINT16)),
    ("bf1", uc.C_BITFIELD(4)),
    ("bf2", uc.C_BITFIELD(4)),
    ("bf3", uc.C_BITFIELD(4)),
))

P.S. Actually, uctypeslib/test.py may look less pretty due to the fact that there's no support for OrderedDict literals (i.e. OrderedDict({...}) doesn't guarantee field order), so tuples of tuples have to be used instead, leading to the proliferation of parens. That's actually the reason why this work was postponed for so long - the whole idea of these additions is to make it easier and more beautiful, but "beautiful" part regresses without OrderedDict literals. Anyway, I decided to do all that to "finish" the uctypes modules, and OD literals are definitely on TODO too.

[pfalcon]

Updated docs for the uctypes module are available at https://pycopy.readthedocs.io/en/latest ... types.html . (So far, these capture the state of the module before the changes described in this thread.)

[pfalcon]

The last thing in queue for this work was implementing the proper OrderedDict literals, which is now done too. So, it's possible to write:

OrderedDict({"foo": "bar"})

And it will be handled as efficient as just

Code: Select all

{"foo": "bar"}

. So, no compromises on both readability and efficiency are now required.

Initial patchset for all these features was posted for the mainline, but by now only 1-2 of patches were processed. So as usual, these and many other great features are available in my fork, https://github.com/pfalcon/micropython .

[pfalcon]

And now project which prompted this work was posted too - FFmpeg bindings: viewtopic.php?f=15&t=5419 .

FFmpeg uses data structures (not just functions) in its public API, so emphasized not implemented/missing features in uctypes quite well.

[pfalcon]

Doc changes mentioned above are now in the mainline: http://docs.micropython.org/en/latest/l ... types.html