Micro Python development wish list

[pfalcon]

While this is great, there seems to be no way for the code to communicate with the Python environment.

That's not how it's usually done - you don't interface simple, low-level system (like Assembler) with complex, high-level system (like Python), that's too cumbersome and error-prone. It's usually done the other way around. So, get a bytearray, pass its address to your assembler routines, communicate both ways via it. If you'd like to work on Python side with more structured representation than just array of bytes, it's possible too - there's uctypes module for that: http://docs.micropython.org/en/latest/l ... types.html .

The official way to get an address of array is btw uctypes.addressof(), which is unfortunately not yet documented on the page above (but otherwise it's the same as with CPython's ctypes).

[Damien]

The problem is that you can't actually pass anything to the callback function, so you can't pass a bytearray address to the assembler routine.

You can do it with a normal Python wrapper, but that defeats the purpose of having an efficient IRQ:

Code: Select all

@micropython.asm_thumb
def f(r0):
    ldr(r1, [r0, 0])
    add(r1, 1)
    str(r1, [r0, 0])

import pyb
b = bytearray(4)
tim = pyb.Timer(4, freq=1)
tim.callback(lambda t: f(b))

for i in range(20):
    print(b)
    pyb.delay(400)

[pythoncoder]

@pfalcon Thanks for the pointer regarding uctypes.addressof() - duly noted.

Regarding uctypes, the constructor for struct requires an argument specifying the address of the data block - a requirement which seems not to have made it to the docs.

[pfalcon]

Yes, uctypes docs are WIP, many things are yet to add. So far, good practical reference how to use it is tests from testsuite: https://github.com/micropython/micropyt ... sts/extmod

[cloudformdesign]

[*] Ability to allocate memory on an interrupt
Interrupts and callbacks play a large role in microcontrollers, especially to get real-time performance, and uPy should have good support for them. Currently you can run code on an interrupt, but you can't allocate heap memory during such an interrupt. The memory allocator can be made re-entrant so that memory can be allocated during an interrupt (steps have been made in this direction already, but more work and testing is needed). With this improvement you could write some very sophisticated control code that responded within a guaranteed time to an external event.[/list]

Improvements for embedded use (eg as replacement for Lua, or its Javascript equivalent: duktape, https://github.com/svaarala/duktape):

The biggest issue with allocating is that it might trigger a garbage collect (as I see it), and that would be bad during an interrupt.

When I worked with one of the PIC series it had a whole bunch of "remapping" features, such as remappable pins, uart, etc (where you could actually move which PIN register C5 or UART1_TX pointed to).

Anyway, one of the features it had was the ability to set interrupt levels on different kinds of interrupts. So you could say that a edge interrupt was more critical than a uart interrupt (for example) etc etc.

Hopefully modern ARM chips have this feature, as what you could do is have levels of interrupts. Level 1 is the highest and you can't use anything but global variables, but the lower levels have various "promises" in terms of how soon they will be available. Let's say level 2 will always be available within 10us. At 168MHz you get 1680 operations in 10us. That's a lot of operations! You could move 6.7kB of data in that time, for instance.

So you could have various levels of interrupts, only the highest one having the requirement of "only global variables"

I think for any of this to work though, you need a thread that continuously keeps everything suitably garbage collected (and defragmented?) that when the system asks for small amounts of memory it can be fulfilled quickly.

[pfalcon]

@pfalcon Thanks for the pointer regarding uctypes.addressof() - duly noted.

Regarding uctypes, the constructor for struct requires an argument specifying the address of the data block - a requirement which seems not to have made it to the docs.

Ok, uctypes docs now should be roughly complete: http://micropython.readthedocs.org/en/l ... types.html . Updates were written in kind of "autopilot" mode though, so it may be possible to improve clarity, style, and grammar. Patches welcome!

[mikaeleiman]

The 2 ideas I have to eliminate fragmentation are: 1) a copying memory manager/garbage collector, that can move parts of the heap around so as to make contiguous regions available; 2)

Here's source code for a copying/compacting memory manager:

https://github.com/contiki-os/contiki/b ... lib/mmem.c

Here's another compacting allocator: https://github.com/mikaelj/rmalloc

Takes up about 4.5 kB Flash with -O1 on ARM7, using gcc.

[pfalcon]

mikaeleiman, great! Seeing you are a memory allocation specialist, I wonder would you be interested to work on adding alternative memory allocation schemes for MicroPython? Note that initial step doesn't require any compactable memory managers and other advanceties, it should be just integration with a standard malloc/free heap.

[chrismas9]

I would like to see a more robust file system and host file sharing than FATFS and MSC. A journalling, wear levelling file system is much more robust.

The first step would be to replace MSC which accesses sectors with a file sharing or transfer protocol. Henrik Solver suggested MTP (Media Transfer Protocol) which is used by most phones and cameras. The downside is you cannot edit scripts directly but you could probably write a script for your editor to make a temporary copy and transfer it back on save. Another possibility would be to use Ethernet over USB (USB Gadget) and file sharing (micro Samba?). This would allow direct editing of files. I don't know if either of these would fit the micro footprint of MicroPython. These options do not require a FAT file system for host access to work.

Implementing file transfer or sharing would go a long way to stop corrupt file systems. The door would then be open to implement a robust file system, possibly using external Flash, EEprom or FRAM for higher endurance.

[dhylands]

I'm not sure why you wouldn't be able to edit files on devices mounted via MTP, at least under Windows and Linux. OSX requires you to use Android File Transfer, but linux mounts the MTP filesystem just like a regular filesystem.

If you had MTP then we could change the underlying filesystem, but with UMS the filesystem neds to be something that the host recognizes, which is why FAT is typically used (WIndows, Mac, and linux all understand FAT).

To use the ethernet gadget, you'd have to implement some type of network file sharing before you could edit files directly (and things like smaba or NFS are not exactly small). Having an ethernet gadget would be interesting from other perspectives. Like you could implement telnet or ftp (which are both fairly simple) and you could use sockets to talk to processes on the pyboard.

I currently disable USB Mass Storage and use rshell.py (under linux) to copy files in and out (and edit).