Porting MicroPython to SwissMicros DM42 / DMCP

[fnord]

Hi,

I would like to port MicroPython to the DMCP platform, and I need a little help getting started.

A bit of background: The SwissMicros DM42 Calculator is an improved version of HPs classic HP42s scientific RPN pocket calculator. It's not made (but apparently toleraded) by HP. It uses a basic operating system called DMCP whis is documented here: http://www.swissmicros.com/dmcp/doc/DMC ... index.html
On top of that runs a modified version of Thomas Okken's Free42 emulator. The emulator is one of potentially multiple programs on the device's FAT filesystem.

I'm working on a Python based RPN Calculator design (currently using PyQt, but the UI layer will be interchangable), and I would love to bring it to the DM42 hardware using MicroPython. Using the DMCP platform (instead of targeting the hardware directly) would have the benefit of being able to switch easily between the Free42 emulator and MicroPython, without having to reflash the device.

I am able to build simple .PGM files for the calculator, and I have also successfully built the unix and pyboard ports of MicroPython, but I'm at a loss at how to put the two systems together. The goal would be having a .PGM file for the calculator that contains the MicroPython interpreter which can then load .py files from the DMCP filesystem, and a module to access the display and keyboard through the DMCP API.

So, if anyone could help me setting up a way to bulid MicroPython as a program for the DMCP OS, or just point me in the right direction, that would be greatly appreciated.

Cheers!
Fnord

[jimmo]

Sounds really cool. I've had my eye on one of these for a while but just can't justify the $$$ so I just use Free42 on my phone.

I seem to remember that the calculator itself is based on a STM32L4 so a bare-metal port would be fairly straightforward, but yeah not terribly useful.

Probably the best way to get started is to look at the "embedding" example, which covers how to add MicroPython to an existing program (which is kind of what you're doing here). Basically you can initialize the VM, give it some heap and stack memory, and then start providing implementations of the various OS-level functionality (like file IO, stdio, etc). To some extent the Unix port is the same thing but has so much other complexity going on that it isn't really a good base.

w.r.t. the embedding example, I'm aware there was some discussion about it recently on github (see https://github.com/micropython/micropython/issues/5909 and https://github.com/micropython/micropython/pull/5912 ) (edit: this looks like it's related to my recent change so I will take a look ASAP).

[fnord]

Ah thank you, I somehow missed the embedding example - I'll have a look at that!

[fnord]

Okay, first I needed to adapt the Makefile so that libmicropython gets built with the arm toolchain, instead of amd64.

Code: Select all

-lmicropython:
        $(MAKE) -f Makefile.upylib MPTOP=$(MPTOP) CC=$(CC) CXX=$(CXX) AS="$(AS)" OBJCOPY=$(OBJCOPY) AR=$(AR) SIZE=$(SIZE)

That worked; now it is using arm-none-eabi-gcc.

However, I now get errors:

Code: Select all

In file included from /usr/arm-none-eabi/include/dirent.h:39,
                 from /home/fnord/data/Develop/DMPY/micropython/ports/unix/modos.c:34:
/usr/arm-none-eabi/include/sys/dirent.h:10:2: error: #error "<dirent.h> not supported"
   10 | #error "<dirent.h> not supported"
      |  ^~~~~
/home/fnord/data/Develop/DMPY/micropython/ports/unix/alloc.c:31:10: fatal error: sys/mman.h: No such file or directory
   31 | #include <sys/mman.h>
      |          ^~~~~~~~~~~~
compilation terminated.
make[1]: *** [/home/fnord/data/Develop/DMPY/micropython/py/mkrules.mk:76: build/genhdr/qstr.i.last] Error 1
make[1]: *** Deleting file 'build/genhdr/qstr.i.last'
make[1]: Leaving directory '/home/fnord/data/Develop/DMPY/DMCP_SDK'
make: *** [Makefile:175: -lmicropython] Error 2

I think the dirent.h problem comes from the fact that the arm toolchain is intended for bare metal use, so there wouldn't be any fiesystem support. I think I'll have to implement my own dirent.h that acts as an interface to the DMCP API.

Not sure yet what's up with mman.h though.

Also, I had to reduce the heap size to ~6k, otherwise I would get "section `.bss' will not fit in region `RAM'" errors.

I'll investigate further; hints are always welcome.

[jimmo]

Yeah, unfortunately the embedding example is kind of focused on "embedding in a Unix program". Which is why it uses a bunch of stuff from the Unix port, that just aren't useful here.

(This issue came up recently -- viewtopic.php?f=3&t=7861 -- there might be some useful hints there)

I'd start with just trying to strip as much stuff as possible out of it, although moduos is probably going to cause the most issues.

I mostly jsut wanted to highlight that it contains an example of how to set up the VM and execute some code. All the additional modules it provides are just distracting from the example (I'd really like to find the time to make a better embedding example)

[fnord]

Okay, I'll try to make this work for a bit longer, but come to think of it, a bare-metal port might be a viable plan b, or maybe even the better choice in the first place. I haven't completely made up my mind about that yet though.

Both approaches have their pros and cons:
Using the DMCP OS makes it easier for less experienced people to try out MicroPython, because it's easier to just switch programs than to fully reflash the calculator with dfu-util. And you can switch back and forth without any additional hardware, which is nice.
On the other hand, a bare-metal port, seems to be easier to do, leaves us more memory to work with, and we can use usb storage as well as usb serial, while the DMCP API does not seem to provide the latter (=> no REPL ).

It seems there are already MicroPython drivers for the Sharp MemoryLcds, and keyboard and buzzer should also not be too hard to implement. As far as I can see, there is no official hardware documentation for the DM42, but I mailed David and Michael from SwissMicros and asked if they could provide me with schematics, or at least the information which pins are used for which devices. I'm curious how they'll respond.

[jimmo]

Okay, I'll try to make this work for a bit longer, but come to think of it, a bare-metal port might be a viable plan b, or maybe even the better choice in the first place. I haven't completely made up my mind about that yet though.

Somewhat inspired by this thread, I put together https://github.com/micropython/micropython/pull/5964 to overhaul the embedding example and hopefully make it a bit simpler to work with.

[fnord]

Hi!

That's awesome, thank you!
I still have a few problems though. The library builds fine for Linux/x64, but when I add "CROSS_COMPILE=arm-none-eabi-" in the Makefile, I get the following error:

Code: Select all

CC ../../py/builtinimport.c
In file included from ../../py/qstr.h:30,
                 from ../../py/lexer.h:32,
                 from ../../py/compile.h:29,
                 from ../../py/builtinimport.c:32:
../../py/builtinimport.c: In function 'mp_builtin___import__':
./mpconfigport.h:43:38: error: 'PATH_MAX' undeclared (first use in this function)
   43 | #define MICROPY_ALLOC_PATH_MAX      (PATH_MAX)
      |                                      ^~~~~~~~

Is it okay to just add "#define PATH_MAX 256" to mpconfigport.h? Is 256 a sensible value or will it cause subtle problems somewhere else? When I add the PATH_MAX definition, I get the following:

Code: Select all

CC libmicropython.c
libmicropython.c:58:14: error: conflicting types for 'mp_import_stat'
   58 | MP_WEAK uint mp_import_stat(const char *path) {
      |              ^~~~~~~~~~~~~~
In file included from ../../py/compile.h:29,
                 from libmicropython.c:29:
../../py/lexer.h:193:18: note: previous declaration of 'mp_import_stat' was here
  193 | mp_import_stat_t mp_import_stat(const char *path);
      |                  ^~~~~~~~~~~~~~
make[1]: *** [../../py/mkrules.mk:63: /home/fnord/data/Develop/DMPY/DMCP_SDK/build-libmicropython/libmicropython.o] Fehler 1
make[1]: Verzeichnis „/home/fnord/data/Develop/DMPY/micropython/ports/libmicropython“ wird verlassen
make: *** [Makefile:86: /home/fnord/data/Develop/DMPY/DMCP_SDK/build-libmicropython/libmicropython.a] Fehler 2

I'm not sure how to resolve this, tbh.

EDIT: Just commenting out the mp_import_stat() definition in libmicropython.c seems to work. Is it actually okay to do that?

[jimmo]

EDIT: Just commenting out the mp_import_stat() definition in libmicropython.c seems to work. Is it actually okay to do that?

Ah sorry it looks like the original hello-embed.c implementation returned uint when it should have returned mp_import_stat_t (and I had just copied that out into libmicropython.c). I've pushed a fix for this.

[fnord]

Okay, after fixing that, I get:

Code: Select all

arm-none-eabi-gcc -c -mthumb -march=armv7e-m -mfloat-abi=hard -mfpu=fpv4-sp-d16 -D__weak="__attribute__((weak))" -D__packed="__attribute__((__packed__))" -Idmcp -Isrc   -Wall -fdata-sections -ffunction-sections -Wno-misleading-indentation -I/home/fnord/data/Develop/DMPY/DMCP_SDK/../micropython/ports/libmicropython -I/home/fnord/data/Develop/DMPY/DMCP_SDK/build-libmicropython -I/home/fnord/data/Develop/DMPY/DMCP_SDK/../micropython -O2 -g  -MD -MP -MF .dep/main.o.d -Wa,-a,-ad,-alms=build/main.lst src/main.c -o build/main.o
arm-none-eabi-gcc build/pgm_syscalls.o build/main.o  build/startup_pgm.o -mthumb -march=armv7e-m -mfloat-abi=hard -mfpu=fpv4-sp-d16 -Tstm32_program.ld   -Wl,-Map=build/TESTPGM.map,--cref -Wl,--gc-sections -Wl,--wrap=_malloc_r -L /home/fnord/data/Develop/DMPY/DMCP_SDK/build-libmicropython -l:libmicropython.a -o build/TESTPGM.elf
/usr/lib/gcc/arm-none-eabi/9.2.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/9.2.0/../../../../arm-none-eabi/lib/thumb/v7e-m+fp/hard/libc.a(lib_a-exit.o): in 
function `exit':
/builddir/build/BUILD/newlib-3.1.0/build-newlib/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/stdlib/../../../../../../../../newlib/libc/stdlib/exit.c:64: undefined reference to `_exit'
/usr/lib/gcc/arm-none-eabi/9.2.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/9.2.0/../../../../arm-none-eabi/lib/thumb/v7e-m+fp/hard/libc.a(lib_a-openr.o): in function `_open_r':
/builddir/build/BUILD/newlib-3.1.0/build-newlib/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/reent/../../../../../../../../newlib/libc/reent/openr.c:50: undefined reference to `_open'
collect2: error: ld returned 1 exit status
make: *** [Makefile:177: build/TESTPGM.elf] Fehler 1

apparently, _exit() and _open() are missing, but where to get them?
When I add

Code: Select all

void _exit(int v) {
    while(1);
}

int _open() {
    return 0;
}

to my main.c, the build succeeds, but that's probably not the right way to do it