Page 1 of 1

Using mpy files in an MP library

Posted: Wed Jun 08, 2022 8:27 pm
by cduran
I'm currently building MicroPython as a library to be used in another application as an embedded python interpreter. I would like to be able to include new mpy files without having to rebuild mp or the application it is embedded in. How would I do this?

Re: Using mpy files in an MP library

Posted: Mon Jun 20, 2022 6:34 am
by jimmo
cduran wrote:
Wed Jun 08, 2022 8:27 pm
I'm currently building MicroPython as a library to be used in another application as an embedded python interpreter. I would like to be able to include new mpy files without having to rebuild mp or the application it is embedded in. How would I do this?
Now I understand your other post (viewtopic.php?f=3&t=12555) :)

You have two options:
- Provide a VFS filesystem implementation, then MicroPython can use that as it's filesystem, and import will "just work". This is how the unix port works, it implements a filesystem backed by posix open/read/stat/etc (see vfs_posix.c).
- Provide an implementation of mp_import_stat and mp_builtin_open and mp_reader_new_file. The import statement will use mp_import_stat to decide whether the import exists, then mp_reader_new_file to get an object that provides readbyte and close (to actually provide the .mpy file bytes).

Re: Using mpy files in an MP library

Posted: Tue Jun 21, 2022 2:39 pm
by cduran
jimmo wrote:
Mon Jun 20, 2022 6:34 am
cduran wrote:
Wed Jun 08, 2022 8:27 pm
I'm currently building MicroPython as a library to be used in another application as an embedded python interpreter. I would like to be able to include new mpy files without having to rebuild mp or the application it is embedded in. How would I do this?
Now I understand your other post (viewtopic.php?f=3&t=12555) :)

You have two options:
- Provide a VFS filesystem implementation, then MicroPython can use that as it's filesystem, and import will "just work". This is how the unix port works, it implements a filesystem backed by posix open/read/stat/etc (see vfs_posix.c).
- Provide an implementation of mp_import_stat and mp_builtin_open and mp_reader_new_file. The import statement will use mp_import_stat to decide whether the import exists, then mp_reader_new_file to get an object that provides readbyte and close (to actually provide the .mpy file bytes).
I decided to go with the unix build, but this still stands. I'm building the unix build as a statically linked library and linking it into my application. When building it as an application I was able to put the mpy file in the same directory as the mp executable, but that doesn't work with my application.

Re: Using mpy files in an MP library

Posted: Wed Jun 22, 2022 12:21 am
by jimmo
cduran wrote:
Tue Jun 21, 2022 2:39 pm
I was able to put the mpy file in the same directory as the mp executable, but that doesn't work with my application.
What do you mean by "doesn't work"... having the file in the same directory isn't what you want (does sys.path help?) or it just flat out doesn't load? As you're running on x86/x64 it should be relatively straightforward to build with DEBUG=1 and trace the builtinimport.c code with gdb.

Re: Using mpy files in an MP library

Posted: Fri Jun 24, 2022 8:08 pm
by cduran
jimmo wrote:
Wed Jun 22, 2022 12:21 am
cduran wrote:
Tue Jun 21, 2022 2:39 pm
I was able to put the mpy file in the same directory as the mp executable, but that doesn't work with my application.
What do you mean by "doesn't work"... having the file in the same directory isn't what you want (does sys.path help?) or it just flat out doesn't load? As you're running on x86/x64 it should be relatively straightforward to build with DEBUG=1 and trace the builtinimport.c code with gdb.
I meant the mpy file wasn't loading. However, I have other issues to fix (with my own application) which could be the problem.

Re: Using mpy files in an MP library

Posted: Thu Jul 21, 2022 5:27 pm
by cduran
jimmo wrote:
Wed Jun 22, 2022 12:21 am
cduran wrote:
Tue Jun 21, 2022 2:39 pm
I was able to put the mpy file in the same directory as the mp executable, but that doesn't work with my application.
What do you mean by "doesn't work"... having the file in the same directory isn't what you want (does sys.path help?) or it just flat out doesn't load? As you're running on x86/x64 it should be relatively straightforward to build with DEBUG=1 and trace the builtinimport.c code with gdb.
So I was able to solve my other issues, but I'm trying to figure out how to change where mp looks for the mpy files. It works just fine if I drop the mpy file in the same directory as the executable using the library, but I really need to be able to use it in a separate directory.

I tried using the paths in mpconfigport.h

Code: Select all

#ifndef MICROPY_PY_SYS_PATH_DEFAULT
#define MICROPY_PY_SYS_PATH_DEFAULT ".frozen:~/.micropython/lib:/usr/lib/micropython"
#endif
However none of them seem to work. I tried adding one at the end of that define "~/Documents" to be exact and that didn't work.

Keep in mind that I am not calling any functions in the unix port's main.c file, I'm simply trying to use any of the directories in MICROPY_PY_SYS_PATH_DEFAULT, I also tried setting the MICROPYPATH environment variable. Is there some function I need to call to register this? I'm running this under Ubuntu.

Re: Using mpy files in an MP library

Posted: Thu Aug 11, 2022 1:42 am
by jimmo
cduran wrote:
Thu Jul 21, 2022 5:27 pm
Keep in mind that I am not calling any functions in the unix port's main.c file, I'm simply trying to use any of the directories in MICROPY_PY_SYS_PATH_DEFAULT, I also tried setting the MICROPYPATH environment variable. Is there some function I need to call to register this? I'm running this under Ubuntu.
MICROPY_PY_SYS_PATH_DEFAULT is used by main.c to set up sys.path.

If you want to make sys.path work then you'll need to configure it explicitly.