Using mpy files in an MP library

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Using mpy files in an MP library

Post by cduran » Wed Jun 08, 2022 8:27 pm

I'm currently building MicroPython as a library to be used in another application as an embedded python interpreter. I would like to be able to include new mpy files without having to rebuild mp or the application it is embedded in. How would I do this?

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Using mpy files in an MP library

Post by jimmo » Mon Jun 20, 2022 6:34 am

cduran wrote:
Wed Jun 08, 2022 8:27 pm
I'm currently building MicroPython as a library to be used in another application as an embedded python interpreter. I would like to be able to include new mpy files without having to rebuild mp or the application it is embedded in. How would I do this?
Now I understand your other post (viewtopic.php?f=3&t=12555) :)

You have two options:
- Provide a VFS filesystem implementation, then MicroPython can use that as it's filesystem, and import will "just work". This is how the unix port works, it implements a filesystem backed by posix open/read/stat/etc (see vfs_posix.c).
- Provide an implementation of mp_import_stat and mp_builtin_open and mp_reader_new_file. The import statement will use mp_import_stat to decide whether the import exists, then mp_reader_new_file to get an object that provides readbyte and close (to actually provide the .mpy file bytes).

cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Re: Using mpy files in an MP library

Post by cduran » Tue Jun 21, 2022 2:39 pm

jimmo wrote:
Mon Jun 20, 2022 6:34 am
cduran wrote:
Wed Jun 08, 2022 8:27 pm
I'm currently building MicroPython as a library to be used in another application as an embedded python interpreter. I would like to be able to include new mpy files without having to rebuild mp or the application it is embedded in. How would I do this?
Now I understand your other post (viewtopic.php?f=3&t=12555) :)

You have two options:
- Provide a VFS filesystem implementation, then MicroPython can use that as it's filesystem, and import will "just work". This is how the unix port works, it implements a filesystem backed by posix open/read/stat/etc (see vfs_posix.c).
- Provide an implementation of mp_import_stat and mp_builtin_open and mp_reader_new_file. The import statement will use mp_import_stat to decide whether the import exists, then mp_reader_new_file to get an object that provides readbyte and close (to actually provide the .mpy file bytes).
I decided to go with the unix build, but this still stands. I'm building the unix build as a statically linked library and linking it into my application. When building it as an application I was able to put the mpy file in the same directory as the mp executable, but that doesn't work with my application.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Using mpy files in an MP library

Post by jimmo » Wed Jun 22, 2022 12:21 am

cduran wrote:
Tue Jun 21, 2022 2:39 pm
I was able to put the mpy file in the same directory as the mp executable, but that doesn't work with my application.
What do you mean by "doesn't work"... having the file in the same directory isn't what you want (does sys.path help?) or it just flat out doesn't load? As you're running on x86/x64 it should be relatively straightforward to build with DEBUG=1 and trace the builtinimport.c code with gdb.

cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Re: Using mpy files in an MP library

Post by cduran » Fri Jun 24, 2022 8:08 pm

jimmo wrote:
Wed Jun 22, 2022 12:21 am
cduran wrote:
Tue Jun 21, 2022 2:39 pm
I was able to put the mpy file in the same directory as the mp executable, but that doesn't work with my application.
What do you mean by "doesn't work"... having the file in the same directory isn't what you want (does sys.path help?) or it just flat out doesn't load? As you're running on x86/x64 it should be relatively straightforward to build with DEBUG=1 and trace the builtinimport.c code with gdb.
I meant the mpy file wasn't loading. However, I have other issues to fix (with my own application) which could be the problem.

cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Re: Using mpy files in an MP library

Post by cduran » Thu Jul 21, 2022 5:27 pm

jimmo wrote:
Wed Jun 22, 2022 12:21 am
cduran wrote:
Tue Jun 21, 2022 2:39 pm
I was able to put the mpy file in the same directory as the mp executable, but that doesn't work with my application.
What do you mean by "doesn't work"... having the file in the same directory isn't what you want (does sys.path help?) or it just flat out doesn't load? As you're running on x86/x64 it should be relatively straightforward to build with DEBUG=1 and trace the builtinimport.c code with gdb.
So I was able to solve my other issues, but I'm trying to figure out how to change where mp looks for the mpy files. It works just fine if I drop the mpy file in the same directory as the executable using the library, but I really need to be able to use it in a separate directory.

I tried using the paths in mpconfigport.h

Code: Select all

#ifndef MICROPY_PY_SYS_PATH_DEFAULT
#define MICROPY_PY_SYS_PATH_DEFAULT ".frozen:~/.micropython/lib:/usr/lib/micropython"
#endif
However none of them seem to work. I tried adding one at the end of that define "~/Documents" to be exact and that didn't work.

Keep in mind that I am not calling any functions in the unix port's main.c file, I'm simply trying to use any of the directories in MICROPY_PY_SYS_PATH_DEFAULT, I also tried setting the MICROPYPATH environment variable. Is there some function I need to call to register this? I'm running this under Ubuntu.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Using mpy files in an MP library

Post by jimmo » Thu Aug 11, 2022 1:42 am

cduran wrote:
Thu Jul 21, 2022 5:27 pm
Keep in mind that I am not calling any functions in the unix port's main.c file, I'm simply trying to use any of the directories in MICROPY_PY_SYS_PATH_DEFAULT, I also tried setting the MICROPYPATH environment variable. Is there some function I need to call to register this? I'm running this under Ubuntu.
MICROPY_PY_SYS_PATH_DEFAULT is used by main.c to set up sys.path.

If you want to make sys.path work then you'll need to configure it explicitly.

Post Reply