Builtin modules are loaded at the point of importing them

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
pulkin
Posts: 44
Joined: Tue Feb 19, 2019 10:22 pm

Builtin modules are loaded at the point of importing them

Post by pulkin » Fri Feb 28, 2020 10:41 pm

When I turn debug messages on I see the following:

Code: Select all

MicroPython v1.11-697-g7b18c0b93-dirty on 2020-02-28; A9/A9G module with RDA8955
Type "help()" for more information.
>>> import sys
...
import name 'sys' level=0
__import__:
  'sys'
  None
  None
  None
  0
Module already loaded
...
This means that there is no way for the user to override this name. How do I fix it?

User avatar
jimmo
Posts: 1681
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Builtin modules are loaded at the point of importing them

Post by jimmo » Fri Feb 28, 2020 11:52 pm

sys is a bit different to the other built-in modules like uos and umachine where "import os" first tries os.py then builtin uos -- it's actually called sys, not usys.

So there's no way to provide a custom sys.py

What are you trying to do?

pulkin
Posts: 44
Joined: Tue Feb 19, 2019 10:22 pm

Re: Builtin modules are loaded at the point of importing them

Post by pulkin » Sat Feb 29, 2020 9:38 pm

It is not only sys: every built-in module is loaded.

User avatar
jimmo
Posts: 1681
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Builtin modules are loaded at the point of importing them

Post by jimmo » Sat Feb 29, 2020 10:27 pm

I don't really understand your question then, what are you trying to do?

If you want a custom "os" you can write os.py.

pulkin
Posts: 44
Joined: Tue Feb 19, 2019 10:22 pm

Re: Builtin modules are loaded at the point of importing them

Post by pulkin » Sat Feb 29, 2020 11:45 pm

I believe the problem is that you cannot override the built-in module. The typical way one would do it is to manipulate sys.path before importing the module. In my case, however, that does not help: whatever import order is the built-in module is always preffered/loaded. I.e. there is no way to import, for example, os.py or machine.py. I traced down the reason down to the output in the header: all built-in modules are loaded already at startup, whatever that means. I am wondering what is the reason for such behavior: the ESP8266 port perfectly allows this kind of hack.

User avatar
jimmo
Posts: 1681
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Builtin modules are loaded at the point of importing them

Post by jimmo » Mon Mar 02, 2020 1:35 am

pulkin wrote:
Sat Feb 29, 2020 11:45 pm
I believe the problem is that you cannot override the built-in module.
This is definitely true of `sys`, but not `uos`, `umachine`, etc and many other built-in modules.

Here's an example interaction of overriding built-in os.

Code: Select all

$ cat /tmp/os.py
print('import os.py')
def fake_os():
  print('hi')

$ pyboard --device /dev/ttyUSB0 -f cp /tmp/os.py :
cp /tmp/os.py :os.py

$ miniterm /dev/ttyUSB0 115200
--- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---

>>> 
MPY: soft reboot
MicroPython v1.12-210-g1993c8cf9 on 2020-03-02; TinyPICO with ESP32-PICO-D4
Type "help()" for more information.
>>> 
>>> import os
import os.py
>>> os.fake_os()
hi
>>> import uos
>>> uos.uname()
(sysname='esp32', nodename='esp32', release='1.12.0', version='v1.12-210-g1993c8cf9 on 2020-03-02', machine='TinyPICO with ESP32-PICO-D4')
>>> os.fake_os()
hi
>>> 
Can you provide a similar example?

User avatar
jimmo
Posts: 1681
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Builtin modules are loaded at the point of importing them

Post by jimmo » Mon Mar 02, 2020 1:40 am

I just noticed you're on ESP8266 not ESP32, but tested it here on an ESP8266 too and same result.

pulkin
Posts: 44
Joined: Tue Feb 19, 2019 10:22 pm

Re: Builtin modules are loaded at the point of importing them

Post by pulkin » Wed Mar 04, 2020 1:07 pm

Right, what I am saying is that I have a port

https://github.com/pulkin/micropython/t ... ts/gprs_a9

where I have to make it behave like you demonstarted. At the firmware level.

User avatar
jimmo
Posts: 1681
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Builtin modules are loaded at the point of importing them

Post by jimmo » Wed Mar 04, 2020 1:20 pm

pulkin wrote:
Wed Mar 04, 2020 1:07 pm
Right, what I am saying is that I have a port
This feature is called "weak links" -- the idea is that "os" is a weak link to "uos". (The name comes from "weak symbol" in a linker).

Might be a good idea to rebase onto the current master (looks like your fork is up to about mid-September 2019). The way the foo -> ufoo imports works changed in mid-October 2019.

(It used to be that each port managed its own set of weak links (e.g. look at stm32/mpconfigport.h for the MICROPY_PORT_BUILTIN_MODULE_WEAK_LINKS definition). This provides the fallback registration for (e.g.) "os" to "uos" (if os.py isn't found). Since late last year, it was just changed that when MICROPY_MODULE_WEAK_LINKS is enabled, any time you import a module named "foo" it will try "ufoo" (i.e. it always just tries to prefix a "u'). See https://github.com/micropython/micropython/pull/5241
pulkin wrote:
Wed Mar 04, 2020 1:07 pm
where I have to make it behave like you demonstarted. At the firmware level.
Hopefully the above explains everything you need...

Note that because the "sys" module is named "sys" (not "usys"), this is not possible for "sys".

pulkin
Posts: 44
Joined: Tue Feb 19, 2019 10:22 pm

Re: Builtin modules are loaded at the point of importing them

Post by pulkin » Wed Mar 04, 2020 4:11 pm

To verify my understanding: does this mean that up-to-date ESP8266 firmware allows importing files named os.py but will not import uos.py?

Post Reply