Module loading times

[Roberthh]

I just made an observation regarding the time it takes to re-load a modules. The follwoing simple program shows the difference between two modules.

Code: Select all

# test import
import pyb
from os import unlink, rename
from _io import StringIO, BytesIO

def imp_os():
    from os import unlink, rename
    return None

def imp_io():
    from _io import StringIO, BytesIO
    return None

start = pyb.millis()

for i in range(1000):
    imp_os()

mid = pyb.millis()

for i in range(1000):
    imp_io()

stop = pyb.millis()
print ("Import os: {} ms, import _io: {} ms".format((mid-start)/1000, (stop - mid)/1000))

The test program shows the following result:
>>> import imptest
Import os: 30.751 ms, import _io: 0.026 ms
It seems that the times differ by a factor of 1000, which I found surprising. if I load the full modules, the difference is even larger (31,5ms vs. 0,019ms)
MicroPython v1.5.1-171-g7ce8860 on 2015-12-25; PYBv1.0 with STM32F405RG
Update1: No substantial change with build MicroPython v1.5.1-190-g521759e on 2015-12-31; PYBv1.0 with STM32F405RG
Update2: If running from internal flash, the times are: Import os: 0.246 ms, import _io: 0.025 ms
Update 3: When loading uos instead of os, the times are similar: Import uos: 0.02 ms, import _io: 0.018 ms

Regards

[dhylands]

When you try to import the module a second time, it notices that the module is already loaded, and just returns a handle to the already imported module.

If you'd like to reload the module, then you need to remove it from sys.modules. Then you can import it a second time and it will reload the module from storage.

For example, I have a hello.py file that contains just a single line:

Code: Select all

print('Hello World')

then from the REPL:

Code: Select all

>>> import hello
Hello World
>>> import hello
>>> del sys.modules['hello']
>>> import hello
Hello World

You'll notice that the second time I did import hello it didn't print 'Hello World'.

[Roberthh]

Hello dhylands, I know the re.load mechanism well. I did not want to reload the module. I was just wondering that re-executing "import os" is so much slower than "import _io". It seems that micropython is looking into the filesystem for the module os first, and determines after that, whether it has to be reloaded at all.

[dhylands]

OK - In MicroPython there is support for something called soft-links for the modules.

For os, the actual module name is uos. However we have a soft-link setup so that if you import os, and os.py isn't found on the filesystem, then it will import uos instead.

So, I'm guessing that each time you import os, its going out to disk to do the check to see if os.py exists, and then falling back to uos at the end.

When you import uos, it doesn't use the soft-link. I would imagine that using import uos as os would also be as fast as import uos.

I notice that when I do import os or import sys, that on MicroPython those modules don't show up in sys.modules, but inCPython they do.

[Roberthh]

Hy dhylands, I fully agree and that is my observation, if you look at the end of my post. The average execution times of these little functions were 35 ms (import os) vs. 20 µs (import uos).
It wrote it as a hint, that one may have unexpected delays if 'import os' (or others) is used at places very often executed, like in a low level function, where one would relay on the fact, that a module is loaded only once, and therefore there is no time penalty on executing the load again.
Regards

[dhylands]

I posted an issue on github that was related to this: https://github.com/micropython/micropython/issues/1759