Put all modules into a single file

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
bokolob
Posts: 11
Joined: Tue Mar 09, 2021 7:03 pm

Put all modules into a single file

Post by bokolob » Thu May 06, 2021 12:56 pm

Hello,I have a project contains several files with modules and classes, it's ok for development, but a bit inconvenient for uploading and getting updates to device.

Is there a "standard" or at least any way to pack all the files into one archive and run it? I know about solution with zip and modulefinder, but it's not implemented (afaik) on Python.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Put all modules into a single file

Post by pythoncoder » Fri May 07, 2021 12:34 pm

I don't know an automated way. I can see two options. Use a text editor to combine the modules into a single Python file. This involves some tweaking, removing redundant import lines and possibly adjusting names of objects.

The other option is to leave the project as it is and change the deployment method. The rshell tool has an rsync command which deploys complex projects very effectively. That is my approach.
Peter Hinch
Index to my micropython libraries.

bokolob
Posts: 11
Joined: Tue Mar 09, 2021 7:03 pm

Re: Put all modules into a single file

Post by bokolob » Sat May 08, 2021 2:52 pm

Thanks. Rshell looks great. But I need solution for getting automatic updates.. I want to download new versions periodically and replace old files. It seems to be easier with a single file.

wangshujun@tom.com
Posts: 61
Joined: Fri Feb 15, 2019 9:22 am

Re: Put all modules into a single file

Post by wangshujun@tom.com » Sun May 09, 2021 1:16 pm

I currently use esp32, and I think it's more convenient to upload files by FTP

SpotlightKid
Posts: 463
Joined: Wed Apr 08, 2015 5:19 am

Re: Put all modules into a single file

Post by SpotlightKid » Mon May 10, 2021 12:02 pm

On the ESP32 you could use the OTA features to send a whole filesystem as partition data and write that to the flash. This technique is used for example here : https://github.com/tve/mqboard/tree/master/mqrepl

SpotlightKid
Posts: 463
Joined: Wed Apr 08, 2015 5:19 am

Re: Put all modules into a single file

Post by SpotlightKid » Mon May 10, 2021 2:25 pm

Also, what you can do is put all the files you want to upload into an uncompressed TAR archive and then use the utarfile module from micropython-lib and the following code to extract it on your MicroPython board:

Code: Select all

import os
from utarfile import TarFile

def exists(path):
    try:
        _ = os.stat(path)
    except:
        return False
    else:
        return True

def untar(filename, overwrite=False, verbose=False, chunksize=4096):
    with open(filename) as tar:
        for info in TarFile(fileobj=tar):
            if info.type == "dir":
                if verbose:
                    print("D %s" % info.name)

                name = info.name.rstrip("/")
                if not exists(name):
                    os.mkdir(name)
            elif info.type == "file":
                if verbose:
                    print("F %s" % info.name)

                if overwrite or not exists(info.name):
                    with open(info.name, "wb") as fp:
                        while True:
                            chunk = info.subf.read(chunksize)
                            if not chunk:
                               break
                            fp.write(chunk)
            elif verbose:
                print("? %s" % info.name)
So the procedure would go something like this:

One time only:

Code: Select all

wget https://raw.githubusercontent.com/micropython/micropython-lib/master/utarfile/utarfile.py
mpy-cross utarfile.py
wget https://raw.githubusercontent.com/SpotlightKid/micropython-stm-lib/master/untar/untar.py
mpy-cross untar.py
rshell cp utarfile.mpy untar.mpy /pyboard/flash
And each time you want to upload a new archive (assuming your code is present in a local subdirectory named mypackage):

Code: Select all

tar -cvf mypackage.tar mypackage/
rshell cp mypackage.tar /pyboard/flash
pyboard -c 'import untar; untar.untar("mypackage.tar", overwrite=True, verbose=True)'
(You may need to add command line options to the rshell and pyboard invocations to set the serial port and baud rate depending on your MicroPython board type.)

Post Reply