upip micropython-os do no work properly

[pinazawa]

Hello,

Im working on a simple OTA update for ESP32, and realized that uos , that comes with esp32 official micropython (i'm using esp32-20190611-v1.11-44-g8b18cfede.bin) , is rather limited.

I searched a lot, and found out that there is a way to put a way bigger and better os library using upip. So I installed it in my esp32 using the normal commands:

<Connect to wifi>
import upip
upip.install('micropython-os')

BUT many things simply do not work in the library, and I keep getting some strange errors, such as this one with ampy:

ampy --port /dev/ttyUSB0 rmdir newapp/
ampy.pyboard.PyboardError: ('exception', b'', b'Traceback (most recent call last):\r\n File "<stdin>", line 17, in <module>\r\n File "<stdin>", line 7, in rmdir\r\n File "/lib/os/__init__.py", line 195, in chdir\r\nNameError: name \'chdir_\' isn\'t defined\r\n')


ampy --port /dev/ttyUSB0 put app

ampy.pyboard.PyboardError: ('exception', b'', b'Traceback (most recent call last):\r\n File "<stdin>", line 6, in <module>\r\n File "/lib/os/__init__.py", line 80, in mkdir\r\nNameError: name \'mkdir_\' isn\'t defined\r\n')


Isn't upip.install already enough for getting this packages?

Second question: can uos and os co-habit the same esp32? or I should take one out to use the other?

Thanks !

[jimmo]

hi,

It's really not clear without looking at the code, but the `os` module in micropython-lib is only designed for the unix port. Specifically it relies on ffi to call the libc implementations of these functions (and you have neither of these things on your ESP32).

So what's going on here is that ampy is trying to use the `os` module on your board, and it's completely non-functional, even for things that work with the built-in module.

What did you need from the `os` module that isn't available in the built-in module?

[pinazawa]

Hey!

Thanks for the response. Sorry for the delay...

In my OTA design, I was trying to remove a hole tree of directories.

I used the shutil lib method called rmtree, but it depends on os.walk (maybe another unix port?)

Code: Select all

>>> shutil.rmtree('new_app')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/shutil.py", line 6, in rmtree
AttributeError: 'module' object has no attribute 'walk

So, I was looking for os.walk or something similar... I saw that uos.ilistdir creates a generator with all files in folder as well , but appearently is not the same format...

(By the way, I tried removing the directory tree applying uos.remove, however i get the following error )

Code: Select all

>>> uos.rmdir('new_app')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 13] EACCES

what do you do when you want to remove a folder with something inside, in ESP32 micropython?

thanks!

[pythoncoder]

There are various ways to do this. The classic approach is a recursive function. This will remove the contents of a specified directory:

Code: Select all

import uos as os
def rmtree(d):
    for f in os.listdir(d):
        p = os.sep.join((d, f))  # Full path
        t = os.stat(p)[0]  # Item type
        if t == 32768:
            os.remove(p)  # File
        elif t == 16384:  # Directory
            c = os.listdir(p)
            if c:  # Non-empty directory
                rmtree(p)
            os.rmdir(p)
        else:
            print('Unknown type. File {:s} type {:d}'.format(f, t))

You can then remove the now empty directory (if that's your intention) with os.rmdir()

Code: Select all

>>> rmtree('/pyboard/my_directory')
>>> os.rmdir('/pyboard/my_directory')

[Roberthh]

@pythoncoder. This is a good script example. The only obstacle: os.sep does not exits in ESP32 micropython. Using "/" instead is suitable (or defining a constant)

[stijn]

the `os` module in micropython-lib is only designed for the unix port. Specifically it relies on ffi to call the libc implementations of these functions (and you have neither of these things on your ESP32).

That's not entirely correct. The ffilib.py file is coded so that when ffi isn't available it just ignores it, hereby obviously rendering all functions relying on it unavailable, but it can just be imported without errors. So functions like os.walk which do not need ffi still just work.

[pythoncoder]

@pythoncoder. This is a good script example. The only obstacle: os.sep does not exits in ESP32 micropython. Using "/" instead is suitable (or defining a constant)

Thanks for pointing that out. I keep getting caught out by these apparently arbitrary differences between ports. The one which baffles me is the fact that some functionality is omitted from the Unix port where RAM is likely to be huge.

So, for @pinazawa, we have

Code: Select all

import uos as os
def rmtree(d):
    for f in os.listdir(d):
        p = '/'.join((d, f))  # Full path
        t = os.stat(p)[0]  # Item type
        if t == 32768:
            os.remove(p)  # File
        elif t == 16384:  # Directory
            c = os.listdir(p)
            if c:  # Non-empty directory
                rmtree(p)
            os.rmdir(p)
        else:
            print('Unknown type. File {:s} type {:d}'.format(f, t))

[Roberthh]

@pythoncoder: For ESP32 or Micropython, i modified your script using ilistdir, shortening it a little bit. It also removes the initial
directory.

Code: Select all

import uos as os
sep = "/"

def rmtree(d):
    for f in os.ilistdir(d):
        p = sep.join((d, f[0]))  # Full path
        if f[1] == 32768:
            os.remove(p)  # File
        elif f[1] == 16384:  # Directory
            rmtree(p)
        else:
            print('Unknown type. File {:s} type {:d}'.format(f[0], f[1]))
    os.rmdir(d)

[pinazawa]

@pythoncoder @roberthh and everybody

Thanks a lot for the help. You are awesome haha

Since ampy also implements this, I found their rmdir in the source code and did some tweeks to adapt it. it also relies on recursive calls. I was a bit insecure whether or not i should use recursivity in my esp32 due to its low ram... But seeing your code, i think it the preferable way.

This code solved my issue, and I do not need to use micropython-os anymore! =)

Code: Select all

def rmdir(directory):
    os.chdir(directory)
    for f in os.listdir():
        try:
            os.remove(f)
        except OSError:
            pass
    for f in os.listdir():
        rmdir(f)
    os.chdir('..')
    os.rmdir(directory)

[pythoncoder]

@Roberthh Neat