rshell can't mkdir or cd into directory on ESP8266

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
michaelhauser
Posts: 7
Joined: Tue Jul 03, 2018 5:54 am

rshell can't mkdir or cd into directory on ESP8266

Post by michaelhauser » Tue Jul 03, 2018 6:39 am

I'm just starting to use Micropython on ESP8266 and really enjoying it. I've been using rshell to copy files and run the REPL over USB.

I successfully connect rshell running on Mac to a D1 Mini Pro (ESP8266 by RobotDyn) using
`rshell -p /dev/cu.SLAB_USBtoUART --buffer-size 30`

I can list files using
`ls /pyboard` and copy files into that directory. I can use mkdir to create directories on the Mac filesystem but when I try `mkdir /pyboard/test` I get `Unable to create /pyboard/test`. However, I can create the directory from Micropython REPL using `uos.mkdir('test')`, and then I can copy files into that directory using rshell.

When I try to cd into a directory on the ESP, rshell crashes giving:

> cd /pyboard
Traceback (most recent call last):
File "//anaconda/bin/rshell", line 11, in <module>
load_entry_point('rshell==0.0.14', 'console_scripts', 'rshell')()
File "//anaconda/lib/python3.6/site-packages/rshell/command_line.py", line 4, in main
rshell.main.main()
File "//anaconda/lib/python3.6/site-packages/rshell/main.py", line 2712, in main
real_main()
File "//anaconda/lib/python3.6/site-packages/rshell/main.py", line 2695, in real_main
shell.cmdloop(cmd_line)
File "//anaconda/lib/python3.6/site-packages/rshell/main.py", line 1589, in cmdloop
cmd.Cmd.cmdloop(self)
File "//anaconda/lib/python3.6/cmd.py", line 138, in cmdloop
stop = self.onecmd(line)
File "//anaconda/lib/python3.6/site-packages/rshell/main.py", line 1619, in onecmd
self.onecmd_exec("".join(group))
File "//anaconda/lib/python3.6/site-packages/rshell/main.py", line 1630, in onecmd_exec
return cmd.Cmd.onecmd(self, line)
File "//anaconda/lib/python3.6/cmd.py", line 217, in onecmd
return func(arg)
File "//anaconda/lib/python3.6/site-packages/rshell/main.py", line 1914, in do_cd
auto(chdir, dirname)
File "//anaconda/lib/python3.6/site-packages/rshell/main.py", line 524, in auto
return dev.remote_eval(func, dev_filename, *args, **kwargs)
File "//anaconda/lib/python3.6/site-packages/rshell/main.py", line 1379, in remote_eval
return eval(self.remote(func, *args, **kwargs))
File "<string>", line 0

^
SyntaxError: unexpected EOF while parsing



Could the problem be related to the fact that I froze a bunch of libraries into the firmware (e.g., picoweb, uasyncio, etc.), increased irom0_0_seg in esp8266.ld to avoid overflow during building, and flashed with `esptool.py --port /dev/cu.SLAB_USBtoUART --baud 460800 write_flash -fm dio -fs 16MB 0 firmware-combined.bin 0xffc000 esp_init_data_default.bin` in order to be able to use 16MB of flash on the board?

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: rshell can't mkdir or cd into directory on ESP8266

Post by pythoncoder » Tue Jul 03, 2018 8:22 am

I appreciate this doesn't help you much but (in rshell) `mkdir /pyboard/test` works here on the reference board.
Peter Hinch
Index to my micropython libraries.

michaelhauser
Posts: 7
Joined: Tue Jul 03, 2018 5:54 am

Re: rshell can't mkdir or cd into directory on ESP8266

Post by michaelhauser » Wed Jul 04, 2018 6:57 am

Thanks for the reply pythoncoder.
I realized that this board is actually 4MB, so I reflashed using `esptool.py --port /dev/cu.SLAB_USBtoUART --baud 460800 write_flash -fm dio --flash_size=detect 0 firmware-combined.bin` but the behavior is as before. Next I'll try with just the stock Micropython distribution (no added modules).

michaelhauser
Posts: 7
Joined: Tue Jul 03, 2018 5:54 am

Re: rshell can't mkdir or cd into directory on ESP8266

Post by michaelhauser » Wed Jul 04, 2018 7:30 am

I just flashed with the prebuilt bin from the website and it seems to fix the problem. It seems like increasing irom0_0_seg in esp8266.ld may have caused the problem behavior. I guess I also need to change something in modesp.c and/or flashbdev.py. I see your post at https://github.com/micropython/micropyt ... -287737238
but it looks like modesp.c has changed since then and I don't yet understand how this stuff works.
So, can someone suggest a recipe for how to accommodate the larger firmware size due to a bunch of frozen modules (with the corresponding decrease in filesystem size)? Thanks.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: rshell can't mkdir or cd into directory on ESP8266

Post by pythoncoder » Thu Jul 05, 2018 8:26 am

From a study of the files and a quick test I think all you have to do is:
  • Edit esp8266/esp8266.ld as below. No other file needs to be changed.
  • Erase the flash.
  • Issue make clean
  • Issue make
  • Flash the result.
Line 8 of esp8266.ld should now read

Code: Select all

    irom0_0_seg :  org = 0x40209000, len = 0xa7000
This will, of course, erase any files you have on the device.

Please report your findings. If you have success I'll post a follow up on the GitHub thread.
Peter Hinch
Index to my micropython libraries.

michaelhauser
Posts: 7
Joined: Tue Jul 03, 2018 5:54 am

Re: rshell can't mkdir or cd into directory on ESP8266

Post by michaelhauser » Thu Jul 05, 2018 6:29 pm

My firmware overflowed a segment size of 0xa7000 so I changed it to 0xb7000 and the firmware built successfully. Then I erased and flashed the board. I get the same problem behavior as before (e.g., unable to create subdirectory of /pyboard).

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: rshell can't mkdir or cd into directory on ESP8266

Post by pythoncoder » Fri Jul 06, 2018 6:31 am

I can't replicate this on the reference board which I flashed with extra space as described above. I created a directory test using rshell:

Code: Select all

/home/adminpete> mkdir /pyboard/test
/home/adminpete> cd /pyboard/test
/pyboard/test> 
Can you create a directory in Python? The following works here:

Code: Select all

>>> import os
>>> os.listdir('/')
['boot.py', 'main.py', 'test']
>>> os.mkdir('/rats')
>>> os.listdir('/')
['boot.py', 'main.py', 'test', 'rats']
>>> 
Peter Hinch
Index to my micropython libraries.

michaelhauser
Posts: 7
Joined: Tue Jul 03, 2018 5:54 am

Re: rshell can't mkdir or cd into directory on ESP8266

Post by michaelhauser » Fri Jul 06, 2018 7:23 am

Here's my rshell session. Maybe one clue is that from MicroPython os.mkdir fails but uos.mkdir works.

Code: Select all

Michaels-iMac:~ michael$ rshell -p /dev/cu.SLAB_USBtoUART --buffer-size 30
Connecting to /dev/cu.SLAB_USBtoUART ...
Welcome to rshell. Use Control-D to exit.
ls /pyboard
boot.py
/Users/michael> mkdir /pyboard/test
Unable to create /pyboard/test
/Users/michael> ls /pyboard
boot.py
/Users/michael> repl
Entering REPL. Use Control-X to exit.
repl_serial_to_stdout dev = <rshell.main.DeviceSerial object at 0x109f85ba8>
>
MicroPython v1.9.4-227-gab02abe-dirty on 2018-07-05; ESP module with ESP8266
Type "help()" for more information.
>>>
>>> import os
>>> os.listdir('/')
['boot.py']
>>> os.mkdir('/rats')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "os/__init__.py", line 80, in mkdir
NameError: name 'mkdir_' is not defined
>>>
>>> import uos
>>> uos.mkdir('/rats')
>>> os.listdir('/')
['boot.py', 'rats']
>>>

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: rshell can't mkdir or cd into directory on ESP8266

Post by Roberthh » Fri Jul 06, 2018 8:12 am

Did you by change copy a module like os.py from micropython_lib onto your device? That is not needed. uos and it's alias os is a built-in module.

michaelhauser
Posts: 7
Joined: Tue Jul 03, 2018 5:54 am

Re: rshell can't mkdir or cd into directory on ESP8266

Post by michaelhauser » Fri Jul 06, 2018 9:07 am

Roberthh wrote:
Fri Jul 06, 2018 8:12 am
Did you by change copy a module like os.py from micropython_lib onto your device? That is not needed. uos and it's alias os is a built-in module.
An "os" directory with an __init.py__ file was apparently created in the modules directory by one of the pip install commands I ran to install the modules. I removed that directory and rebuilt and reflashed the firmware and it fixed the problem! Now I can get back to my IoT endeavors. Thanks!

Post Reply