How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
Post Reply
sunbear
Posts: 9
Joined: Thu Jan 31, 2019 5:17 pm

How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

Post by sunbear » Mon Feb 04, 2019 9:56 am

How can the above be achieved from a Linux system? Purpose: copy python modules and data files into and out from the flash? Thanks.

kevinkk525
Posts: 409
Joined: Sat Feb 03, 2018 7:02 pm

Re: How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

Post by kevinkk525 » Mon Feb 04, 2019 10:12 am

Use rshell

User avatar
Roberthh
Posts: 1455
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

Post by Roberthh » Mon Feb 04, 2019 1:11 pm

Either dhylands/rshell, or Adafruit ampy or one of the other tools, like mpfshell. Alternatively, I made myself a small ftp server script based on the work of @chrisgp, which I start when needed. And then I can use FilZilla top copy files in & out. https://github.com/robert-hh/FTP-Server ... -and-ESP32
There is a "foreground" version, ftp.py, and a background version (uftpd.py).
See also https://github.com/pfalcon/awesome-micropython

sunbear
Posts: 9
Joined: Thu Jan 31, 2019 5:17 pm

Re: How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

Post by sunbear » Mon Feb 04, 2019 5:04 pm

Thanks. I was experimenting using `esptool.py` to copy a .py file into flash.
For example, after installing micropython with :

Code: Select all

esptool.py --chip esp32 --port /dev/ttyUSB0 write_flash -z --erase-all --compress 0x1000 firmwares/esp32-20190125-v1.10.bin
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting........__
Chip is ESP32D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Erasing flash (this may take a while)...
Chip erase completed successfully in 10.0s
Compressed 1087456 bytes to 687409...
Wrote 1087456 bytes (687409 compressed) at 0x00001000 in 60.9 seconds (effective 142.8 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
Following that, I ran another write_flash command. This time to write "login_network.py" into flash. It is written at a location after file "esp32-20190125-v1.10.bin":

Code: Select all

esptool.py --chip esp32 --port /dev/ttyUSB0 write_flash 0x110000 login_network.py
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting........___
Chip is ESP32D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 1020 bytes to 408...
Wrote 1020 bytes (408 compressed) at 0x00110000 in 0.0 seconds (effective 199.5 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
The writes were successful. However, when in the REPL, I could only see boot.py:

Code: Select all

>>> import os
>>> os.listdir()
['boot.py']
Why is "login_network.py" not visible in REPL?

User avatar
Roberthh
Posts: 1455
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

Post by Roberthh » Mon Feb 04, 2019 5:56 pm

Because the device has a FAT file system. When you write data to place where the file system starts, you actually destroy it, and on the next boot, it will be reinitialized with a default boot.py file.

sunbear
Posts: 9
Joined: Thu Jan 31, 2019 5:17 pm

Re: How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

Post by sunbear » Wed Feb 06, 2019 2:05 am

@Roberthh, can you elaborate on what you mean? E.g. what do mean with
write data to place where the file system starts
? In my example, I had thought the FAT32 filesystem started from 0x0 to end of the flash, consisted of one partition, with micropython occupying 0x1000 to say 0x109800. Thereafter, all the space available on the flash is available for a user to write to.

I just learnt from https://docs.espressif.com/projects/esp ... ables.html that a esp32 flash can have multiple partitions. So how does micropython setup the partitions of the SSD? How many partition does it create? At what flash address can a user write their python files to? When in REPL, when a user uses the "open" method to write a file, how does micropython determine what flash address to write the file to? Thank you.

loboris
Posts: 317
Joined: Fri Oct 02, 2015 6:19 pm

Re: How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

Post by loboris » Wed Feb 06, 2019 9:24 am

Fat file system is a complex structure.

Christian Walther
Posts: 47
Joined: Fri Aug 19, 2016 11:55 am

Re: How to copy files (e.g. .py, .txt, .dat) to and from the esp32 flash?

Post by Christian Walther » Wed Feb 06, 2019 11:52 am

You are confused about how the flash is organized. The MicroPython firmware is not a file, it lives outside of the FAT filesystem in raw flash. The FAT filesystem is located in a different part of flash. Python files that you want to run need to live inside the filesystem, and you don't install them by just writing to a particular flash address (unless afterwards you very carefully repair all the filesystem structures you have destroyed by doing that), just like you wouldn’t save a file on your PC by writing to a particular raw block of the hard drive, but by using filesystem calls like open().

esptool.py writes to raw flash, you use it to install firmware. rshell, ampy and all the other mentioned tools (I would add WebREPL to the list) work inside the filesystem, you use them to install Python files.

Post Reply