OSError(84,)

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
edf012
Posts: 14
Joined: Sun May 09, 2021 8:51 pm

Re: OSError(84,)

Post by edf012 » Mon May 16, 2022 5:43 pm

It seems I went down the wrong rabbit hole. It was not the writeblocks that failed. It was the file read. I "fixed it" as shown below.

Code: Select all

        chunk = ()
        import esp32
        part = esp32.Partition("nvs")
        block_num = int(part.info()[2] / 4096)
        try:
            with open('ubinvs.bin', 'rb') as f:
                log.info('opened ubinvs.bin..iterating through the file')
                while True:
                    try:
                        while chunk := f.read(4096):
                            position = f.tell()
                            self.wdt.feed()
                            gc.collect()
                            log.info('block_num is {}, chunk size is {}, position is {}'.format(block_num, len(chunk), position))
                            try:
                                esp32.Partition.writeblocks(block_num, chunk)
                            except:
                                pass
                            block_num = block_num + 1
                        os.rename('ubinvs.bin','ubinvs.save')
                        log.info('renamed ubinvs file')
                        break
                    except Exception as e:
                        log.info('f.read error {}' + repr(e))
                        log.info('skipping block_num {}'.format(block_num))
                        block_num = block_num + 1
                        position = position + 4096
                        f.seek(position)
                        pass
        except:
            pass
There is a block that gets skipped because of the EILSEQ error. But after that, the rest of the binary file is read and written to the NVS partition. I will keep investigating until I find out where the EILSEQ error is being raised - unless someone out there finds it first :)

marcidy
Posts: 133
Joined: Sat Dec 12, 2020 11:07 pm

Re: OSError(84,)

Post by marcidy » Mon May 16, 2022 8:01 pm

well, that's is some progress at least

Code: Select all

                            try:
                                esp32.Partition.writeblocks(block_num, chunk)
                            except:
                                pass
Are you sure this isn't swallowing a problem? i am relatively lenient on try/except, so i'm not arguing from a purist point of view on bare excepts. i would definitely want to know if my data was written correctly, especially while debugging, but if you are confident based on your testing i can accept it. based on your previous post, it seemed pretty clear writeblocks was failing. However, the information that it fails both ways is also interesting if true. Can you please clarify if writeblocks definitely succeeds?

If you can identify the block that's causing the problem, you should be able to reproduce the illegal sequence error with just that block in the file, and not the whole file. From there, you should be able to modify byte by byte and find the problematic byte sequence.

however im not sure what would be illegal about it, as im not clear what is trying to interpret the byte sequence as legal or illegal at this stage.
I'm not even clear at what level the sequence is invalid. Bytes? Words? Blocks? unclear right now.

edit:
also, for the sake of argument, try a non-nvs partition like fat. nvs has special properties (https://docs.espressif.com/projects/esp ... flash.html), and it's unclear if this is causing issues.

There is an nvs partition generator, are you using this?
https://docs.espressif.com/projects/esp ... n_gen.html

edf012
Posts: 14
Joined: Sun May 09, 2021 8:51 pm

Re: OSError(84,)

Post by edf012 » Mon May 16, 2022 8:55 pm

Hi Marcidy,

Yes writeblocks is working (as was spi_flash_write and esp_partition_write I suspect) and no, I am not satisfied that 1 block out of 768 fails. It is the file read that is giving me the OSError(84,).

The partition table, the ubinvs.bin file and the NVS keys file, were created with the appropriate Espressif tools, and, if you recall, I initially used esptool.py to flash the generated files via serial interface and the device ran with no errors.

As far as the illegal sequence, I'm with you. I know of no sequence of bytes that should cause a file read to fail. When I figure out what is causing the failure, I'll post it here.

marcidy
Posts: 133
Joined: Sat Dec 12, 2020 11:07 pm

Re: OSError(84,)

Post by marcidy » Mon May 16, 2022 11:45 pm

OK, thanks for clarifying.

You can also use machine.mem32 (or the 8 bit, or 16 bit variants) to directly read bytes / half-words / words from memory directly.

The syntax is

Code: Select all

>>> print("{:04X}".format(machine.mem32[0x3ff54900]))
so you could loop over the region and read the bytes also. not sure if that could help, but it's another tool in the toolbox.

note that you can cause a core panic by triggering a prohibited load if you target the wrong regions.

edf012
Posts: 14
Joined: Sun May 09, 2021 8:51 pm

Re: OSError(84,)

Post by edf012 » Tue May 17, 2022 10:08 pm

Hi Marcidy, thanks for the machine.mem32 tip. I think I'll try that because I really don't feel like debugging Espressif's file system code!

marcidy
Posts: 133
Joined: Sat Dec 12, 2020 11:07 pm

Re: OSError(84,)

Post by marcidy » Tue May 17, 2022 10:14 pm

good luck, i may have given you a new headache rather than solved your current one, please test it carefully.

i consider this extremely low level access, and a foot-gun.

At the same time, if all you need is a bunch of bytes from memory, this can do it.

i intended it more of a debugging tool, but a tool is a tool.

edf012
Posts: 14
Joined: Sun May 09, 2021 8:51 pm

Re: OSError(84,)

Post by edf012 » Thu May 26, 2022 3:46 pm

update: There is a bug emanating from the MP HAL layer, although I do not think it is the root cause. Unfortunately I do not have time to debug this issue further. It seems there is a ring buffer between espressif IDF and MP that collects the file data. After some time (or file size), the get pointer catches up to the put pointer, which throws an error.
I solved my issue by moving all of the functionality to C, that is, from micropython I call a C function to open the binary file, read it block by block and call spi_flash_write to flash the NVS partition block by block.

Post Reply