Invalid boot sector?

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
Post Reply
User avatar
fdushin
Posts: 32
Joined: Thu Jul 21, 2016 5:38 pm

Invalid boot sector?

Post by fdushin » Fri Aug 19, 2016 2:11 pm

I am starting this thread to help diagnose what I think may be the root cause of several symptoms already described on this forum, specifically, issues related to the Micropython file system, such as ENODEV issues, "FAT filesystem appears to be corrupted" issues, and errors reading or writing to the file system after flashing the device.

I have been able to reproduce these problems fairly reliably on my ESP-12 boards, but not always on my ESP-01 (which has a nice adapter with a capacitor close to the chip), so I understand when leads on this forum have pointed to issues with power supply. However, I have been able to reproduce the issue on different chips and board configuration, using different machines, so I wanted to investigate what might be happening on the software side, if not to find an error in the software, but maybe to point to something I might be doing wrong. I am including my investigation here.

This has been happening on my boards since I started playing with this (which was around or near after the 1.8.2 release -- I'm a noob). Basically, I could always flash the device successfully, but the file system would end up being corrupt. In some cases I could do an os.listdir(), and it would return empty ("[]"), but in other cases I would get a list of hex characters (usually zeros). In any case, trying to write a file would fail, usually crashing the device.

With 1.8.3, and the addition of the esp.check_fw() command, I can verify that at least the firmware image is correct:

Code: Select all

>>> import esp
>>> esp.check_fw()
size: 558748
md5: f8d6b701f96d5bf51b5f0709b4add1ea
True
But the file system still seems to be damaged.

Code: Select all

>>> import os                                                                                                                   
>>> os.listdir()                                                                                                                
['\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00', '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00', '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00', '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\]
So I looked into the ini setup code, and ran inisetup.check_bootsec() which I think might be run at initial startup (I am still not fully familiar with the micro python launch sequence). And sure enough, the file system is reported to be corrupted:

Code: Select all

>>> import inisetup
>>> inisetup.check_bootsec()
FAT filesystem appears to be corrupted. If you had important data there, you
may want to make a flash snapshot to try to recover it. Otherwise, perform
factory reprogramming of MicroPython firmware (completely erase flash, followed
by firmware programming).
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "inisetup.py", line 21, in check_bootsec
  File "inisetup.py", line 32, in fs_corrupted
KeyboardInterrupt: 
But why? Looking at the code [1], the check_bootsec function is walking over the initial 4096 bytes of the boot sector (FlashBdev.START_SEC, or 0x90000, as defined at [2]), and making sure that all bytes are 0xFF.

If I do that manually, however, I see that the first two bytes are 0x00:

Code: Select all

>>> from flashbdev import bdev
>>> buf = bytearray(bdev.SEC_SIZE)
>>> bdev.readblocks(0, buf)
>>> buf
bytearray(b'\x00\x00\x90MSDOS5.0\x00\x10\x01\x01\x00\x01\x00\x02k\x00\xf0\x01\x00?\x00\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x00)\x0f\x00!(NO NAME    FAT     \x00\x00\x00\x00\x00\x0)
I am not sure why this is the case. Was this file system already initialized? I always do an erase_flash (using esptool.py) to the device before flashing. And in fact in this case I created an image of all 0xFF and write that to the flash beforehand!

In any event, if I manually proceed with the inisetup.setup function:

Code: Select all

>>> import uos
>>> uos.VfsFat.mkfs(bdev)
>>> vfs = uos.VfsFat(bdev, "")
I then get a file system I can write to.

I will continue to dig into how I ended up with an initial file system that is not all 0xFF, as I think is expected. I am certainly open to the issue still being due to power supply issues, but I am less certain that this is the root cause. I do find it interesting that this occurs with:

* 2 different ESP-12 devices, using a breadboard and CP2102 UART adapter (no caps, though the power supply has some), both USB and wall wart powered.
* An ESP-12 with a resoldered 1MB flash from an ESP-01
* Wemos D1 Mini

and does not happen with:

* ESP-01 on breadboard with ESP-01 breadboard adapter (and capacitor on the adapter)

Any debugging hints appreciated!

[1] https://github.com/micropython/micropyt ... tup.py#L11
[2] https://github.com/micropython/micropyt ... bdev.py#L6

User avatar
pythoncoder
Posts: 3599
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Invalid boot sector?

Post by pythoncoder » Fri Aug 19, 2016 3:54 pm

The odd thing about this is that some people seem plagued with these problems while others don't. Because I'm using frozen bytecode I've been flashing firmware several times a day: entirely without issue. This on a Feather Huzzah, a Wemos D1 mini and (on occasion) on a Huzzah board (the UART rather than USB variant).

I still suspect an electrical issue, but I guess the jury's out until someone with problems has the means of thoroughly checking their electrical integrity.
Peter Hinch

User avatar
deshipu
Posts: 1348
Joined: Thu May 28, 2015 5:54 pm

Re: Invalid boot sector?

Post by deshipu » Mon Aug 22, 2016 11:19 pm

Hmm, what version of the esptool.py are you using, and where did you get it from? Maybe that's the problem? Also, what operating system? Which drivers for the CP2102? Did you try this on a different computer? With a different usb2ttl?

jms
Posts: 108
Joined: Thu May 05, 2016 8:29 pm
Contact:

Power supply decoupling Re: Invalid boot sector?

Post by jms » Thu Aug 25, 2016 3:06 pm

Not saying it's the problem here but I want to scribble a quick note on power supply decoupling.

It's not about what current the power supply can deliver (which will almost always be enough as the ESP takes 300mA max) or its own capacitors.

It is about decoupling to deal with demand changes roughly of the order of a microsecond and these really need to be tens of uF, on your breadboard and of reasonable quality. Why ? Because the module itself will have its own upto 1uF capacitors and they will cope with faster stuff but there's a middle ground where neither the on-module capacitors or the (relatively) distant power supply will cope with transients.

(Cope meaning not cause a supply drop of hundreds of millivolts with a step change in load of hundreds of milliamps and distant referring to cable resistance and inductance rather than length.)

User avatar
pythoncoder
Posts: 3599
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Invalid boot sector?

Post by pythoncoder » Fri Aug 26, 2016 5:57 am

There's also the issue of USB cable length. Short is good, to avoid excessive voltage drops!
Peter Hinch

User avatar
fdushin
Posts: 32
Joined: Thu Jul 21, 2016 5:38 pm

Re: Invalid boot sector?

Post by fdushin » Fri Aug 26, 2016 12:31 pm

What I find telling is that flashing my ESP-01 never seems result in a bad file system, and it always seems to happen on an ESP-12 (using an ESP-12 on a breadboard of my own, as well as a Wemos D1 mini).

However, if I read the full contents of my ESP-01 (all 1MB) and then write those contents back to my ESP-12, my file system is fully in tact and I can use the ESP-12 normally. I can list the directory, open files, read from them, write to them, etc. and for now this is a kind of workaround.

So my personal suspicion is around the initial setup of the file system on these chips (or again, possibly related to my hardware configuration, which as a software engineer, and having no formal training in hardware, I cannot discount!).

The call order, and I understand it, is something like:

* _boot.py imports bdev from Flashbdev, which has some interesting code at https://github.com/micropython/micropyt ... dev.py#L50
* _boot.py tries to get an reference to the vFAT file system. If that fails, it tries to set up the file system. All of this is at
https://github.com/micropython/micropyt ... boot.py#L6
* setup will try to make the fat file system at https://github.com/micropython/micropyt ... tup.py#L38

I have found that after flashing an ESP-12, the initialization sequence will hang (or sometimes take a very long time) after printing "Performing initial setup", but I have not verified which function we are hung in.

To answer deshipu, I am running MacOS X 10.11.5, using SILibs signed driver:

Code: Select all

shell$ kextstat | grep -i silabs
358 0 0xffffff7f83374000 0x6000 0x6000 com.silabs.driver.CP210xVCPDriver (4.10.11) CF83A369-EA70-3E24-B8FE-7351DCF03430
Incidentally, I have been blogging some details of my setup at [1,2]. I am using esptool.py cloned from github and built/installed using tag v1.1.

Many thanks for your advice, and especially for micro python, generally.

[1] http://blog.dushin.net/2016/07/getting- ... n-the-mac/
[2] http://blog.dushin.net/2016/07/running- ... e-esp8266/

User avatar
deshipu
Posts: 1348
Joined: Thu May 28, 2015 5:54 pm

Re: Invalid boot sector?

Post by deshipu » Fri Aug 26, 2016 3:14 pm

Hmm, the fact that an image copied from esp-01 "works" doesn't mean it's actually correct -- as you noticed, if a filesystem is already created, it doesn't do a check for it. It can be still subtly corrupted.

However, from what you are saying, it seems that there is some bad interaction between the mkfs call and your module, which can be either a bug or a problem with your setup somewhere. I wonder if you could add more messages to your _boot.py to pin-point the place where it hangs better.

Finally, if you reset the esp8266 while it's formatting the filesystem, it will most likely get corrupted. I just realized that this can happen when you reset your board after flashing it -- if you reset it twice or more in a row... Perhaps the solution is as simple as adding debouncing to the reset button?

jms
Posts: 108
Joined: Thu May 05, 2016 8:29 pm
Contact:

Re: Invalid boot sector?

Post by jms » Fri Aug 26, 2016 3:19 pm

However, if I read the full contents of my ESP-01 (all 1MB) and then write those contents back to my ESP-12, my file system is fully in tact and I can use the ESP-12 normally.
Then you should read the contents of your ESP-12 as well and compare the two. That should reveal whether it's a small number of dodgy blocks or what.

Also read back both devices several times and compare in case there are bits close to the edge.

Personally I gave up working on the Mac and use the windows box and NodeMCU flashing tool exclusively now. No trouble at all.

Jon

JumpZero
Posts: 27
Joined: Mon Oct 30, 2017 5:54 am
Location: Arcachon - France

Re: Invalid boot sector?

Post by JumpZero » Thu Mar 22, 2018 5:33 pm

Hello,
I dig out this old post, because I had the exact same problem, and searching the forum brought me here.
I normally use NodeMcu devkit because they are convenient and I have no problem flashing the MicroPython firmware.
But recently I use plain ESP12F exactly these ones
https://www.aliexpress.com/item/1PCS-Es ... 03423.html
I wired all the needed circuits around, flashed MicroPython firmware but almost every time ended with a corrupted filesystem.
As noticed by OP esp.check_fw() command, reports firmware image is correct.
I can connect to REPL but cannot use the filesystem.
But if I flash Lua NodeMCU firmware on the same ESP with the same circuit on the breadboard with the same esptool command... success!
I did approx. 10 times and everytime same result: success with Lua NodeMCU firmware, corrupted filesystem with MicroPython.


However I solved it, I found that the USB to serial converter was the culprit. I changed this one:
https://www.aliexpress.com/item/CP2102- ... 44759.html
to this one
https://www.adafruit.com/product/70
And no more problem, again same ESP12-F on same breadboard with same circuit around, only changed the USB to serial converter.

This isn't obvious to understand why... but that's it.
Maybe this can help others.

--
Jmp0

Post Reply