[SOLVED] Problem running latest GENERIC_1M firmware on Sonoff

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
User avatar
pythoncoder
Posts: 4268
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Problem running latest GENERIC_1M firmware on Sonoff

Post by pythoncoder » Wed May 06, 2020 4:31 am

The beauty of open source is that you can answer these questions by studying the source ;)

If you look at the code in flashbdev.py I think you'll find the answer. The chip is reporting a flash size too small to support a filesystem so, instead of creating a filesystem it sets bdev to None. This causes _boot.py to skip mounting the nonexistent filesystem and to skip initialising it.

The question is what does esp.flash_size() return on the ESP8255? There may be a bug here, and if so, it needs to be reported.
Peter Hinch

Ev Quink
Posts: 9
Joined: Thu Apr 30, 2020 9:32 pm

Re: Problem running latest GENERIC_1M firmware on Sonoff

Post by Ev Quink » Wed May 06, 2020 10:36 am

Code: Select all

import uos
from flashbdev import bdev
print(esp.flash_size())
returns 1048576 on this board

So it seems like flashbdev.py is ok.

This only works if I also first do:

Code: Select all

import esp
Otherwise I get this:

Code: Select all

NameError: name 'esp' isn't defined

User avatar
pythoncoder
Posts: 4268
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Problem running latest GENERIC_1M firmware on Sonoff

Post by pythoncoder » Thu May 07, 2020 5:06 am

I'd expect you to have to import esp, and flashbdev.py does this.

I can't explain why bdev is being set to None. Very odd.
Peter Hinch

Ev Quink
Posts: 9
Joined: Thu Apr 30, 2020 9:32 pm

Re: Problem running latest GENERIC_1M firmware on Sonoff

Post by Ev Quink » Fri May 08, 2020 1:30 am

I found another clue that may help.

Using my build from the latest repo (unmodified - the one that does not work) I see the following message in the serial terminal:

Code: Select all

Traceback (most recent call last):
  File "_boot.py", line 13, in <module>
  File "inisetup.py", line 45, in setup
  File "inisetup.py", line 16, in check_bootsec
  File "flashbdev.py", line 13, in readblocks
OSError: [Errno 5] EIO
I think I didn't see this initiallly because I was interacting with the board through a Jupyter Notebook.

So maybe the inisetup routine _is_ getting called after all?

The offending line in flashbdev.py is this:

Code: Select all

esp.flash_read((n + self.start_sec) * self.SEC_SIZE + off, buf)
So I executed this from my terminal:

Code: Select all

from flashbdev import bdev
buff=bytearray(bdev.SEC_SIZE)
esp.flash_read(n,buff)
And did not get any errors for values of n < 1044480 but do get the EIO error for larger values

I am not sure what start_sec gets initialized to with FlashBdev but could this be related to the problem? I am also not sure if this is consistent with the behavior of the v1.12 _boot.py working while the latest _boot.py does not.

In a web search I found some indications that EIO errors could be related to timing which I am guessing could explain why it works on the Wemos on not on the Sonoff.

Thanks again for the help with this.

Evan

Ev Quink
Posts: 9
Joined: Thu Apr 30, 2020 9:32 pm

Re: Problem running latest GENERIC_1M firmware on Sonoff

Post by Ev Quink » Fri May 08, 2020 11:53 pm

The root cause of this problem was with the way I was building the firmware - not in the firmware itself.

When I built the firmware I used

make board=GENERIC_1M

rather than

make BOARD=GENERIC_1M :oops:

Unfortunately this defaults to building the GENERIC firmware rather than the GENERIC_1M firmware and does not throw an error. This is why the firmware worked on the Wemos and not on the Sonoff. I compounded the problem by copying the wrong binary file when moving it between machines which led me down a wrong path.

Embarrassing but ultimately solved.

kevinkk525
Posts: 739
Joined: Sat Feb 03, 2018 7:02 pm

Re: Problem running latest GENERIC_1M firmware on Sonoff

Post by kevinkk525 » Sat May 09, 2020 6:53 am

Good to know! Thanks for the feedback. Glad it works now.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

mcL
Posts: 9
Joined: Wed Dec 25, 2019 1:36 am

Re: [SOLVED] Problem running latest GENERIC_1M firmware on Sonoff

Post by mcL » Tue Jun 16, 2020 12:37 am

Thank you for this discussion! I had a similar issue where I was getting memory errors and trying to learn how to freeze modules (unsuccessfully thus far). I was able to build ok via jimmo's docker instructions but had no apparent filesystem until I also included the BOARDS=GENERIC_1M to make. Now I have what seems a proper build.

(Maybe I should make another thread?)
I'm able to flash my Sonoff R3 and copy my .py files over no problem. It seems to have a lot more free memory now (was on v1.12 release), but I still get a memory error during the initial module imports (where I didn't with the release code). There seems to be some issue with umqtt.simple (or more likely my understand). I thought it was available without using upip to install it but my import couldn't find it so I used upip to install it. Now the import dies on 'from umqtt.simple import MQTTClient'. upip created a /lib folder during the install which I'm pretty sure I didn't have with 1.12-release.

Isn't umqtt included in the build? I see it in the github micropython-lib.
Also, how do I determine if littlefs succeeded? I thought it doesn't have file timestamps, but 'ls -l' shows dates ('course they're all Dec 31, 1999 so maybe that's the indication).

Thanks for taking a look-see!

Code: Select all

MPY: soft reboot
id: sonoffr3
--- reading config-C0.json
--- Loaded file config-C0.json
Traceback (most recent call last):
  File "main.py", line 5, in <module>
  File "atticfan01.py", line 17, in <module>
  File "mqttsetup.py", line 2, in <module>
MemoryError: memory allocation failed, allocating 314 bytes
MicroPython v1.12-537-gecd782631-dirty on 2020-06-15; ESP module (1M) with ESP8266

>>> gc.mem_free()
23216

>>> micropython.mem_info(1)
stack: 2128 out of 8192
GC: total: 37952, used: 14960, free: 22992
 No. of 1-blocks: 293, 2-blocks: 53, max blk sz: 41, max free sz: 297
GC memory layout; from 3ffeeec0:
00000: MDhhhhSMBhDhBhDBSBMB=MhhB=h===h===h==h=================h=======h
00400: =======h=DhDSSh=========h==h=====Dhh=h=======MBh=hB..h=h=h.h=...
00800: ..h=.......hD..h=hh=.Bh.....B.M.MD.S..h===h===h=======.ShShh=h=D
00c00: LhDSh=MD.BBBBBBh=h===h===DSh=DhSShhShSh.h=======h===h=DBSBBBBB=B
01000: Bh========BBBShBLhhSShShh===hh=Sh==========h========h=...Th=ShSh
01400: =Sh=hSSSh=======SBBDhSSh=======h=hS..hShh......h==ShS....h======
01800: =...BMD.hSh=.hh...B...h=MDh=========hAh.hS...h..SST=MShh..SS.LS.
01c00: .h==============hhh=Sh.BhBBL.....DhBh=Bh=====...............BBBB
02000: B..S.....hh=BBDBL...................ShhBLhBLS...h===h======hBLh.
02400: Bh=======LhBLhBLhBLhBLhBLhS.....h==BLShhh=......................
02800: hBLSh=hBLh.....ShhShh=h.Sh....S...h=hS...hShS..h==============h=
02c00: ===.....................h..................h=======.............
03000: .......h========..............Sh..........Sh................Sh..
03400: ...................h=============h======S...........h===========
03800: =========..........h=============...............h===============
03c00: ====h========================================.....h=======Sh====
04000: ===.............................................................
04400: .................................................h..............
04800: ...........................S.................Sh....Sh.....h=====
04c00: ==h.......ShShS..........h.....S....h................h=.........
05000: ............................h.............ShSh...h=======.......
05400: .....................................ShSh=.h====================
05800: ===========........Sh.........ShShSh...........S................
05c00: ........h..........Sh...........................................
       (3 lines all free)
06c00: ....................hh..........h=....hh....hS....hSh=======....
       (2 lines all free)
07800: ...........................h==============h=====h===h=======h===
07c00: =======h=======h=====h==h=====h==hh=====h=Shh=====h=Shh======h=S
08000: hh=h=h=h=h=h=h=h=h=h=h=h=h=.....................................
       (4 lines all free)
09400: ....
>>>  
/pyboard> ls -l
  9288 Dec 31 1999  atticfan01.py
   230 Dec 31 1999  boot.py
  6204 Dec 31 1999  commands.py
   342 Dec 31 1999  config-C0.json
   342 Dec 31 1999  lib/
   142 Dec 31 1999  main.py
  1642 Dec 31 1999  mqttsetup.py
  5432 Dec 31 1999  params.py
  1021 Dec 31 1999  validaterange.py
/pyboard> 
The project, BTW, is an attic fan controller that has internal/external temp sensors powered by solar and a DCPS. Based on temperature and time of day I'm switching between DCPC and solar. I have other features I want to add (Solar V sense) but I'm out of memory :cry: I'm spitting the stats out to mqtt broker on my Synology NAS (docker image). It's all been working fine but I keep coming up with new great ideas to add! Trying to freeze a couple modules but they seem to be ignored in the build...

User avatar
pythoncoder
Posts: 4268
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: [SOLVED] Problem running latest GENERIC_1M firmware on Sonoff

Post by pythoncoder » Tue Jun 16, 2020 3:59 am

Learning to freeze modules is vital on ESP8266. I wrote a guide here which may be of some help in understanding the new manifest system.
Peter Hinch

mcL
Posts: 9
Joined: Wed Dec 25, 2019 1:36 am

Re: [SOLVED] Problem running latest GENERIC_1M firmware on Sonoff

Post by mcL » Tue Jun 16, 2020 5:36 pm

Thanks pythoncoder for the reference -- that got me going! After re-re-reading that I relized that I had the format slightly wrong: I was using the direct file as the freeze arg instead of dir, file.

In ../boards/manifest.py I added:

Code: Select all

# INCORRECT
freeze("/home/greg/PycharmProjects/Boiler/validaterange.py")
freeze("/home/greg/PycharmProjects/Boiler/umqtt.simple/umqtt/simple.py")

# CORRECT
freeze("/home/greg/PycharmProjects/Boiler","validaterange.py")
freeze("/home/greg/PycharmProjects/Boiler/umqtt.simple")
I need mqtt (and want if frozen) and couldn't find it in the uPy repo that I cloned to do the builds so I made a dir in my Boiler folder and downloaded micropyton/micropython-lib/umqtt.simple to it (see the freeze above). It all works fine (modules are show up in help('modules' and my programs runs normally), but I feel like there's a better way to use a micropython-lib module, as it's not include in my micropython cloned repo now. I didn't make a unix build so maybe that's part of it.

Next step is to create manifest.py in my project Boiler folder, for git tracking, and direct make to it via the FROZEN_MANIFEST directive. When I do that will the build use only that manifest or will is append my project one onto the one in boards? ie. Would I need to add an 'include' in mine to use boards version or should I copy the entries in the board manifest into my project one so there's a single file?

Post Reply