Help with upip

[gmeloy]

Hi all,

I'm trying to use an ESP8266 for several new products and I'm trying to make sure that we'll have a reasonable update scheme once units are in the field. I've been trying to make upip work but so far it's been very unreliable.

It works for some very small files, but more often than not I get a get a memory allocation error or an [Errno 22] message. I think that perhaps the Errno 22 messages are for projects that weren't built properly for distribution to Micropython. If that's the case then I'm not too concerned about that error because presumably I can figure out the right way to build ours when the time comes. Does anybody know if that's what the Errno 22 message means?

The memory allocation issue seems huge, though. What confuses me is that I appear to have TONS of heap available. Maybe it's not contiguous and that's why the allocation fails?

See an example of what I'm getting below. Note that I modified upip to take an address parameter so that in the future I can move off of pypi.python.org to another location, but this issue was happening when I was running from the precompiled esp8266-20170725-v1.9.1-191-gf3687109.bin file as well, so I don't think my customization has anything to do with it:


>>> import gc
>>> import upip
>>> gc.mem_free()
21200
>>> gc.collect()
>>> gc.mem_free()
21232
>>> upip.install('https://pypi.python.org/pypi/','micropython-binascii')
Installing to: /lib/
Warning: pypi.python.org SSL certificate is not validated
Installing micropython-binascii 2.4.0.post5 from https://pypi.python.org/packa
ges/9c/eb/4bd24214bbd2eb9bfeddac3db4473de686ad9c6bb56449856b7e688f748c/micropy
thon-binascii-2.4.0.post5.tar.gz
Error installing 'micropython-binascii': memory allocation failed, allocating
4096 bytes, packages may be partially installed
>>>


I'm really hoping that I'm doing something wrong and it's simple to fix. If upip doesn't really work then I don't know if we can use this device.

Thanks very much,
Glen

[pythoncoder]

You can get further information on RAM with

Code: Select all

import micropython
micropython.mem_info(1)

This will show if RAM is fragmented. With a USB connection to an ESP8266 I see 26080 bytes free so I guess you have something running (webrepl?).

[gmeloy]

Well, I was going to say that I have "almost nothing" running, but apparently that's not true. My main.py imports the websockets code that I got from here:
https://github.com/danni/uwebsockets

I'm using that for a simple websocket interface to be able to send and receive data. I tried removing that import and rebooting the ESP. Without that import I have about 26k free and the upip call from my first post succeeds. So I guess it's 5-6k in size. I suppose that's why the git page for websockets says that it should be 'baked' into the micropython code, though I'm hesitant to do that as anything I compile in with micropython can't be upgraded in the field.

I changed my main.py to not automatically import that and I can really see a difference in the memory map before and after loading it. I think I can find a way to use this where I don't load the websockets module when I'm doing pip updates.

>>> import micropython
>>> micropython.mem_info(1)
stack: 2128 out of 8192
GC: total: 35968, used: 9696, free: 26272
No. of 1-blocks: 107, 2-blocks: 38, max blk sz: 264, max free sz: 1629
GC memory layout; from 3ffef4e0:
00000: h=hhhhMLSDh=hDBB=BBBh===h====hhBhh==h===========================
00400: ================================================================
00800: ================================================================
00c00: ================================================================
01000: ============================================h=h=h==AMDBB..h.h===
01400: ====h===========================================================
01800: ============================================Shh=h==...A.Sh=Sh=SS
01c00: h==T=Sh=.....hShh=T=Sh=ShT=ShSh.T=SSSShh=hT=ShShT=T=hShT=ShShT=S
02000: hShT=ShShT=ShShShT=Sh=ShT=ShShT=Sh=Shhh==T=Sh=ShT=ShShT=Sh======
02400: ========h=hShT=ShShSh=h=======hT=ShShT=Sh=S.....................
(25 lines all free)
08c00: ........
>>> import ws
>>> micropython.mem_info(1)
stack: 2128 out of 8192
GC: total: 35968, used: 12512, free: 23456
No. of 1-blocks: 92, 2-blocks: 29, max blk sz: 264, max free sz: 727
GC memory layout; from 3ffef4e0:
00000: MDShhSMhBDh=hD.B=BBBh===h====hhhhh==h===========================
00400: ================================================================
00800: ================================================================
00c00: ================================================================
01000: ============================================h=h=h==.MDBSh.h.h===
01400: ====h===========================================================
01800: ============================================.....Sh.MDh.Sh=h==SS
01c00: h=.h.Sh=....D.Shh=MD....h====....S..SSh..B.DB.h====B....BB.B=BB.
02000: B.B=B=h===h=h=B...B...............h==hh==h===h==h===A....h======
02400: ========h=Sh====....h=h==================..........MDhh==...hh==
02800: h==Sh...h========.h=.h==.hh=======h===..Sh=..........S...h====..
02c00: ........h======...................................h=h===========
03000: ====.h=h=======h==============..................................
03400: .................................Sh.........Sh........SSh=....Sh
03800: =....Sh=.....Sh=.....h=.......................h...............hh
03c00: ...............h=======.........................................
(2 lines all free)
04800: ...............................Sh.............h.................
(3 lines all free)
05800: .........h====........h=h=hh=hh=hh================hh============
05c00: =====h=h==============hh======hh====hh==hh==hh==h...............
(11 lines all free)
08c00: ........
>>>

Thanks for the reply

[pythoncoder]

...I suppose that's why the git page for websockets says that it should be 'baked' into the micropython code, though I'm hesitant to do that as anything I compile in with micropython can't be upgraded in the field...

Have a look at this thread which may be relevant. viewtopic.php?f=16&t=3374