If I call this in a loop:
def doit(arg):
soc = esp.socket()
soc.__del__()
gc.collect()
And call gc.collect() in my outer loop, the memory is slowly used up. Is this expected?
Should I be trying to add code to the del so it manually marks some of the allocated elements in the socket free?
Are there any good ways to work out the object types in the managed memory?
ESP8266 sockets - memory being all used up. FIXED
ESP8266 sockets - memory being all used up. FIXED
Last edited by mianos on Sun Oct 04, 2015 11:43 pm, edited 1 time in total.
Re: ESP8266 sockets - memory being all used up.
Me again. I am trying to track down why, when I use the external timer from my code https://github.com/micropython/micropython/pull/1453
It seems the garbage collector never actually has anything freeable.
For example, I have enabled the gc debugging. Once the string is printed I would assume it would be collectable.
I may be on the wrong track here. Once I come out of the uart loop and back to the system scheduled loop
maybe there is some other issue with the stack? It seems the garbage collector should just run through the object list that has been built on the pre-allocated heap.
The garbage collect does not seem to have any knowledge of the stack in the existing thread.
(ps. I bought a micropython board to contribute but I really want to get the ESP board going well).
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(3fff4310)
gc_alloc(18 bytes -> 2 blocks)
gc_alloc(3fff4450)
b'HTTP/1.0 200 OK\r\n'
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(3fff4350)
gc_alloc(146 bytes -> 10 blocks)
gc_alloc(3fff47c0)
b'Date: Mon, 07 Sep 2015 11:42:17 GMT\r\nServer: WSGIServer/0.1 Python/2.7.3\r\nContent-type: text/html\r\nContent-Length: 24\r\n\r\n<html>Hello world</html>'
disconnected
gc_alloc(40 bytes -> 3 blocks)
gc_alloc(3fff4640)
gc_alloc(28 bytes -> 2 blocks)
gc_alloc(3fff4600)
gc_alloc(32 bytes -> 2 blocks)
gc_alloc(3fff4670)
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(3fff4370)
gc_sweep(3fff4310)
gc_sweep(3fff4350)
gc_sweep(3fff4370)
gc_sweep(3fff4420)
gc_sweep(3fff5600)
gc_sweep(3fff5610)
gc_sweep(3fff5620)
gc_sweep(3fff5630)
gc_sweep(3fff5640)
gc_sweep(3fff5650)
gc_sweep(3fff5660)
gc_sweep(3fff5670)
gc_sweep(3fff5680)
gc_sweep(3fff5690)
GC: total: 16128, used: 2224, free: 13904
No. of 1-blocks: 27, 2-blocks: 13, max blk sz: 11
connected
It seems the garbage collector never actually has anything freeable.
For example, I have enabled the gc debugging. Once the string is printed I would assume it would be collectable.
I may be on the wrong track here. Once I come out of the uart loop and back to the system scheduled loop
maybe there is some other issue with the stack? It seems the garbage collector should just run through the object list that has been built on the pre-allocated heap.
The garbage collect does not seem to have any knowledge of the stack in the existing thread.
(ps. I bought a micropython board to contribute but I really want to get the ESP board going well).
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(3fff4310)
gc_alloc(18 bytes -> 2 blocks)
gc_alloc(3fff4450)
b'HTTP/1.0 200 OK\r\n'
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(3fff4350)
gc_alloc(146 bytes -> 10 blocks)
gc_alloc(3fff47c0)
b'Date: Mon, 07 Sep 2015 11:42:17 GMT\r\nServer: WSGIServer/0.1 Python/2.7.3\r\nContent-type: text/html\r\nContent-Length: 24\r\n\r\n<html>Hello world</html>'
disconnected
gc_alloc(40 bytes -> 3 blocks)
gc_alloc(3fff4640)
gc_alloc(28 bytes -> 2 blocks)
gc_alloc(3fff4600)
gc_alloc(32 bytes -> 2 blocks)
gc_alloc(3fff4670)
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(3fff4370)
gc_sweep(3fff4310)
gc_sweep(3fff4350)
gc_sweep(3fff4370)
gc_sweep(3fff4420)
gc_sweep(3fff5600)
gc_sweep(3fff5610)
gc_sweep(3fff5620)
gc_sweep(3fff5630)
gc_sweep(3fff5640)
gc_sweep(3fff5650)
gc_sweep(3fff5660)
gc_sweep(3fff5670)
gc_sweep(3fff5680)
gc_sweep(3fff5690)
GC: total: 16128, used: 2224, free: 13904
No. of 1-blocks: 27, 2-blocks: 13, max blk sz: 11
connected
Re: ESP8266 sockets - memory being all used up.
It would be collectable, yes. It wouldn't be collected right away, because uPy uses garbage collection, not reference counting (which means deallocating old objects when there's not enough memory, or on exlicit gc.collect()). And it uses conservative GC, which means any particular object may be not freed (as eagerly as you expect, or at all).Once the string is printed I would assume it would be collectable.
Maybe.Once I come out of the uart loop and back to the system scheduled loop
maybe there is some other issue with the stack?
The effect of this would be opposite of what you describe - some yet-live objects would be collected (because GC misses refs to them on the stack).The garbage collect does not seem to have any knowledge of the stack in the existing thread.
Code: Select all
gc_alloc(16 bytes -> 1 blocks)
gc_alloc(3fff4310)
gc_sweep(3fff4310)
Everyone would, it's just with all the effort already spent on the port, even more is required to get something really working out of it, and that will require outgrowing vendor framework, which is actually more disabling, than enabling, to the capabilities of the chip. That means many-many days, or if you have dayjob, many-many nights. Welcome to the club .(ps. I bought a micropython board to contribute but I really want to get the ESP board going well).
In my list, next thing which should be done to esp8266 port is making it run testsuite (using pyboard.py tool, like other boards). That will allow to really assess what works in the port, and what not.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Re: ESP8266 sockets - memory being all used up.
Yes, same story for me, I'd like to find some more time to pick up an issue or functionality that needs work on ESP port but it's a little difficult to see what's being worked on at the moment and what actually needs doing. I don't know if that deserves a separate board on the forum or perhaps it would be good to set up irc or similar.mianos wrote: (ps. I bought a micropython board to contribute but I really want to get the ESP board going well).
Re: ESP8266 sockets - memory being all used up. FIXED
With this:
https://github.com/micropython/micropython/issues/1479
and this:
https://github.com/micropython/micropython/issues/1486
My branch works perfectly now. I have just run 10 GETs a second for 3 days and it has been faultless.
You need to call the garbage collector if you are using the ESP internal scheduler. I am doing at the end of my os_timer callback.
I'll not be bothering with trying to get this pushed as the changes required to get the push approved will leave it not usable for me.
I'll try and keep my fork up to date with the master.
https://github.com/mianos/micropython
With my esp.os_timer and these changes this board is becoming really nice to use.
https://github.com/micropython/micropython/issues/1479
and this:
https://github.com/micropython/micropython/issues/1486
My branch works perfectly now. I have just run 10 GETs a second for 3 days and it has been faultless.
You need to call the garbage collector if you are using the ESP internal scheduler. I am doing at the end of my os_timer callback.
I'll not be bothering with trying to get this pushed as the changes required to get the push approved will leave it not usable for me.
I'll try and keep my fork up to date with the master.
https://github.com/mianos/micropython
With my esp.os_timer and these changes this board is becoming really nice to use.