help with leaking memory management
Posted: Sat Jan 23, 2016 9:38 am
I am at a halt with my ESP8266 version.
I can't get a long running program that uses lists to keep running as it runs out of memory.
I have found if I have a struct that contains a list
I have a mini state based web server that receives multiple requests and, hopefully dispatches the request type, headers as a list and body to a dispatcher.
In my code I was planning to add the header keys and values to a ist.
(The keys and values themselves should be mostly the same so interning them should use next to no memory),
I found if I create a list, as follows (I have reduced the running code and it fails even without any items added to the list).
and at the end of each loop reset the value as follows:
As far as I can tell, this should orphan the mp_obj_new_list item and the collector should find it unreferenced and free it up.
If I do a few hundred requests and stop, a pub.info(1)
shows this:
Then after a gc.collect()
I can only guess at what this means, as far as I can see the list is getting freed, but the 'h' records are filling up the memory.
The basic file is here:
https://github.com/mianos/micropython/b ... d_esp_ws.c
Can anyone help me with an idea here? I ran into this issue with the modesp sockets so I thought I'd try something similar in a separate file but I seem to have hit the same problem.
It would be a shame to give in at this point as my ESP8266 version is really fun now I pretty much everything on it.
I can't get a long running program that uses lists to keep running as it runs out of memory.
I have found if I have a struct that contains a list
Code: Select all
typedef struct _esp_ws_obj_t {
mp_obj_base_t base;
mp_obj_t header_key;
mp_obj_t header_val;
mp_obj_t header_kp;
mp_obj_t headers;
In my code I was planning to add the header keys and values to a ist.
(The keys and values themselves should be mostly the same so interning them should use next to no memory),
I found if I create a list, as follows (I have reduced the running code and it fails even without any items added to the list).
Code: Select all
...
pesp->header_val = mp_obj_new_str(header_val, strlen(header_val), true);
pesp->header_kp = mp_obj_new_list(0, NULL);
Code: Select all
ctx->header_val = mp_const_none;
ctx->header_kp = mp_const_none;
If I do a few hundred requests and stop, a pub.info(1)
shows this:
Code: Select all
qstr:
n_pool=1
n_qstr=9
n_str_data_bytes=123
n_total_bytes=219
GC:
19520 total
9424 : 10096
1=534 2=5 m=13
GC memory layout; from 3fff4d70:
f4d70: hthhhBhtBhhtttLhLhtthttttttthtttttttttttthLhLhLhLhthLhLhLhLhLhLh
f5170: LhttttttthttttthLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLh
f5570: LhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLh
f5970: LhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLh
f5d70: LhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLh
f6170: LhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLh
f6570: LhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLh
f6970: LhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLh
f6d70: LhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhLhBh
f7170: hBhhB...h.ht....ht....htt.......................................
(9 lines all free)
f9970: ....
Code: Select all
qstr:
n_pool=1
n_qstr=9
n_str_data_bytes=123
n_total_bytes=219
GC:
19520 total
5312 : 14208
1=272 2=6 m=13
GC memory layout; from 3fff4d70:
f4d70: hthhhhhtB.htttLh.htthttttttthtttttttttttthLh.h.h.hth.h.h.h.h.h.h
f5170: .httttttthttttth.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h
f5570: .h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h
f5970: .h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h
f5d70: .h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h
f6170: .h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h
f6570: .h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h
f6970: .h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h
f6d70: .h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h.h
f7170: htth.hhB..ht....ht..hthtt.......................................
(9 lines all free)
f9970: ....
The basic file is here:
https://github.com/mianos/micropython/b ... d_esp_ws.c
Can anyone help me with an idea here? I ran into this issue with the modesp sockets so I thought I'd try something similar in a separate file but I seem to have hit the same problem.
It would be a shame to give in at this point as my ESP8266 version is really fun now I pretty much everything on it.