Page 1 of 1
garbage collector free memory strange?
Posted: Tue Oct 07, 2014 12:30 am
by mosi
Can someone confirm this behavior, or is it just me?
Code: Select all
>>> import gc
>>> gc.enable()
>>>
>>>
>>>
>>>
>>>
>>>
>>> gc.mem_free()
FATAL: uncaught exception 804b09c
FATAL ERROR:
This comes after having 260 lines of code in main.py, so maybe it is just my implementation...
Thank you.
Re: garbage collector free memory strange?
Posted: Tue Oct 07, 2014 8:11 am
by dhylands
Calling gc.enable() without a previous balanced call to gc.disable() will actually wind up disabling gc (just an FYI probably not related to your problem)
I'm going to guess that you have some heap/stack corruption occurring.
Is this for a port to a different processor? Or board?
What does the address of the exception map to?
Re: garbage collector free memory strange?
Posted: Tue Oct 07, 2014 1:45 pm
by blmorris
I just confirmed this behavior on my pyboard, with a different exception number (and running much less code and a very recent firmware build):
Code: Select all
Micro Python v1.3.1-190-gb6ca397 on 2014-10-06; PYBv1.0 with STM32F405RG
Type "help()" for more information.
>>> import gc
>>> gc.enable()
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> gc.mem_free()
FATAL: uncaught exception 804ec64
FATAL ERROR:
Strange, I haven't seen a hard fault like this in several weeks.
-Bryan
Edit - in spite of the hard fault, my mac is still able to cleanly unmount the filesystem. After all the filesystem hangs I've seen on osx, that was an even bigger surprise. Thanks @dhylands, nice work!
Re: garbage collector free memory strange?
Posted: Tue Oct 07, 2014 3:39 pm
by dhylands
It turns out that you can get the hard crash by disabling the GC and then trying to run any statment:
Code: Select all
>>> gc.disable()
>>> x=1
FATAL: uncaught exception 200002d0
FATAL ERROR:
What's happening is that because the heap is locked, it can't allocate memory to execute the next statement.
Re: garbage collector free memory strange?
Posted: Tue Oct 07, 2014 6:22 pm
by mosi
Does it mean the garbage collector should be active from the start?
I usually check free memory with:
However,
after each REPL command, the free memory decreases, until "Out of Memory".
Tried to solve this with garbage collector and it works only in manual mode:
Code: Select all
import gc
# low memory here, e.g. 28000 bytes free
pyb.info()
gc.collect()
pyb.info()
# freed memory, e.g. 98000 bytes free
Any ideas how to test if automatic garbage collection occurs?
Re: garbage collector free memory strange?
Posted: Tue Oct 07, 2014 7:13 pm
by blmorris
mosi wrote:Does it mean the garbage collector should be active from the start?
In normal usage the garbage collector is always enabled, and it collects memory automatically whenever it tries to allocate memory and finds that there isn't enough free. This is true even if you don't explicitly import gc or call gc.enable(). (The garbage collector is also responsible for allocating memory, which is why you see these failures when it is disabled and a function tries to allocate memory.)
Normally the garbage collector only gets disabled during interrupt handler routines, which if I understand correctly is why they are not allowed to allocate memory.
The only reason I know of to call gc.collect() explicitly is if you need to call a function that is going to allocate memory and you would like to ensure that a collection cycle (taking ~2-4ms) doesn't occur at that time.
If you are running code that is continuously allocating memory, then you will be able to see when garbage collection occurs by calling pyb.info() or gc.mem_free() periodically during execution (this can also be done from the REPL) and you can watch when suddenly much more memory becomes available.
One simple way to deliberately chew up memory to observe this process is a floating point adder loop:
Code: Select all
def consume_memory():
x = 0
for i in range(10000):
x = x + 1.0
print(x)
I then see the following on my pyboard without ever calling gc.collect():
Code: Select all
>>> gc.mem_free()
42560
>>> consume_memory()
10000.0
>>> gc.mem_free()
25504
>>> consume_memory()
10000.0
>>> gc.mem_free()
8448
>>> consume_memory()
10000.0
>>> gc.mem_free()
92448
>>>
-Bryan
Re: garbage collector free memory strange?
Posted: Thu Oct 09, 2014 3:58 pm
by dhylands
The Fatal error problem has now been fixed (in the latest source).
You're still hooped if you call gc.disable() (or gc.enable()) from the REPL, but you can hit Control-D tto do a soft-reset.