garbage collector free memory strange?

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
mosi
Posts: 28
Joined: Tue Oct 07, 2014 12:07 am

garbage collector free memory strange?

Post by mosi » Tue Oct 07, 2014 12:30 am

Can someone confirm this behavior, or is it just me?

Code: Select all

>>> import gc
>>> gc.enable()
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> gc.mem_free()
FATAL: uncaught exception 804b09c

FATAL ERROR:

This comes after having 260 lines of code in main.py, so maybe it is just my implementation...

Thank you.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: garbage collector free memory strange?

Post by dhylands » Tue Oct 07, 2014 8:11 am

Calling gc.enable() without a previous balanced call to gc.disable() will actually wind up disabling gc (just an FYI probably not related to your problem)

I'm going to guess that you have some heap/stack corruption occurring.

Is this for a port to a different processor? Or board?

What does the address of the exception map to?

blmorris
Posts: 348
Joined: Fri May 02, 2014 3:43 pm
Location: Massachusetts, USA

Re: garbage collector free memory strange?

Post by blmorris » Tue Oct 07, 2014 1:45 pm

I just confirmed this behavior on my pyboard, with a different exception number (and running much less code and a very recent firmware build):

Code: Select all

Micro Python v1.3.1-190-gb6ca397 on 2014-10-06; PYBv1.0 with STM32F405RG
Type "help()" for more information.
>>> import gc
>>> gc.enable()
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> gc.mem_free()
FATAL: uncaught exception 804ec64

FATAL ERROR:
Strange, I haven't seen a hard fault like this in several weeks.
-Bryan

Edit - in spite of the hard fault, my mac is still able to cleanly unmount the filesystem. After all the filesystem hangs I've seen on osx, that was an even bigger surprise. Thanks @dhylands, nice work!

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: garbage collector free memory strange?

Post by dhylands » Tue Oct 07, 2014 3:39 pm

It turns out that you can get the hard crash by disabling the GC and then trying to run any statment:

Code: Select all

>>> gc.disable()
>>> x=1
FATAL: uncaught exception 200002d0

FATAL ERROR:
What's happening is that because the heap is locked, it can't allocate memory to execute the next statement.

mosi
Posts: 28
Joined: Tue Oct 07, 2014 12:07 am

Re: garbage collector free memory strange?

Post by mosi » Tue Oct 07, 2014 6:22 pm

Does it mean the garbage collector should be active from the start?

I usually check free memory with:

Code: Select all

import pyb
pyb.info()
However,
after each REPL command, the free memory decreases, until "Out of Memory".

Tried to solve this with garbage collector and it works only in manual mode:

Code: Select all

import gc
# low memory here, e.g. 28000 bytes free
pyb.info()
gc.collect()
pyb.info()
# freed memory, e.g. 98000 bytes free
Any ideas how to test if automatic garbage collection occurs?

blmorris
Posts: 348
Joined: Fri May 02, 2014 3:43 pm
Location: Massachusetts, USA

Re: garbage collector free memory strange?

Post by blmorris » Tue Oct 07, 2014 7:13 pm

mosi wrote:Does it mean the garbage collector should be active from the start?
In normal usage the garbage collector is always enabled, and it collects memory automatically whenever it tries to allocate memory and finds that there isn't enough free. This is true even if you don't explicitly import gc or call gc.enable(). (The garbage collector is also responsible for allocating memory, which is why you see these failures when it is disabled and a function tries to allocate memory.)

Normally the garbage collector only gets disabled during interrupt handler routines, which if I understand correctly is why they are not allowed to allocate memory.
The only reason I know of to call gc.collect() explicitly is if you need to call a function that is going to allocate memory and you would like to ensure that a collection cycle (taking ~2-4ms) doesn't occur at that time.

If you are running code that is continuously allocating memory, then you will be able to see when garbage collection occurs by calling pyb.info() or gc.mem_free() periodically during execution (this can also be done from the REPL) and you can watch when suddenly much more memory becomes available.

One simple way to deliberately chew up memory to observe this process is a floating point adder loop:

Code: Select all

def consume_memory():     
    x = 0               
    for i in range(10000):
        x = x + 1.0       
    print(x)
I then see the following on my pyboard without ever calling gc.collect():

Code: Select all

>>> gc.mem_free()   
42560
>>> consume_memory()
10000.0
>>> gc.mem_free()   
25504
>>> consume_memory()
10000.0
>>> gc.mem_free()   
8448
>>> consume_memory()
10000.0
>>> gc.mem_free()   
92448
>>> 
-Bryan

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: garbage collector free memory strange?

Post by dhylands » Thu Oct 09, 2014 3:58 pm

The Fatal error problem has now been fixed (in the latest source).

You're still hooped if you call gc.disable() (or gc.enable()) from the REPL, but you can hit Control-D tto do a soft-reset.

Post Reply