Page 1 of 1
[SOLVED] mutex deadlock during garbage collection
Posted: Wed Feb 21, 2018 9:02 am
by ThomasK
Hi,
I'm currently facing a fairly interesting problem with the garbage collection when running the micropython kernel inside of an RTOS.
as micropython runs inside of an RTOS, I'm making use of GC_ENTER and GC_EXIT to lock & unlock mutexes.
I'm wondering now if this is due to something I did in my port, or if this is an issue that exists within the standard micropython builds too?
Has anyone else seen this behavior, or am I missing something / did I make a mistake when implementing an alternative filesystem?
I started from MicroPython 1.9.3
Scenario:
Code: Select all
call gc_alloc()
GC_ENTER()
...
GC_EXIT()
call gc_collect()
call gc_collect_start()
GC_ENTER()
call gc_collect_end()
call gc_sweep()
call "__del__" method of object if it exists (in this example an old file pointer)
file_close() throws a "file not open" OS Exception
OS Exception calls gc_alloc()
gc_alloc() calls GC_ENTER()
Result: The thread waits for the mutex to be released by itself.
Re: mutex deadlock during garbage collection
Posted: Wed Feb 21, 2018 7:10 pm
by jickster
ThomasK wrote: ↑Wed Feb 21, 2018 9:02 am
Hi,
I'm currently facing a fairly interesting problem with the garbage collection when running the micropython kernel inside of an RTOS.
as micropython runs inside of an RTOS, I'm making use of GC_ENTER and GC_EXIT to lock & unlock mutexes.
I'm wondering now if this is due to something I did in my port, or if this is an issue that exists within the standard micropython builds too?
Has anyone else seen this behavior, or am I missing something / did I make a mistake when implementing an alternative filesystem?
I started from MicroPython 1.9.3
Scenario:
Code: Select all
call gc_alloc()
GC_ENTER()
...
GC_EXIT()
call gc_collect()
call gc_collect_start()
GC_ENTER()
call gc_collect_end()
call gc_sweep()
call "__del__" method of object if it exists (in this example an old file pointer)
file_close() throws a "file not open" OS Exception
OS Exception calls gc_alloc()
gc_alloc() calls GC_ENTER()
Result: The thread waits for the mutex to be released by itself.
Either you're not supposed to throw an exception while you're cleaning up (i.e. in __del__) OR
it's a bug in uPy.
m_malloc_maybe appears as if it's supposed to be a conditional allocation but its implementation does not actually provide that feature; it just calls malloc and does a debug print.
Code: Select all
void *m_malloc_maybe(size_t num_bytes) {
void *ptr = malloc(num_bytes);
#if MICROPY_MEM_STATS
MP_STATE_MEM(total_bytes_allocated) += num_bytes;
MP_STATE_MEM(current_bytes_allocated) += num_bytes;
UPDATE_PEAK();
#endif
DEBUG_printf("malloc %d : %p\n", num_bytes, ptr);
return ptr;
}
Re: mutex deadlock during garbage collection
Posted: Fri Feb 23, 2018 7:08 am
by Damien
The GC_ENTER/GC_EXIT macros are really only intended to be used by the GC for the case that multithreading is enabled and the GIL is disabled. It's probably going to be safer in general to enable the GIL so that things like list or dict update appear atomic to MicroPython scripts.
Also, according to standard CPython behaviour, a file should be allowed to be closed multiple times without error. That really helps the case at hand, where the user may have closed the file explicitly but then it also has the finaliser called when the memory for the file is cleaned up.
But apart from those two points, it does look like there is a bug in uPy that you've exposed: when doing a garbage collection the memory allocator is locked and so it shouldn't try to allocate, but in this case it does.
Re: mutex deadlock during garbage collection
Posted: Fri Feb 23, 2018 9:43 am
by ThomasK
Great, thank you for the feedback, jickster & Damien!
Re: mutex deadlock during garbage collection
Posted: Fri Feb 23, 2018 9:39 pm
by jickster
ThomasK wrote: ↑Fri Feb 23, 2018 9:43 am
Great, thank you for the feedback, jickster & Damien!
Please update your question title by prepending
[SOLVED] to make the forum easier to read
Re: [SOLVED] mutex deadlock during garbage collection
Posted: Mon Feb 26, 2018 8:12 am
by ThomasK
Done, as the part that directly applied to my question was resolved,
although I don't quite agree that it's solved when Damien comes to the conclusion that "there indeed appears to be a bug".
Re: [SOLVED] mutex deadlock during garbage collection
Posted: Mon Feb 26, 2018 6:01 pm
by jickster
ThomasK wrote: ↑Mon Feb 26, 2018 8:12 am
Done, as the part that directly applied to my question was resolved,
although I don't quite agree that it's solved when Damien comes to the conclusion that "there indeed appears to be a bug".
Personally, I'm not clear what the GIL is and how it works and its relationship to GC_ENTER.
Re: [SOLVED] mutex deadlock during garbage collection
Posted: Tue Feb 27, 2018 9:13 am
by pythoncoder
GIL stands for Global Interpreter Lock and is a rather contentious aspect of Python. Most implementations have a GIL. Google is your friend here
