help is appreciated - memory allocated by m_malloc() is automatically free'ed

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
User avatar
dhylands
Posts: 3228
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by dhylands » Mon Apr 16, 2018 3:37 pm

I'd recommend that you try to reproduce under the unix build. It's much easier to debug there. It would mean removing any hardware access from your module. If that doesn't work, then I'd recommend that you publish your full source, preferably in a micropython tree so others can try to reproduce.

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by jickster » Mon Apr 16, 2018 3:53 pm

dhylands wrote:
Mon Apr 16, 2018 3:37 pm
I'd recommend that you try to reproduce under the unix build. It's much easier to debug there. It would mean removing any hardware access from your module. If that doesn't work, then I'd recommend that you publish your full source, preferably in a micropython tree so others can try to reproduce.
If I understood how the gc works, I could solve it.

It obviously doesn't work as I've read: "as long as there's a chain of pointers from root pointer to your memory block, your memory won't get deleted"

That part is clear. I've used mp_store_name/mp_store_global - both store to dict_main - and mp_module_register which all store to a dictionary in the root pointer section.


KEY ISSUE
The one relevant difference is that I've built-up my module hierarchy completely dynamically, and to tell you the truth, I'm beginning to suspect the garbage collector does NOT traverse the "chain" correctly and/or completely.

I'm going to come up with a super basic example and post it. Gimme a few minutes.

User avatar
dhylands
Posts: 3228
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by dhylands » Mon Apr 16, 2018 4:11 pm

One possible "chain" that I don't believe that the gc follows is if you have a heap object pointing back into non-heap (i.e. static globals) and those static globals have a pointer into the heap. In that case, the gc wouldn't see the pointer from the static data. The gc only follows pointers within the heap itself.

Since you're building things dynamically, and your module object is declared statically, this might be the cause.

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by jickster » Mon Apr 16, 2018 4:36 pm

dhylands wrote:
Mon Apr 16, 2018 4:11 pm
One possible "chain" that I don't believe that the gc follows is if you have a heap object pointing back into non-heap (i.e. static globals) and those static globals have a pointer into the heap. In that case, the gc wouldn't see the pointer from the static data. The gc only follows pointers within the heap itself.

Since you're building things dynamically, and your module object is declared statically, this might be the cause.
That makes perfect sense. I'll allocating the module object dynamically.
Thanks.

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by jickster » Mon Apr 16, 2018 6:54 pm

dhylands wrote:
Mon Apr 16, 2018 4:11 pm
One possible "chain" that I don't believe that the gc follows is if you have a heap object pointing back into non-heap (i.e. static globals) and those static globals have a pointer into the heap. In that case, the gc wouldn't see the pointer from the static data. The gc only follows pointers within the heap itself.

Since you're building things dynamically, and your module object is declared statically, this might be the cause.
BINGO! You are correct.
I placed my "device_module" in root pointers section and it works!!!!!
I can now do "dir(device.CAN)" after a "gc.collect()"

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by jickster » Wed Aug 29, 2018 6:58 pm

dhylands wrote:
Mon Nov 20, 2017 7:11 pm
Yes - soft reset causes the entire heap to be re-initialized.
That's not true: gc_init() only zeros-out the below items and I think it's a source of a bug

Code: Select all

// clear ATBs
    memset(MP_STATE_MEM(gc_alloc_table_start), 0, MP_STATE_MEM(gc_alloc_table_byte_len));

#if MICROPY_ENABLE_FINALISER
    // clear FTBs
    memset(MP_STATE_MEM(gc_finaliser_table_start), 0, gc_finaliser_table_byte_len);
#endif

User avatar
dhylands
Posts: 3228
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by dhylands » Thu Aug 30, 2018 3:02 pm

The ATB is where the heap keeps track of which blocks are allocated. The value 0 corresponds to AT_FREE. What do you think the bug is?

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by jickster » Thu Aug 30, 2018 3:54 pm

dhylands wrote:The ATB is where the heap keeps track of which blocks are allocated. The value 0 corresponds to AT_FREE. What do you think the bug is?
The actual memory is not zeroed.
If you do a soft-reset, same object COULD be in same memory location if you execute same code.

Sent from my iPhone using Tapatalk Pro

User avatar
dhylands
Posts: 3228
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by dhylands » Thu Aug 30, 2018 6:52 pm

If you call m_malloc from C, then no, the memory is not cleared. This is consistent with the behaviour from the C runtime library function malloc. If you want cleared memory, then you should be using m_malloc0 (which clears the returned memory to zero).

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: help is appreciated - memory allocated by m_malloc() is automatically free'ed

Post by jickster » Thu Aug 30, 2018 7:12 pm

dhylands wrote:
Thu Aug 30, 2018 6:52 pm
If you call m_malloc from C, then no, the memory is not cleared. This is consistent with the behaviour from the C runtime library function malloc. If you want cleared memory, then you should be using m_malloc0 (which clears the returned memory to zero).
Why shouldn't we add a clearing of the heap in gc_init()?

Post Reply