Page 1 of 1
Use of gc over and over?
Posted: Fri Jul 29, 2022 2:43 am
by Jibun no kage
I was reviewing the following code...
Code: Select all
import gc
import usocket as socket
import ustruct as struct
gc.collect()
from ubinascii import hexlify
import uasyncio as asyncio
gc.collect()
from utime import ticks_ms, ticks_diff
from uerrno import EINPROGRESS, ETIMEDOUT
gc.collect()
from micropython import const
from machine import unique_id
import network
gc.collect()
from sys import platform
Initially this stuck me as a bit odd, given that modern heap management should optimize and deference heap pointers, but above code suggests that gc is not that flexible or otherwise limited. Is that really the case? Would not setting a gc threshold service as a method to encourage gc to do the work, versus calling it explicitly over and over as above? Or is it the case that the loading the various modules severely (can) fragment the heap, and gc can't handle it without explicit invocation? Just looking for a bit of confirmation or clarity on why the above was done, as it was done.
Re: Use of gc over and over?
Posted: Fri Jul 29, 2022 7:59 am
by pythoncoder
When you import a module which exists as Python sourcecode the compiler converts it to bytecode. This process involves allocations and running an explicit gc.collect() reclaims the allocations. The aim is to reduce RAM fragmentation. This can be an issue if the application needs to allocate large buffers. That said, the example above does seem rather enthusiastic in its use.
Another reason to explicitly issue gc.collect() is that it runs faster if it doesn't have much to do. Some of my code periodically does this to ensure that the blocking period is short and occurs at times when it has least impact on the application.
Re: Use of gc over and over?
Posted: Fri Jul 29, 2022 9:40 am
by p_j
pythoncoder wrote: ↑Fri Jul 29, 2022 7:59 am
When you import a module which exists as Python sourcecode the compiler converts it to bytecode. This process involves allocations and running an explicit
gc.collect() reclaims the allocations. The aim is to reduce RAM fragmentation. This can be an issue if the application needs to allocate large buffers. That said, the example above does seem rather enthusiastic in its use.
Another reason to explicitly issue
gc.collect() is that it runs faster if it doesn't have much to do. Some of my code periodically does this to ensure that the blocking period is short and occurs at times when it has least impact on the application.
If the code is pre-compiled to bytecode, does this remove the need to run gc.collect?
Re: Use of gc over and over?
Posted: Fri Jul 29, 2022 9:46 am
by Roberthh
No, because Python uses a dynamic memory allocation approach. Objects are created and freed at runtime.
Re: Use of gc over and over?
Posted: Sat Jul 30, 2022 3:46 pm
by Jibun no kage
So... the long and short, my understanding was correct, explicit gc.collect forces consolidation, and the benefit of frequent invocation can improve over all responsive/speed of collect since it has less to do.
Does setting a gc threshold not help with this after initialization? I recall in my past ESP coding is I set the go threshold to a given value, which I determined by watching the heap during testing, would ensure that gc did collections based on need versus explicit invocation at various points in code. At the time this seemed more logical and elegant, and peppering collects in code. Is my perception of how the threshold feature still valid, IMHO, so, no?
Re: Use of gc over and over?
Posted: Sat Jul 30, 2022 4:15 pm
by Roberthh
Setting a GC threshold helps to start the automatic GC collection earlier. And in my experience manual optimisation of the GC behaviour is only required at devices with small heaps, to avoid allocation fails, and those with very large heaps, if GC collection time matters. In any case it is better to allocate large buffers early and keep them.