Clearing RAM

[Minyiky]

I am running a task that is quite ram hungry and at the start I am loading an image onta a screen but this is cutting into my available RAM

This is the funtion I am using to push the image to the screen:

Code: Select all

def draw_image(self, path, x=0, y=0, w=320, h=240, chunk_height=3200):
    """Draw image from flash.
    Args:
        path (string): Image file path.
        x (int): X coordinate of image left.  Default is 0.
        y (int): Y coordinate of image top.  Default is 0.
        w (int): Width of image.  Default is 320.
        h (int): Height of image.  Default is 240.
    """
    x2 = x + w - 1
    y2 = y + h - 1

    with open(path, "rb") as f:
        chunk_height = chunk_height // w
        chunk_count, remainder = divmod(h, chunk_height)
        chunk_size = chunk_height * w * 2
        chunk_y = y
        buf = bytearray(chunk_size)

        if chunk_count:
            for c in range(0, chunk_count):
                f.readinto(buf, chunk_size)
                self.block(x, chunk_y,
                           x2, chunk_y + chunk_height - 1,
                           buf)
                chunk_y += chunk_height

        if remainder:
            f.readinto(buf, remainder * w * 2)
            self.block(x, chunk_y,
                       x2, chunk_y + remainder - 1,
                       buf)

When running gc.mem_free() before and after the creation of buf the memory taken is as expected but I cannot find any way to get that memory back, including setting buf=None and gc.collect() calls.

Does anyone know how this can be achieved?

Thanks in advance

[stijn]

What does self.block() do with the buf argument?

In any case, sometimes multiple gc.collect() calls seem to be required before everything gets collected. Also you're sure the memory which remains taken is effectively the memory allocated for buf, not something else?

[Minyiky]

The block function sends position commands to the screen and then writes the contents of buf to the screen.

When I check the mem_free it reduces immediately after initiating the buffer and the amount used is equal to its size, that memory is then never freed even with multiple calls of gc. collect.

[jimmo]

including setting buf=None and gc.collect() calls.

if buf has been set to None (you can also use "del buf"), but gc.collect() isn't subsequently catching it then it suggests that something else has a reference to it. Could there be a memoryview pointing to it or something?

Are you able to share the full code? Which board are you using?

In any case, sometimes multiple gc.collect() calls seem to be required before everything gets collected.

That's surprising... I can't see why this would be the case... perhaps a complicated interaction with a finaliser?

[stijn]

@Minyiky so self.block() does not keep a reference to buf, right? Could you try gradually removing statements to see if that fixes something? And try calling draw_image() then run some other statements, then call gc.collect()?

That's surprising... I can't see why this would be the case... perhaps a complicated interaction with a finaliser?

Not sure, but I now realize my wording was actually wrong: it's not so much needing multiple collect() calls, but rather needing extra statements before collect() effectively collects everything.
See e.g. https://github.com/stinos/micropython-w ... nalizer.py and also look at the history of that file: it took a couple of iterations to get collect() to get rid of x, which is an instance of a fairly normal class implemented in C which doesn't hold references to anything else, it just has a finalizer so it can print when it gets finalized. If I had to take a wild guess, I'd say it has something to do with the fastn array in the VM which stores pointers but might not remove them until other statements come along. Again, wild guess, I admit I should just take the time to figure it out

[Minyiky]

I cannot see anywhere where a reference would be kept, I am now wondering if simply passing the variable is causing the issue. If I try to delete the buffer before calling block I am able to recover the space but not after. To try and see if block was the issue I created a new empty function:

Code: Select all

def _block(self, x0, y0, x1, y1, data):
    pass

Even if I call this function instead I do not seem to be able to free up the allocated memory. For reference I am using an esp32 (dfrobot firebeetle).

I have tried to cut out the actual writing process as I am currently testing out lvgl and there is a lot of burried code which may well have been causing the problem.

[pythoncoder]

If you are performing GC outside of the function draw_image the memory should be reclaimed, so long as no reference is retained. For example:

Code: Select all

import gc
def bar(a):
    pass

def foo():
    gc.collect()
    print(gc.mem_free())
    a = bytearray(5_000)
    bar(a)
    gc.collect()
    print(gc.mem_free())

foo()
gc.collect()
print(gc.mem_free())

Outcome:

Code: Select all

101168
96144
101168

[Minyiky]

Hmm firstly I want to say thank you, this seems to have fixed the problem, although I am a bit confused as that is what I was doing when I first encountered the problem. I believe what has happened is a combination of the problem mentioned by stijn where the collect call was not freeing the deleted space when called immediately after. This has probably hidden a fix I made though experimenting and simplifying my code.

Thank you to everyone though you have all been very helpful.

[Minyiky]

Now comming across another memory issue, I seem to be having fragmentation issues, when swithing between screens I am checking mem_free and it is remaining pretty consistant with over 45kb free but after a few switches it will throw a memory error saying it is unable to allocate 2048bytes.

1) Am I correct in saying that it is an issue of fragmentation?

2) If so is there any way to resolve this?

[pythoncoder]

1. Almost certainly.
2. Only with a good deal of attention to detail. The first step is to read this doc.