Unexpected behavior on memory allocation

[Roberthh]

When setting up a test regarding the discussion here: http://forum.micropython.org/viewtopic.php?f=3&t=1328
I ran into an unexpected behavior on WiPy. I wrote a little test program which allocates memory. That should run into an exception at a certain point. With PyBoard, it does. With WiPy, it does NOT. WiPy simply stalls. No reaction on Telnet, not heartbeat LED.
This is the test code:

Code: Select all

import gc, micropython
micropython.alloc_emergency_exception_buf(400)

def alloctest(n):
    gc.collect()
    list = []
    for i in range(n):
        try:
            list.append("{} *************************************************".format(i))
        except:
            print("Exception at count ", i)
            return
    del list
    gc.collect()
    print("Fine run of {} cycles, free memory {}".format(n, gc.mem_free()))

This is the output from Pyboard:

Code: Select all

MicroPython v1.5.2-89-g7417ccf on 2016-01-29; PYBv1.0 with STM32F405RG
>>> import alloctest
>>> alloctest.alloctest(100)
Fine run of 100 cycles, free memory 85296
>>> alloctest.alloctest(800)
Exception at count  726

And this the output from WiPy:

Code: Select all

MicroPython v1.5.2-97-gef5f266 on 2016-01-30; WiPy with CC3200
>>> import alloctest
>>> alloctest.alloctest(100)
Fine run of 100 cycles, free memory 41968
>>> alloctest.alloctest(400)

After the last line, WiPy stalls, so there is no response. The Code was a little bit longer before, because initially I wanted to test, whether after the statement "del list" memory is freed again. B.T-W.: not surprisingly, Linux micropython also behaves as expected, failing at count 12118.
Regards

[danicampora]

Hi!

What version of the WiPy software are you running?

Code: Select all

import os
os.uname()

Cheers,
Daniel

[Roberthh]

os.uname() results in:

>>> os.uname()
(sysname='WiPy', nodename='WiPy', release='1.1.1', version='v1.5.2-97-gef5f266 on 2016-01-30', machine='WiPy with CC3200')

Regards

[Roberthh]

I have a variant of the test script, which runs into an exception and gets back to the REPL prompt, which is however not usable any more. Any Python command I enter causes an error.

Code: Select all

import micropython, machine
micropython.alloc_emergency_exception_buf(400)

def alloctest(n):
    stars="*************************************************"
    list = []
    for i in range(n):
        list.append(str(i)+stars[:(machine.rng() % len(stars)) + 1])
    print("Fine run")

Test run:

Code: Select all

PYB: soft reboot
MicroPython v1.5.2-73-g42a6364 on 2016-01-24; WiPy with CC3200
Type "help()" for more information.
>>> import alloctest
>>> alloctest.alloctest(500)
Fine run
>>> alloc.alloctest(550)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "alloc.py", line 9, in alloctest
MemoryError: memory allocation failed, allocating 4096 bytes
>>> dir
Traceback (most recent call last):
  File "<stdin>", line 1
MemoryError: parser could not allocate enough memory
>>> 

[dhylands]

If you enter Control-D that should do a soft-reset and reinitialize the heap.

[Roberthh]

That's obvious, and Ctrl-D works as long as there is a REPL prompt.
But the topic is more the initial script, where WiPy stalls (no prompt, no heartbeat) and can be revived only by a physical reset.
Regards

[Roberthh]

Maybe I was not clear in my second example;
What I found interesting is, that WiPy stalls if like in the first example all list elements have the same length (at least from element 100 on), but raises an exception if the length of the list elements vary. In the second example I used the rng to create varying length records.
That's funny, isn't it?

[Damien]

It could be down to the fact that WiPy has a different size heap to other ports and this size just by coincidence means that a memory error is not detected properly. To test this you can try running unix port with different sized heaps.

[dhylands]

I think that there is a bug somewhere. I hit a hard fault with one particular test I was doing, but it stopped happening after a rebuild. When I was hitting it I could change the number of iterations and sometimes get a memory exception. So not readily repriducible.

[Roberthh]

Hello Folks,
just a few bits more of information. Using micropython on Windows and Linux woith reduced heapsize (50000), I get the exception pretty fast. But I've seen something else: without heapsize reduction (heap ~ 1M) the loop count up to 8552. But it takes very long to raise the exception. The interesting stuff is, that up to that limit of 8552, the functions end almost immediately, within less than a second. But when I go beyond that limit, let's say to 8553, it takes about 20 seconds to get the exception message. I looks like the garbage collector is desperately trying to find an empty slot and gives in after a while. That's my test code:

Code: Select all

import gc
def alloctest(n):
    gc.collect()
    list = []
    for i in range(n):
        try:
            list.append(str(i) + " *************************************************")
        except:
            print("Exception at count ", i)
            return
    del list
    print("Fine run of", n, "cycles")

Interestingly, if I comment out the 'gc.collect()' and 'del list' statements, the behavior gets more irregular. If the gc.collect() is omitted, sometimes it event returns immediately (at the high count) with the memory error exception.