Unexpected behavior on memory allocation

Questions and discussion about The WiPy 1.0 board and CC3200 boards.
Target audience: Users with a WiPy 1.0 or CC3200 board.
Post Reply
User avatar
Roberthh
Posts: 2759
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Unexpected behavior on memory allocation

Post by Roberthh » Sat Jan 30, 2016 7:52 am

When setting up a test regarding the discussion here: http://forum.micropython.org/viewtopic.php?f=3&t=1328
I ran into an unexpected behavior on WiPy. I wrote a little test program which allocates memory. That should run into an exception at a certain point. With PyBoard, it does. With WiPy, it does NOT. WiPy simply stalls. No reaction on Telnet, not heartbeat LED.
This is the test code:

Code: Select all

import gc, micropython
micropython.alloc_emergency_exception_buf(400)

def alloctest(n):
    gc.collect()
    list = []
    for i in range(n):
        try:
            list.append("{} *************************************************".format(i))
        except:
            print("Exception at count ", i)
            return
    del list
    gc.collect()
    print("Fine run of {} cycles, free memory {}".format(n, gc.mem_free()))
This is the output from Pyboard:

Code: Select all

MicroPython v1.5.2-89-g7417ccf on 2016-01-29; PYBv1.0 with STM32F405RG
>>> import alloctest
>>> alloctest.alloctest(100)
Fine run of 100 cycles, free memory 85296
>>> alloctest.alloctest(800)
Exception at count  726
And this the output from WiPy:

Code: Select all

MicroPython v1.5.2-97-gef5f266 on 2016-01-30; WiPy with CC3200
>>> import alloctest
>>> alloctest.alloctest(100)
Fine run of 100 cycles, free memory 41968
>>> alloctest.alloctest(400)

After the last line, WiPy stalls, so there is no response. The Code was a little bit longer before, because initially I wanted to test, whether after the statement "del list" memory is freed again. B.T-W.: not surprisingly, Linux micropython also behaves as expected, failing at count 12118.
Regards

User avatar
danicampora
Posts: 342
Joined: Tue Sep 30, 2014 7:20 am
Contact:

Re: Unexpected behavior on memory allocation

Post by danicampora » Sat Jan 30, 2016 9:25 am

Hi!

What version of the WiPy software are you running?

Code: Select all

import os
os.uname()
Cheers,
Daniel

User avatar
Roberthh
Posts: 2759
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Unexpected behavior on memory allocation

Post by Roberthh » Sat Jan 30, 2016 11:57 am

os.uname() results in:

>>> os.uname()
(sysname='WiPy', nodename='WiPy', release='1.1.1', version='v1.5.2-97-gef5f266 on 2016-01-30', machine='WiPy with CC3200')

Regards

User avatar
Roberthh
Posts: 2759
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Unexpected behavior on memory allocation

Post by Roberthh » Sat Jan 30, 2016 7:13 pm

I have a variant of the test script, which runs into an exception and gets back to the REPL prompt, which is however not usable any more. Any Python command I enter causes an error.

Code: Select all

import micropython, machine
micropython.alloc_emergency_exception_buf(400)

def alloctest(n):
    stars="*************************************************"
    list = []
    for i in range(n):
        list.append(str(i)+stars[:(machine.rng() % len(stars)) + 1])
    print("Fine run")
Test run:

Code: Select all

PYB: soft reboot
MicroPython v1.5.2-73-g42a6364 on 2016-01-24; WiPy with CC3200
Type "help()" for more information.
>>> import alloctest
>>> alloctest.alloctest(500)
Fine run
>>> alloc.alloctest(550)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "alloc.py", line 9, in alloctest
MemoryError: memory allocation failed, allocating 4096 bytes
>>> dir
Traceback (most recent call last):
  File "<stdin>", line 1
MemoryError: parser could not allocate enough memory
>>> 

User avatar
dhylands
Posts: 3669
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Unexpected behavior on memory allocation

Post by dhylands » Sat Jan 30, 2016 7:24 pm

If you enter Control-D that should do a soft-reset and reinitialize the heap.

User avatar
Roberthh
Posts: 2759
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Unexpected behavior on memory allocation

Post by Roberthh » Sat Jan 30, 2016 7:28 pm

That's obvious, and Ctrl-D works as long as there is a REPL prompt.
But the topic is more the initial script, where WiPy stalls (no prompt, no heartbeat) and can be revived only by a physical reset.
Regards

User avatar
Roberthh
Posts: 2759
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Unexpected behavior on memory allocation

Post by Roberthh » Mon Feb 01, 2016 7:52 pm

Maybe I was not clear in my second example;
What I found interesting is, that WiPy stalls if like in the first example all list elements have the same length (at least from element 100 on), but raises an exception if the length of the list elements vary. In the second example I used the rng to create varying length records.
That's funny, isn't it?

Damien
Site Admin
Posts: 642
Joined: Mon Dec 09, 2013 5:02 pm

Re: Unexpected behavior on memory allocation

Post by Damien » Tue Feb 02, 2016 8:08 pm

It could be down to the fact that WiPy has a different size heap to other ports and this size just by coincidence means that a memory error is not detected properly. To test this you can try running unix port with different sized heaps.

User avatar
dhylands
Posts: 3669
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Unexpected behavior on memory allocation

Post by dhylands » Tue Feb 02, 2016 10:38 pm

I think that there is a bug somewhere. I hit a hard fault with one particular test I was doing, but it stopped happening after a rebuild. When I was hitting it I could change the number of iterations and sometimes get a memory exception. So not readily repriducible.

User avatar
Roberthh
Posts: 2759
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Unexpected behavior on memory allocation

Post by Roberthh » Wed Feb 03, 2016 7:59 am

Hello Folks,
just a few bits more of information. Using micropython on Windows and Linux woith reduced heapsize (50000), I get the exception pretty fast. But I've seen something else: without heapsize reduction (heap ~ 1M) the loop count up to 8552. But it takes very long to raise the exception. The interesting stuff is, that up to that limit of 8552, the functions end almost immediately, within less than a second. But when I go beyond that limit, let's say to 8553, it takes about 20 seconds to get the exception message. I looks like the garbage collector is desperately trying to find an empty slot and gives in after a while. That's my test code:

Code: Select all

import gc
def alloctest(n):
    gc.collect()
    list = []
    for i in range(n):
        try:
            list.append(str(i) + " *************************************************")
        except:
            print("Exception at count ", i)
            return
    del list
    print("Fine run of", n, "cycles")
Interestingly, if I comment out the 'gc.collect()' and 'del list' statements, the behavior gets more irregular. If the gc.collect() is omitted, sometimes it event returns immediately (at the high count) with the memory error exception.

Post Reply