Leveraging second core, code hangs

[torpi]

Hello everyone,

I am trying to leverage the second core of the Raspberry Pico but for some reason I cannot explain, the code hangs after few seconds. Calling the second loop directly instead of calling it with a thread on the second core works properly.

The second loop, should run 100 times but hangs at 2nd iteration. The loop gets 4000 random numbers as fast as possible, compare (I just created a small function for the sake of the example) and print the time it took to compute. Very basic.

I cannot understand what is happening there? Probably missing something very basic here, but what?

Have a lovely day,
Torpi

Code: Select all

from machine import Timer, Pin
import random
import time
import _thread

RNDBOUND = 2097152
smallWords = 0
bigWords = 0

pin = Pin(2, Pin.OUT)

def function_called_by_second_thread(word:int):
    global smallWords, bigWords, RNDBOUND
    if word < int(RNDBOUND / 2):
        smallWords += 1
    else:
        bigWords += 1


def second_thread():
    global smallWords, bigWords, RNDBOUND

    for j in range(100):
        start_time = time.ticks_ms()
        for i in range(4000):
            function_called_by_second_thread(random.randrange(RNDBOUND))

        print(f"time to compute: {time.ticks_diff(time.ticks_ms(), start_time)}")
        time.sleep_ms(500)


def toggleLED():
    global pin
    pin.toggle()


def main():
    # Doing something on the main thread
    tim = Timer(period=10, mode=Timer.PERIODIC, callback=lambda t: toggleLED())
    
    # Doing something on the second thread (calling second_thread() directly on the main thread works properly)
    _thread.start_new_thread(second_thread, ()) # This hangs after 2 iterations, why?


main()

[pythoncoder]

In my opinion using the second core is hazardous. I have raised this issue but I have observed crashing behaviour too. It can work with a very simple (yet computationally demanding) thread, but I do think there are bugs still lurking.

[torpi]

Hum. That is sad. The second core works really well for the same code with C/C++ implementation. So my take is that it's a problem in MicroPython, right?

[pythoncoder]

Yes. I'm sure it will be fixed before too long.

[manpowre]

replace int(RNDBOUND / 2) with this: RNDBOUND // 2 and it works.

Basically the code is creating 4k floats every iteration, and it floods the memory. by just calculate with integer as you used the int() function to force the division to int anyway, the double // does the trick as true integer calculation instead.

also its a good idea to implement forced garbage collection in threads, and this code can easily do a gc.collect() before the time.sleep(500). I did first try this with the float calculation, but it wouldnt garbage collect the objects, so theres something with floats and garbage collector.

I enjoyed the debugging on this after I woke up today.. seems like micropython has an issue with floats in threads ?

I kept looking at this, so I forced the deletion of those floats and now its working with those floats. gc definetly not removing those floats:

Code: Select all

from machine import Timer, Pin
import random
import time
import _thread
import gc
gc.enable()

RNDBOUND = 2097152
smallWords = 0
bigWords = 0

pin = Pin(2, Pin.OUT)

def function_called_by_second_thread(word:int):
    global smallWords, bigWords, RNDBOUND
    c=RNDBOUND / 2
    b=int(c)
    if word < b:
        smallWords += 1
    else:
        bigWords += 1
    del b,c

def second_thread():
    global smallWords, bigWords, RNDBOUND

    for j in range(100):
        start_time = time.ticks_ms()
        for i in range(4000):
            #print(i)
            function_called_by_second_thread(random.randrange(RNDBOUND))

        print(f"time to compute: {time.ticks_diff(time.ticks_ms(), start_time)}")
        gc.collect()
        #time.sleep_ms(500)


def toggleLED():
    #global pin
    #pin.toggle()
    print(gc.mem_free())


def main():
    # Doing something on the main thread
    #tim = Timer(period=10, mode=Timer.PERIODIC, callback=lambda t: toggleLED())
    
    # Doing something on the second thread (calling second_thread() directly on the main thread works properly)
    _thread.start_new_thread(second_thread, ()) # This hangs after 2 iterations, why?
    while True:
        toggleLED()
        time.sleep(1)

main()

[manpowre]

I did recompile the firmware as armv6 for my arduino rp2040 connect instead of armv7m found in cmakelists.txt (manifest) but same happened.

then I set the threshold to 4096, and the gc collects automatically those floats.

Code: Select all

from machine import Timer, Pin
import random
import time
import _thread
import gc
gc.enable()
gc.threshold(4096)

RNDBOUND = 2097152
smallWords = 0
bigWords = 0

pin = Pin(2, Pin.OUT)

def function_called_by_second_thread(word:int):
    global smallWords, bigWords, RNDBOUND
    if word < int(RNDBOUND / 2):
        smallWords += 1
    else:
        bigWords += 1

def second_thread():
    global smallWords, bigWords, RNDBOUND

    for j in range(100):
        start_time = time.ticks_ms()
        for i in range(4000):
            #print(i)
            function_called_by_second_thread(random.randrange(RNDBOUND))

        print(f"time to compute: {time.ticks_diff(time.ticks_ms(), start_time)}")
        
        #gc.collect()
        #time.sleep_ms(500)


def toggleLED():
    #global pin
    #pin.toggle()
    print(gc.mem_free())


def main():
    # Doing something on the main thread
    #tim = Timer(period=10, mode=Timer.PERIODIC, callback=lambda t: toggleLED())
    
    # Doing something on the second thread (calling second_thread() directly on the main thread works properly)
    _thread.start_new_thread(second_thread, ()) # This hangs after 2 iterations, why?
    while True:
        toggleLED()
        time.sleep(1)

main()

[pythoncoder]

I did recompile the firmware as armv6 for my arduino rp2040 connect instead of armv7m found in cmakelists.txt (manifest)...

If the build system specifies the wrong arch, that's a bug that should be reported. However, if that were the case surely floating point operations wouldn't work at all on rp2 because V7 has hardware FP. Whereas in fact FP works fine.

Aside from that the code is standard Python and should run without tweaks to GC. It might be worth writing a simple test case so it can be reported.

[manpowre]

the threshold is set to -1 when I print out the default threashold. the highest threashold I could use before it crashes is gc.threshold(5423).
It crashes at 5424.

It could be as simple as the rp2 port doesnt have default threashold set.

[Zoot]

I believe that Floats are heap-allocated objects unlike (machine sized) ints, so computing new Float values involves doing heap allocations which will not be safe in a multi-threaded context.

There's a docs section on writing interrupt handlers which have similar restrictions as an interrupt can occur on any machine cycle which could be in the middle of updating a python object (maintaining a list say), doing object allocation or deallocation on the heap, etc. and it's worth a read through as most of these restrictions will probably apply to using the second core.

https://docs.micropython.org/en/latest/ ... rules.html

[samneggs]

Get rid of the print(f that allocates runtime memory. Convert it to a simple print()


I've been fairly successful using _thread with some severe compromises. I pre-allocate whatever memory/objects I need and I avoid garbage collection once my repetitive loop starts. That means no floating point, string concatenation, any functions that create objects, etc.
The test is to put gc.mem_free() in the main loop and it should not decrease. It's not always fun but it works for many things.
Sam