Multicore FIFO
Multicore FIFO
Hello everyone,
I am wondering if we can access the multicore FIFO so thread one on core 0 could send data to thread 2 on core 1?
Basically, the equivalent of the C++ SDK (here) multicore_fifo_push_blocking() and multicore_push_pop_blocking() would do the job but I cannot find anything similar in the MicroPython documentation.
Thank you very much,
Torpi
I am wondering if we can access the multicore FIFO so thread one on core 0 could send data to thread 2 on core 1?
Basically, the equivalent of the C++ SDK (here) multicore_fifo_push_blocking() and multicore_push_pop_blocking() would do the job but I cannot find anything similar in the MicroPython documentation.
Thank you very much,
Torpi
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Multicore FIFO
You may be breaking new ground with this one, but it's a problem well worth solving. It is easy to make a FIFO in Python but I guess the concern is about thread safety. Would it be possible to use a structure with inherent thread safety such as a ring buffer? Could something clever be done with the PIO?
[EDIT]
Why not use the lock object? The following seems to work, calculating Fibonacci numbers on one core and printing them on the other:
[EDIT]
Why not use the lock object? The following seems to work, calculating Fibonacci numbers on one core and printing them on the other:
Code: Select all
import _thread
from time import sleep_ms
lock = _thread.allocate_lock()
def other(d):
while True:
lock.acquire()
n = d[0]
l = d[1]
q = l[n] + l[n -1]
d[1].append(q)
d[0] += 1
lock.release()
sleep_ms(100)
def main():
d = [1, [1, 1]]
_thread.start_new_thread(other, (d,))
while True:
lock.acquire()
print(d)
lock.release()
sleep_ms(103)
main()
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Multicore FIFO
Hello and thank you pythoncoder!
I first thought of locking mechanisms but my understanding was that Queue() has a thread-safe implementation with all the required locking mechanism and I could basically pass a queue as an argument to the second thread:
In Main:
q = queue.Queue()
_thread.start_new_thread(second_core_thread, (q) )
while True:
msg = q.get()
In second_core_thread:
def writer(q):
queue.put(....)
What I was thinking is leveraging the two FIFO buffers of the RP2040 so I can squeeze out all the juice of the Pico. My understanding is that I need to go C/C++ for that since uPython does not yet offer it, is that a good assumption?
The image comes from this site
I first thought of locking mechanisms but my understanding was that Queue() has a thread-safe implementation with all the required locking mechanism and I could basically pass a queue as an argument to the second thread:
In Main:
q = queue.Queue()
_thread.start_new_thread(second_core_thread, (q) )
while True:
msg = q.get()
In second_core_thread:
def writer(q):
queue.put(....)
What I was thinking is leveraging the two FIFO buffers of the RP2040 so I can squeeze out all the juice of the Pico. My understanding is that I need to go C/C++ for that since uPython does not yet offer it, is that a good assumption?
The image comes from this site
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Multicore FIFO
As far as I know that is true. You may achieve results by doing register level coding in Python using machine.mem32 and uctypes.addressof.My understanding is that I need to go C/C++ for that since uPython does not yet offer it, is that a good assumption?
I'd suggest that it is only a subset of Python applications which will receive a significant performance boost using hardware FIFOs compared to a shared data structure. The golden rule of any optimisation is to identify where the bottleneck actually is.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Multicore FIFO
Thank you for your message @pythoncoder!
100% with you on this one.
.I'd suggest that it is only a subset of Python applications which will receive a significant performance boost using hardware FIFOs compared to a shared data structure. The golden rule of any optimisation is to identify where the bottleneck actually is
100% with you on this one.
Re: Multicore FIFO
That should work and I have had success using 'mem32' to directly interact with RP2040 registers, but I saw some odd behaviour when I tried that with the Inter-Core FIFO. Results were inconstant and varied depending on sequence of what was done and values pushed to the FIFO.pythoncoder wrote: ↑Wed Nov 03, 2021 11:01 amYou may achieve results by doing register level coding in Python using machine.mem32
For example, when nothing should be running on the second core, I am able to read two items with 'get', and was able to 'put' 15 items to the FIFO -
Code: Select all
Get test
Get 0
Get 0
Code: Select all
Blind Put test
Put 3203334144
Put 3203334145
Put 3203334146
Put 3203334147
Put 3203334148
Put 3203334149
Put 3203334150
Put 3203334151
Put 3203334152
Put 3203334153
Put 3203334154
Put 3203334155
Put 3203334156
Put 3203334157
Put 3203334158
Traceback (most recent call last):
File "<stdin>", line 59, in <module>
File "<stdin>", line 31, in put
Exception: put when not ready
Code: Select all
# intercore.py
from machine import mem32
# Register Base Addresses
SIO_BASE = 0xD0000000
# Processor ID
CPUID = SIO_BASE + 0x000
# Inter-core FIFO - 8 words deep
FIFO_STATUS = SIO_BASE + 0x050
FIFO_WR = SIO_BASE + 0x054
FIFO_RD = SIO_BASE + 0x058
FIFO_STATUS_ROE_BIT = 3 # Was read when empty - 1=Yes (sticky)
FIFO_STATUS_WOF_BIT = 2 # Was write when full - 1=Yes (sticky)
FIFO_STATUS_RDY_BIT = 1 # Can write to fIFO - 1=Yes
FIFO_STATUS_ANY_BIT = 0 # Can read from FIFO - 1=Yes
def rdy():
return (mem32[FIFO_STATUS] >> FIFO_STATUS_RDY_BIT ) & 1
def put(n):
if rdy():
mem32[FIFO_WR] = n
else:
raise Exception("put when not ready")
return n
def any():
return (mem32[FIFO_STATUS] >> FIFO_STATUS_ANY_BIT ) & 1
def get():
if any():
return mem32[FIFO_RD]
else:
raise Exception("get when none available")
if __name__ == "__main__":
n = 0xBEEF_0000
if True:
print("Blind Get test")
while True:
print("Get {:x}".format(get()))
elif True:
print("Get test")
while True:
if any():
print("Get {:x}".format(get()))
elif True:
print("Blind Put test")
while True:
print("Put {:x}".format(put(n)))
n += 1
elif True:
print("Put test")
while True:
if rdy():
print("Put {:x}".format(put(n)))
n += 1
else:
print("Put and Get test")
while True:
if rdy():
print("Put {:x}".format(put(n)))
n += 1
while any():
print("Get {:x}".format(get())
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Multicore FIFO
I'm getting odd results with dual core coding when accessing a hardware resource (in my case a UART). I think there's a bug lurking somewhere.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Multicore FIFO
I can confirm I am also getting odd results on my side.
Torpi
Torpi
Re: Multicore FIFO
A belated update. I created a C Extension Module which allows access to the inter-core FIFO using the standard Pico SDK methods, nothing complicated, nothing out of the ordinary -
Without anything explicitly started on the second core, it is behaving as oddly as it was when I tried via registers using 'mem32'; I can 'get' at least one zero, and I can 'put' more than eight items. When I 'put' things it seems I can 'get' more things back.
It all suggests to me something is interacting with the FIFO, something in MicroPython (1.17-220) or something from Pico SDK (1.3), or there is something broken somewhere.
Unless the core can read its own FIFO, and the hardware definition indicates it cannot, something on the other core must be running, but I can't see how it is, or where that would be being launched.
I set "#define MICROPY_PY_THREAD (0)" in 'mpconfigport.h' but same result. Looking through the '.dis' file I can't see anywhere anything is linking to "multicore_*", any indication MicroPython itself is using it.
I guess the next thing to do is see if a Pico SDK C program exhibits the same behaviour.
Update : Easier than I thought. Same odd behaviour so I'll chase that up on the Raspberry Pi forum.
Code: Select all
STATIC mp_obj_t core1_put(mp_obj_t int_obj) {
multicore_fifo_push_blocking(mp_obj_get_int(int_obj));
return mp_const_none;
}
MP_DEFINE_CONST_FUN_OBJ_1(core1_put_obj, core1_put);
Code: Select all
STATIC mp_obj_t core1_get(void) {
return mp_obj_new_int(multicore_fifo_pop_blocking());
}
MP_DEFINE_CONST_FUN_OBJ_0(core1_get_obj, core1_get);
It all suggests to me something is interacting with the FIFO, something in MicroPython (1.17-220) or something from Pico SDK (1.3), or there is something broken somewhere.
Unless the core can read its own FIFO, and the hardware definition indicates it cannot, something on the other core must be running, but I can't see how it is, or where that would be being launched.
I set "#define MICROPY_PY_THREAD (0)" in 'mpconfigport.h' but same result. Looking through the '.dis' file I can't see anywhere anything is linking to "multicore_*", any indication MicroPython itself is using it.
I guess the next thing to do is see if a Pico SDK C program exhibits the same behaviour.
Update : Easier than I thought. Same odd behaviour so I'll chase that up on the Raspberry Pi forum.
Re: Multicore FIFO
Mystery solved
As part of booting, Core 1 runs a program implementing a 'launch protocol' via the FIFO which allows something else to be launched on Core 1.
What I was seeing was interaction with that program, it reading the FIFO, filling the return FIFO, when I didn't think anything was or would be running on Core 1.
So, the lesson is, don't interact with the FIFO unless your own code is running on Core 1.
As part of booting, Core 1 runs a program implementing a 'launch protocol' via the FIFO which allows something else to be launched on Core 1.
What I was seeing was interaction with that program, it reading the FIFO, filling the return FIFO, when I didn't think anything was or would be running on Core 1.
So, the lesson is, don't interact with the FIFO unless your own code is running on Core 1.