Wifi bridge for RPI Pico

RP2040 based microcontroller boards running MicroPython.
Target audience: MicroPython users with an RP2040 boards.
This does not include conventional Linux-based Raspberry Pi boards.
kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: Wifi bridge for RPI Pico

Post by kevinkk525 » Fri Feb 05, 2021 11:07 am

Yeah I didn't use standard pins with the hardware SPI, so maybe there's a bug in there. But for this project it makes no difference, SoftSPI at 8MHz works just fine.

Exactly, we need basic socket support but not neccessarily controlling esp32 pins. If someone wants that, it can easily be added later, PR are always welcome :D

CRC will definitely come last once everything else works well enough. It looks like the protocol uses a single bit for some kind of checksum but I'll check that later. Resetting the ESP is part of the protocol. The host resets the ESP whenever it starts and it could be reset if it doesn't react.
If the ESP crashes, then the sockets won't be available anymore. Therefore a request to read from those sockets will result in an error which should be raised in the same way a normal socket raises those errors, e.g. a non-blocking socket would return b'' (or raise an OSError?). The user application can then handle that situation like every other case of a broken connection.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: Wifi bridge for RPI Pico

Post by kevinkk525 » Sat Feb 06, 2021 5:38 pm

Update: After porting way too much I started testing more than just the communication and getting the current time and what can I say.. I just can't get sockets working.. they just won't connect to anything and I'm out of ideas now.. The Nina-FW Esp32 just doesn't return what is expected. But debugging that is going to be difficult as I'd have to fully understand their C++ code.
With so much effort I could just as well write a micropython version of nina-fw for the esp32 using UART instead of SPI..

If you know a library for resilient UART communication between 2 devices, let me know. I don't need to reinvent the wheel if I don't have to.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Wifi bridge for RPI Pico

Post by pythoncoder » Sun Feb 07, 2021 8:03 am

UARTS are inherently no more resilient than SPI but they are much easier to use because of their proven support in C. Their lack of resilience is for exactly the same reason: if the transmitter crashes, the receiver continues to clock in garbage. However this is easily fixed with a CRC. As you have pointed out, this is a feature which can be added later.

I like the idea of an all-Python solution. Good luck!
Peter Hinch
Index to my micropython libraries.

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: Wifi bridge for RPI Pico

Post by kevinkk525 » Thu Feb 11, 2021 11:50 am

I do have a working minimal codebase now between 2 esp32 over uart but I'm wondering about the performance.
I might have overdone the protocol and the processing time for optimizations takes longer than just sending the frames without optimizations :lol: It was a premature decision coming from ardunio but can be changed later.
I was wondering about the general speed, currently it takes ~50ms to send something over the socket (8 Byte or 400Bytes makes 6ms difference) and 50-80ms to receive 8Bytes and 400Bytes. (Native that would only take 2-6ms)

What do you think about it? future improvements will get rid of execution time for trying to receive data from a socket that has no data yet so all that will be left is the pure time for sending and receiving actual data but still sounds rather slow to me.

If you want to peek at the code (far from done but basic socket works without any optimizations or buffering, so every command gets executed at the host, which takes time): https://github.com/kevinkk525/micropython_wlan_link
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Wifi bridge for RPI Pico

Post by pythoncoder » Thu Feb 11, 2021 2:15 pm

Interesting. You've evidently been busy. I can't claim to fully understand it but it looks promising.

I'm not sure what to say about performance. In many applications (e.g. logging outside temperatures or air pressure) it really doesn't matter. I'm also doubtful about talking to a remote broker, especially with TLS: do we know what would be the likely turnround for a PUBACK?

However you're building a general purpose socket, so ideally it wants to be as fast as possible

I guess your own application is MQTT remote control with a local broker where a fast turnround matters. Realistically, though, would a 200ms delay matter? I tested mqtt_as with a local broker, using one ESPx to control an LED on another. There was visible (but IMO acceptable) latency between pressing the button and seeing the LED change state. Adding a UART link can only add to that.

Re the recent discussion about CRC: in this application I'd use the fastest, simplest algorithm you can find. If the ESP32 reboots in the middle of a transfer, the received data will be complete garbage. It is possible that the receiving UART will lock up completely waiting on a start bit in which case (hopefully) a timeout will trigger. I guess this behaviour needs testing on the Pico. In any event CRC is looking for gross errors not for a single bit. If the UART doesn't behave properly I gather there is a UART based on the PIO which could be adapted.

I like your profiler by the way ;)

I have ported my MQTT-only solution to the Pico. I wrote it four years ago for Pyboard 1.x: it turned out that there was quite a lot of work required to make the client portable and to use uasyncio V3 effectively. It used my workrounds for the bugs in uasyncio V2 and the client API was a bit of a dog's breakfast. I put all the effort into the ESP8266 code and didn't really give enough attention to the API. But the ESP8266 code was OK and is almost unchanged. The client is hopefully much improved. It worked first time when I put it on the Pico. I believe it could be ported to other hosts with similar ease.

So that is an option for those wanting MQTT only. One advantage is that the resilience of the ESP8266 solution is already proven.
Peter Hinch
Index to my micropython libraries.

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: Wifi bridge for RPI Pico

Post by kevinkk525 » Thu Feb 11, 2021 2:41 pm

Yeah I was more busy than I expected :D

Yes in many applications the performance might not be too important. In general I'd say the current state will add 50-100ms to communication. Haven't tested it with mqtt_as yet because in the current state there is one major problem: With non-blocking sockets you read every 20ms which currently is each a call to the host taking 50ms so that's not good :D But I already know how to solve that and it'll be fine. I'll test it then.

I'm more concerned about sending, maybe you have an idea. Mqtt_as e.g. has 4 write instructions per publish, sometimes just a single byte. Currently every instruction would be sent to the host resulting in a simple publish needing like 200ms only for the communication between client and host.
I don't know how to buffer those writes unless I use an interrupt driven timer to send them every 5ms if data is waiting. But this has 2 downsiedes: 1) I return True but don't know if that data will actually be sent, which is ok with non-blocking sockets but not with blocking socket. 2) That would also mean that I add a background task to micropython that uses up to 20-50ms. That would crazily noticeable in every application. By using more interrupts (or uasyncio :D) I could drive that time down to maybe 10-20ms but that is still noticeable. It might be acceptable but it's not transparent to the application because it will occur e.g. 5ms after you write to the socket.

You're right about crc. Current implementation takes at most 1ms. All that profiling adds some overhead so maybe 500us. Could make it faster but doesn't seem to be the slowest part of my code. But could definitely be faster and less complex, detecing errors on uart should be rather easy.

Generally this might not be a very fast solution but I guess if you need a fast wlan solution you will probably pick a board with integrated wlan?
pythoncoder wrote:
Thu Feb 11, 2021 2:15 pm
I have ported my MQTT-only solution to the Pico.
Guess you've been rather busy yourself :D Thought you had retired that project. But it's good to see it alive and working!
Seeing your latency of about 250ms (according to your docs) makes me feel a little bit better but is expected with a software bitbanging communication.
pythoncoder wrote:
Thu Feb 11, 2021 2:15 pm
So that is an option for those wanting MQTT only. One advantage is that the resilience of the ESP8266 solution is already proven.
True. For needing mqtt only this might be a nice solution. My library will depend on the host application to be resilient so should work just fine with mqtt_as but we'll see. It has to prove itself first :)
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

python4me2ever
Posts: 1
Joined: Sat Feb 13, 2021 12:41 pm

Re: Wifi bridge for RPI Pico

Post by python4me2ever » Sat Feb 13, 2021 1:21 pm

Hello All,
I have been a long time user of MicroPython and a first time poster to these forums!

I started with the ESP8266 implementation, followed by the ESP32 and now the RPi-PICO!

I have read this thread and I had a question about an alternative implementaton?

When I first started playing around with the ESP8266, I started with the original Espressif supported "AT" command set running in an ESP-01 module talking to some other MCU. Before to long, I ended up transitioning to the ESP8266 running LUA prior to the availability of MicroPython!

So my question to this group is why not consider using the Espressif "AT Command Set Over Serial port" firmware for either the ESP8266 or the ESP32? It seems to support everything that is desired and the "only" thing would need to be created is the sockets object wrapper (shim) for the AT protocol in MicroPython - as has benn stated, that will allow all existing libraries and user code to continue to work unmodified! This would open up WiFi connectivity to a whole set of MicroPython based systems that are running on hardware that do not have native networking hardware...

If it is possible, I think it gives MicroPython on the PICO the greatest amount of capability: It uses the serial port, it supports TCP/IP, WiFi (both STA and AP modes) and BLE (in the ESP32), it even supports OTA updates of the "AT command set" firmware (if desired)! And of course, the source code is available in Espressif's GITHUB repository!


Here are the latest copies of the command references for the ESP8266 and ESP32:
ESP8266:
https://www.espressif.com/sites/default ... set_en.pdf

ESP32:
https://www.espressif.com/sites/default ... les_en.pdf

Thoughts?

Thanks...

P.S.
The old "workhorse" ESP8266 based ESP-01 module is shown here for reference: https://en.wikipedia.org/wiki/ESP8266

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: Wifi bridge for RPI Pico

Post by kevinkk525 » Sat Feb 13, 2021 2:56 pm

Thanks for the suggestion. The though crossed my mind and I took a look at the documentation you linked. It's not quite easy to implement either using a compatible socket object. Would also need to create buffers and receive the data interrupt driven as the AT firmware just sends whatever it receives. So basically on the client side it's just as much work writing code to support the AT firmware as it is with a custom firmware, but it needs more time to understand their protocol. Moreover it might be more complex due to all the interpretation of their string commands. On the other hand the communication should be faster using their firmware since it's written in C++ and not in micropython. But I like to be able to expand on the firmware in micropython and it makes it easier to have a fully compatible wifi module since the host runs the same commands as the client and returns the same output. With the AT firmware I'd always have to see how I can make their output compatible to the expected output of the existing network and socket modules, which could be a considerable amount of work too.

So to sum it up: The amount of effort needed might be rather similar but I'm not sure the advantages outweigh the disadvantages.
It certainly is an option, but none I'll be following.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

frustr8dlec
Posts: 2
Joined: Fri Apr 09, 2021 2:25 pm

Re: Wifi bridge for RPI Pico

Post by frustr8dlec » Fri Apr 09, 2021 2:45 pm

Hi Sorry to hijack the topic slightly

Peter is there any possibility that you would be able to clarify if an ESP-01 module would be useable with your MQTT system, I have downloaded the scripts flashed the binary to the ESP-01 and setup the network params. the ESP is wired to the pico GPIO pins identified in the script plus 3.3 and GND but I am just getting SYNCOMM timed out.

'reset' : Pin(16, Pin.OPEN_DRAIN), to ESP-01 RST
'stx' : Pin(17, Pin.OUT), to ESP-01 RX
'sckout' : Pin(18, Pin.OUT, value = 0), Is this possible to be ESP-01 IO2
'srx' : Pin(19, Pin.IN), ESP-01 to ESP-01 TX
'sckin' : Pin(20, Pin.IN), Is this possible to be ESP-01 IO0
'led' : Pin(25, Pin.OUT),

When looking at the bridge code it is referring to gpio
mtx = Pin(14, Pin.OUT) # Define pins
mckout = Pin(15, Pin.OUT, value=0) # clocks must be initialised to 0
mrx = Pin(13, Pin.IN)
mckin = Pin(12, Pin.IN)

ESP-01 looks to expose IO2 and IO0 plus rx and tx

I am pretty sure I am not following things correctly and it most likely a lack of understanding on my part. I am happy to purchase additional hardware if the 01 is not appropriate. I am trying to get this working for my students to create some MQTT labs around the Pico. Your help identifying the correct wiring and implementation would be very gratefully received. I would also be happy to contribute to the documentation on the GitHub repository once I have it sorted.

gregwa
Posts: 1
Joined: Thu Mar 25, 2021 4:01 pm

Re: Wifi bridge for RPI Pico

Post by gregwa » Sun Apr 11, 2021 9:24 am

I just wanted to thank everyone for this topic. It helped me a lot.

I was able to get the Wifi bridge working correctly on the Pico to an NodeMCU8266 and it worked correctly "out of the box".

I too am frustrated by the lack of a true networking solution for MicroPython on the Pico. Yes, there is no hardware for doing that, so on one hand I understand the reluctance for the MicroPython for Pico team to invest the time into some type of socket library. On the other hand, as it has been presented here, it is something that would be helpful.

I have tried the CircuitPython to AirLift solution and it does work and seems to work pretty well. But as it has been pointed out, the Airlift has received custom firmware.

Again, I appreciate all the help this topic has given me.

Greg

Post Reply