Pico W Micro Python MQTT of data to a RP4B

RP2040 based microcontroller boards running MicroPython.
Target audience: MicroPython users with an RP2040 boards.
This does not include conventional Linux-based Raspberry Pi boards.
Post Reply
Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Sun Aug 14, 2022 5:14 pm

As some will be aware, I was initially struggling with getting some good data out of a Temp/Humidity sensor I bought for my new Pico W.
If you're not aware, but interested, you can find that thread here: viewtopic.php?f=21&t=12900

The end result of that, with LOTS of very good help from "Roberthh" (Thank you very much for that Roberthh!) is that I am now getting useful data from that sensor.

Now on to the next challenge!

Now, I want to transmit these Temperature and Humidity readings to my Raspberry Pi 4B (RP4B) elsewhere in the house, using my indoor Wi-Fi.

After many hours yesterday, I managed to cobble together some code from the official Pico W guidance which allows for me to switch on the Pico W Wi-Fi and connect it to my router with a DHCP IP address. I've yet to figure out how to make it a fixed IP address, but to be honest i've also got that same issue with mt RP4B.

I believe now I need to have my Pico W and my RP4B set up together with something called "MQTT" to enable me to transmit my Pico W data and receive it at my RP4B.

I believe that my Pico W has to be a "Publisher" and my RP4B has to be a "Broker" AND a "Subscriber". That's all I know.

Can I ask for guidance on what route to take to achieve my goal?

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Mon Aug 15, 2022 10:04 pm

So i've made some headway on MQTT today.

I've installed umqtt simple on to my Pico W, and Mosquitto with config file adjustment on to my RP4B. My mobile phone with an appropriate app (MQTTools) recognises my RP4B as a Broker now, which is good. But i'm not having success in trying to send my temperature/humidity measurments sent to my Broker. I've had various differerent errors according to trying to adjust the decimal formatted readings into something which the MQTT code will pass on to the broker. I've done some reading on the internet based on the different errors i've had, and even the solutions for this (as others have clearly had this same issue), but the simple fact is, I don't understand the solutions either.

Can someone help me understand what I need to do to fix my issue?

This is an excerpt of the code where i'm having issues. I basically juts want to send "temperature", "humidity" and "picotemp" via MQTT to my RP4B Broker.

The data for each of these takes this raw format (float?):
Temperature: 26.81049
Humidity: 46.69566
Pico W Temperature: 29.85327

No matter what manipulation I try on these, I can't make them be sent using these excerpts as my example code:

Code: Select all

    tpm1 = round(temperature,2)
    tpm2 = round(humidity,2)
    tpm3 = round(picotemp,2)
    
    tpm4 = tpm1*100
    tpm5 = tpm2*100
    tpm6 = tpm3*100
    
    tpm7 = int(tpm4)
    tpm8 = int(tpm5)
    tpm9 = int(tpm6)
    
    topic_msg1 = len(str(tpm7))
    topic_msg2 = len(str(tpm8))
    topic_msg3 = len(str(tpm9))
    
    client.publish(topic_pub, topic_msg1, topic_msg2, topic_msg3)
For the above, I get:

Traceback (most recent call last):
File "<stdin>", line 162, in <module>
File "/lib/umqtt/simple.py", line 112, in publish
TypeError: object of type 'int' has no len()

But i have also had the following too (through different manipulation of the raw floats in attempts to make them be "acceptable":

Traceback (most recent call last):
File "<stdin>", line 151, in <module>
File "/lib/umqtt/simple.py", line 112, in publish
TypeError: object of type 'int' has no len()

Traceback (most recent call last):
File "<stdin>", line 143, in <module>
TypeError: function expected at most 1 arguments, got 2

Traceback (most recent call last):
File "<stdin>", line 151, in <module>
File "/lib/umqtt/simple.py", line 111, in publish
TypeError: unsupported types for __lshift__: 'float', 'int'


Ultimately, at my subscriber end to this data, I want to be able to get decimal formatted numbers like this (according to the raw floats above):

Temperature: 26.8
Humidity: 46.7
Pico W Temperature: 29.9


Please help me if you know how. Thanks in advance.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Pico W Micro Python MQTT of data to a RP4B

Post by pythoncoder » Tue Aug 16, 2022 8:39 am

The msg parameter for publish should be a bytes object:

Code: Select all

msg = f"{tpm7},{tpm8},{tpm9}".encode()
client.publish(topic_pub, msg)
This will send the values separated by commas, as per

Code: Select all

>>> a = 101
>>> b = 77
>>> c = 88
>>> f"{a},{b},{c}".encode()
b'101,77,88'
Peter Hinch
Index to my micropython libraries.

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Tue Aug 16, 2022 3:07 pm

pythoncoder wrote:
Tue Aug 16, 2022 8:39 am
The msg parameter for publish should be a bytes object:

Code: Select all

msg = f"{tpm7},{tpm8},{tpm9}".encode()
client.publish(topic_pub, msg)
This will send the values separated by commas, as per

Code: Select all

>>> a = 101
>>> b = 77
>>> c = 88
>>> f"{a},{b},{c}".encode()
b'101,77,88'
Hi Peter, thanks again. Much appreciated having your feedback on my call for help.

So if I'm understanding your solution by using

Code: Select all

f
followed by

Code: Select all

""
with variables encased, within using

Code: Select all

{}
and separated by

Code: Select all

,
and then finishing off with

Code: Select all

.encode()
are we saying that this line is saying here's a bunch of floats, please convert these into bytes which are acceptable for transmission over MQTT?

Is it reasonable also/or alternatively to make three different publish topics and make three individual publish transmissions instead/too? Rather than only one message with the variable comma separated.

In this case, with three separate topics and transmissions, would I also still have to convert the floats to bytes as per your method (if this is what it's doing), or could I leave the variables to be sent as floats, or even convert these into 4 numbered integers after moving the decimal place (as per my original thoughts in my last post).

A colleague of mine says you can only MQTT string type variables and said "look at all the examples online, they are ALL strings. Nobody ever passes numbers let alone floats!".

Is this true?

I find this very hard to believe as most of the time it's numbers people are more interested in using rather than some text based message, surely!?

What I don't want to have, is any issue with using these variables (as decimalized numbers) at the receiving end of the transmission.

I know that's a lot to lay on you, but I'm just trying to grow my understanding and I have a hard time understanding these complicated instruction manuals due to not being up on all the lingo and understanding at your sort of level.

DeaD_EyE
Posts: 19
Joined: Sun Jul 17, 2022 12:57 pm
Contact:

Re: Pico W Micro Python MQTT of data to a RP4B

Post by DeaD_EyE » Tue Aug 16, 2022 3:57 pm

A colleague of mine says you can only MQTT string type variables and said "look at all the examples online, they are ALL strings. Nobody ever passes numbers let alone floats!".
Yes. The implementation from paho.mqtt requires for publishing messages one str for the Topic (which is converted to bytes) and bytes for the payload. With micropython-umqtt the payload could also be a str. I don't know why, but it's easier to use.
Otherwise, you have to convert the float to str and then encode the str, which returns bytes.


If you want to send 3 floats, you can use the hierarchy to differentiate which float is which sensor.

For example, you choose these topics for each float:
Float1: /temp/sensor1
Float2: /temp/sensor2
Float3: /temp/sensor3


Then publishing them:

Code: Select all

from umqtt.robust2 import MQTTClient

mqtt = MQTTClient("ESP32", "192.168.10.130", keepalive=60)
mqtt.connect()


def publish_sensor_values(v1, v2, v3):
    mqtt.publish("/temp/sensor1", str(v1))
    mqtt.publish("/temp/sensor2", str(v2))
    mqtt.publish("/temp/sensor3", str(v3))

publish_sensor_values(1.0, 42.0, 133.7)
Or you just send 3 floats and on the receiver side you deconstruct it.
The cool feature of MQTT is the hirarchy. You can listen for /temp/+ and you'll receive only /temp/xxx but not /temp/xxx/xxx.
The # is like the wildcard. Take everything what comes after the slash.

If you want to listen to all /temp/xxx values, then subscribe:

Code: Select all

/temp/+
With mosquitto_sub on Broker/PC-Side

Code: Select all

mosquitto_sub -t "/temp/+" -F %j --pretty
This will print prettyfied json strings for each message.

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Tue Aug 16, 2022 5:09 pm

DeaD_EyE wrote:
Tue Aug 16, 2022 3:57 pm
A colleague of mine says you can only MQTT string type variables and said "look at all the examples online, they are ALL strings. Nobody ever passes numbers let alone floats!".
Yes. The implementation from paho.mqtt requires for publishing messages one str for the Topic (which is converted to bytes) and bytes for the payload. With micropython-umqtt the payload could also be a str. I don't know why, but it's easier to use.
Otherwise, you have to convert the float to str and then encode the str, which returns bytes.
oh right. That sounds complicated an not uniformly standard propositions across the different toolsets for MQTT. It's no wonder I get lost all the time from one topic (general coding topic, not an MQTT topic!) to another! So you're saying, that although Peter is getting me to convert my floats into byte format; these will still be transmitted as strings? So confused.
DeaD_EyE wrote:
Tue Aug 16, 2022 3:57 pm
If you want to send 3 floats, you can use the hierarchy to differentiate which float is which sensor.

For example, you choose these topics for each float:
Float1: /temp/sensor1
Float2: /temp/sensor2
Float3: /temp/sensor3


Then publishing them:

Code: Select all

from umqtt.robust2 import MQTTClient

mqtt = MQTTClient("ESP32", "192.168.10.130", keepalive=60)
mqtt.connect()


def publish_sensor_values(v1, v2, v3):
    mqtt.publish("/temp/sensor1", str(v1))
    mqtt.publish("/temp/sensor2", str(v2))
    mqtt.publish("/temp/sensor3", str(v3))

publish_sensor_values(1.0, 42.0, 133.7)
Or you just send 3 floats and on the receiver side you deconstruct it.
The cool feature of MQTT is the hirarchy. You can listen for /temp/+ and you'll receive only /temp/xxx but not /temp/xxx/xxx.
The # is like the wildcard. Take everything what comes after the slash.

If you want to listen to all /temp/xxx values, then subscribe:

Code: Select all

/temp/+
Ah right, this is clever. I'll try that!
DeaD_EyE wrote:
Tue Aug 16, 2022 3:57 pm
With mosquitto_sub on Broker/PC-Side

Code: Select all

mosquitto_sub -t "/temp/+" -F %j --pretty
This will print prettyfied json strings for each message.
Okay..... ii'm not sure I get what you're saying here. "Prettyfied"? JSON? Once i've got the publisher side working (checking my phone with the MQTTool app), I'll be getting on to my RP4B Python side for trying to set that up to subscribe to my Pico output messages. So i'm sure lots more headaches are on the road ahead for me there too.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Pico W Micro Python MQTT of data to a RP4B

Post by pythoncoder » Tue Aug 16, 2022 5:20 pm

This

Code: Select all

f"{a},{b},{c}"
is a Python "f-string". If a, b and c are floats it converts them to a string. Note the commas between the pairs of curly braces. This ensures that the string has a comma between each float. You could equally well separate them with any other text string.

Code: Select all

f"a = {a} b = {b} c = {c}"
The .encode() converts the string to a bytes object: this deals with any possible Unicode problems.

MQTT topics and messages can only be strings. I am not sure how the official MicroPython libraries cope with Unicode so issuing .encode() is a precaution especially if you use non-ASCII characters. It will be transmitted as a string.
Peter Hinch
Index to my micropython libraries.

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Tue Aug 16, 2022 5:37 pm

Great success!!!! Woo-hoo!

Thank you to you guys for supporting me and helping me out, it's much appreciated!

It's working!

If only I could show you my phone screen with the MQTTool app showing my readings!

It's a thing of beauty seeing these readings coming in!

Here's my code as I have it running now:

Code: Select all

#Necessary library imports for this code to function
from secrets import SSID
from secrets import PASSWORD
from secrets import MQTTSERVE
from umqtt.simple import MQTTClient
#import time
import network
import ubinascii
import socket
from machine import SoftI2C, Pin
import sht30

#Define the interface of the SHT30 Temp/Humidity sensor with the Pico W
i2c=SoftI2C(sda=Pin(4), scl=Pin(5))
sht=sht30.SHT30(i2c=i2c, i2c_address=68)
sensor = sht.measure()
temperature, humidity = sensor

#Define the onboard Pico Temperature sensor and necessary conversion factor
sensor_temp = machine.ADC(4)
conversion_factor = 3.3 / (65535)
 
#Active the onboard Pico W Wi-Fi and connect to home network defined through "secrets.py"
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
#print(wlan.scan())
wlan.connect(SSID, PASSWORD)
mac = ubinascii.hexlify(network.WLAN().config('mac'),':').decode()
print("MAC address for this Pico W:", mac)
# Wait for connect or fail
max_wait = 10
while max_wait > 0:
 if wlan.status() < 0 or wlan.status() >= 3:
  break
  max_wait -= 1
  print('waiting for connection...')
  time.sleep(1)

# Handle connection error
if wlan.status() != 3:
 raise RuntimeError('network connection attempt failed')
else:
 print("Pico W Wi-Fi is connected to: ",SSID)
 status = wlan.ifconfig()
 print( "ip address for this Pico W: " + status[0] )
 print()
 print("Channel: ", wlan.config('channel'))
 print("SSID: ", wlan.config('essid'))
 print("TXPOWER: ", wlan.config('txpower'))

wlan.config(pm = 0xa11140)

#Show some onboard LED flashing to show that attempting to connect to the Wi-Fi has been completed
time.sleep(5)
led = machine.Pin("LED", machine.Pin.OUT)
#led = Pin(15, Pin.OUT)
led.off()
time.sleep(2)
led.toggle()
time.sleep(0.2)
led.toggle()
time.sleep(1)
led.toggle()
time.sleep(0.2)
led.toggle()
time.sleep(1)
led.toggle()
time.sleep(0.2)
led.toggle()
time.sleep(1)
led.toggle()
time.sleep(2)

#Setup MQTT server, local client ID (not really required),
#and published data transmissions hierarchy
mqtt_server = MQTTSERVE
client_id = b'PICOW'
topic_pub1 = b'PicoW/OS_Temp'
topic_pub2 = b'PicoW/OS_Hum'
topic_pub3 = b'PicoW/Pico_Temp'

#For the above topics, use "PicoW/+"
#at the subscriber side to receive all
#three messages at once or use the full
#titles individually i.e PicoW/OS_Temp for
#just the outside temperature variable

def mqtt_connect():
    client = MQTTClient(client_id, mqtt_server, keepalive=3600)
    client.connect()
    print('Connected to %s MQTT Broker'%(mqtt_server))
    return client

def reconnect():
    print('Failed to connect to the MQTT Broker. Reconnecting...')
    time.sleep(5)
    machine.reset()

try:
    client = mqtt_connect()
except OSError as e:
    reconnect()
    

#The main script of repeativeness for reading in from the connected sensor
#and taking in onboard temperature readings
while True: #do stuff here
    led.on()
    time.sleep(0.5)
    
    sensor = sht.measure()
    temperature, humidity = sensor
    
    reading = sensor_temp.read_u16() * conversion_factor 
    picotemp = 27 - (reading - 0.706)/0.001721
    
    #make readings 1 decimal place
    tpm1 = round(temperature,1)
    tpm2 = round(humidity,1)
    tpm3 = round(picotemp,1)
    
    #convert float reading values into bytes ready for MQTT transmission
    tpm4 = f"{tpm1}".encode()
    tpm5 = f"{tpm2}".encode()
    tpm6 = f"{tpm3}".encode()
    
    topic_msg1 = tpm4
    topic_msg2 = tpm5
    topic_msg3 = tpm6
    
    #Message displayed to the screen
    print()
    print('SHT Temperature: {:3.1f}ºC'.format(tpm1))
    print('SHT Humidity: {:3.1f}%'.format(tpm2))
    print('Pico W Temperature: {:3.1f}ºC'.format(tpm3))
    
    #Equivalent transmission of displayed readings above
    #(reverse order to match above order on subscriber receipt)
    client.publish(topic_pub3, topic_msg3)
    client.publish(topic_pub2, topic_msg2)
    client.publish(topic_pub1, topic_msg1)
    
    led.off()
    
    time.sleep(10)
I have time imported in boot.py before anyone gets confused how time.sleep() is working ok. :lol:

I'm sure my code is somewhat long winded and clumsy compared to what you guys could come up with, but at least it works.

I might want someone to offer up suggestions of how to better do the flashing of the LED because that's quite clumsy looking i'll admit.

Interestingly enough, you'll see I had to publish my messages to the opposite way around to how they're displayed on-screen to get them to appear in that same order on my phone as on the screen. Can anyone answer me to why this should be? Weird one!

Speaking of LEDs. Is there a clever way to flash the LED at intervals (maybe once a minute or ever 30 secs perhaps) during the end time.sleep() at the bottom of the code because i'll eventually be pushing that time.sleep() up to 900 (every 15 mins) to match the reading interval of my other readings being measured at my RP4B. It would be nice to be able to look at th Pico W and see a regular "heartbeat" to confirm it's still alive and working, and then do the more lengthy flashing routine I have it doing as it is transmitting data to my Pi.


Speaking of Pi, THAT'S my next challenge. Trying to set up the subscription to these newly transmitted variables. I guess i'm going to have to convert them all back to floats at the Pi end again now!?

danjperron
Posts: 51
Joined: Thu Dec 27, 2018 11:38 pm
Location: Québec, Canada

Re: Pico W Micro Python MQTT of data to a RP4B

Post by danjperron » Tue Aug 16, 2022 11:07 pm

Hi Rissy,


Looks very good. I also made a mqtt device with pico.

The big difference is that I'm adding a tiny solar cell with a lipo battery. This mean that I'm using lightsleep often to keep the battery.

This is my github https://github.com/danjperron/PicoWSolar

Maybe it will give you some more idea.

Daniel

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Wed Aug 17, 2022 7:37 am

danjperron wrote:
Tue Aug 16, 2022 11:07 pm
Hi Rissy,


Looks very good. I also made a mqtt device with pico.

The big difference is that I'm adding a tiny solar cell with a lipo battery. This mean that I'm using lightsleep often to keep the battery.

This is my github https://github.com/danjperron/PicoWSolar

Maybe it will give you some more idea.

Daniel
Thanks Daniel,

Looking at your stuff....

This is awesome! Very complex looking. Very impressive. You're clearly quite good at this coding business. I think you'd have to identify the low power bits of the code to me before I'd have any chance of trying that out with my setup. Putting the Pico W on to a small battery is a good idea. That would give me much more flexibility than my current thoughts of literally dropping my sensor out of a house window long-term and plugged into the wall!

If nothing else, you've immediately shown me how to set a static IP address in my Pico W now, so thank you for that. I'll try that tonight perhaps! :)

Post Reply