Pico W Micro Python MQTT of data to a RP4B

RP2040 based microcontroller boards running MicroPython.
Target audience: MicroPython users with an RP2040 boards.
This does not include conventional Linux-based Raspberry Pi boards.
Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Fri Aug 19, 2022 10:10 am

I've had a further thought on this since posting.

I'm wondering if where I'm going wrong is that i need to used the "decode()" function in my RP4B mqtt subscriber since i used the "encode()" function in the Pico W to publish the data.

I'm also wondering if I should have a simple mqtt subscription script like the one above, and then "tap" into that with another separate script which runs my more "fancy" coding for displaying to the screen and writing to a file?

Using something like this at the beginning of my fancy script:

from MQTT.py import OS_Temp, OS_Hum, Pico_Temp

(Can i do this?)

Anyone got any helpful comments or insights?

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Sat Aug 20, 2022 9:03 am

Hi all,

I have a BIG problem which i'm really begging for help on.

I'm afraid this is still Python at this point, not uPython. I've got my Pico W sending data reliably and before i try to make it more fancy, I simply want to complete the data path life-cycle from there to my overall end Python script for my desired functionality.

I've got my MQTT code working in my RP4B. Not the way i wanted, but it's collecting three individual messages, and i *think* i've managed to associate each of these readings with global variables for use outside of the defined message variables (please tell me if i've not done this at all, because i clearly have no way of testing this due to the next issue.)

I'm trying to call in my global variables in the script where i want to use these from the MQTT message collecting script.

This is a copy of my MQTT script called "MQTT_PW_Vars.py":

Code: Select all

#!/usr/bin/env python
#!/bin/bash
#MQTT_PW_Vars.py

import time
import paho.mqtt.client as mqtt

broker = '192.168.0.51'
port = 1883
#keepalive = 120
mqttclient = "PiLogger"
client = mqtt.Client(mqttclient)

topic1 = "PicoW/OS_Temp"
topic2 = "PicoW/OS_Hum"
topic3 = "PicoW/Pico_Temp"



def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print(f"Connected to MQTT Broker with result: {rc}")
    else:
        print("Failed to connect to Broker, return code = ", rc)

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Unexpected disconnection!")

def on_message1(client, userdata, message):
    print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
    GVAR = message.payload.decode("utf-8")
    global OS_Temp
    OS_Temp = float(GVAR)

def on_message2(client, userdata, message):
    print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
    GVAR = message.payload.decode("utf-8")
    global OS_Hum
    OS_Hum = float(GVAR)

def on_message3(client, userdata, message):
    print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
    GVAR = message.payload.decode("utf-8")
    global Pico_Temp
    Pico_Temp = float(GVAR)

client.connect(broker, port)
client.on_connect = on_connect
client.on_disconnect = on_disconnect
time.sleep(3)
client.subscribe(topic1)
client.subscribe(topic2)
client.subscribe(topic3)
client.on_message = on_message1
client.on_message = on_message2
client.on_message = on_message3
time.sleep(2)
client.loop_forever()
This produces the following:

Connected to MQTT Broker with result: 0
Received: 25.2 from Topic: PicoW/Pico_Temp
Received: 57.3 from Topic: PicoW/OS_Hum
Received: 22.2 from Topic: PicoW/OS_Temp
Received: 25.2 from Topic: PicoW/Pico_Temp
Received: 57.3 from Topic: PicoW/OS_Hum
Received: 22.1 from Topic: PicoW/OS_Temp

When I use the following lines in my overall desired script, called "PicoW_Vars.py":

Code: Select all

from MQTT_PW_Vars.py import OS_Temp as PW_Temp
from MQTT_PW_Vars.py import OS_Hum as PW_Hum
from MQTT_PW_Vars.py import Pico_Temp as Pi_Temp
The above "MQTT_PW_Vars.py" code completely hijacks the "PicoW_Vars.py" programs actual programming for its own!?

I only want the three variables as a product of the MQTT scripted code, not the functionality to completely take over anything else i have written in my end script!

Please. I'm really confused. I'm not sure where else to ask for help.

beetle
Posts: 51
Joined: Sat Oct 16, 2021 11:35 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by beetle » Sat Aug 20, 2022 11:09 am

I think you are assigning the client.on_message callback incorrectly. As far as I know only one callback is assigned and the effect of your code it to reassign the callback function, ending up with the on_message3 as being the callback that will be used for all messages received.

The callback function can examine the topics part of the message to decide what function to then call to deal with the data.
So the following could be what you put in your callback function:

Code: Select all

topics = message.topic.split('/') # topics is now a list containing the topic segments
payload = message.payload.decode()
if topic[1] == "Pico_Temp":
    func_to_handle_Pico_Temp(payload)
elif topic[1] == "OS_hum":
    func_to_handle_OS_hum(payload)
etc.
or you could create and send mqtt payload data as a delimited string such as "27.3, 22.1, 88.996" where you know the the first data represents the Pico temp and the second represents the os temp etc.
Then you might code it as follows

Code: Select all

if topic[0] == "PicoW":
    func_to handle(payload)
and in func_to_handle

Code: Select all

def funct_to_handle(pico_data):
    pico_temp, os_temp, os_hum = pico_data.split(',')
    print('Pico Temperature is: ', pico_temp)
etc.

beetle
Posts: 51
Joined: Sat Oct 16, 2021 11:35 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by beetle » Sat Aug 20, 2022 11:40 am

Regarding imports - study the following

https://realpython.com/python-import/

which should give you a proper understanding of the subject. It may take a while but it will be worth the effort :D

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Sat Aug 20, 2022 11:42 am

Hi beetle,

First thank you very much for giving me some of your time to help me try and fix my problem.

You're right in saying that it always appears as though my third message "callback" (if that's what it is) seems to be the master one above all.

It seems to be that no matter what i try to do within the other messages, except for displaying separate readings and topic titles, every thing else gets blended into only the functionality of the third message.

I've spent hours today trying to even get the individual readings sent to separate .txt files so that i can get my main code to get the variables independently via separate .txt files. No such luck. It always lumps them into the one text file, and its always the .txt file opened up in the third message definition logic.

I'm becoming very disheartened over this now. I'm clearly not good enough at trying to do what i want my code to do.

This suggestion of your looks promising.

Code: Select all

topics = message.topic.split('/') # topics is now a list containing the topic segments
payload = message.payload.decode()
if topic[1] == "Pico_Temp":
    func_to_handle_Pico_Temp(payload)
elif topic[1] == "OS_hum":
    func_to_handle_OS_hum(payload)
Where would this code go please?

beetle
Posts: 51
Joined: Sat Oct 16, 2021 11:35 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by beetle » Sat Aug 20, 2022 11:51 am

The code will go into your callback function.

def MyMQTT_Callback():
....the function code

and you nominate this function as your MQTT callback by

client.on_message = MyMQTT_Callback

just as you ended up doing with your client.on_message = on_message3 bit of code. Of course name the function to your desire, on_message3 or MyMQTT_Callback or whatever.

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Sat Aug 20, 2022 11:52 am

Relating to global variables.

I've been trying to define global variables within functions to a complete loss for some reason.

This is my latest file writing attempt of code:

Code: Select all

#!/usr/bin/env python
#!/bin/bash
#MQTT_PW_Vars.py

import time
import datetime
import sys
import os
import paho.mqtt.client as mqtt

#initial screen message with delay attached before running program
print("---------------------------------------------------------")
print("---Pico W Receiving Program is loading, please wait...---")
print("---------------------------------------------------------")
time.sleep(2)

#====================================================================
#sleeptimesecs is time setting for program to record samples in secs
sleeptimesecs=10
#====================================================================

#program_name = "PicoW_Vars.py"
#readingsfilename1 = 'PicoW_Vars.xls'
#readingsfilename2 = '/home/rissypi/BME280_Envir_Sen/PicoW_Vars.xls'

#PicoSampleFile = 'Pico_Sample.txt'
#PicoMAXFile = 'Pico_MAX.txt'
#PicoMINFile = 'Pico_MIN.txt'
#PicoTempFile = 'PicoTempFile.txt'
#PicoHumFile = 'PicoHumFile.txt'
#PicoFile =  'PicoFile.txt'

#send initial message to terminal to allow quiting later with CTRL+C
current_time = datetime.datetime.now()
time_stamp = current_time.timestamp()
from datetime import datetime
timestamp = time_stamp
date_time = datetime.fromtimestamp(timestamp)
str_date_time1 = date_time.strftime("%d-%m-%Y, %H:%M:%S")
starttimedate1 = str_date_time1
timestamp = time
minsperhour=60
sleeptimemins=sleeptimesecs/minsperhour

#set destination output for printing (original = screen)
orig_out = sys.stdout

loop = 1
sample = 0
loop2 = 1
initial = 0

#MQTT
broker = '192.168.0.51'
port = 1883
#keepalive = 120
mqttclient = "PiLogger"
client = mqtt.Client(mqttclient)

topic1 = "PicoW/OS_Temp"
topic2 = "PicoW/OS_Hum"
topic3 = "PicoW/Pico_Temp"

msg1=False
msg2=False
msg3=False

def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print(f"Connected to MQTT Broker with result: {rc}")
    else:
        print("Failed to connect to Broker, return code = ", rc)

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Unexpected disconnection!")

client.connect(broker, port)
client.on_connect = on_connect
client.on_disconnect = on_disconnect
time.sleep(3)

def on_message1(client, userdata, message1):
    global GVAR11
    #global msg1
    print(f"Received: {(message1.payload.decode('utf-8'))} from Topic: {message1.topic}")
    time.sleep(1)
    GVAR1 = str(message1.payload.decode("utf-8"))
    GVAR11 = GVAR1
    #client.unsubscribe(topic1)
    time.sleep(1)
    #return GVAR1

def on_message2(client, userdata, message2):
    global GVAR22
    #global msg2
    print(f"Received: {(message2.payload.decode('utf-8'))} from Topic: {message2.topic}")
    time.sleep(1)
    GVAR2 = str(message2.payload.decode("utf-8"))
    GVAR22 = GVAR2
    #client.unsubscribe(topic2)
    time.sleep(1)
    #return GVAR2

def on_message3(client, userdata, message3):
    global GVAR33
    #global msg3
    print(f"Received: {(message3.payload.decode('utf-8'))} from Topic: {message3.topic}")
    time.sleep(1)
    GVAR3 = str(message3.payload.decode("utf-8"))
    GVAR33 = GVAR3
    #client.unsubscribe(topic3)
    time.sleep(1)
    #return GVAR3

while msg1 == False:
    client.subscribe(topic1)
    client.on_message = on_message1
    time.sleep(3)
    PWTfile = open('PicoTempFile.txt', 'w')
    PWTfile.write(GVAR11)
    PWTfile.close()
    msg1 = True

while msg2 == False:
    client.subscribe(topic2)
    client.on_message = on_message2
    time.sleep(3)
    PWHfile = open('PicoHumFile.txt', 'w')
    PWHfile.write(GVAR22)
    PWHfile.close()
    msg2 = True

while msg3 == False:
    client.subscribe(topic3)
    client.on_message = on_message3
    time.sleep(3)
    PicoFile = open('PicoFile.txt', 'w')
    PicoFile.write(GVAR33)
    PicoFile.write("\n")
    PicoFile.close()
    msg3 = True

time.sleep(2)
client.loop_forever()
Why is the result this?:

Code: Select all

---------------------------------------------------------
---Pico W Receiving Program is loading, please wait...---
---------------------------------------------------------
Traceback (most recent call last):
  File "/home/rissypi/BME280_Envir_Sen/MQTT_PW_Vars2.py", line 121, in <module>
    PWTfile.write(GVAR11)
NameError: name 'GVAR11' is not defined
This is simply doing my head in. This all seems completely logical to me. Yet it refuses to work.

(I should add that there are definitions of files and variables at the top not used now, because i've been hacking a code all day today and it's sometimes bearing previous code attempts)
Last edited by Rissy on Sat Aug 20, 2022 11:56 am, edited 1 time in total.

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Sat Aug 20, 2022 11:53 am

beetle wrote:
Sat Aug 20, 2022 11:51 am
The code will go into your callback function.

def MyMQTT_Callback():
....the function code

and you nominate this function as your MQTT callback by

client.on_message = MyMQTT_Callback

just as you ended up doing with your client.on_message = on_message3 bit of code. Of course name the function to your desire, on_message3 or MyMQTT_Callback or whatever.
I'm sorry. I'm not very good at talking the lingo. So a lot of what you said here goes over my head. Can you place your suggestion into my code and paste it back showing me?

Sorry.

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Sat Aug 20, 2022 12:34 pm

Okay, i've had a go at what i think you're suggesting.

This is what i've got:

Code: Select all

#!/usr/bin/env python
#!/bin/bash
#MQTT_PW_Vars4.py

import time
import paho.mqtt.client as mqtt

#initial screen message with delay attached before running program
print("---------------------------------------------------------")
print("---Pico W Receiving Program is loading, please wait...---")
print("---------------------------------------------------------")
time.sleep(2)

broker = '192.168.0.51'
port = 1883
mqttclient = "PiLogger"
client = mqtt.Client(mqttclient)

topic1 = "PicoW/OS_Temp"
topic2 = "PicoW/OS_Hum"
topic3 = "PicoW/Pico_Temp"

def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print(f"Connected to MQTT Broker with result: {rc}")
    else:
        print("Failed to connect to Broker, return code = ", rc)

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Unexpected disconnection!")

def on_message(client, userdata, message):
    #topics = message.topic.split('/') # topics is now a list containing the topic segments
    topics = message.topic.split() # topics is now a list containing the topic segments
    print(topics)
    #print(topic[1])
    payload = message.payload.decode("utf-8")
    #print(payload)
    if topics == topic1:
        global OS_Temp
        print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
        GVAR = payload
        OS_Temp = float(GVAR)
    elif topics == topic2:
        global OS_Hum
        print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
        GVAR = payload
        OS_Hum = float(GVAR)
    elif topics == topic3:
        global Pico_Temp
        print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
        GVAR = payload
        Pico_Temp = float(GVAR)

client.connect(broker, port)
client.on_connect = on_connect
client.on_disconnect = on_disconnect
time.sleep(3)
client.subscribe(topic1)
client.subscribe(topic2)
client.subscribe(topic3)
client.on_message = on_message
time.sleep(2)
client.loop_forever()
This produces the result:

Code: Select all

--------------------------------------------------------
---Pico W Receiving Program is loading, please wait...---
---------------------------------------------------------
Connected to MQTT Broker with result: 0
['PicoW/Pico_Temp']
['PicoW/OS_Hum']
['PicoW/OS_Temp']
['PicoW/Pico_Temp']
['PicoW/OS_Hum']
['PicoW/OS_Temp']
It just repeats this now, over and over:

Code: Select all

['PicoW/Pico_Temp']
['PicoW/OS_Hum']
['PicoW/OS_Temp']
I've made it also print the actual readings, as you can see from what is hashed out now, but ultimately, it never prints out the lines within the if loops!? Therefore it wont also be assigning the global variables etc either (despite the fact that this way of defining global variables doesn't seem to work at all - i'm not sure how else to do it right now!?

Where's it going wrong now please?

Rissy
Posts: 116
Joined: Sun Aug 14, 2022 8:15 am

Re: Pico W Micro Python MQTT of data to a RP4B

Post by Rissy » Sat Aug 20, 2022 12:56 pm

Right, i've removed unnecessary complication from my Pico topic strings.

Now just:

topic1 = 'OS_Temp'
topic2 = 'OS_Hum'
topic3 = 'Pico_Temp'

No "/" any more.

Then i've altered my code to this:

Code: Select all

#!/usr/bin/env python
#!/bin/bash
#MQTT_PW_Vars4.py

import time
import paho.mqtt.client as mqtt

#initial screen message with delay attached before running program
print("---------------------------------------------------------")
print("---Pico W Receiving Program is loading, please wait...---")
print("---------------------------------------------------------")
time.sleep(2)

broker = '192.168.0.51'
port = 1883
mqttclient = "PiLogger"
client = mqtt.Client(mqttclient)

#topic1 = "PicoW/OS_Temp"
#topic2 = "PicoW/OS_Hum"
#topic3 = "PicoW/Pico_Temp"

#topic1 = 'PicoW/OS_Temp'
#topic2 = 'PicoW/OS_Hum'
#topic3 = 'PicoW/Pico_Temp'

topic1 = 'OS_Temp'
topic2 = 'OS_Hum'
topic3 = 'Pico_Temp'

def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print(f"Connected to MQTT Broker with result: {rc}")
    else:
        print("Failed to connect to Broker, return code = ", rc)

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Unexpected disconnection!")

def on_message(client, userdata, message):
    #topics = message.topic.split('/') # topics is now a list containing the topic segments
    #topics = message.topic.split() # topics is now a list containing the topic segments
    topics = message.topic
    #print(topic[1])
    payload = message.payload.decode("utf-8")
    #print(payload)
    if topics == topic1:
        global OS_Temp
        print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
        GVAR = payload
        OS_Temp = float(GVAR)
    if topics == topic2:
        global OS_Hum
        print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
        GVAR = payload
        OS_Hum = float(GVAR)
    if topics == topic3:
        global Pico_Temp
        print(f"Received: {(message.payload.decode('utf-8'))} from Topic: {message.topic}")
        GVAR = payload
        Pico_Temp = float(GVAR)

client.connect(broker, port)
client.on_connect = on_connect
client.on_disconnect = on_disconnect
time.sleep(3)
client.subscribe(topic1)
client.subscribe(topic2)
client.subscribe(topic3)
client.on_message = on_message
print(OS_Temp, OS_Hum, Pico_Temp)
time.sleep(2)
client.loop_forever()
This produces this output when i don't try to call my attempted global variables:

Code: Select all

---------------------------------------------------------
---Pico W Receiving Program is loading, please wait...---
---------------------------------------------------------
Connected to MQTT Broker with result: 0
Received: 27.0 from Topic: Pico_Temp
Received: 54.7 from Topic: OS_Hum
Received: 24.0 from Topic: OS_Temp
Received: 27.0 from Topic: Pico_Temp
Received: 54.7 from Topic: OS_Hum
Received: 24.1 from Topic: OS_Temp
So now we're starting to look more promising. It's clearly dipping into the "if topics" conditions now, which is good.

But back to the global variables again. What am I doing wrong there?

This is the result with me trying to print out the global variables at the end to prove they're working (they're not of course):

Code: Select all

---------------------------------------------------------
---Pico W Receiving Program is loading, please wait...---
---------------------------------------------------------
Traceback (most recent call last):
  File "/home/rissypi/BME280_Envir_Sen/MQTT_PW_Vars4.py", line 72, in <module>
    print(OS_Temp, OS_Hum, Pico_Temp)
NameError: name 'OS_Temp' is not defined

Post Reply