Efficiently sending data over wifi. Am I doing this right?

The official PYBD running MicroPython, and its accessories.
Target audience: Users with a PYBD
Post Reply
Batman
Posts: 13
Joined: Sun Aug 26, 2018 6:04 pm

Efficiently sending data over wifi. Am I doing this right?

Post by Batman » Wed Aug 11, 2021 3:35 pm

Hello all,

I have attached sensors to my Pyboard D, and I am trying to collect the data from the sensors over wifi from a computer. This is the idea in brief.

1) The pyboard and my laptop connect to my local wifi network
2) The pyboard opens a socket and waits for a single connection
3) My laptop opens a socket connecting to the pyboard
4) If the board receives a connection, it collects that data from the sensor and places the data in a buffer
5) The data are sent to the laptop over the socket
6) The socket is closed, both on the computer and the pyboard

Steps 2-6 are repeated every time I want to get some data (~every 2 seconds).

This works 90% of the time. However, sometimes steps 2-3 raise an issue. Usually, I can open the socket almost immediately. But sometimes, it takes seconds (>10). And sometimes, the connection fails. This made me wonder whether opening and closing the socket for each data exchange is maybe not the best way to do this.

I've thought about keeping the socket open. However, that would mean I need to accept a new connection to the socket every 2 seconds in step 2, leading to many open connections.

This is the main code running on my pyboard. The Server() object is defined below.

Code: Select all

sta_if = network.WLAN(network.STA_IF)
if not sta_if.isconnected():
    print('connecting to network...')
    sta_if.active(True)
    sta_if.connect('MyWIFINetworkName', 'MyWIFIpassword')
    while not sta_if.isconnected(): pass
    print('network config:', sta_if.ifconfig())

while True:
        data_server = server.Server()
        buffer = get_data()
        data_server.send_data(buffer)
        data_server.disconnect()

This is the code for the Server class:

Code: Select all

class Server:
    def __init__(self):
        self.break_character = '*'
        self.buffer = 1024
        self.connection = None
        self.address = None
        self.skt = socket.socket()
        self.skt.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.skt.bind(('', 1000))
        self.skt.listen(1)
        connection, address = self.skt.accept()
        self.connection = connection
        self.address = address
        
        
    def disconnect(self):
        self.connection.close()
        
    
    def receive_data(self):
        data = ''
        while 1:
            packet = self.connection.recv(self.buffer)
            packet = packet.decode()
            #if not packet: break
            data += packet
            if data.endswith(self.break_character): break
        data = data.rstrip(self.break_character + '\n')
        return data
    
    def send_data(self, message):
        if not message.endswith(self.break_character): message = message +  self.break_character
        encoded_message = message.encode()
        self.connection.sendall(encoded_message)

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: Efficiently sending data over wifi. Am I doing this right?

Post by davef » Wed Aug 11, 2021 9:35 pm

I not an expert but have struggled this situation. After countless hours I came to the conclusion that keeping on an ESP32 connected to a hotspot as the possible cause of random re-boots. I almost got to the stage of buying a Pyboard D to see how it coped.

I found only connecting to WiFi, in my case once a day for about 30 seconds, has worked properly for the last month or so.
I've thought about keeping the socket open. However, that would mean I need to accept a new connection to the socket every 2 seconds in step 2, leading to many open connections.
Does this only happen if you keep the socket open at one end? If so, why can't you keep the socket open at both ends?

Batman
Posts: 13
Joined: Sun Aug 26, 2018 6:04 pm

Re: Efficiently sending data over wifi. Am I doing this right?

Post by Batman » Thu Aug 12, 2021 7:58 pm

Does this only happen if you keep the socket open at one end? If so, why can't you keep the socket open at both ends?
If I keep the socket open on the pyboard, I get into trouble if the program on the laptop fails or restarts. In that case, the pyboard would have to accept a new connection because the first one has not been closed at the side of the pyboard.

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: Efficiently sending data over wifi. Am I doing this right?

Post by davef » Fri Aug 13, 2021 6:49 am

Hi Batman,

Thanks for the explanation. I have two links, one where I control a pump, the other controlling a microhydro both using UDP sockets on port 10000. As part of range-testing this system I am pretty sure that one or both ends can go "down" and then when back in range pick-up where they left off.

I am pretty sure the same behaviour takes place on some ESP-NOW links that I have been testing.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Efficiently sending data over wifi. Am I doing this right?

Post by pythoncoder » Fri Aug 13, 2021 1:20 pm

@Batman you might like to look at two existing solutions, both of which cope with intermittent WiFi connectivity. asynchronous mqtt and micropython-iot. Either of these should serve your purpose.
Peter Hinch
Index to my micropython libraries.

Post Reply