Webserver send() + POST data reception

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
Post Reply
trap16
Posts: 1
Joined: Tue Dec 04, 2018 8:23 pm

Webserver send() + POST data reception

Post by trap16 » Tue Dec 04, 2018 8:47 pm

Hello,

I'm currently developping a soft on micropython for ESP8266 board.

I was currently working on a minimalist webserver to access and configure the board and notices 2 behaviors i need expertise.

Current software if a simple http server (I create a socket TCP on port 80 then listen + accept connection), the initial GET request from the client browser is replied with a formular with text input field in POST mode, the subsequent POST request should send my text to the ESP8266

I checked the class on my local computer with python/idle, this works perfectly, I see my formular, can fill it and process the user input. But when uploading it on the ESP and running it I see strange behavior.

The initial problem I faced is with closing the connection my code was somewhat like

while True:
conn ... = self.sock.accept()
query = conn.recv(2048)
...
conn.send(reply)
conn.close()

The client browser was hanging, I then added a time.sleep(1) before the conn.close() and saw it works (means the connection is closed before the reply is sent), is there a better way to handle that (I thought send would be a blocking call) ?

The second issue is very weird : when i print(query) on my local computer I see the POST data, but not on the ESP8266.
The contents of the datas are stricly the same, except the last line (containing the post data) which does not exist when printed on the server running on ESP8266.
I don't know if it is because of my web browser (chrome) or if it is an issue on the ESP8266, anyone by chance had experienced similar issue to give a hint ?

Best regards.

[EDIT] : added the dump of buffer of the POST request in both cases :

Local computer :

b'POST / HTTP/1.1\r\nHost: 172.16.2.1\r\nConnection: keep-alive\r\nContent-Length: 28\r\nCache-Control: max-age=0\r\nOrigin: http://172.16.2.1\r\nUpgrade-Insecure-Requests: 1\r\nContent-Type: application/x-www-form-urlencoded\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8\r\nReferer: http://172.16.2.1/\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7\r\n\r\nSSID=toto&PASS=&ACTION=Check'

ESP8266 :

b'POST / HTTP/1.1\r\nHost: 172.16.1.30\r\nConnection: keep-alive\r\nContent-Length: 28\r\nCache-Control: max-age=0\r\nOrigin: http://172.16.1.30\r\nUpgrade-Insecure-Requests: 1\r\nContent-Type: application/x-www-form-urlencoded\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8\r\nReferer: http://172.16.1.30/\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: fr-FR,fr;q=0.9,en-US'

As you can see the local computer environment has a trailing ";q=0.8,en;q=0.7\r\n\r\nSSID=toto&PASS=&ACTION=Check" which is not present when i run the same software on ESP8266

cefn
Posts: 230
Joined: Tue Aug 09, 2016 10:58 am

Re: Webserver send() + POST data reception

Post by cefn » Sat Dec 15, 2018 12:20 pm

Hi, trap16

Without seeing all the reference code you are running I think it'll be hard for anyone to help.

nikitto
Posts: 2
Joined: Sun Nov 03, 2019 11:48 pm

Re: Webserver send() + POST data reception

Post by nikitto » Mon Nov 04, 2019 12:06 am

Hello everyone.

Same problem here.

This is a part of the code:

Code: Select all

try:
  import usocket as socket
except:
  import socket
import network
import esp
esp.osdebug(None)
import gc
gc.collect()

....

def web_page():
  html = """<!DOCTYPE html>
  <html>
  <head> <title>nodeMCU PVControl+</title> </head>
  <link rel="icon" href="data:;base64,=">  <!-- para evitar 2 conexiones, http y favicon -->
  <center><h2>PVControl+</h2></center>
  <center><h3>Configurar nodeMCU Lolin V3</h3></center><br>
  <form action="/" method="POST"><center>
    <input type="text" name="ssid" placeholder="ssid"><br> 
    <input type="text" name="password" placeholder="Password Wifi"><br>
    <input type="text" name="ip_servidor" placeholder="IP servidor mqtt"><br>
    <input type="text" name="puerto_serv" placeholder="Puerto servidor mqtt"><br>
    <input type="text" name="usuario_mqtt" placeholder="Usuario mqtt"><br> 
    <input type="text" name="pass_mqtt" placeholder="Password mqtt"><br>
    <input type="text" name="ip_router" placeholder="IP router"><br>
    <input type="text" name="ip_nodemcu" placeholder="IP nodemcu"><br><br>
    <left><button type="submit">Guardar en archivo</button></left>
  </center></form>     
  </html>
  """
  return html
...

if not exists:    # a file, AP mode	
  ssid = "nodeMCU"
  password = "12345678" 
  ap_if = network.WLAN(network.AP_IF)
  if not ap_if.isconnected():
    print("Configuracion AP")
    ap_if.active(True)
    ap_if.config(essid=ssid, password=password)
    while not ap_if.isconnected():
      pass

  print('Connection successful')
  print(ap_if.ifconfig())
 
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
  s.bind(('', 80))
  s.listen(1) 
 
  while True:
    conn, addr = s.accept()
    print('Got a connection from %s' % str(addr))
    request = conn.recv(1024) #1024
    request = str(request)
    print('Content = %s' % request)

    response = web_page()
    conn.send(response)
    conn.close()
    
If I change POST for GET in the form, the output is right.

Thanks in advance.

nikitto
Posts: 2
Joined: Sun Nov 03, 2019 11:48 pm

Re: Webserver send() + POST data reception

Post by nikitto » Tue Nov 05, 2019 10:40 am

Hi again.

It's imposible to read all buffer just in one time, if you send much data. So, the best way is read few bytes every time, till buffer is empty.

Code: Select all

...
  s.listen(1)   # max 1 conexion

  
  while True:
    conn, addr = s.accept()
    print('Got a connection from %s' % str(addr))
    
    while True:
      request = conn.recv(8)
      print('%s' % request)
      if len(request) < 8:
        print('The End') 
        break

    
    #print('%s' % request)
    
    response = web_page()
    conn.sendall(response)
    #conn.send(response)
    conn.close()

As you can see, if len(request) is smaller than 8 bytes then breaks the loop. Just need a variable where to save all data.
For example: buffer += request

User avatar
Ventran
Posts: 24
Joined: Sun Jun 21, 2020 4:28 pm
Location: Poland, Europe

Re: Webserver send() + POST data reception

Post by Ventran » Wed Sep 09, 2020 10:51 pm

Hello everyone!
I have a similar problem with read all buffer using recv() function. So, decided to use trick gave by @nikitto but now i have another problem.

Sometimes the last chunk of data have a exactly the same length which previous chunk (8 bytes). Everything is ok when the last chunk have a other size (less than 8 bytes). So, how i can recognize last chunk and break forever while loop?

Code: Select all

        while True:
            try:
                self.conn, addr = self.s.accept()
                # for i in str(self.conn).split():
                #     print(i)
            except Exception as exc:
                print("Socket Accept Error ", exc.args[0])
                reset()
                pass
            print('Connected by', addr)
            try:

                while True:
                  self.request = self.conn.recv(8)
                #   print('%s, :: len -> %d' % (self.request, len(self.request)))
                  if len(self.request) < 8:
                    print('The End') 
                    break

            except Exception as exc:
                print("recv -------------", exc.args[0])
                reset()
                pass
            if not self.request: break
            self.connection(html)

svensche
Posts: 2
Joined: Thu Jan 06, 2022 10:42 am

Re: Webserver send() + POST data reception

Post by svensche » Thu Jan 06, 2022 10:46 am

Hi guys,
I'm searching for a tiny little www-server (module) to communicate with my ESP8266-12F in order to read/set some values.
I've tried MicroWebSrv and MicroWebSrv2, but they are to big and producing memory errors.
Your posts made me think, that you are dealing with the same problem. I was wondering if you found a solution for this problem?
Happy hearing from you.

Post Reply