Bullet-Proof HTTP server.

[devnull]

I am trying to create a bullet proof web server that won't just die when there is a socket error, here's the main code loop that handles the socket listen and connect:

Code: Select all

  def close(self,conn):
    try: conn.close()
    except Exception as err:
      print('Close error:',err)

   while True:
      print('HTTP serve on IP:', wlan.net.ifconfig()[0],'PORT:', self.port )
      
      try:
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.sock.bind(('0.0.0.0', self.port))
        self.sock.listen(1)
  
        while True:
          try:
            conn, addr = self.sock.accept()
            raw = None
            while True:
              line = conn.readline()
              if not raw and len(line) > 10:
                raw = line.decode('UTF-8')
              if not line or line == b'\r\n':
                break
    
            if not raw:
              self.close(conn)
            else:
              self.request(conn,raw)
          
          except Exception as e:
            self.close(conn) 
            print('Sock Conn Err:',e)
            break
          
          gc.collect()
  
      except Exception as e:
        self.active = False
        print('Sock Listen Err:',e)
        break

Now this may run fine for an hour or a daay or 2 days, but eventually it will fail always the result of a broken socket of one sort or another.

Is it possible to make this bullet-proof such that it automatically recovers from socket errors and does not require a hard reset ?

After a listen failure, attempts to establish a new listen always results in "[Errno 12] ENOMEM".

[Capstan]

I'm just feeling my way through this currently, no expert, but I think you need to explicitly close the socket after handling each request. Here's a code fragment from my (incomplete) experiment that does something similar to yours I think, never have seen it run out of memory. Doesn't mean it won't ever do it though.

Code: Select all

server = usocket.socket()

def mainLoop():
    server.bind(('0.0.0.0', 80))
    server.listen(1)
    while True:
        try:
            (socket, sockaddr) = server.accept()
            handleQuery(socket)
        except OSError as ose:
            print(ose)
            code = ose.args[0]
            if code == 104 or code == 103 or code == 103: 
                print('OSError:', code)
                # ECONNRESET or ECONNABORTED or EALREADY
                return code
            socket.write("HTTP/1.1 500 Internal Server Error\r\n\r\n")
            socket.write("<h1>Internal Server Error</h1>")
        socket.close()	# << see this right here

while True:
    mainLoop()

[devnull]

I am still chasing this 'bullet proof web server' and have yet to find a solution.

Is there any way to prevent the server from just going offline without any reported errors or reason, it just appears to stop responding and there's only one connection, where the page might be loaded / reloaded once a minute.

I have added code to always close the connection but it seems to make little difference to the reliability, and I am beginning to think that this problem cannot be solved and without the esp8266 having a watchdog, it means that this device could never run autonomously as a web server ?

Code: Select all

  def stop(self):
    if self.sock:
      self.sock.close()
      self.sock = None

  def run(self):
    while True:
      print('HTTP serve on IP:', wlan.net.ifconfig()[0],'PORT:', self.port )
      
      try:
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.sock.bind(('0.0.0.0', self.port))
        self.sock.listen(1)
  
        while True:
          try:
            conn, addr = self.sock.accept()
            raw = None
            while True:
              line = conn.readline()
              if not raw and len(line) > 10:
                raw = line.decode('UTF-8')
              if not line or line == b'\r\n':
                break
    
            if not raw:
              self.close(conn)
            else:
              self.request(conn,raw)
          
          except Exception as e:
            self.close(conn) 
            print('Conn Err:',e)
            break
          
          gc.collect()
  
      except Exception as e:
        self.stop()
        self.active = False
        print('Listen Err:',e)
        gc.collect()
        time.sleep(1)

[pythoncoder]

I have no experience with web servers but I have put in some effort into achieving a reliable MQTT client. The problem is that wireless links can never be as reliable as wired Ethernet. The connection can be disrupted by radio interference or in extremis the device moving out of range of the AP. I found that coding round this to ensure reasonable application behaviour in the event of outages was a non trivial task. But sockets on any wireless connected device cannot be assumed to be reliable.

There are faster ways to test these things than waiting for days for a socket to fail. I power the ESP8266 from one of those USB battery packs intended for recharging mobile phones. I put the ESP and battery in a microwave oven and shut the door - the microwave acts as a very effective Faraday cage, cutting the WiFi signal. This tests sudden outages. If it consistently passes that test I then walk the ESP/battery until it is out of range, then bring it back. Depending on your code, a gradual dropout can trigger different error conditions compared to a sudden one.

I agree that a working watchdog would be very useful.

I doubt my code will be of much use to you as the applications are so very different but for completeness it is here https://github.com/peterhinch/micropython-mqtt.git.

[devnull]

Hi Peter, thanks for replying.

Microwave is a good idea, I have put it in a aluminium foil covered box in the past and it made no difference, and also put it in a steel cabinet and there appeared to be no change in the signal strength.

I've been working on this intermittently now for over 6 months, so far nothing I do can resolve it.

It works 100% great, no problem at all until you don't refresh a page for say 15 minutes (not sure of exact time) and it will stop responding, there are no errors and the console just sits there showing the last successful access log, it appears to fail silently.

I have tried an interrupt timer to try and ping itself once a minute, but I am guessing that you cannot have a client and socket running on the same device at the same time as it is never able to connect.

You could be right about the cause, but, there is a socked setting which is supposed (if I understand correctly) to solve this long term open connection, but it appears to have no effect:

Code: Select all

s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

If you have the time, here's an example of a http server, you should be able to cut & paste and run it (no dependancies) and connect to it with a web browser, just leave the connection idle for some time and you should see the same things that I do:

https://github.com/micropython/micropyt ... _server.py

[devnull]

I just setup a timer to get the sock status:

Code: Select all

<socket state=0 timeout=-1 incoming=0 off=0>

It shows this exact same status when the connection is alive and after it decides to stop responding, there's no errors and no indiction that anything has failed.

[Capstan]

I don't see your network setup code. Do you have the ESP serving up the webpage running as a WiFi client, or as an access point?

I am assuming it is a client? When the webserver stops responding can you check to see if the ESP has WiFi connectivity? It would be nice if the network API generated an event when WiFi connectivity gets lost, but I am not seeing that in the docs. Apparently we just have to check it periodically in a timer loop? If so you could do that, and reconnect if needed. Or just reset and let it boot back up fresh.

[pythoncoder]

The ESP8266 is remarkably sensitive. Other approaches to shielding it failed probably caused by the power lead carrying RF. I tried putting the ESP8266 in a metal box and then shutting the box in the microwave with the door open just enough for the USB cable but it still communicated unless I pushed rather unreasonably hard against the door.

The only reliable testing approach was as described above.

[jimeer]

I have a central heating controller where an RPi communicates with an ESP8266 over a TCP socket very similar to the code at the top of this topic. It has been running over a year quite reliably. The only differences I can see are as follows:
First I check the connection with this:
[code]
# ******************************************************************************
# checks for a client connection request
# returns true, client (connected socket), address of connecting client
# if true cl can be used to read and write
# ******************************************************************************
def check_con(s):
try:
cl, addr = s.accept()
return True, cl, addr
except:
return False, 0, 0
[/code]

if all okay then instead of using 'conn, addr = self.sock.accept()'
I have:
[code]
# ******************************************************************************
# returns the socket state
# 2 = open
# 3 = closed by other side
# ******************************************************************************
def s_state(s):
state = 0
incoming = False
ss = (str)(s).split(' ')
if 'state' in ss[1]:
try:
state = int(ss[1].split('=')[1])
except:
state = -9
if 'incoming' in ss[3]:
try:
incoming = len(ss[3].split('=')[1]) > 1
except:
incoming = False
return state, incoming
[/code]

Hope that helps

[deonis]

I am not sure if you still interested in a simple but stable socket server to serve a webpage. The code below displays a simple webpage to control a NeoPixel led array using a socket server and micro ajax framework. The schematics on how I connected the board is also available in the link below.

Do not forget to set your IP.

https://github.com/deonis1/nphttpd