Running webserver and other code simultaneously
Running webserver and other code simultaneously
Hi, I’m using the ws.py (http://jacobstoner.com/ws.py) minimalist webserver and it works well in a forever loop with ws.serve(1000). I’m using the web service for configuration values.
However, for my application I also want to have a tcp client running too, and I want it to be reactive, so I call select() there, and normally that is a blocking call. Is my only option to change that to a non blocking call, and call the ws.serve() when the other socket has nothing incoming?
I hope I explained that reasonably well, basically I’ve got two normally blocking things that I want to run in one thread - any pointers welcome!
However, for my application I also want to have a tcp client running too, and I want it to be reactive, so I call select() there, and normally that is a blocking call. Is my only option to change that to a non blocking call, and call the ws.serve() when the other socket has nothing incoming?
I hope I explained that reasonably well, basically I’ve got two normally blocking things that I want to run in one thread - any pointers welcome!
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Running webserver and other code simultaneously
This is something of a FAQ. I suggest you read this thread and look at this example code.
The consensus view is to use uasyncio. A tutorial on this MicroPython asyncio subset may be found here.
The consensus view is to use uasyncio. A tutorial on this MicroPython asyncio subset may be found here.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Running webserver and other code simultaneously
Peter, my apologies for asking an FAQ, and thanks for your pointer, I'll take a look. I had looked at the asyncio stuff briefly but it didn't work for me on my D1 mini - maybe the same issue with the outdated firmware I started off with.
I'll look a bit further into it. It seems the ws.py isnt the silver bullet either, I left it running overnight and it seems to have disconnected - program looks like it's running, but it isn't accepting connections on port 80 anymore (until a reboot)
Thanks again.
I'll look a bit further into it. It seems the ws.py isnt the silver bullet either, I left it running overnight and it seems to have disconnected - program looks like it's running, but it isn't accepting connections on port 80 anymore (until a reboot)
Thanks again.
Re: Running webserver and other code simultaneously
Here is a simple web server based on the uasyncio example, that I know works on the ESP8266 and the D1 Mini. I have yet to implement another coro but perhaps Peter's tutorial will help you with that.
Good luck!
Code: Select all
#
# Simple HTTP server based on the uasyncio example script
# by J.G. Wezensky (joewez@gmail.com)
#
import uasyncio as asyncio
import uos
import pkg_resources
webroot = 'wwwroot'
default = 'index.html'
# Breaks an HTTP request into its parts and boils it down to a physical file (if possible)
def decode_path(req):
cmd, headers = req.decode("utf-8").split('\r\n', 1)
parts = cmd.split(' ')
method, path = parts[0], parts[1]
# remove any query string
query = ''
r = path.find('?')
if r > 0:
query = path[r:]
path = path[:r]
# check for use of default document
if path == '/':
path = default
else:
path = path[1:]
# return the physical path of the response file
return webroot + '/' + path
# Looks up the content-type based on the file extension
def get_mime_type(file):
if file.endswith(".html"):
return "text/html", False
if file.endswith(".css"):
return "text/css", True
if file.endswith(".js"):
return "text/javascript", True
if file.endswith(".png"):
return "image/png", True
if file.endswith(".gif"):
return "image/gif", True
if file.endswith(".jpeg") or file.endswith(".jpg"):
return "image/jpeg", True
return "text/plain", False
# Quick check if a file exists
def exists(file):
try:
s = uos.stat(file)
return True
except:
return False
@asyncio.coroutine
def serve(reader, writer):
try:
file = decode_path((yield from reader.read()))
if exists(file):
mime_type, cacheable = get_mime_type(file)
yield from writer.awrite("HTTP/1.0 200 OK\r\n")
yield from writer.awrite("Content-Type: {}\r\n".format(mime_type))
if cacheable:
yield from writer.awrite("Cache-Control: max-age=86400\r\n")
yield from writer.awrite("\r\n")
f = open(file, "rb")
buffer = f.read(512)
while buffer != b'':
yield from writer.awrite(buffer)
buffer = f.read(512)
f.close()
else:
yield from writer.awrite("HTTP/1.0 404 NA\r\n\r\n")
except:
raise
finally:
yield from writer.aclose()
def start():
import logging
logging.basicConfig(level=logging.ERROR)
loop = asyncio.get_event_loop()
loop.call_soon(asyncio.start_server(serve, "0.0.0.0", 80, 20))
loop.run_forever()
loop.close()
Re: Running webserver and other code simultaneously
bitninja wrote: ↑Sat May 26, 2018 7:01 pmHere is a simple web server based on the uasyncio example, that I know works on the ESP8266 and the D1 Mini. I have yet to implement another coro but perhaps Peter's tutorial will help you with that.
Good luck!Code: Select all
# # Simple HTTP server based on the uasyncio example script # by J.G. Wezensky (joewez@gmail.com) # import uasyncio as asyncio import uos import pkg_resources webroot = 'wwwroot' default = 'index.html' # Breaks an HTTP request into its parts and boils it down to a physical file (if possible) def decode_path(req): cmd, headers = req.decode("utf-8").split('\r\n', 1) parts = cmd.split(' ') method, path = parts[0], parts[1] # remove any query string query = '' r = path.find('?') if r > 0: query = path[r:] path = path[:r] # check for use of default document if path == '/': path = default else: path = path[1:] # return the physical path of the response file return webroot + '/' + path # Looks up the content-type based on the file extension def get_mime_type(file): if file.endswith(".html"): return "text/html", False if file.endswith(".css"): return "text/css", True if file.endswith(".js"): return "text/javascript", True if file.endswith(".png"): return "image/png", True if file.endswith(".gif"): return "image/gif", True if file.endswith(".jpeg") or file.endswith(".jpg"): return "image/jpeg", True return "text/plain", False # Quick check if a file exists def exists(file): try: s = uos.stat(file) return True except: return False @asyncio.coroutine def serve(reader, writer): try: file = decode_path((yield from reader.read())) if exists(file): mime_type, cacheable = get_mime_type(file) yield from writer.awrite("HTTP/1.0 200 OK\r\n") yield from writer.awrite("Content-Type: {}\r\n".format(mime_type)) if cacheable: yield from writer.awrite("Cache-Control: max-age=86400\r\n") yield from writer.awrite("\r\n") f = open(file, "rb") buffer = f.read(512) while buffer != b'': yield from writer.awrite(buffer) buffer = f.read(512) f.close() else: yield from writer.awrite("HTTP/1.0 404 NA\r\n\r\n") except: raise finally: yield from writer.aclose() def start(): import logging logging.basicConfig(level=logging.ERROR) loop = asyncio.get_event_loop() loop.call_soon(asyncio.start_server(serve, "0.0.0.0", 80, 20)) loop.run_forever() loop.close()
Thanks!!
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Running webserver and other code simultaneously
@simonmcc I can confirm that uasyncio does work on ESP8266. This repo uses it extensively.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Running webserver and other code simultaneously
so I got uasyncio loaded and running, and I got one of my existing modules converted to it, but I keep running into issues with memory. My program requires quite a few modules - TCP connection to server, ssd1306 for display, 1-wire, etc, etc.
After reading another thread, I tried to precompile some modules, but when I put them onto my Wemos D1 mini, it just tells me it cant find them. I also tried changing the order of my imports, that helped a bit. I reset the board between each run.
when I call the micropython.mem_info() it gives:
stack: 4112 out of 8192
GC: total: 35968, used: 34048, free: 1920
No. of 1-blocks: 349, 2-blocks: 75, max blk sz: 264, max free sz: 33
and FYI now running:
MicroPython v1.9.4-8-ga9a3caad0 on 2018-05-11; ESP module with ESP8266
any pointers welcome
After reading another thread, I tried to precompile some modules, but when I put them onto my Wemos D1 mini, it just tells me it cant find them. I also tried changing the order of my imports, that helped a bit. I reset the board between each run.
when I call the micropython.mem_info() it gives:
stack: 4112 out of 8192
GC: total: 35968, used: 34048, free: 1920
No. of 1-blocks: 349, 2-blocks: 75, max blk sz: 264, max free sz: 33
and FYI now running:
MicroPython v1.9.4-8-ga9a3caad0 on 2018-05-11; ESP module with ESP8266
any pointers welcome
Re: Running webserver and other code simultaneously
You can use frozen modules for the code that is stable and definitely required. Pre-compiled modules may be accomplishing the same thing... I have not used those yet. I just got comfortable building my own firmware and then added the modules I needed into the build.
Second... garbage collect often... you already have an uphill battle with the ESP8266's limited RAM. Try to only use the memory when you need to and give the gc time to do its work.
It's tough, but I have seen some surprisingly large apps work on the ESP8266.
Good luck.
Second... garbage collect often... you already have an uphill battle with the ESP8266's limited RAM. Try to only use the memory when you need to and give the gc time to do its work.
It's tough, but I have seen some surprisingly large apps work on the ESP8266.
Good luck.
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Running webserver and other code simultaneously
As @bitninja said, using frozen bytecode is vital for larger applications. In the esp8266 directory issue make clean. Then, in your source tree, copy the modules into ports/esp8266/modules. Compile and flash. Then at the REPL issue
Code: Select all
>>> help('modules')
Most of us have been there at some point.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Running webserver and other code simultaneously
Peter, as you've probably detected I'm at the 'silly' stage alright. This is all new to me, and I'm loving it, although it's (clearly) taking a bit of time to get used to! I appreciate your patience
I definitely will try building my own firmware soon (if only there were more hours in the day) but what I was referring to above is using mpy-cross, to create 'mpy' files - I thought I could just dump these into the directory, and load them up like regular .py files?
I definitely will try building my own firmware soon (if only there were more hours in the day) but what I was referring to above is using mpy-cross, to create 'mpy' files - I thought I could just dump these into the directory, and load them up like regular .py files?