Simple HTTP framework

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
User avatar
ernitron
Posts: 84
Joined: Fri Jun 03, 2016 5:53 pm
Location: The Netherlands

Re: Simple HTTP framework

Post by ernitron » Tue Dec 13, 2016 5:03 pm

Hi @fdushin,

I had a look at your http server framework and it's amazing. I am impressed, really.

I developed my own and it's ugly (I know) but it makes its job and it's extremely compact. But the temptation to use an elegant, robust and versatile httpd framework is huge.

For my purposes this is something I am looking for in a microhttpd framework (let's call these requirements ;)
- Handlers for dynamic functions. Let's say I want to call a function that gets temperature/humidity and format them in nice html using template.
- HTML templating - I wonder if (the remarkable) utemplate by @pfalcon https://github.com/pfalcon/utemplate could be integrated or just another templating mechanism that is small enough to be competitive with custom made dynamic functions that adds html strings to the code.

Of course frozen byte code instead of cross-compiled should be better. But this is up to the developers ;)

Thank you

User avatar
fdushin
Posts: 32
Joined: Thu Jul 21, 2016 5:38 pm

Re: Simple HTTP framework

Post by fdushin » Wed Dec 14, 2016 3:02 pm

@ernitron I appreciate the comments, but really the credit goes to the makers of the ESP8266, the ESP SDK, the developers of micropython, and the broader micropython community, which makes all of this possible.

As far as adding handlers for templates, the framework certainly supports doing something like that. I personally prefer moving as much of the rendering and business logic of the web application to the client-side, since the resources are pretty constrained in an ESP8266. And which the plethora of rich client-side MVC frameworks you can chose from these days, one can theoretically write very rich user experiences without impacting the ESP8266 very much -- it just becomes a file server and a REST endpoint, serving up JSON. Let the browser do all the work!

I also think that for many applications (though maybe not all), web UIs are useful for two modes of operation: 1) initial setup, and 2) troubleshooting. For example, a new device may need to be configured to set up the network and other parameters, and for that, a web UI is suitable for a user who is disinclined to use UART or webrepl. From that point forward (for many applications), the web UI should not be needed until something goes wrong that requires human intervention. At least that is how I have been looking at it, in designing this framework.

Finally, I'd be happy to contribute this framework where it could be of more use to the community (micropython-lib, for example), if that would make it more useful and maintainable down the road, but I am not engaged enough yet with this community to know where that would be.

toddeye
Posts: 1
Joined: Thu Dec 22, 2016 9:19 pm

Re: Simple HTTP framework

Post by toddeye » Thu Dec 22, 2016 11:13 pm

Has anyone got uhttpd to run under micropython unix?

Thanks,
Todd

User avatar
fdushin
Posts: 32
Joined: Thu Jul 21, 2016 5:38 pm

Re: Simple HTTP framework

Post by fdushin » Fri Dec 23, 2016 10:44 pm

I have not. I did a quick test to see if it would work, and while I could get some stuff to work by importing uos instead of os and usocket instead of socket, I did hit a snag with uos.listdir not being defined in unix micro python, which is strange, because I see it in the docs. We could work on fixing these issues, if there is sufficient interest. I have started a branch unix-port, in case anyone is interested.

User avatar
deshipu
Posts: 1348
Joined: Thu May 28, 2015 5:54 pm

Re: Simple HTTP framework

Post by deshipu » Fri Dec 23, 2016 10:50 pm

I think you need to use the os module from here in the linux version: https://github.com/micropython/micropyt ... /master/os

User avatar
fdushin
Posts: 32
Joined: Thu Jul 21, 2016 5:38 pm

Re: Simple HTTP framework

Post by fdushin » Sat Dec 24, 2016 3:49 pm

Using the micropython-lib versions of os and socket gets me further, without using the unix-port branch, though os pull in a lot of additional micropython-lib modules, but that's okay.

The problem is in our use of socket.setsockopt, which the uhttpd.TCPServer class is using (lifted shamelessly from the telnet code). In order to do asynchronous accepts, the uhttpd.TCPServer passes in a function here. The unix implementation of usocket, which the micropython-lib socket module inherits, does not support passing in functions in this way. I suspect there is support for select in the unix port, which is not (yet?) in the esp8266 implementation, but I don't know what the roadmap is for select on esp8266.

So, short answer, no, uhttpd does not run on the unix version of micropython, but it would be nice if it could!

User avatar
pythoncoder
Posts: 3710
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Simple HTTP framework

Post by pythoncoder » Mon Dec 26, 2016 8:47 am

I assume you're aware of the uselect module which is available on the ESP8266 ( source in extmod/moduselect.c)? I think this is an area under active development in relation to performing asynchronous I/O with uasyncio.
Peter Hinch

Adam5Wu
Posts: 6
Joined: Wed Dec 28, 2016 3:30 am

Re: Simple HTTP framework

Post by Adam5Wu » Thu Dec 29, 2016 4:29 am

Fantastic work, fdushin!
You project saves me considerable time and efforts in building a small Web application on my ESP8266.

I do have one small suggestion:
Since current implementation is blocking and handles one request at a time, downgrading to HTTP/1.0 will yield a better user experience.

Some browsers, like Google Chrome likes to fire multiple parallel connections to HTTP/1.1 servers to speed up loading.
But for current implementation, it is doing the reverse -- parallel connections are not acknowledged, and eventually have to timeout, before the next request can proceed, meanwhile, the browser simple "hang" while spinning the loading animation.

But if the server report HTTP/1.0, then Google Chrome only use one connection at a time, which makes consecutive request load much faster.

-------
Update: Sorry I was wrong. Changing to HTTP/1.0 does not solve the problem.
Somehow I was a bit lucky during a brief test, but eventually the problem comes back.

It seems non-blocking HTTP request handling is necessary to make Google Chrome work nicely.

User avatar
fdushin
Posts: 32
Joined: Thu Jul 21, 2016 5:38 pm

Re: Simple HTTP framework

Post by fdushin » Fri Dec 30, 2016 1:45 am

Interesting. You might try increasing the backlog on TCP connections. I've pushed a change to master which allows you to specify the backlog via config, e.g.,

Code: Select all

server = uhttpd.Server([
        ('/api', api_handler),
        ('/test', file_handler)
    ], {
        'require_auth': True,
        'backlog': 5
    })
The backlog is now defaulted to 5 on master (it was hard coded to 0, before).

However, I am definitely getting spurious errors testing this with a simple concurrent test here

E.g.,

Code: Select all

Traceback (most recent call last):
  File "ulog.py", line 99, in do_log
  File "console_sink.py", line 37, in log
RuntimeError: maximum recursion depth exceeded
Traceback (most recent call last):
  File "uhttpd.py", line 157, in handle_request
  File "http_file_handler.py", line 106, in handle_request
  File "http_file_handler.py", line 113, in create_file_response
  File "http_file_handler.py", line 124, in generate_file
  File "http_file_handler.py", line 221, in file_size
RuntimeError: maximum recursion depth exceeded
which doesn't make any sense to me, as I have not defined any recursive functions (to my knowledge). I wonder if I am just running out of stack space.

User avatar
Roberthh
Posts: 1739
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Simple HTTP framework

Post by Roberthh » Fri Dec 30, 2016 9:07 am

I could result from the fact, that the code is not using threading, but handles incoming requests in a callback mode. You use the callback mechanism, introduced by @pfalcon a a kind of hack for webrepl, and then was re-used by my ftp-server and @chrisgp's telnet-server. Being a callback, it cannot really serve nested requests, especially if these call the underlying library functions of micropython.
In my ftp-server, I therefore reject requests immediately if already one request is in process. That works reasonably well. Most clients can deal with that, except OS X's finder.
@pythoncoder made a comment about using asyncio and select, which might be the technically better solution for a server being the main task on the device.

Post Reply