urllib / urequests for Micropython

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
Damien
Site Admin
Posts: 640
Joined: Mon Dec 09, 2013 5:02 pm

Re: urllib for Micropython

Post by Damien » Fri Oct 30, 2015 3:50 pm

pfalcon wrote:I thought about implementing "requests" lib work-alike instead (especially as there're talks that requests may be merged into stdlib).
+1

I would much prefer to see a urequests library for uPy than uurllib (which has too many double letters!).

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: urllib for Micropython

Post by pfalcon » Fri Oct 30, 2015 3:56 pm

Code: Select all

sock = ssl.wrap_socket(sock, ...)
# use sock as usual, set timeout, connect, bind, etc
That's an option I also had in mind, but I believe it doesn't work currently with ussl - ssl.wrap_socket() provides object with just stream interface, nothing else. I'd consider allowing to call thru to underlying socket, but then it indeed will require "educating" users that's how it should be done on wipy. Of course, people won't anticipate or remember that, so they will keep doing it otherwise, so there will be need for extensive error checking. In the end, overhead will be same or more than what I proposed, which also impose much less restrictions.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

lucien2k
Posts: 24
Joined: Sun Oct 11, 2015 5:34 pm

Re: urllib for Micropython

Post by lucien2k » Fri Oct 30, 2015 4:21 pm

Pretty easy to change it to work like requests instead.

It is unlikely that we'll end up with 1 monolithic library, it'll probably end up being a bunch of separate ones so you can pick and choose the functions you need (not everyone needs auth, cookies etc. especially when it is likely just API calls).

Damien
Site Admin
Posts: 640
Joined: Mon Dec 09, 2013 5:02 pm

Re: urllib for Micropython

Post by Damien » Fri Oct 30, 2015 4:45 pm

pfalcon wrote:

Code: Select all

sock = ssl.wrap_socket(sock, ...)
# use sock as usual, set timeout, connect, bind, etc
That's an option I also had in mind, but I believe it doesn't work currently with ussl - ssl.wrap_socket() provides object with just stream interface, nothing else.
It does work. cc3200's ssl sockets "inheret" from normal sockets, so they do provide all the methods (eg settimeout).

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: urllib for Micropython

Post by pfalcon » Fri Oct 30, 2015 5:25 pm

So that's what I mean - it may work on cc3200, and does not work on ports which use ussl module based on axTLS.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

lucien2k
Posts: 24
Joined: Sun Oct 11, 2015 5:34 pm

Re: urllib for Micropython

Post by lucien2k » Sun Nov 08, 2015 5:17 pm

I have added a first version of urequests.

Supports get, post, custom http headers, cookies, basic http auth etc. Works in a very similar way to python-requests.

Available at https://github.com/lucien2k/wipy-urllib ... equests.py

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: urllib / urequests for Micropython

Post by pfalcon » Sat Nov 21, 2015 12:31 am

Absolutely minimal implementation of urlopen() is now available in micropython-lib's urllib.urequest module: https://pypi.python.org/pypi/micropytho ... b.urequest
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

JoePyC
Posts: 2
Joined: Sat Jun 11, 2016 5:12 pm

Re: urllib / urequests for Micropython

Post by JoePyC » Sat Jun 11, 2016 7:51 pm

I'm fairly new to python in general so I apologize if I'm missing something obvious here but I can't seem to get the urlopen command to work at all. Just using the original example of f=urllib.urlopen('http://www.microsoft.com') or any other url in its place gives me errors:

File "/flash/lib/urllib.py", line 82, in urlopen
File "/flash/lib/urllib.py", line 25, in __init__

I also don't understand the difference between the different urequest libraries that I see. The one that's packaged with wipy-urllib-master and then further in the thread there's one that's called urequest that's slightly different. I'm just trying to use it to load json data from a url so a simple load of the url is all I need.

I've looked through documentation and all the different posts related to the micropython urllib but I don't know enough about it to figure out what's wrong.

Any guidance would be greatly appreciated.

~Joe

(I'm running a Wipy board w/expansion, OS 1.2.0 connected to my home wifi)

User avatar
deshipu
Posts: 1385
Joined: Thu May 28, 2015 5:54 pm

Re: urllib / urequests for Micropython

Post by deshipu » Sat Jun 11, 2016 9:51 pm

That error you pasted is truncated, can you post the full traceback?

JoePyC
Posts: 2
Joined: Sat Jun 11, 2016 5:12 pm

Re: urllib / urequests for Micropython

Post by JoePyC » Sun Jun 12, 2016 8:50 pm

Sure:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/flash/lib/urllib.py", line 82, in urlopen
File "/flash/lib/urllib.py", line 25, in __init__
OSError: 111

Post Reply