urequests and SSL on WiPy

Questions and discussion about The WiPy 1.0 board and CC3200 boards.
Target audience: Users with a WiPy 1.0 or CC3200 board.
danielm
Posts: 167
Joined: Mon Oct 05, 2015 12:24 pm

urequests and SSL on WiPy

Post by danielm » Fri Sep 02, 2016 10:38 am

I am trying to perform HTTP GET request on https://www.google.com:

Code: Select all

urequests.get("https://www.google.com")
Empty response is read from the socket which causes error I already encountered before: http://forum.micropython.org/viewtopic.php?f=15&t=2191

Anyway I think that SSL socket is not established correctly. I exported CA certificate in DER format as described here: http://processors.wiki.ti.com/index.php ... pplication
CA Certificate can be downloaded using various methods. For Example, On Windows 7 machine, procedure to Download CA Certificate of http://www.google.com is:

Press Start button
Typing certmgr.msc into the Search box, and then pressing ENTER.
Double Click Trusted Root Certificate Authorities
Double Click Certificate
Look for "Equifax Secure CA"
Double Click on it. It will open the Certificate.
Select Details Tab
Click on “Copy to File” button and export the certificate as .cer Format
Then I tried to flash it first time as "/cert/ca.pem" and second time as "/cert/129.der" as it is suggested in that wiki page. I always got empty response from the server.

Is there any way how to verify ssl socket?

Sending GET request to "http://www.google.com" returns redirect which is ok. I used urequests module version 0.4 from micropython-lib.

danielm
Posts: 167
Joined: Mon Oct 05, 2015 12:24 pm

Re: urequests and SSL on WiPy

Post by danielm » Fri Sep 02, 2016 12:16 pm

Looking at urequests.py module it wraps the socket without any parameters on line 56:

Code: Select all

s = ussl.wrap_socket(s)
In my understanding that means that ussl.wrap_scoket() is called with default values:

Code: Select all

ssl.wrap_socket(sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, ca_certs=None)
To use flashed certificates I think it should be called with cert_reqs=ssl.CERT_REQUIRED, ca_certs='/flash/cert/ca.pem'

Not sure what this causes on server side.

jgmdavies
Posts: 57
Joined: Tue Aug 09, 2016 2:39 pm

Re: urequests and SSL on WiPy

Post by jgmdavies » Fri Sep 02, 2016 1:01 pm

When creating the cert file, Win10 gives two options for .CER files:
  • - DER encoded binary X.509 (.CER)
    - Base-64 encoded X.509 (.CER)
Which is best for the WiPy code please?

Thanks,
Jim

ubiq_01
Posts: 26
Joined: Mon Feb 08, 2016 1:45 pm
Location: Dresden, Germany

Re: urequests and SSL on WiPy

Post by ubiq_01 » Fri Sep 02, 2016 2:43 pm

I also worked with the urequests module yesterday (see https://twitter.com/_rac01/status/771378341270159360 ) - and in order to use it with SSL on the WiPy I had to modify
# s = usocket.socket()
s = usocket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_SEC)
(I used the module from a stock micropython-lib unix insstallation - maybe it has this SSL specific in the WiPy version already.)

best regards
Ralf

danielm
Posts: 167
Joined: Mon Oct 05, 2015 12:24 pm

Re: urequests and SSL on WiPy

Post by danielm » Fri Sep 02, 2016 3:40 pm

Jim, DER format is mentioned in TI wiki I posted before.

Ralf, I will try that modification on Monday, thanks.

jgmdavies
Posts: 57
Joined: Tue Aug 09, 2016 2:39 pm

Re: urequests and SSL on WiPy

Post by jgmdavies » Fri Sep 02, 2016 7:35 pm

Thanks, danielm.

danielm
Posts: 167
Joined: Mon Oct 05, 2015 12:24 pm

Re: urequests and SSL on WiPy

Post by danielm » Sun Sep 04, 2016 7:02 pm

Jim, no problem.

Ralf, I did modify socket constructor as you suggested and you were right, it worked.

On top of that, I did modify SSL wrapper:

Code: Select all

s = ussl.wrap_socket(s, keyfile=None, certfile=None, server_side=False, cert_reqs=ussl.CERT_REQUIRED, ca_certs='/flash/cert/ca.pem')
The problem now is that it seems that my SSL client does not validate server's certificate - I am able to successfully perform GET request also with incorrect CA certificate.
Last edited by danielm on Sun Sep 04, 2016 7:24 pm, edited 1 time in total.

danielm
Posts: 167
Joined: Mon Oct 05, 2015 12:24 pm

Re: urequests and SSL on WiPy

Post by danielm » Sun Sep 04, 2016 7:17 pm

Maybe it has something to do with server_side parameter set to False. However setting it to True throws this error:
ValueError: invalid argument(s) value

jgmdavies
Posts: 57
Joined: Tue Aug 09, 2016 2:39 pm

Re: urequests and SSL on WiPy

Post by jgmdavies » Mon Sep 05, 2016 1:34 pm

@ubiq_01

Hi Ralf,

Many thanks for the tip about

Code: Select all

s = usocket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_SEC)
in urequests.py. I can now use HTTPS and POST to services like ThingSpeak, which was failing before with an empty response from the server (leading to "wrong number of values to unpack' errors in urequests.py).

I ended up with:

Code: Select all

	if proto == "https:":
		s = usocket.socket(usocket.AF_INET, usocket.SOCK_STREAM, usocket.IPPROTO_SEC)
	else:
		s = usocket.socket()
Best,
Jim

danielm
Posts: 167
Joined: Mon Oct 05, 2015 12:24 pm

Re: urequests and SSL on WiPy

Post by danielm » Mon Sep 05, 2016 3:55 pm

Jim, did you also test server certificate validation with locally stored CA certificate?

Post Reply