Using HTTPS and sending mail

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
cool-RR
Posts: 4
Joined: Thu May 30, 2019 2:52 pm

Using HTTPS and sending mail

Post by cool-RR » Thu May 30, 2019 2:57 pm

Hi!

Complete newbie here, using MicroPython on an ESP32 board.

Here are a couple of things I want to do:

1. Access HTTPS URLs. I experimented with `urequests` and it doesn't seem to do any checking on the certificate, which is important.
2. Sending email, hopefully with TLS. I saw that the `smtplib` module from Python isn't part of MicroPython.

Any idea how I can do these two things with MicroPython?

Thanks for your help,
Ram Rachum.

cool-RR
Posts: 4
Joined: Thu May 30, 2019 2:52 pm

Re: Using HTTPS and sending mail

Post by cool-RR » Sat Jun 01, 2019 12:20 pm

Hi, does anyone have a clue about this?

User avatar
jimmo
Posts: 521
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia

Re: Using HTTPS and sending mail

Post by jimmo » Sat Jun 01, 2019 1:30 pm

cool-RR wrote:
Thu May 30, 2019 2:57 pm
1. Access HTTPS URLs. I experimented with `urequests` and it doesn't seem to do any checking on the certificate, which is important.
HTTPS is supported.

Cert validation depends on which board you have. e.g. ESP8266 no, ESP32 & PYBD apparently yes.
cool-RR wrote:
Thu May 30, 2019 2:57 pm
2. Sending email, hopefully with TLS. I saw that the `smtplib` module from Python isn't part of MicroPython.
There's a few mentions of this in the forum or searching Google for "micropython smtp". Have you looked at https://github.com/shawwwn/uMail

cool-RR
Posts: 4
Joined: Thu May 30, 2019 2:52 pm

Re: Using HTTPS and sending mail

Post by cool-RR » Sat Jun 01, 2019 1:49 pm

Thank you for helping Jimmo.
jimmo wrote:
Sat Jun 01, 2019 1:30 pm
HTTPS is supported.

Cert validation depends on which board you have. e.g. ESP8266 no, ESP32 & PYBD apparently yes.
I'm using an ESP32 board and I tested a bad certificate using https://untrusted-root.badssl.com/ and the request succeeded, meaning that certificate validation isn't implemented or is buggy.
jimmo wrote:
Sat Jun 01, 2019 1:30 pm
There's a few mentions of this in the forum or searching Google for "micropython smtp". Have you looked at https://github.com/shawwwn/uMail
uMail looks good, thank you! I'd still have the same SSL problem because of no validation, but I guess I'll have to live with that.

Thanks Jimmo!

User avatar
jimmo
Posts: 521
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia

Re: Using HTTPS and sending mail

Post by jimmo » Sat Jun 01, 2019 2:25 pm

cool-RR wrote:
Sat Jun 01, 2019 1:49 pm
I'm using an ESP32 board and I tested a bad certificate using https://untrusted-root.badssl.com/ and the request succeeded, meaning that certificate validation isn't implemented or is buggy.
Ah yeah sorry I misinterpreted a comment I saw somewhere. It seems that even though these ports use mbedtls, cert validation is not enabled by default.

If you're willing to try building your own ESP32 firmware, you might want to try changing

Code: Select all

    mbedtls_ssl_conf_authmode(&o->conf, MBEDTLS_SSL_VERIFY_NONE);
to

Code: Select all

    mbedtls_ssl_conf_authmode(&o->conf, MBEDTLS_SSL_VERIFY_REQUIRED);
in modussl_mbedtls.c and see if that works for you.

Might be a pretty simple change to add the same handling of ussl.CERT_NONE / ussl.CERT_OPTIONAL / ussl.CERT_REQUIRED as used by the cc3200 port.

cool-RR
Posts: 4
Joined: Thu May 30, 2019 2:52 pm

Re: Using HTTPS and sending mail

Post by cool-RR » Sat Jun 01, 2019 2:44 pm

If I were to open an issue for this, should it be on MicroPython or MicroPython-lib?

User avatar
jimmo
Posts: 521
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia

Re: Using HTTPS and sending mail

Post by jimmo » Sun Jun 02, 2019 1:05 pm

cool-RR wrote:
Sat Jun 01, 2019 2:44 pm
If I were to open an issue for this, should it be on MicroPython or MicroPython-lib?
MicroPython.

I think the actual bug is https://github.com/micropython/micropython/issues/3687 (pretty much just the title is the relevant detail).

The mbedtls implementation of ussl.wrap_socket (which is used by micropython-lib/urequests) currently ignores the `cert_reqs` and `ca_certs` kwargs.

For anyone who finds this thread looking for more info about "why isn't there certificate validation" (and the somewhat cryptic warning at https://docs.micropython.org/en/latest/ ... /ussl.html -- "Some implementations of ussl module do NOT validate server certificates, which makes an SSL connection established prone to man-in-the-middle attacks.")

There are four scenarios:
1 - Boards with axTLS (e.g. ESP8266, the Unix port default config) (extmod/modussl_axtls.c)
2 - Boards with mbedtls provided externally (e.g. ESP32 (provided by ESP-IDF), Unix port with mbedtls enabled (provided by system package manager)) (extmod/modussl_mbedtls.c)
3 - Boards with mbedtls build by micropython (e.g. pybd) (also extmod/modussl_mbedtls.c, with additional config in ports/stm32)
4 - Boards with their own TLS (e.g. CC3200 on the WiPy) (ports/cc3200/modussl.c)

(1) seems likely will never support validation
(2) does not currently enable look at cert_reqs / ca_certs,
(3) is not yet merged, but uses (2) anyway
(4) is the one implementation currently where the ussl module does provide validation (use the ussl.CERT_REQUIRED flag and the ca_certs argument)

As far as I can tell, fixing (2) above should be fairly straightforward. Then the only requirement is for the user to get the appropriate root CA .pem file onto their device (which is something you're going to need to think about anyway if you care about cert validation).

Post Reply