Page 1 of 1

[SOLVED] [Unix port] examples/network/ only works for

Posted: Sat Apr 17, 2021 11:23 am
by HermannSW
Both, Unix port MicroPython as well as python3, only work for example going against -- any other https target does not:

OSError -40 for "".

Domain not found for "":

Code: Select all

<_SSLSocket 11f9cf0>
b'HTTP/1.1 500 Domain Not Found\r\nConnection: close\r\nContent-Length: 221\r\nServer: Varnish\r\nRetry-After: 0\r\ncontent-type: text/html\r\nCache-Control: private, no-cache\r\nX-Served-By: cache-fra19157-FRA\r\nAccept-Ranges: bytes\r\nDate: Sat, 17 Apr 2021 11:21:18 GMT\r\nVia: 1.1 varnish\r\n\r\n\n<html>\n<head>\n<title>Fastly error: unknown domain </title>\n</head>\n<body>\n<p>Fastly error: unknown domain: . Please check that this domain has been added to a service.</p>\n<p>Details: cache-fra19157-FRA</p></body></html>'

Bad request for "":

Code: Select all

<_SSLSocket 5c65f8>
b'HTTP/1.1 400 Bad Request\r\nServer: nginx\r\nDate: Sat, 17 Apr 2021 11:24:55 GMT\r\nContent-Type: text/html; charset=iso-8859-1\r\nContent-Length: 226\r\nConnection: close\r\n\r\n<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>400 Bad Request</title>\n</head><body>\n<h1>Bad Request</h1>\n<p>Your browser sent a request that this server could not understand.<br />\n</p>\n</body></html>\n'
What is wrong with the example or with MicroPython?

Re: [Unix port] examples/network/ only works for

Posted: Sat Apr 17, 2021 7:34 pm
by HermannSW
I think the example is wrong (works only for, needs other code for other websites).

Does not work for http without ussl either:

Code: Select all

pi@raspberrypi4B:~/pico/micropython/ports/unix $ cat 
import usocket as _socket

s.write(b"GET / HTTP/1.0\r\n\r\n")
pi@raspberrypi4B:~/pico/micropython/ports/unix $

Something is missing in that short script for not getting HTTP 403 forbidden response:

Code: Select all

pi@raspberrypi4B:~/pico/micropython/ports/unix $ ./micropython 
b'HTTP/1.1 403 Forbidden\r\nServer: CloudFront\r\nDate: Sat, 17 Apr 2021 19:27:15 GMT\r\nContent-Type: text/html\r\nContent-Length: 915\r\nX-Cache: Error from cloudfront\r\nX-Amz-Cf-Pop: HAM50-C1\r\nX-Amz-Cf-Id: Q0E5gegX2lk3nb_TA6adbFNmnWJ68LVkaUihFkI7dr_vHzE79GSUgQ==\r\nX-Cache: MISS from\r\nX-Cache-Lookup: MISS from\r\nVia: 1.1 (CloudFront), 1.1 (squid/4.4)\r\nConnection: close\r\n\r\n<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "">\n<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">\n<TITLE>ERROR: The request could not be satisfied</TITLE>\n</HEAD><BODY>\n<H1>403 ERROR</H1>\n<H2>The request could not be satisfied.</H2>\n<HR noshade size="1px">\nBad request.\nWe can\'t connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.\n<BR clear="all">\nIf you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.\n<BR clear="all">\n<HR noshade size="1px">\n<PRE>\nGenerated by cloudfront (CloudFront)\nRequest ID: Q0E5gegX2lk3nb_TA6adbFNmnWJ68LVkaUihFkI7dr_vHzE79GSUgQ==\n</PRE>\n<ADDRESS>\n</ADDRESS>\n</BODY></HTML>'
pi@raspberrypi4B:~/pico/micropython/ports/unix $

"curl" does not have any issue to access ...

Code: Select all

$ curl
        <title>NeverSSL - helping you get online</title>

        body {
             font-family: Montserrat, helvetica, arial, sans-serif; 
             font-size: 16x;
             color: #444444;
             margin: 0;
        h2 {
            font-weight: 700;
            font-size: 1.6em;
            margin-top: 30px;
        p {
            line-height: 1.6em;
        .container {
            max-width: 650px;
            margin: 20px auto 20px auto;
            padding-left: 15px;
            padding-right: 15px
        .header {
            background-color: #42C0FD;
            color: #FFFFFF;
            padding: 10px 0 10px 0;
            font-size: 2.2em;
        <!-- CSS from Mark Webster -->

    <div class="header">
        <div class="container">
    <div class="content">
    <div class="container">

    <p>This website is for when you try to open Facebook, Google, Amazon, etc
    on a wifi network, and nothing happens. Type ""
    into your browser's url bar, and you'll be able to log on.</p>

    <p> will never use SSL (also known as TLS). No
    encryption, no strong authentication, no <a
    no HTTP/2.0, just plain old unencrypted HTTP and forever stuck in the dark
    ages of internet security.</p> 

    <p>Normally, that's a bad idea. You should always use SSL and secure
    encryption when possible. In fact, it's such a bad idea that most websites
    are now using https by default.</p>
    <p>And that's great, but it also means that if you're relying on
    poorly-behaved wifi networks, it can be hard to get online.  Secure
    browsers and websites using https make it impossible for those wifi
    networks to send you to a login or payment page. Basically, those networks
    can't tap into your connection just like attackers can't. Modern browsers
    are so good that they can remember when a website supports encryption and
    even if you type in the website name, they'll use https.</p> 

    <p>And if the network never redirects you to this page, well as you can
    see, you're not missing much.</p>


Re: [Unix port] examples/network/ only works for

Posted: Sat Apr 17, 2021 9:50 pm
by HermannSW domain is really different.

Doing HTTP request with slightly modified script ...

Code: Select all

pi@raspberrypi4B:~/pico/micropython/ports/unix $ diff 
< ai=_socket.getaddrinfo('',80)
> ai=_socket.getaddrinfo('',80)
pi@raspberrypi4B:~/pico/micropython/ports/unix $ 

... shows HTTP 301 response ...

Code: Select all

pi@raspberrypi4B:~/pico/micropython/ports/unix $ ./micropython
b'HTTP/1.1 301 Moved Permanently\r\nLocation:\r\nContent-Type: text/html; charset=UTF-8\r\nDate: Sat, 17 Apr 2021 21:44:58 GMT\r\nExpires: Mon, 17 May 2021 21:44:58 GMT\r\nCache-Control: public, max-age=2592000\r\nServer: gws\r\nContent-Length: 219\r\nX-XSS-Protection: 0\r\nX-Frame-Options: SAMEORIGIN\r\nX-Cache: MISS from\r\nX-Cache-Lookup: MISS from\r\nVia: 1.1 (squid/4.4)\r\nConnection: close\r\n\r\n<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\n<TITLE>301 Moved</TITLE></HEAD><BODY>\n<H1>301 Moved</H1>\nThe document has moved\n<A HREF="">here</A>.\r\n</BODY></HTML>\r\n'
pi@raspberrypi4B:~/pico/micropython/ports/unix $

... as "curl" does as well:

Code: Select all

$ curl
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<H1>301 Moved</H1>
The document has moved
<A HREF="">here</A>.

[SOLVED] Re: [Unix port] examples/network/ only works for

Posted: Thu Apr 22, 2021 8:53 am
by HermannSW
I identified root cause: is a bit sloppy, other http[s] servers need at least "Host" header presented!

I fixed both network http[s] client examples, see this commit on my micropython fork for details: ... cf85fbb86d

Now both work with unix micropython port (eg. http example)

Code: Select all

pi@raspberrypi4B:~/pico/micropython/ports/unix $ ./micropython ../../examples/network/ 
Address infos: [(2, 1, 6, None, bytearray(b'\x02\x00\x00PA\tE\xb1\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 2, 17, None, bytearray(b'\x02\x00\x00PA\tE\xb1\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 3, 0, None, bytearray(b'\x02\x00\x00PA\tE\xb1\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 1, 6, None, bytearray(b'\x02\x00\x00PA\tE\xb8\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 2, 17, None, bytearray(b'\x02\x00\x00PA\tE\xb8\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 3, 0, None, bytearray(b'\x02\x00\x00PA\tE\xb8\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 1, 6, None, bytearray(b'\x02\x00\x00PA\tEx\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 2, 17, None, bytearray(b'\x02\x00\x00PA\tEx\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 3, 0, None, bytearray(b'\x02\x00\x00PA\tEx\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 1, 6, None, bytearray(b'\x02\x00\x00PA\tE\x05\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 2, 17, None, bytearray(b'\x02\x00\x00PA\tE\x05\x00\x00\x00\x00\x00\x00\x00\x00')), (2, 3, 0, None, bytearray(b'\x02\x00\x00PA\tE\x05\x00\x00\x00\x00\x00\x00\x00\x00'))]
Connect address: bytearray(b'\x02\x00\x00PA\tE\xb1\x00\x00\x00\x00\x00\x00\x00\x00')
b'HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nContent-Length: 2536\r\nLast-Modified: Thu, 19 Nov 2020 22:29:21 GMT\r\nAccept-Ranges: bytes\r\nServer: AmazonS3\r\nDate: Thu, 22 Apr 2021 06:35:14 GMT\r\nETag: "e8bb9152091d61caa9d69fed8c4aebc6"\r\nVary: Accept-Encoding\r\nX-Cache: Hit from cloudfront\r\nX-Amz-Cf-Pop: FRA56-C1\r\nX-Amz-Cf-Id: KjXhCDjU-DpeNgjhqq-75fSvu2uyc3C5-FZWYsJWpw5Jh20YMuG7TA==\r\nAge: 8063\r\nX-Cache: HIT from\r\nX-Cache-Lookup: HIT from\r\nVia: 1.1 (CloudFront), 1.1 (squid/4.4)\r\nConnection: close\r\n\r\n<html>\n    <head>\n        <title>NeverSSL - helping you get online</title>\n\n        <style>\n        body {\n             font-family: Montserrat, helvetica, arial, sans-serif; \n             font-size: 16x;\n             color: #444444;\n             margin: 0;\n        }\n        h2 {\n            font-weight: 700;\n            font-size: 1.6em;\n            margin-top: 30px;\n        }\n        p {\n            line-height: 1.6em;\n        }\n        .container {\n            max-width: 650px;\n            margin: 20px auto 20px auto;\n            padding-left: 15px;\n            padding-right: 15px\n        }\n        .header {\n            background-color: #42C0FD;\n            color: #FFFFFF;\n            padding: 10px 0 10px 0;\n            font-size: 2.2em;\n        }\n        <!-- CSS from Mark Webster'
pi@raspberrypi4B:~/pico/micropython/ports/unix $ 

as well as with python3 (eg. https example):

Code: Select all

pi@raspberrypi4B:~/pico/micropython/ports/unix $ python3 ../../examples/network/ 
Address infos: [(2, 1, 6, '', ('', 443)), (2, 2, 17, '', ('', 443)), (2, 3, 0, '', ('', 443))]
Connect address: ('', 443)
<ssl.SSLSocket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('', 58898), raddr=('', 443)>
b'HTTP/1.1 200 OK\r\nServer: nginx\r\nDate: Thu, 22 Apr 2021 08:50:49 GMT\r\nContent-Type: text/html; charset=UTF-8\r\nContent-Length: 290\r\nConnection: close\r\nVary: Accept-Encoding\r\nAccess-Control-Allow-Origin: *\r\nVary: Accept-Encoding\r\n\r\n<pre>60000\n\n.........\n...   ...\n...   ...\n.       .\n.    oo .\n.       .\n...   ...\n...   ...\n.........\n\n.........\n...   ...\n...   ...\n.       .\n.    oX .\n.       .\n...   ...\n...   ...\n.........\n\n.........\n...   ...\n...   ...\n.       .\n.   o   .\n.       .\n...   ...\n...   ...\n.........\n</pre>'
pi@raspberrypi4B:~/pico/micropython/ports/unix $