Page 1 of 1

Resolving local network domain name

Posted: Thu Nov 25, 2021 5:38 pm
by liudr
I have an ESP32 board running MP 1.17. I use urequests to get and post to a server on my local network So far if I give it an ip address such as 192.168.0.123, it works. If I post to a remote host with address like abc.duckdns.org, it works too. This means MP successfully did domain name lookup, I guess.

But, I installed a Debian machine and left it with a default domain name "debian". On PC, using requests, I was able to reach it by just providing debian as address. I was also able to reach debian's web server with http://debian in my address bar on firefox as well.

But, if I use debian as address for ESP32, it reports an error:

Code: Select all

>>> import usocket
>>> usocket.getaddrinfo('debian', 80, 0, usocket.SOCK_STREAM)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: -202
>>> usocket.getaddrinfo('192.168.0.191', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 0, '192.168.0.191', ('192.168.0.191', 80))]
>>> 
As you can see above, looking up debian failse with -202, while looking up an ip address got the right response.

I wonder if this is only on ESP32 port or all MCU ports.

I also installed the unix port on my debian machine using a snap command. Did the same test and seems fine:

Code: Select all

>>> import usocket
>>> usocket.getaddrinfo('debian', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 6, None, bytearray(b'\x02\x00\x00P\x7f\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00'))]
>>> usocket.getaddrinfo('192.168.0.191', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 6, None, bytearray(b'\x02\x00\x00P\xc0\xa8\x00\xbf\x00\x00\x00\x00\x00\x00\x00\x00'))]

Re: Resolving local network domain name

Posted: Thu Nov 25, 2021 6:02 pm
by liudr
One improvement:

I renamed my debian machine to debian.local. This passed the getaddinfo()

Code: Select all

>>> import usocket
>>> usocket.getaddrinfo('debian.local', 80, 0, usocket.SOCK_STREAM)
[(2, 0, 0, 'debian.local', ('192.168.0.191', 80))]
But I still can't get requests.get() to work:

Code: Select all

>>> import urequests as requests
>>> results=requests.get('http://debian.local:80')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "urequests.py", line 116, in get
  File "urequests.py", line 58, in request
OSError: [Errno 22] EINVAL
Again I tested with IP address and it works:

Code: Select all

>>> results=requests.get('http://192.168.0.191:80')
>>> results
<Response object at 3f82e0c0>
>>> results.content
b'\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.

Re: Resolving local network domain name

Posted: Thu Nov 25, 2021 7:01 pm
by scruss
.local is a mDNS/DNS-SD thing (aka Bonjour). Trying to resolve it from a client that doesn't know these protocols won't work.

Your Debian box will have avahi running, which is doing all the multicast DNS magic for you.

Re: Resolving local network domain name

Posted: Thu Nov 25, 2021 7:50 pm
by liudr
OK, so I was going through the source code of urequests, specifically request():

https://github.com/micropython/micropyt ... equests.py

I followed its logic in REPL more or less but didn't do the " s = ussl.wrap_socket(s, server_hostname=host)" line. That might be the source of the problem when I supplied a "debian.local".

So what I did was to call usocket.getaddrinfo(host, port, 0, usocket.SOCK_STREAM) to return the (ip address, port) tuple and replaced the "debian.local" with this ip address. Then it works.

Re: Resolving local network domain name

Posted: Thu Nov 25, 2021 8:02 pm
by liudr
scruss wrote:
Thu Nov 25, 2021 7:01 pm
.local is a mDNS/DNS-SD thing (aka Bonjour). Trying to resolve it from a client that doesn't know these protocols won't work.

Your Debian box will have avahi running, which is doing all the multicast DNS magic for you.
Thanks scruss. I figured out the issue was the return values of getaddrinfo ai[1] passed to s = usocket.socket(ai[0], ai[1], ai[2])

If you see the results I got with "debian.local" vs "192.168.0.191", the value ai[1] is 0 in former and 1 in latter (2,0,0) vs (2,1,0):

Code: Select all

>>> usocket.getaddrinfo('debian.local', 80, 0, usocket.SOCK_STREAM)
[(2, 0, 0, 'brain.local', ('192.168.0.191', 80))]

>>> usocket.getaddrinfo('192.168.0.191', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 0, '192.168.0.191', ('192.168.0.191', 80))]
I wish I knew what that value means. It's used to create a socket and that fails.

Re: Resolving local network domain name

Posted: Fri Nov 26, 2021 10:48 pm
by karfas
liudr wrote:
Thu Nov 25, 2021 8:02 pm

Code: Select all

>>> usocket.getaddrinfo('debian.local', 80, 0, usocket.SOCK_STREAM)
[(2, 0, 0, 'brain.local', ('192.168.0.191', 80))]
>>> usocket.getaddrinfo('192.168.0.191', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 0, '192.168.0.191', ('192.168.0.191', 80))]
This looks like a bug for me.
I will have to dig out one of my ESPs to confirm this, but the first call returns a SOCK_DGRAM (contrary to the requested SOCK_STREAM in the getaddrinfo() call) in the second element of the outer tuple.
Most likely, the socket.connect() will succeed, but it will not be useable for a HTTP request.

What is your exact micropython version and ESP variant?
Did you compile yourself (if yes: with which ESP-IDF version) or are you using a prebuilt image ?

Re: Resolving local network domain name

Posted: Fri Nov 26, 2021 10:54 pm
by liudr
1.17 released version for ESP32 generic with SPIRAM
Thanks for looking over it.

Re: Resolving local network domain name

Posted: Sat Nov 27, 2021 10:45 am
by karfas
I just added the following issues: However, it might last weeks/months/years until this gets fixed.

For your urequest problem, there is an easy workaround:
in urequest.py the socket gets created as

Code: Select all

    
    ai = usocket.getaddrinfo(host, port, 0, usocket.SOCK_STREAM)
    ai = ai[0]

    s = usocket.socket(ai[0], ai[1], ai[2])
    try:
        s.connect(ai[-1])
You can change the socket() call to

Code: Select all

    s = usocket.socket(ai[0], usocket.SOCK_STREAM, ai[2])
Regards,
Thomas

Re: Resolving local network domain name

Posted: Sun Nov 28, 2021 5:25 pm
by liudr
Thanks @karfas for raising the issues for me. From my tests, calling getaddrinfo() if the host name is just "debian" caused exceptions so I went for a dotted name and that got me past getaddrinfo(). So I first call this and then extract the number-dot IP address and use that to call request().

It would be nice to resolve this problem in urequests though.