Resolving local network domain name

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Resolving local network domain name

Post by liudr » Thu Nov 25, 2021 5:38 pm

I have an ESP32 board running MP 1.17. I use urequests to get and post to a server on my local network So far if I give it an ip address such as 192.168.0.123, it works. If I post to a remote host with address like abc.duckdns.org, it works too. This means MP successfully did domain name lookup, I guess.

But, I installed a Debian machine and left it with a default domain name "debian". On PC, using requests, I was able to reach it by just providing debian as address. I was also able to reach debian's web server with http://debian in my address bar on firefox as well.

But, if I use debian as address for ESP32, it reports an error:

Code: Select all

>>> import usocket
>>> usocket.getaddrinfo('debian', 80, 0, usocket.SOCK_STREAM)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: -202
>>> usocket.getaddrinfo('192.168.0.191', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 0, '192.168.0.191', ('192.168.0.191', 80))]
>>> 
As you can see above, looking up debian failse with -202, while looking up an ip address got the right response.

I wonder if this is only on ESP32 port or all MCU ports.

I also installed the unix port on my debian machine using a snap command. Did the same test and seems fine:

Code: Select all

>>> import usocket
>>> usocket.getaddrinfo('debian', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 6, None, bytearray(b'\x02\x00\x00P\x7f\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00'))]
>>> usocket.getaddrinfo('192.168.0.191', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 6, None, bytearray(b'\x02\x00\x00P\xc0\xa8\x00\xbf\x00\x00\x00\x00\x00\x00\x00\x00'))]

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Resolving local network domain name

Post by liudr » Thu Nov 25, 2021 6:02 pm

One improvement:

I renamed my debian machine to debian.local. This passed the getaddinfo()

Code: Select all

>>> import usocket
>>> usocket.getaddrinfo('debian.local', 80, 0, usocket.SOCK_STREAM)
[(2, 0, 0, 'debian.local', ('192.168.0.191', 80))]
But I still can't get requests.get() to work:

Code: Select all

>>> import urequests as requests
>>> results=requests.get('http://debian.local:80')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "urequests.py", line 116, in get
  File "urequests.py", line 58, in request
OSError: [Errno 22] EINVAL
Again I tested with IP address and it works:

Code: Select all

>>> results=requests.get('http://192.168.0.191:80')
>>> results
<Response object at 3f82e0c0>
>>> results.content
b'\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.

User avatar
scruss
Posts: 360
Joined: Sat Aug 12, 2017 2:27 pm
Location: Toronto, Canada
Contact:

Re: Resolving local network domain name

Post by scruss » Thu Nov 25, 2021 7:01 pm

.local is a mDNS/DNS-SD thing (aka Bonjour). Trying to resolve it from a client that doesn't know these protocols won't work.

Your Debian box will have avahi running, which is doing all the multicast DNS magic for you.

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Resolving local network domain name

Post by liudr » Thu Nov 25, 2021 7:50 pm

OK, so I was going through the source code of urequests, specifically request():

https://github.com/micropython/micropyt ... equests.py

I followed its logic in REPL more or less but didn't do the " s = ussl.wrap_socket(s, server_hostname=host)" line. That might be the source of the problem when I supplied a "debian.local".

So what I did was to call usocket.getaddrinfo(host, port, 0, usocket.SOCK_STREAM) to return the (ip address, port) tuple and replaced the "debian.local" with this ip address. Then it works.

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Resolving local network domain name

Post by liudr » Thu Nov 25, 2021 8:02 pm

scruss wrote:
Thu Nov 25, 2021 7:01 pm
.local is a mDNS/DNS-SD thing (aka Bonjour). Trying to resolve it from a client that doesn't know these protocols won't work.

Your Debian box will have avahi running, which is doing all the multicast DNS magic for you.
Thanks scruss. I figured out the issue was the return values of getaddrinfo ai[1] passed to s = usocket.socket(ai[0], ai[1], ai[2])

If you see the results I got with "debian.local" vs "192.168.0.191", the value ai[1] is 0 in former and 1 in latter (2,0,0) vs (2,1,0):

Code: Select all

>>> usocket.getaddrinfo('debian.local', 80, 0, usocket.SOCK_STREAM)
[(2, 0, 0, 'brain.local', ('192.168.0.191', 80))]

>>> usocket.getaddrinfo('192.168.0.191', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 0, '192.168.0.191', ('192.168.0.191', 80))]
I wish I knew what that value means. It's used to create a socket and that fails.

User avatar
karfas
Posts: 193
Joined: Sat Jan 16, 2021 12:53 pm
Location: Vienna, Austria

Re: Resolving local network domain name

Post by karfas » Fri Nov 26, 2021 10:48 pm

liudr wrote:
Thu Nov 25, 2021 8:02 pm

Code: Select all

>>> usocket.getaddrinfo('debian.local', 80, 0, usocket.SOCK_STREAM)
[(2, 0, 0, 'brain.local', ('192.168.0.191', 80))]
>>> usocket.getaddrinfo('192.168.0.191', 80, 0, usocket.SOCK_STREAM)
[(2, 1, 0, '192.168.0.191', ('192.168.0.191', 80))]
This looks like a bug for me.
I will have to dig out one of my ESPs to confirm this, but the first call returns a SOCK_DGRAM (contrary to the requested SOCK_STREAM in the getaddrinfo() call) in the second element of the outer tuple.
Most likely, the socket.connect() will succeed, but it will not be useable for a HTTP request.

What is your exact micropython version and ESP variant?
Did you compile yourself (if yes: with which ESP-IDF version) or are you using a prebuilt image ?
A few hours of debugging might save you from minutes of reading the documentation! :D
My repositories: https://github.com/karfas

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Resolving local network domain name

Post by liudr » Fri Nov 26, 2021 10:54 pm

1.17 released version for ESP32 generic with SPIRAM
Thanks for looking over it.

User avatar
karfas
Posts: 193
Joined: Sat Jan 16, 2021 12:53 pm
Location: Vienna, Austria

Re: Resolving local network domain name

Post by karfas » Sat Nov 27, 2021 10:45 am

I just added the following issues: However, it might last weeks/months/years until this gets fixed.

For your urequest problem, there is an easy workaround:
in urequest.py the socket gets created as

Code: Select all

    
    ai = usocket.getaddrinfo(host, port, 0, usocket.SOCK_STREAM)
    ai = ai[0]

    s = usocket.socket(ai[0], ai[1], ai[2])
    try:
        s.connect(ai[-1])
You can change the socket() call to

Code: Select all

    s = usocket.socket(ai[0], usocket.SOCK_STREAM, ai[2])
Regards,
Thomas
A few hours of debugging might save you from minutes of reading the documentation! :D
My repositories: https://github.com/karfas

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Resolving local network domain name

Post by liudr » Sun Nov 28, 2021 5:25 pm

Thanks @karfas for raising the issues for me. From my tests, calling getaddrinfo() if the host name is just "debian" caused exceptions so I went for a dotted name and that got me past getaddrinfo(). So I first call this and then extract the number-dot IP address and use that to call request().

It would be nice to resolve this problem in urequests though.

Post Reply