Questions about ntptime

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
Post Reply
User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Questions about ntptime

Post by liudr » Wed Dec 01, 2021 5:58 am

I'm looking for ways to get time with ntptime and this is the only official reference I found:

http://docs.micropython.org/en/latest/e ... -clock-rtc

So here is how to do it:

Code: Select all

ntptime.settime() # set the rtc datetime from the remote server
The issue is, this function calls another function time() and that function has no exception handling:
This whole thing is under ESP8266:

Code: Select all

def time():
    NTP_QUERY = bytearray(48)
    NTP_QUERY[0] = 0x1B
    addr = socket.getaddrinfo(host, 123)[0][-1]
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    try:
        s.settimeout(1)
        res = s.sendto(NTP_QUERY, addr)
        msg = s.recv(48)
    finally:
        s.close()
    val = struct.unpack("!I", msg[40:44])[0]
    return val - NTP_DELTA
See there is no except clause. This at times works but eventually throws errors:

Traceback (most recent call last):
File "main.py", line 1, in <module>
File "logger.py", line 130, in <module>
File "ntptime.py", line 34, in settime
File "ntptime.py", line 25, in time
OSError: [Errno 116] ETIMEDOUT

Line 25 is s.recv(48). I guess the magic number of 1 second didn't cut it. I wonder if I should just handle the exception or increase 1 second to some other magic number instead.

Also, I would like to convert this datetime tuple into timestamp. I tried a few ways but could get it right.

I know utime.mktime() should be the right tool but it seems to only create a time stamp based off 2000, not 1970.

So I came up with this:

Code: Select all

0x100000000-a1970+time.mktime(time.localtime())
Where:

Code: Select all

a1970=time.mktime((1970,1,1,0,0,0,0,0))
This seems to give me the correct timestamp.

So is there any more proper way to get a unix time stamp? Thanks

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: Questions about ntptime

Post by davef » Wed Dec 01, 2021 6:45 am

My solution to ETIMEOUT:

Code: Select all

try:
    import usocket as socket
except:
    import socket
try:
    import ustruct as struct
except:
    import struct
import utime


# (date(2000, 1, 1) - date(1900, 1, 1)).days * 24*60*60
NTP_DELTA = 3155673600

# The NTP host can be configured at runtime by doing: ntptime.host = 'myhost.org'
host = "pool.ntp.org"


def time():
    NTP_QUERY = bytearray(48)
    NTP_QUERY[0] = 0x1B
    addr = socket.getaddrinfo(host, 123)[0][-1]
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    counter = 0

 #  try 5 times to get the time
    while True:
        counter +=1

        try:
            s.settimeout(10) #  was 1 second
            res = s.sendto(NTP_QUERY, addr)
            msg = s.recv(48)
            counter = 0
            s.close()
            break
        except Exception:
            try:
                with open('errors.txt', 'a') as outfile:
                    outfile.write('getting the time failed' + '\n')
            except OSError:
                pass

        utime.sleep(5)

        if counter == 5:
            counter = 0
            utime.sleep(80) #  force a machine.reset()

    val = struct.unpack("!I", msg[40:44])[0]
    return val - NTP_DELTA


# There's currently no timezone support in MicroPython, and the RTC is set in UTC time.
def settime():
    t = time()
    if t < 0:
        return False
    import machine
    import utime

    tm = utime.gmtime(t)
    machine.RTC().datetime((tm[0], tm[1], tm[2], tm[6] + 1, tm[3], tm[4], tm[5], 0))
not sure it helps.

P.S. I have a software wdt running.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Questions about ntptime

Post by dhylands » Wed Dec 01, 2021 7:34 pm

The difference between the Micropython 2000 epoch and the unix 1970 is a constant. You can calculate that constant in CPython by using:

Code: Select all

import calendar
epoch_1970 = (1970, 1, 1, 0, 0, 0, 0, 0)
epoch_2000 = (2000, 1, 1, 0, 0, 0, 0, 0)
time_offset = calendar.timegm(epoch_2000) - calendar.timegm(epoch_1970)
and that will give a time_offset of 946684800 which you can code into your program, and add it to the epoch 2000 timestamp to come up with an epoch 1970 timestamp.

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Questions about ntptime

Post by liudr » Wed Dec 01, 2021 11:54 pm

dhylands wrote:
Wed Dec 01, 2021 7:34 pm
The difference between the Micropython 2000 epoch and the unix 1970 is a constant. You can calculate that constant in CPython by using:

Code: Select all

import calendar
epoch_1970 = (1970, 1, 1, 0, 0, 0, 0, 0)
epoch_2000 = (2000, 1, 1, 0, 0, 0, 0, 0)
time_offset = calendar.timegm(epoch_2000) - calendar.timegm(epoch_1970)
and that will give a time_offset of 946684800 which you can code into your program, and add it to the epoch 2000 timestamp to come up with an epoch 1970 timestamp.
Thanks! I didn't know about the calendar module. It's not built-in, or in the stdlib or ecosys. Is it a 3rd-party contributed module for Python or MP?

I checked my offset with 0x100000000-a1970, exactly the same value as yours.

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Questions about ntptime

Post by liudr » Wed Dec 01, 2021 11:56 pm

Thanks @davef!

I'll see if I can understand your code and incorporate into my script.

Curious, are those exception handling in import supposed to make your script run on both MP and Python (or another platform)?

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: Questions about ntptime

Post by davef » Thu Dec 02, 2021 12:01 am

I only modified the existing script. The change in def settime was by Peter Hinch.

The import statements ... good question ... I think it gives the option to call some Python libraries into your Micropython script. Ones with more features.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Questions about ntptime

Post by dhylands » Thu Dec 02, 2021 12:04 am

liudr wrote:
Wed Dec 01, 2021 11:54 pm
Thanks! I didn't know about the calendar module. It's not built-in, or in the stdlib or ecosys. Is it a 3rd-party contributed module for Python or MP?
calendar comes with CPython. You can find the documentation here: https://docs.python.org/3/library/calendar.html

The documentation for the time module: https://docs.python.org/3/library/time.html#module-time has links to it as well.

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Questions about ntptime

Post by liudr » Thu Dec 02, 2021 12:16 am

dhylands wrote:
Thu Dec 02, 2021 12:04 am
liudr wrote:
Wed Dec 01, 2021 11:54 pm
Thanks! I didn't know about the calendar module. It's not built-in, or in the stdlib or ecosys. Is it a 3rd-party contributed module for Python or MP?
calendar comes with CPython. You can find the documentation here: https://docs.python.org/3/library/calendar.html

The documentation for the time module: https://docs.python.org/3/library/time.html#module-time has links to it as well.
So did you just load the calendar module in a lib/ folder on your MP board?

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Questions about ntptime

Post by dhylands » Thu Dec 02, 2021 1:51 am

I only use calander on the host (under CPython). And since the offset you're looking for is a constant that can't ever change you can just hard-code it into your micropython code. No need to use calendar to calculate it.

User avatar
liudr
Posts: 211
Joined: Tue Oct 17, 2017 5:18 am

Re: Questions about ntptime

Post by liudr » Thu Dec 02, 2021 2:38 am

dhylands wrote:
Thu Dec 02, 2021 1:51 am
I only use calander on the host (under CPython). And since the offset you're looking for is a constant that can't ever change you can just hard-code it into your micropython code. No need to use calendar to calculate it.
Right thanks. I really don't need the module, just the offset.

Post Reply