ntptime not included with PyBoard firmware

The official PYBD running MicroPython, and its accessories.
Target audience: Users with a PYBD
kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

ntptime not included with PyBoard firmware

Post by kwiley » Mon Apr 05, 2021 10:05 pm

I see mention of an ntptime module in the ESP8266 MicroPython docs, but it doesn't appear to be included with MicroPython on the PyBoard D firmware. Importing it fails. How are other people synchronizing the RTCs of their PyBoard Ds (which have built-in wifi) or PyBoard 1.1s (perhaps with a wifi shield or ethernet module or something to that effect)?

Code: Select all

MicroPython v1.12 on 2019-12-20; PYBD-SF2W with STM32F722IEK
Type "help()" for more information.
>>> import ntptime
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: no module named 'ntptime'
>>> 
Thanks.

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: ntptime not included with PyBoard firmware

Post by davef » Tue Apr 06, 2021 12:52 am

I see quite a few threads talking about external RTC for some STM32 based product. Try searching <external RTC>. The ESP8266 RTC is very inaccurate. Even with using a system timer on the ESP8266 I run ntptime once a day. Probably that explains why it is in the ESP8266 and ESP32 builds.

kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: ntptime not included with PyBoard firmware

Post by kwiley » Tue Apr 06, 2021 2:06 am

Why do I need an external RTC to set the time on the PyBoard? Why not use NTP the way it was always intended to be used? I was just curious why it isn't built into the PyBoard's MicroPython. I've already copied over the version from the ESP8266 subdirectory and it mostly works -- with the exception that the utime.gmtime() it relies on doesn't seem to exist in the PyD's firmware, so now I have to figure out how to to timezone offsets, which is always super tricky.

kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: ntptime not included with PyBoard firmware

Post by kwiley » Tue Apr 06, 2021 2:40 am

Ah. gmtime() is a fairly recent addition. My PyBoard D came with 1.12. Updating the firmware has gotten it all to work. Now I need to figure out how to do timezone offsets.

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: ntptime not included with PyBoard firmware

Post by davef » Tue Apr 06, 2021 3:03 am

viewtopic.php?f=16&t=3675
Especially the last few posts.

User avatar
scruss
Posts: 360
Joined: Sat Aug 12, 2017 2:27 pm
Location: Toronto, Canada
Contact:

Re: ntptime not included with PyBoard firmware

Post by scruss » Tue Apr 06, 2021 4:06 pm

but it's always going to be a bit of a hack, since full Unix tz support requires more memory than we'll ever have available.

Non-wifi MicroPython boards can get their RTC set from rshell and the host computer's time. The RTC might drift a bit, but it's usually good enough for data logging work. NTP needs a whole network stack. You might be able to do something creative with GPS, WWV or R(B)DS radio clocks that's a lot lighter than networking

kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: ntptime not included with PyBoard firmware

Post by kwiley » Tue Apr 06, 2021 6:49 pm

Topics are getting crossed here. My fault since I brought up multiple issues. I'm less concerned with the timezone correction, which I realize is a pretty complex issue, and more concerned with proper NTP synchronization. The basic NTP method offered in the ESP8266 subproject seems to work pretty well on the PyBoardD. I'm curious why it hasn't been folded into the main branch, or at least the PyBoardD branch so that it would be natively available on PyD without copying the file over, as I did (also, see my other post about a serious error I'm experiencing. It still isn't working properly: viewtopic.php?f=20&t=10221).

Once I get basic NTP working without crashing, as per that other thread, I am still curious about subsecond synchronization. I am going to start experimenting with a GPS approach later this week in fact, just as you mentioned in the previous comment.

Cheers!

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: ntptime not included with PyBoard firmware

Post by pythoncoder » Wed Apr 07, 2021 4:39 pm

You might be interested in these notes on Pyboard D RTC accuracy and calibration.
Peter Hinch
Index to my micropython libraries.

kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: ntptime not included with PyBoard firmware

Post by kwiley » Wed Apr 07, 2021 6:04 pm

That's cool! I wasn't aware of that device. You still have to set the external device though. I'm going to try with a GPS later this week when the module arrives.

BTW, the timezone correction is a piece of cake. It doesn't require all the memory in the world, thereby overloading the little PyBoard's capacity, as was implied in some other responses. All you do is convert the GMT time to an epoch time (bearing in mind that PyBoard epochs start at the millennium), roll the epoch by your timezone offset in seconds (timezone offset in hours * 3600), then convert that epoch time back to a datetime tuple and feed the result to the RTC initialization routine (you have to be careful here, since for perplexing and confusing reasons, the datetime tuple isn't in the same order now! and incurs an off-by-one discrepancy in the weekday value! -- WHY?!).

Anyway, that sets your RTC to a local timezone for practically no effort. It's so straightforward that I suspect I missed something in the earlier discussion about how hard this ought to be. What's the problem? Here's how it works:

Code: Select all

content = http_get('http://micropython.org/ks/test.html')
# t1 = utime.ticks_ms()
start_idx = content.index("Date: ")
end_idx = content.index("\r\n", start_idx+1)
date = content[start_idx+len("Date: "):end_idx]
print("HTTP GM time: {}".format(date))
# Tue, 06 Apr 2021 06:26:18 GMT
words = date.split()
# print(words)
month = months[words[2]]
weekday = weekdays[words[0][:3]]
hms = words[4].split(':')
datetime = [
int(words[3]),
month,
int(words[1]),
int(hms[0]),
int(hms[1]),
int(hms[2]),
weekday + 1,
0,
]
print("HTTP GM time: {}".format(datetime))
jan_1_2000_secs = utime.mktime(datetime)
SECS_PER_HOUR = 3600
TZ = -7
jan_1_2000_secs += SECS_PER_HOUR * TZ
# calc_ms = utime.ticks_ms() - t1
# calc_s = round(calc_ms / 1000)
# jan_1_2000_secs += calc_s  # Add in any time lost while performing these calculations (removed since the cost turned out to be negligible)
datetime = utime.localtime(jan_1_2000_secs)
print("Local time:   {}".format(datetime))
# machine.RTC().datetime(datetime)
machine.RTC().datetime((datetime[0], datetime[1], datetime[2], datetime[6] + 1, datetime[3], datetime[4], datetime[5], 0))
# print("Calc time:    {} {}".format(calc_ms, calc_s))
print("Local time:   {}".format(utime.localtime()))
print("GM time:      {}".format(utime.gmtime()))
You'll notice I'm grabbing the time from an HTTP request. I'm doing that because the DataGram protocol required for an NTP query appears to be broken in MicroPython. I explained this in another thread, even including drop-in copy/paste REPL code to reproduce the problem, but it hasn't received much interest yet (viewtopic.php?f=20&t=10221&sid=e248a328 ... 710d528095). :-(

User avatar
scruss
Posts: 360
Joined: Sat Aug 12, 2017 2:27 pm
Location: Toronto, Canada
Contact:

Re: ntptime not included with PyBoard firmware

Post by scruss » Wed Apr 07, 2021 6:26 pm

I think the off-by-one in the day of the week field might be something to do with traditional RTCs being developed before Posix standards became universal in computers. Almost every RTC I've used has been modelled after the MC146818 from 1984: all BCD fields and two-digit years.

Post Reply