Page 1 of 2

ntptime not included with PyBoard firmware

Posted: Mon Apr 05, 2021 10:05 pm
by kwiley
I see mention of an ntptime module in the ESP8266 MicroPython docs, but it doesn't appear to be included with MicroPython on the PyBoard D firmware. Importing it fails. How are other people synchronizing the RTCs of their PyBoard Ds (which have built-in wifi) or PyBoard 1.1s (perhaps with a wifi shield or ethernet module or something to that effect)?

Code: Select all

MicroPython v1.12 on 2019-12-20; PYBD-SF2W with STM32F722IEK
Type "help()" for more information.
>>> import ntptime
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: no module named 'ntptime'
>>> 
Thanks.

Re: ntptime not included with PyBoard firmware

Posted: Tue Apr 06, 2021 12:52 am
by davef
I see quite a few threads talking about external RTC for some STM32 based product. Try searching <external RTC>. The ESP8266 RTC is very inaccurate. Even with using a system timer on the ESP8266 I run ntptime once a day. Probably that explains why it is in the ESP8266 and ESP32 builds.

Re: ntptime not included with PyBoard firmware

Posted: Tue Apr 06, 2021 2:06 am
by kwiley
Why do I need an external RTC to set the time on the PyBoard? Why not use NTP the way it was always intended to be used? I was just curious why it isn't built into the PyBoard's MicroPython. I've already copied over the version from the ESP8266 subdirectory and it mostly works -- with the exception that the utime.gmtime() it relies on doesn't seem to exist in the PyD's firmware, so now I have to figure out how to to timezone offsets, which is always super tricky.

Re: ntptime not included with PyBoard firmware

Posted: Tue Apr 06, 2021 2:40 am
by kwiley
Ah. gmtime() is a fairly recent addition. My PyBoard D came with 1.12. Updating the firmware has gotten it all to work. Now I need to figure out how to do timezone offsets.

Re: ntptime not included with PyBoard firmware

Posted: Tue Apr 06, 2021 3:03 am
by davef
viewtopic.php?f=16&t=3675
Especially the last few posts.

Re: ntptime not included with PyBoard firmware

Posted: Tue Apr 06, 2021 4:06 pm
by scruss
but it's always going to be a bit of a hack, since full Unix tz support requires more memory than we'll ever have available.

Non-wifi MicroPython boards can get their RTC set from rshell and the host computer's time. The RTC might drift a bit, but it's usually good enough for data logging work. NTP needs a whole network stack. You might be able to do something creative with GPS, WWV or R(B)DS radio clocks that's a lot lighter than networking

Re: ntptime not included with PyBoard firmware

Posted: Tue Apr 06, 2021 6:49 pm
by kwiley
Topics are getting crossed here. My fault since I brought up multiple issues. I'm less concerned with the timezone correction, which I realize is a pretty complex issue, and more concerned with proper NTP synchronization. The basic NTP method offered in the ESP8266 subproject seems to work pretty well on the PyBoardD. I'm curious why it hasn't been folded into the main branch, or at least the PyBoardD branch so that it would be natively available on PyD without copying the file over, as I did (also, see my other post about a serious error I'm experiencing. It still isn't working properly: viewtopic.php?f=20&t=10221).

Once I get basic NTP working without crashing, as per that other thread, I am still curious about subsecond synchronization. I am going to start experimenting with a GPS approach later this week in fact, just as you mentioned in the previous comment.

Cheers!

Re: ntptime not included with PyBoard firmware

Posted: Wed Apr 07, 2021 4:39 pm
by pythoncoder
You might be interested in these notes on Pyboard D RTC accuracy and calibration.

Re: ntptime not included with PyBoard firmware

Posted: Wed Apr 07, 2021 6:04 pm
by kwiley
That's cool! I wasn't aware of that device. You still have to set the external device though. I'm going to try with a GPS later this week when the module arrives.

BTW, the timezone correction is a piece of cake. It doesn't require all the memory in the world, thereby overloading the little PyBoard's capacity, as was implied in some other responses. All you do is convert the GMT time to an epoch time (bearing in mind that PyBoard epochs start at the millennium), roll the epoch by your timezone offset in seconds (timezone offset in hours * 3600), then convert that epoch time back to a datetime tuple and feed the result to the RTC initialization routine (you have to be careful here, since for perplexing and confusing reasons, the datetime tuple isn't in the same order now! and incurs an off-by-one discrepancy in the weekday value! -- WHY?!).

Anyway, that sets your RTC to a local timezone for practically no effort. It's so straightforward that I suspect I missed something in the earlier discussion about how hard this ought to be. What's the problem? Here's how it works:

Code: Select all

content = http_get('http://micropython.org/ks/test.html')
# t1 = utime.ticks_ms()
start_idx = content.index("Date: ")
end_idx = content.index("\r\n", start_idx+1)
date = content[start_idx+len("Date: "):end_idx]
print("HTTP GM time: {}".format(date))
# Tue, 06 Apr 2021 06:26:18 GMT
words = date.split()
# print(words)
month = months[words[2]]
weekday = weekdays[words[0][:3]]
hms = words[4].split(':')
datetime = [
int(words[3]),
month,
int(words[1]),
int(hms[0]),
int(hms[1]),
int(hms[2]),
weekday + 1,
0,
]
print("HTTP GM time: {}".format(datetime))
jan_1_2000_secs = utime.mktime(datetime)
SECS_PER_HOUR = 3600
TZ = -7
jan_1_2000_secs += SECS_PER_HOUR * TZ
# calc_ms = utime.ticks_ms() - t1
# calc_s = round(calc_ms / 1000)
# jan_1_2000_secs += calc_s  # Add in any time lost while performing these calculations (removed since the cost turned out to be negligible)
datetime = utime.localtime(jan_1_2000_secs)
print("Local time:   {}".format(datetime))
# machine.RTC().datetime(datetime)
machine.RTC().datetime((datetime[0], datetime[1], datetime[2], datetime[6] + 1, datetime[3], datetime[4], datetime[5], 0))
# print("Calc time:    {} {}".format(calc_ms, calc_s))
print("Local time:   {}".format(utime.localtime()))
print("GM time:      {}".format(utime.gmtime()))
You'll notice I'm grabbing the time from an HTTP request. I'm doing that because the DataGram protocol required for an NTP query appears to be broken in MicroPython. I explained this in another thread, even including drop-in copy/paste REPL code to reproduce the problem, but it hasn't received much interest yet (viewtopic.php?f=20&t=10221&sid=e248a328 ... 710d528095). :-(

Re: ntptime not included with PyBoard firmware

Posted: Wed Apr 07, 2021 6:26 pm
by scruss
I think the off-by-one in the day of the week field might be something to do with traditional RTCs being developed before Posix standards became universal in computers. Almost every RTC I've used has been modelled after the MC146818 from 1984: all BCD fields and two-digit years.