Adding adjtime() method to utime

[MostlyHarmless]

The esp-idf used for the esp32 has a function adjtime(3) that can be used to gradually apply a correction value to the system clock. This keeps the clock monotonic (moving forward at all times) while the correction happens.

I was not able to find an equivalent function for the esp8266, so this is esp32 specific and therefore belongs into ports/esp32.

Since utime.localtime() is based on gettimeofday(3) declared in the same header as adjtime(3) (sys/time.h), I think it is appropriate to add it to ports/esp32/modutime.c.

This branch has the implementation. Here is the diff.

This function is a fundamental building block of an effort to provide DS3231, NEO-6M (gps module) and NTP based time synchronization. So it would be good to get this into the next official release. The other parts can probably all be built as pure Python modules based on uasyncio.

I tested it by syncing an esp32 every 10 seconds in a uasyncio based task to the time provided by a DS3231. The result is that (on this particular specimen) the clock on average is running ahead by about 37us per second, then is slewed back by 370us over the course of about 24ms. I didn't find a good way to actually adjust the speed of the clock more permanently. The only way seems to be to switch to the internal 150kHz oscillator and use the esp_clk_slowclk_cal_set() function. But that oscillator is so inaccurate and jumpy that it is almost useless. This sort of saw tooth pattern is certainly much better.

Another task running every 1000 milliseconds produced a square wave. Comparing the two square waves on the oscilloscope there is the occasional +/- 1ms jitter in the Python generated square wave, which is entirely expected. But the two square waves don't drift any more, and that was the goal for this test.

@pythoncoder: I did take a look at your DS3231 driver. Borrowed the bcd2dec and dec2bcd stuff from it. But I ended up writing my own that is using the PPS signal from the DS3231 to get the beginning of a second at microsecond precision. I might end up offering both methods, in case someone doesn't want to or cannot connect the SQW pin. Then the change of second is the only way to sync by. Anyhow, it is WIP and will be published when ready.

Comments and questions please.


Regards, Jan

[pythoncoder]

To have any chance of getting this into the official build you will need to raise a PR.

The idea of a monotonic correction is great but I'm a little sceptical of an ESP32-only solution. But I'm not a maintainer

[MostlyHarmless]

To have any chance of getting this into the official build you will need to raise a PR.

Last time I raised a PR I was told to discuss and argue for the change on the forum first. That discussion (DHT improvements towards uasyncio) is still sitting there without being discussed though. So I wonder what really is the right way of getting these changes into some sort of merge process.

The idea of a monotonic correction is great but I'm a little sceptical of an ESP32-only solution.

If a monotonic correction is not supported on a given port and the port doesn't provide a method to otherwise slew the clock (like adjusting the oscillator speed), then hard setting the clock to a new value every now and then is the only way to correct drift. This should always be the last option, but it is sometimes better than drifting away by minutes over a month or so.


Thanks for the feedback, Jan

[MostlyHarmless]

Update (long post):

I currently have an NTP client prototype that works with the proposed adjtime(3) function on an ESP32. It works as follows:

• If on start the RTC is more than 1s off (default) it will first set the RTC hard to the first NTP server timestamp received. In my environment that is usually somewhere +/- 5ms of the "real time" (see below).
• A uasyncio task polls with varying interval between 64s and 1024s, depending on the spread in the required adjustment.
• Each poll loop it queries the NTP server 5 times with 2s in between, discards outliers and calculates the average delta of the RTC to the server's time, taking network round trip into account. From that and some limited history it calculates what the current RTC drift is and determines a new value for the adjtime(3) call.
• A second uasyncio task calls adjtime(3) every 2s to slew the RTC to where it should be.
• A good "lock" to within +/- 2ms of the server time is usually achieved after 10 minutes.
• From there usually the poll interval just goes up and reaches 1024s after about an hour total.

I will publish the NTP client and test code on github after a bit of cleanup.

What I meant with "real time" and all the offsets requires explaining my setup a bit. I have a NEO-6M GPS module that produces a PPS. Long term goal is to build this into a GPS based NTP server, but that is another project. According to my oscilloscope the PPS is 1Hz with a StdDev <100nHz. I consider that good enough for home use. The NTP source most things here at home use is my main server, which is a Dell PowerEdge 2950 running CentOS 7. It runs plenty of stuff, including firewalld (since it is my default GW), ntpd, dhcpd, named and a bunch of QEMU/KVM machines for other stuff. The test script on the ESP32 has a third uasyncio task that tries to produce a 100ms pulse at the beginning of every second, based on RTC.datetime(). So when I look at the "ntpq -np" output of the server and compare that to all the signals on the oscilloscope, it is clear that none of the ESP32s is more than 2ms off compared to the server. They do differ quite a bit with respect to their adjustment. One needs -60us adjustment every 2s, the other almost nothing. A Raspberry Pi 3, running a similar script producing the same 100ms pulse at the beginning of the second isn't any better, and that Pi 3 is synced to the same server via ntpd 4.2.6. So I am quite confident that this thing works in a decent WLAN only setup.

I also seem to be blessed/cursed with a rather good internet connection. Not only are the up- and downlink speeds identical, but the roundtrip is apparently very even most of the time. The server is often stratum 2 and seldom shows an offset of more than 5ms. With typical 40-80ms round trip times to the next stratum this is a blessing for normal operations, a curse for testing. I have two VLANs on another server that are behind a WANEM3. Plan is to do stress testing with that to introduce varying delays, packet loss/dups, corruption and the like. But that won't get done tonight.

As for ESP32 only: I ported the whole thing to an ESP8266 and while the NTP client itself works, the instability of the ESP8266's RTC makes running it questionable at best. To keep the RTC even within +/- 50ms one would have to recalculate the slew rate at least once a minute. Constantly polling public NTP servers at that rate is simply rude, so I am not going to aim for that. I have not played with the ESP8266's RTC calibration value yet, but since that value would have to be adjusted as often, by itself it isn't better. My idea for that platform for now is to add a DS3231 to the mix. Slew the DS3231 via its oscillator aging so that it keeps in sync with NTP, then try to keep the ESP8266 in sync with that by manipulating its RTC calibration every few seconds. But at this point this is just handwaving with no code behind it.

Anyhow, at the end of the day, even though I missed the 1.12 release, I still want this code merged. So I will publish the NTP client code and then open a PR for adjtime(3).


Regards, Jan

[MostlyHarmless]

I will publish the NTP client and test code on github after a bit of cleanup.

https://github.com/wieck/micropython-ntpclient


Questions and comments please, Jan

[MostlyHarmless]

I added a second test that uses an SSD1306 OLED to display the date and time, NTP synchronized to milliseconds.

https://github.com/wieck/micropython-ntpclient


Regards, Jan

[seonr]

Great job Jan! Thanks for working on this and sharing it!

[MostlyHarmless]

Just for the record,

I just looked at the O-Scope and though "what a happy family ... so unlike real families this TIME of the year" (pun intended).

ScreenImg_003.png (28.2 KiB) Viewed 4556 times

Signal 1 (yellow) is an actual GPS PPS, so that is the official time pulse.
Signal 2 (magenta) is a Raspberrry Pi 3 running ntpd 4.6.
Signal 3 and 4 (cyan and green) are ESP32s running ntpclient_test2.py.

Signals 2-4 are all syncing to the same NTP server in the basement, which happens to be stratum 2 at the moment. Each of them just produces a PPS signal on an OUT pin.

For those unfamiliar with oscilloscopes, there is a (very faint) grid in the image. You need to crank up the contrast to see it. It is clearly visible on the actual oscilloscope screen. One square of that grid in horizontal direction is 1ms (see top of the display). This means that all clocks are in sync to better than +/- 1ms right now.

[tve]

FYI, I'm trying to get this in: https://github.com/micropython/micropython/pull/5973 "esp32/time: make utime conform to CPython".