timezone support in MicroPython ?

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
andydatablocks
Posts: 2
Joined: Wed Sep 05, 2018 1:21 pm

Re: timezone support in MicroPython ? DST/EST

Post by andydatablocks » Wed Sep 05, 2018 1:24 pm

def dstTime():
year = time.localtime()[0] #get current year
# print(year)
HHMarch = time.mktime((year,3 ,(14-(int(5*year/4+1))%7),1,0,0,0,0,0)) #Time of March change to DST
HHNovember = time.mktime((year,10,(7-(int(5*year/4+1))%7),1,0,0,0,0,0)) #Time of October change to EST
# print(HHNovember)
now=time.time()
if now < HHMarch : # we are before last sunday of march
dst=time.localtime(now-18000) # EST: UTC-5H
elif now < HHNovember : # we are before last sunday of october
dst=time.localtime(now-14400) # DST: UTC-4H
else: # we are after last sunday of october
dst=time.localtime(now-18000) # EST: UTC-5H
return(dst)

kwtf
Posts: 1
Joined: Fri Apr 03, 2020 10:01 am

Re: timezone support in MicroPython ?

Post by kwtf » Fri Apr 03, 2020 10:14 am

Easiest way I found is changing ntptimes internal NTP_DELTA Variable, so it calculates it right.
Run ntptime.settime() after boardinit to have the correct time before execution.
Get tuple afterwards with utime.localtime()

Code: Select all

def getntptime(timer):
    year = utime.localtime()[0]       #get current year
    now=ntptime.time()
    HHMarch   = utime.mktime((year,3 ,(31-(int(5*year/4+4))%7),1,0,0,0,0,0)) #Time of March change to CEST
    HHOctober = utime.mktime((year,10,(31-(int(5*year/4+1))%7),1,0,0,0,0,0)) #Time of October change to CET
    if now < HHMarch :               # we are before last sunday of march
        ntptime.NTP_DELTA = 3155673600-3600 # CET:  UTC+1H
    elif now < HHOctober :           # we are before last sunday of october
        ntptime.NTP_DELTA = 3155673600-7200 # CEST: UTC+2H
    else:                            # we are after last sunday of october
        ntptime.NTP_DELTA = 3155673600-3600 # CET:  UTC+1H
    
    ntptime.settime() # set the rtc datetime from the remote server

enzo
Posts: 40
Joined: Thu Jun 11, 2020 1:03 am

Re: timezone support in MicroPython ?

Post by enzo » Thu Jun 11, 2020 1:08 am

kwtf wrote:
Fri Apr 03, 2020 10:14 am
Easiest way I found is changing ntptimes internal NTP_DELTA Variable, so it calculates it right.
Run ntptime.settime() after boardinit to have the correct time before execution.
Get tuple afterwards with utime.localtime()

Code: Select all

def getntptime(timer):
    year = utime.localtime()[0]       #get current year
    now=ntptime.time()
    HHMarch   = utime.mktime((year,3 ,(31-(int(5*year/4+4))%7),1,0,0,0,0,0)) #Time of March change to CEST
    HHOctober = utime.mktime((year,10,(31-(int(5*year/4+1))%7),1,0,0,0,0,0)) #Time of October change to CET
    if now < HHMarch :               # we are before last sunday of march
        ntptime.NTP_DELTA = 3155673600-3600 # CET:  UTC+1H
    elif now < HHOctober :           # we are before last sunday of october
        ntptime.NTP_DELTA = 3155673600-7200 # CEST: UTC+2H
    else:                            # we are after last sunday of october
        ntptime.NTP_DELTA = 3155673600-3600 # CET:  UTC+1H
    
    ntptime.settime() # set the rtc datetime from the remote server
Great workaround!
Please, can you tell why I get this error:
" File "ntptime.py", line 35, in settime
OverflowError: overflow converting long int to machine word"
When I run it on my ESP 8266 NODEMCU?
Thanks for your time!

diggy
Posts: 1
Joined: Wed Aug 19, 2020 8:52 am

Re: timezone support in MicroPython ?

Post by diggy » Wed Aug 19, 2020 9:19 am

Same error here.

Code: Select all

  File "ntptime.py", line 35, in settime
OverflowError: overflow converting long int to machine word

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: timezone support in MicroPython ?

Post by pythoncoder » Wed Aug 19, 2020 12:45 pm

What firmware are you guys using? I can't replicate this with firmware built today.
Peter Hinch
Index to my micropython libraries.

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: timezone support in MicroPython ?

Post by davef » Tue Sep 29, 2020 9:38 am

kwtf,

Thanks for posting your solution.

Unfortunately, NZ change back to NZST on the first Sunday in April at 3AM, which left me wondering if:

Code: Select all

HHApril   = utime.mktime((year,4,(7-(int(5*year/4+4))%7),3,0,0,0,0,0))
would be correct. I couldn't find any Google hits on what appears to be a "magic" formula:

Code: Select all

31-(int(5*year/4+4))%7
Did find some information suggesting that Robert Harry van Gent was responsible for the CET forumlaes but searched all around his website without finding out how to calculate the mday number.

Can someone post a link to how to work this out?

Thanks.

Stephen Betts
Posts: 1
Joined: Sun Oct 11, 2020 9:58 pm

Re: timezone support in MicroPython ?

Post by Stephen Betts » Thu Oct 22, 2020 8:38 am

the timzone had not bothered me with my projects until i made a neopixel clock and we switched to daylight savings recently.

So this is what I have come up with for my timezone calculations based on the input from the earlier posts by davef and enzo
The different months, March, April, September and October / first and last sunday all have a different offsets based so you will need to play around with those for your timezone. This has have been check against https://www.timeanddate.com/ for the NZ timezone up to 2030.

there is probably some additions that could be included but this met my needs for now

https://en.wikipedia.org/wiki/List_of_UTC_time_offsets
https://www.iana.org/time-zones

I basically run this code on boot, and once a day to keep the RTC in sync correctly

Code: Select all

# ###################################################################### #
# Set RTC localTime from UTC and apply DST offset
# Requires import machine, utime, ntptime
# assumes active network, otherwise will raise error whch is not currently dealt with
# Note: dstOffset should be between -12 and +14 
# ###################################################################### #
def setLocalTime ( dstOffset ):
  # get the YEAR to use in calculation
  year, month, day, hour, minute, second, ms, dayinyear = utime.localtime()
  
  if year < 2020:
    # probably just booted, so get the time for calculation
    now=ntptime.time()
    # reload current time 
    year, month, day, hour, minute, second, ms, dayinyear = utime.localtime()

  # szTime = "{:4}-{:02}-{:02} {:02}:{:02}:{:02}".format( year, month, day, hour, minute, second )
  # print( "Time : " , szTime )

# define DST change dates for this year
  # NZDT Starts : 2:00 Last Sunday September 
  dstDT = utime.mktime((year, 9, (30 -(int(5*year/4+4))%7),2,0,0,0,0,0)) #Time of September change to NZDT
  # NZST Starts : 3:00 1st Sunday April 
  dstST = utime.mktime((year, 4, ( 7 -(int(5*year/4+5))%7),3,0,0,0,0,0)) #Time of April change to NZST
    
  if dstOffset >= 0 : # 
    # print( "Positive DST timezones" )
    if now > dstST and now < dstDT:
      # print( "Standard Time")
      ntptime.NTP_DELTA = 3155673600-( dstOffset * 3600) 
    else:
      # print( "Daylight Time")
      ntptime.NTP_DELTA = 3155673600-( (dstOffset+1) * 3600)
  else:
    # print( "Negative DST timezones" )
    if now > dstDT and now < dstST  :
      # print( "Standard Time")
      ntptime.NTP_DELTA = 3155673600-( dstOffset * 3600) 
    else:
      # print( "Daylight Time")
      ntptime.NTP_DELTA = 3155673600-( (dstOffset+1) * 3600)

  # set the RTC to correct time 
  ntptime.settime()
  # year, month, day, hour, minute, second, ms, dayinyear = utime.localtime()
  # szTime = "{:4}-{:02}-{:02} {:02}:{:02}:{:02}".format( year, month, day, hour, minute, second )
  # print( "setTime : " , szTime )

Divergentti
Posts: 67
Joined: Fri Sep 04, 2020 9:27 am
Location: Hanko, Finland
Contact:

Re: timezone support in MicroPython ?

Post by Divergentti » Mon Feb 22, 2021 10:10 am

Stephen Betts wrote:
Thu Oct 22, 2020 8:38 am
the timzone had not bothered me with my projects until i made a neopixel clock and we switched to daylight savings recently.

So this is what I have come up with for my timezone calculations based on the input from the earlier posts by davef and enzo
The different months, March, April, September and October / first and last sunday all have a different offsets based so you will need to play around with those for your timezone. This has have been check against https://www.timeanddate.com/ for the NZ timezone up to 2030.

there is probably some additions that could be included but this met my needs for now

https://en.wikipedia.org/wiki/List_of_UTC_time_offsets
https://www.iana.org/time-zones

I basically run this code on boot, and once a day to keep the RTC in sync correctly

Code: Select all

# ###################################################################### #
# Set RTC localTime from UTC and apply DST offset
# Requires import machine, utime, ntptime
# assumes active network, otherwise will raise error whch is not currently dealt with
# Note: dstOffset should be between -12 and +14 
# ###################################################################### #
def setLocalTime ( dstOffset ):
  # get the YEAR to use in calculation
  year, month, day, hour, minute, second, ms, dayinyear = utime.localtime()
  
  if year < 2020:
    # probably just booted, so get the time for calculation
    now=ntptime.time()
    # reload current time 
    year, month, day, hour, minute, second, ms, dayinyear = utime.localtime()

  # szTime = "{:4}-{:02}-{:02} {:02}:{:02}:{:02}".format( year, month, day, hour, minute, second )
  # print( "Time : " , szTime )

# define DST change dates for this year
  # NZDT Starts : 2:00 Last Sunday September 
  dstDT = utime.mktime((year, 9, (30 -(int(5*year/4+4))%7),2,0,0,0,0,0)) #Time of September change to NZDT
  # NZST Starts : 3:00 1st Sunday April 
  dstST = utime.mktime((year, 4, ( 7 -(int(5*year/4+5))%7),3,0,0,0,0,0)) #Time of April change to NZST
    
  if dstOffset >= 0 : # 
    # print( "Positive DST timezones" )
    if now > dstST and now < dstDT:
      # print( "Standard Time")
      ntptime.NTP_DELTA = 3155673600-( dstOffset * 3600) 
    else:
      # print( "Daylight Time")
      ntptime.NTP_DELTA = 3155673600-( (dstOffset+1) * 3600)
  else:
    # print( "Negative DST timezones" )
    if now > dstDT and now < dstST  :
      # print( "Standard Time")
      ntptime.NTP_DELTA = 3155673600-( dstOffset * 3600) 
    else:
      # print( "Daylight Time")
      ntptime.NTP_DELTA = 3155673600-( (dstOffset+1) * 3600)

  # set the RTC to correct time 
  ntptime.settime()
  # year, month, day, hour, minute, second, ms, dayinyear = utime.localtime()
  # szTime = "{:4}-{:02}-{:02} {:02}:{:02}:{:02}".format( year, month, day, hour, minute, second )
  # print( "setTime : " , szTime )
I made this for Finland. I fixed second last in date tuple to 6, because 0 is Monday. TIMEZONE_DIFFERENCE is normal (winter) time difference, like 2 for Finland.

Global dst_on is needed for Suntime calculation, perhaps I change it so that timezone is not actually needed.

Code: Select all


def resolve_dst_and_set_time():
    global TIMEZONE_DIFFERENCE
    global dst_on
    # This is most stupid thing what humans can do!
    # Rules for Finland: DST ON: March last Sunday at 03:00 + 1h, DST OFF: October last Sunday at 04:00 - 1h
    # Sets localtime to DST localtime
    if network.WLAN(network.STA_IF).config('essid') == '':
        now = mktime(localtime())
        if DEBUG_ENABLED == 1:
            print("Network down! Can not set time from NTP!")
    else:
        now = ntptime.time()

    (year, month, mdate, hour, minute, second, wday, yday) = localtime(now)

    if year < 2020:
        if DEBUG_ENABLED == 1:
            print("Time not set correctly!")
        return False

    dstend = mktime((year, 10, (31 - (int(5 * year / 4 + 1)) % 7), 4, 0, 0, 0, 6, 0))
    dstbegin = mktime((year, 3, (31 - (int(5 * year / 4 + 4)) % 7), 3, 0, 0, 0, 6, 0)))

    if TIMEZONE_DIFFERENCE >= 0:
        if (now > dstbegin) and (now < dstend):
            dst_on = True
            ntptime.NTP_DELTA = 3155673600 - ((TIMEZONE_DIFFERENCE + 1) * 3600)
        else:
            dst_on = False
            ntptime.NTP_DELTA = 3155673600 - (TIMEZONE_DIFFERENCE * 3600)
    else:
        if (now > dstend) and (now < dstbegin):
            dst_on = False
            ntptime.NTP_DELTA = 3155673600 - (TIMEZONE_DIFFERENCE * 3600)
        else:
            dst_on = True
            ntptime.NTP_DELTA = 3155673600 - ((TIMEZONE_DIFFERENCE + 1) * 3600)
    if dst_on is None:
        if DEBUG_ENABLED == 1:
            print("DST calculation failed!")
            return False
    else:
        ntptime.settime()

Divergentti
Posts: 67
Joined: Fri Sep 04, 2020 9:27 am
Location: Hanko, Finland
Contact:

Re: timezone support in MicroPython ?

Post by Divergentti » Sun Feb 28, 2021 8:25 am

pythoncoder wrote:
Wed Aug 19, 2020 12:45 pm
What firmware are you guys using? I can't replicate this with firmware built today.
npttime.settime() error overflow converting long int to machine word

with ESP32, sys.version = 3.4.0

esp32-idf4-20200902-v1.13.bin

How to repeat? Execute multiple times ntptime.settime()

>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
>>> ntptime.settime()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "ntptime.py", line 39, in settime
OverflowError: overflow converting long int to machine word
>>> ntptime.settime()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "ntptime.py", line 35, in settime
File "ntptime.py", line 25, in time
OSError: [Errno 110] ETIMEDOUT
>>> ntptime.settime()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "ntptime.py", line 35, in settime
File "ntptime.py", line 25, in time
OSError: [Errno 110] ETIMEDOUT
>>> ntptime.settime()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "ntptime.py", line 35, in settime
File "ntptime.py", line 25, in time
OSError: [Errno 110] ETIMEDOUT


-----

Not related to ntphosts:

>>> ntptime.host = 'test.nothin.com'
>>> ntptime.settime()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "ntptime.py", line 35, in settime
File "ntptime.py", line 20, in time
OSError: -202
>>>
>>>
>>>
>>> resolve_dst_and_set_time()
NTP time not set! Error -202
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "main.py", line 322, in resolve_dst_and_set_time
ValueError:
>>> resolve_dst_and_set_time()
NTP time not set! Error -202
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "main.py", line 322, in resolve_dst_and_set_time
ValueError:
>>> resolve_dst_and_set_time()
NTP time not set! Error -202
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "main.py", line 322, in resolve_dst_and_set_time
ValueError:
>>> ntptime.host = 'pool.ntp.org'
>>> resolve_dst_and_set_time()
>>> resolve_dst_and_set_time()
>>> ntptime.settime()
>>> ntptime.settime()
>>> localtime()
(2021, 2, 28, 10, 30, 40, 6, 59)

----

trying this:

Code: Select all

n = 0

    while (settime_complete is False) and (n <= 20):
        if n >= 20:
            f4.write("npttime.settime() tried 20 times. Giving up\n")
            f4.close()
            reset()
        else:
            try:
                ntptime.settime()
            except Exception as e:
                n += 1
                # npttime.settime() error overflow converting long int to machine word
                f4.write("npttime.settime() error %s\n" % e)
                sleep(10)
            settime_complete = True


davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: timezone support in MicroPython ?

Post by davef » Tue Apr 06, 2021 3:21 am

Hi Divergentti,

Sorry I missed this. I had a lot of grief getting ntptime on the ESP32. Finally ended up doing this:

boot.py

Code: Select all

# This file is executed on every boot (including wake-boot from deepsleep)
#import network

import esp
esp.osdebug(None)
from wifi_functions import connect
import gc

connect()
gc.collect()
which calls:

Code: Select all

import network
import utime
import machine

def connect():

 #  connect to the internet
    sta_if = network.WLAN(network.STA_IF)
    count = 0

    if not sta_if.isconnected():
        print('connecting to hotspot...')
        sta_if.active(True)
        sta_if.ifconfig(('192.168.10.99', '255.255.255.0', '192.168.10.1', '8.8.8.8'))
        sta_if.connect('HUAWEI-E8231-c4fd', 'your password')

        while (count < 5):
            count += 1

            status = sta_if.status()

            try:
                with open('errors.txt', 'a') as outfile:
                    outfile.write('connect status = ' + str(status) + '\n')
            except OSError:
                print('oops')

            if (sta_if.isconnected()):
                count = 0
                break

            print('.', end = '')
            utime.sleep(1)

    if (count == 5):
        count = 0

        try:
            with open('errors.txt', 'a') as outfile:
                outfile.write('did NOT connect to internet' + '\n')
        except OSError:
            print('oops')

        machine.reset()

    print(' network config:', sta_if.ifconfig())
then:
main.py

Code: Select all

from getUTC import getntptime
import utime
import webrepl
import gc
from watchdog_timer import wdt_callback
from watchdog_timer import wdt_feed


wdt_callback()

getntptime()

#  found that waiting for awhile, especially if
#  NOT making a static connection helps to remove
#  a OSError [Errno 110] TIMEOUT error in ntptime.py
print('waiting 5 seconds')
utime.sleep(5)

wdt_feed()

webrepl.start()

gc.collect()

import logger
which calls:
getUTC.py

Code: Select all

import ntptime
import utime
import machine


def getntptime():
    now = 0
    ntptime.host = 'nz.pool.ntp.org'
    count = 0

    while (count < 10):
        count += 1

        try:
            now = ntptime.time()
        except OSError as error:
            machine.reset()

        print('.', end = '')
        utime.sleep(1)

        if (now > 660000000):
            count = 0
            break

    if (count == 10):
        count = 0
        machine.reset()

    print(' got epoch')
    utime.sleep(1)


 #  only correct for 2021 New Zealand
    HHApril = utime.mktime((2021,4,4,3,0,0,0,0,0)) #  time of April change to NZST
    HHSept = utime.mktime((2021,9,26,2,0,0,0,0,0)) #  time of Sept change to NZDT

    if now < HHApril:               # we are before the first Sunday of April
        ntptime.NTP_DELTA = 3155673600 - (13 * 3600) #  NZDT: UTC-13H
    elif now < HHSept:              # we are before the last Sunday of Sept
        ntptime.NTP_DELTA = 3155673600 - (12 * 3600) #  NZST: UTC-12H
    else:                           # we are after the last Sunday of Sept
        ntptime.NTP_DELTA = 3155673600 - (13 * 3600) #  NZDT: UTC-13H

 #  set the time
    count = 0

    while (count < 10):
        count += 1

        my_time = ntptime.settime() # set the rtc datetime from the remote server

        print('.', end = '')
        utime.sleep(1)

        if (str(my_time) == 'None'):
            count = 0
            break

    if (count == 10):
        count = 0
        machine.reset()

    print(' time has been set')
    utime.sleep(1)
and finally my datalogger program runs.

I see the ESP32 try to connect 5 and then connects on the 2nd try in the 2nd call. I am connecting to a Huwaei hotspot so maybe you will need to go through different "hoops" to get consistent connection. If I don't stop at 5 attempts it just keeping trying to connect. It appears I need to try between 3-5 times before doing the machine.reset().

Snippets of information picked up from Peter Hinch and RobertHH.

Good luck or have you sorted it by now.

P.S. something else quite important I think sometimes you get an error when ntptime() is called that it not obvious to me. A software watchdog timer sorts that out or the system freezes.

Post Reply