asynycio as_GPS

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
User avatar
pythoncoder
Posts: 3936
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: asynycio as_GPS

Post by pythoncoder » Tue Jan 28, 2020 6:42 pm

Consider the line
'$GPGGA,123519,4807.038,N,01131.000,E,1,08,0.9,545.4,M,46.9,M,,*47'
Then
segs == ['$GPGGA', '123519', '4807.038', 'N', '01131.000', 'E', '1', '08', '0.9', '545.4', 'M', '46.9', 'M', '', '', '47']
and
seg0 == 'GPGGA'
segx == 'GGA'
After executing

Code: Select all

segs = segs[:-1]
segs == ['$GPGGA', '123519', '4807.038', 'N', '01131.000', 'E', '1', '08', '0.9', '545.4', 'M', '46.9', 'M', '', '']

So I think that line is correct. However the original code removed the '$' from segs[0], rather than just from a copy so they got the following as their arg
['GPGGA', '123519', '4807.038', 'N', '01131.000', 'E', '1', '08', '0.9', '545.4', 'M', '46.9', 'M', '', '']

None of the parse routines use element 0, but I guess we should remove the '$' in case anyone has subclassed AS_GPS and implemented their own parse routine. With the following change the parse routines will receive exactly the same input as before the changes:

Code: Select all

        segs[0] = segs[0][1:]  # discard $
        segs = segs[:-1]  # and checksum
        seg0 = segs[0]  # e.g. GPGLL
        segx = seg0[2:]  # e.g. GLL
I have also spotted an optimisation so I'll post an update later.
Peter Hinch

User avatar
romeotango
Posts: 26
Joined: Tue Jun 16, 2015 10:52 am
Location: Germany

Re: asynycio as_GPS

Post by romeotango » Wed Jan 29, 2020 7:40 am

Good morning!
I fell asleep at night thinking...
Please to consider:
Isn't it too much effort?
Receiving the data set.
Check for illegal characters. And checking for CRC.
This proves that the data set was transmitted correctly and received completely.
Then the first 3 characters ($GN") are removed. Or "$PU" for my PUBX records.
The unavoidable check in the supported_sentences query is yet another safety net.
I can't imagine a constellation that would lead to errors.

User avatar
romeotango
Posts: 26
Joined: Tue Jun 16, 2015 10:52 am
Location: Germany

Re: asynycio as_GPS

Post by romeotango » Wed Jan 29, 2020 7:55 am

This is the added function for Parsing PUBX:
Unfortunately when starting gps-callback.py with import of my extended as_GPS.py the heap is now too small again.
I will switch to .mpy for now.
Can you detect an error?
I always have a little trouble with the logical links &= GSV | PUB
PUB is declared as PUB = const (64)

Why is this double allocation necessary?
course = gfloat(gps_segments[12]) und anschliessend erst
self.course = course

Code: Select all

    def _pubx(self, gps_segments):  # Parse PUBX00 sentence
        if gps_segments[1] != '00':
            raise ValueError # Not PUBX, 00 sentence
        self._valid &= ~PUB
        # Check Receiver Data Valid
        if not self.battery:
            if gps_segments[8] == 'NF' or int (gfloat(gps_segments[9])) > 10:  
                raise ValueError # No Fix or horizontal accuracy > 10 Meters
        self._fix(gps_segments, 3, 5)
        # Altitude#
        altitude = gfloat(gps_segments[7])
        # Speed    
        spd_knt = gfloat(gps_segments[11])
        # Course
        course = gfloat(gps_segments[12])
        # HDOP,VDOP
        hdop = gfloat(gps_segments[15])
        vdop = gfloat(gps_segments[16])
        # Update Object Data
        self.satellites_used = int (gps_segments[18])
        self.altitude = altitude
        self.hdop = hdop
        self.vdop = vdop
        self._speed = spd_knt
        self.course = course
        self._valid |= PUB | GSA
 

User avatar
pythoncoder
Posts: 3936
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: asynycio as_GPS

Post by pythoncoder » Wed Jan 29, 2020 9:37 am

romeotango wrote:
Wed Jan 29, 2020 7:40 am
...
Isn't it too much effort?
Receiving the data set.
Check for illegal characters. And checking for CRC.
This proves that the data set was transmitted correctly and received completely.
Then the first 3 characters ($GN") are removed. Or "$PU" for my PUBX records.
The unavoidable check in the supported_sentences query is yet another safety net.
I can't imagine a constellation that would lead to errors.
You can skip the CRC check:

Code: Select all

from as_GPS import AS_GPS
AS_GPS.FULL_CHECK = False
gps = AS_GPS(args)
There are situations where bad data can be acquired. During a baudrate change. If the GPS unit is powered independently from the Pyboard, or if the Pyboard is rebooted while the GPS is running, partial sentences may be received. If you power them from the same source, keep the baudrate constant, and never reboot the Pyboard then you should only ever see good data.
I always have a little trouble with the logical links &= GSV | PUB
Sorry, I don't understand your point here.
Why is this double allocation necessary?
course = gfloat(gps_segments[12]) und anschliessend erst
self.course = course
[EDIT]
I have been guilty of forgetting how my own code works :( The original float conversion was correct. If a GPS returns an empty field the float conversion will throw a ValueError. This is trapped here. This is the reason for the double allocation: if the exception occurs, the bound variable is not updated.

I am reverting this gfloat change.

I would be grateful if you could explain why this behaviour is problematic to you: it seems to me that if the unit returns an empty field it is safest to assume no change rather than to assume a value of 0.
Peter Hinch

User avatar
romeotango
Posts: 26
Joined: Tue Jun 16, 2015 10:52 am
Location: Germany

Re: asynycio as_GPS

Post by romeotango » Fri Jan 31, 2020 5:45 pm

I'm glad you're still thinking about it here!

In the first version of as_GPS.py I worked with an empty value of magnetic variation caused a value error.
So RMC was never evaluated.
You have changed that with this if-statement:
# Add Magnetic Variation if firmware supplies it
if gps_segments[10]: ...

So you are right that float alone is enough!

Have a nice evening!
Rainer

User avatar
romeotango
Posts: 26
Joined: Tue Jun 16, 2015 10:52 am
Location: Germany

Re: asynycio as_GPS

Post by romeotango » Sat Feb 01, 2020 6:30 am

romeotango wrote:
Fri Jan 31, 2020 5:45 pm
I'm glad you're still thinking about it here!

In the first version of as_GPS.py, I worked with, an empty value of magnetic variation caused a value error.
So RMC was never evaluated as true and no callback was called.
You have changed that with this if-statement:
# Add Magnetic Variation if firmware supplies it
if gps_segments[10]: ...

So you are right that float alone is enough!

Have a nice evening!
Rainer

User avatar
romeotango
Posts: 26
Joined: Tue Jun 16, 2015 10:52 am
Location: Germany

Re: asynycio as_GPS

Post by romeotango » Sat Feb 08, 2020 12:22 pm

Hello,
after many detours I have now managed to create a stable version of the as_GPS method for LOPY.
Heap overflow always occurred with the standard files of PeterHinch.
I had to change the very elegant method of parsing to a more primitive method.
My GPS receiver now only sends PUBX, 00, which is position data with timestamp.
Additionally PUBX, 04, which is essentially date and time. Only every 10 seconds.

All other data records are suppressed.
5 Hz, 115200baud

The class as_PUBRT is like as_GPS without the elegance. Although the file size is not significantly reduced, there are no problems with HEAP-allocation.

In my research, I found:
https://github.com/dpgeorge/micropython ... d/uasyncio . Peter, You are certainly not uninvolved in this version ;-)
This new version of uasyncio I can now import without heap allocation error.

After these changes the asynchronous GPS acquisition can run.
I have solved the handling of time-UTC data differently. Especially to solve the handling of msec faster. Now the epoch_time is completely output as integer of the msec. Hopefully this will make the further processing of the data a bit faster.
The RTC-Clock is reset every 10sec with the time data from the Reveiver. I don't know if this is useful or necessary.

I have not implemented the callback function. Await works very well and the callback does not work reliably for me. I also read from PeterHinch, who was critical of the callback, because blocking is a big risk.

I am looking forward to constructive criticism...
https://gitlab.com/romeotango/lopyrt/-/ ... evelopment

Post Reply