Reading raw binary GPS from serial and writing to SD card with pyboard

The official pyboard running MicroPython.
This is the reference design and main target board for MicroPython.
You can buy one at the store.
Target audience: Users with a pyboard.
Post Reply
katesfb
Posts: 22
Joined: Sun Dec 18, 2016 8:09 pm

Reading raw binary GPS from serial and writing to SD card with pyboard

Post by katesfb » Wed Aug 08, 2018 11:32 am

HI,
We have a project to read raw binary GPS (in a remote environment) from a ublox NEO M8N and write it to the sd card using a pyboard and then convert the resultant binary file using the RTK library into the Rinex. I did it first with a conventional python3 program on pc (linux) and this worked fine however when i do the same thing using the pybaord the resultant binary file does not convert correctly implying that there is something wrong with the binary file that has been created.

Below is the python3 script that works and the micropython script that doesnt. Can anybody see where i may have gone wrong in the micropython code. What values should i set for timeout and read_buf_len in the uart setup.

Any help is much appreciated.

Cheers.

Code: Select all

import serial, time, os

def setupSerial():
    print("Setting up serial object")
    ser = serial.Serial(
        port=GPSPort,
        baudrate=115200,         
        bytesize=8,     
        parity='N',     
        stopbits=1,  
        #timeout=0,
        xonxoff=0,              
        rtscts=0
        )               
    print(ser)
    return ser

print('UbloxRawGPSLog')
time.sleep(2)
newFile = open("/home/richard/Temp/GPSBinTest-Py.ubx","ab")
GPSPort='/dev/ttyUSB0'
ser=setupSerial()
newFileBytes=[]

while True:
  newFileBytes=[]
  while ser.inWaiting()==0:
    pass

  z=ser.readline()
  newFileBytes.append(z)

  for x in range(len(newFileBytes)):
      newFile.write(newFileBytes[x])

Code: Select all

import time
from pyb import UART, LED

def setupUart():
    print('Setting up uart')
    ser=UART(1,115200) #x10=uart_RX
    ser.init(115200,timeout=50,read_buf_len=64) #check this

    return ser

time.sleep(2)
newFile = open("GPSBinTest.ubx","ab")
ser=setupUart()
newFileBytes=[]

led=LED(1)
led2=LED(2)
led.off()
led2.off()

while True:
  newFileBytes=[]
  while ser.any()==0:
    led2.toggle()  
 
  z=ser.readline()
  newFileBytes.append(z)
  led.toggle()
    
  for x in range(len(newFileBytes)):
    newFile.write(newFileBytes[x])

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Reading raw binary GPS from serial and writing to SD card with pyboard

Post by Roberthh » Wed Aug 08, 2018 12:29 pm

Do you have examples on how the files should look like and how it looks instead?
b.t.w.:
This code looks overly complicated. Why, in the inner loop, do you not simply write:

Code: Select all

while True:
  while ser.any()==0:
    led2.toggle()  
 
  z=ser.readline()
  led.toggle()
  newFile.write(z)

katesfb
Posts: 22
Joined: Sun Dec 18, 2016 8:09 pm

Re: Reading raw binary GPS from serial and writing to SD card with pyboard

Post by katesfb » Mon Aug 13, 2018 1:53 am

Hi,
And thanks for the reply - much appreciated.

The files actually look very similar. The first difference is the size - the correct file is much larger and converts correctly using the convbin.exe program (a part of RTKlib, a collection of utility files for manipulating raw binary GPS data files - rtklib.com). The second difference is that during the conversion process an incorrectly converted file creates errors in the conversion process.

I thought the problem may have been something to do with the way i was initializing the UART to receive the binary data (the binary data comes in at 115200 baud) and is continuous while the GPS receiver is on with a 1s gap between each block of data/line which is terminated with a carriage return.

I tried:

Code: Select all

ser.init(115200,timeout=50,read_buf_len=64)
This creates a file that will convert but has errors and the file is very small. I also tried a timeout of 1000ms but this results in no data being written to the file at all. Basically i am not really sure how i should be initializing the UART which i think is resulting in input data being missed therefor resulting in a file that doesn't convert properly.

Interestingly, if use a serial terminal like 'cutecom' to log the data, the created file converts perfectly as does the PC python script previously mentioned which further leads me to believe that it is the way i am initialing the UART.

Any help you can give with this is much appreciated.

Cheers.

Post Reply