Sending large array of numbers to host computer

The official pyboard running MicroPython.
This is the reference design and main target board for MicroPython.
You can buy one at the store.
Target audience: Users with a pyboard.
Post Reply
Batman
Posts: 1
Joined: Sun Aug 26, 2018 6:04 pm

Sending large array of numbers to host computer

Post by Batman » Sun Sep 02, 2018 8:45 pm

I use the Pyboard's ADC's to fill up an array with 7500 unsigned short integers. Now I want to send the data to the host computer.

At the side of the Pyboard I'm using

port = pyb.USB_VCP()
port.send(data)

or

port = pyb.USB_VCP()
port.write(data)

I am having (at least two problems):

1) the data gets truncated. I am not sure what the difference between write() and send() is. However, both of the commands allow me to read out some of the data on the host computer. However, not all: the sent data seems to get truncated. This is, only part of the data arrives at the host computer. For example, only 3000 out of 7500 bytes. Does anybody have an example of how to send larger arrays of data to the host computer?

2) All print statements in my program seem to be sent to the serial port as well. This is great for debugging. However, for my use case, all the print statements end up making it more difficult to find my data in whether arrives at the host computer. It seems I need to implement my own 'mini protocol' that allows me to identify where my data (the 75000 numbers) start and end. Has anybody run into this issue before and found a robust solution?

jickster
Posts: 401
Joined: Thu Sep 07, 2017 8:57 pm

Re: Sending large array of numbers to host computer

Post by jickster » Sun Sep 02, 2018 11:38 pm

(1) try increasing the time out; currently it’s set to five seconds


Sent from my iPhone using Tapatalk Pro

User avatar
pythoncoder
Posts: 2888
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Sending large array of numbers to host computer

Post by pythoncoder » Mon Sep 03, 2018 5:55 am

@Batman It's not clear from your post how you're doing this. I created the following script and transferred it to the Pyboard as rats10.py:

Code: Select all

import array, ujson
a = array.array('h', range(7500))
print(ujson.dumps(a))
At the shell I issued:

Code: Select all

pyboard.py --device /dev/pyboard -c "import rats10"
I got back the full array without issue.

Re distinguishing between data and other print statements, I assume you're running Python on the PC. I would attempt a json load in a try-except block. This should reject all but valid json data.
Peter Hinch

starter111
Posts: 15
Joined: Wed Mar 08, 2017 7:24 am

Re: Sending large array of numbers to host computer

Post by starter111 » Thu Sep 06, 2018 5:15 pm

Here's what I did on pyboard side. Is slow but works, I was able to send larger data back to computer.

boot.py

Code: Select all

import machine
import pyb
sw = pyb.Switch()
pyb.LED(2).on()
pyb.delay(1500)
pyb.LED(2).off()

if not sw():    
    pyb.main('pyb_server.py')
pyb_server.py

Code: Select all

import pyb

class USB_Port:
    def __init__(self):
        self.serial = pyb.USB_VCP()
        self.serial.setinterrupt(-1)
        
    def any(self):
        if self.serial.any():
            return True
        else:
            return False

    def read_until(self, ending, timeout=100):
        data = self.serial.read(1)
        timeout_count = 0
        while True:
            if data.endswith(ending):
                data=data.replace(ending, b'')
                break
            elif self.serial.any():
                new_data = self.serial.read(1)
                data = data + new_data
                timeout_count = 0
            else:
                timeout_count += 1
                if timeout is not None and timeout_count >= timeout:
                    break
                pyb.delay(1)
        return data
                
    def send(self, data):
        self.serial.write(data)
        self.serial.write(b'END')

def run_cmd(cmd):
    try:
        exec('d={}'.format(cmd))
        return locals()['d']
    except Exception as e:
        return e    
    
def main(vcp):    
    while True:
        if vcp.any():
            cmd=vcp.read_until(b'END').decode('utf-8')
            data=run_cmd(cmd) 
            if data is not None:
                if not isinstance(data, bytes):
                    data = bytes(data, encoding='utf8')
                vcp.send(data)
        pyb.delay(1)
        
main(USB_Port())

Last edited by starter111 on Thu Sep 06, 2018 5:18 pm, edited 1 time in total.

jickster
Posts: 401
Joined: Thu Sep 07, 2017 8:57 pm

Re: Sending large array of numbers to host computer

Post by jickster » Thu Sep 06, 2018 5:16 pm

It’s easier to read your posts if you surround code with the code tags


Sent from my iPhone using Tapatalk Pro

Post Reply