Page 1 of 1
Sending large array of numbers to host computer
Posted: Sun Sep 02, 2018 8:45 pm
by Batman
I use the Pyboard's ADC's to fill up an array with 7500 unsigned short integers. Now I want to send the data to the host computer.
At the side of the Pyboard I'm using
port = pyb.USB_VCP()
port.send(data)
or
port = pyb.USB_VCP()
port.write(data)
I am having (at least two problems):
1) the data gets truncated. I am not sure what the difference between write() and send() is. However, both of the commands allow me to read out some of the data on the host computer. However, not all: the sent data seems to get truncated. This is, only part of the data arrives at the host computer. For example, only 3000 out of 7500 bytes. Does anybody have an example of how to send larger arrays of data to the host computer?
2) All print statements in my program seem to be sent to the serial port as well. This is great for debugging. However, for my use case, all the print statements end up making it more difficult to find my data in whether arrives at the host computer. It seems I need to implement my own 'mini protocol' that allows me to identify where my data (the 75000 numbers) start and end. Has anybody run into this issue before and found a robust solution?
Re: Sending large array of numbers to host computer
Posted: Sun Sep 02, 2018 11:38 pm
by jickster
(1) try increasing the time out; currently it’s set to five seconds
Sent from my iPhone using Tapatalk Pro
Re: Sending large array of numbers to host computer
Posted: Mon Sep 03, 2018 5:55 am
by pythoncoder
@Batman It's not clear from your post how you're doing this. I created the following script and transferred it to the Pyboard as rats10.py:
Code: Select all
import array, ujson
a = array.array('h', range(7500))
print(ujson.dumps(a))
At the shell I issued:
Code: Select all
pyboard.py --device /dev/pyboard -c "import rats10"
I got back the full array without issue.
Re distinguishing between data and other print statements, I assume you're running Python on the PC. I would attempt a json load in a try-except block. This should reject all but valid json data.
Re: Sending large array of numbers to host computer
Posted: Thu Sep 06, 2018 5:15 pm
by starter111
Here's what I did on pyboard side. Is slow but works, I was able to send larger data back to computer.
boot.py
Code: Select all
import machine
import pyb
sw = pyb.Switch()
pyb.LED(2).on()
pyb.delay(1500)
pyb.LED(2).off()
if not sw():
pyb.main('pyb_server.py')
pyb_server.py
Code: Select all
import pyb
class USB_Port:
def __init__(self):
self.serial = pyb.USB_VCP()
self.serial.setinterrupt(-1)
def any(self):
if self.serial.any():
return True
else:
return False
def read_until(self, ending, timeout=100):
data = self.serial.read(1)
timeout_count = 0
while True:
if data.endswith(ending):
data=data.replace(ending, b'')
break
elif self.serial.any():
new_data = self.serial.read(1)
data = data + new_data
timeout_count = 0
else:
timeout_count += 1
if timeout is not None and timeout_count >= timeout:
break
pyb.delay(1)
return data
def send(self, data):
self.serial.write(data)
self.serial.write(b'END')
def run_cmd(cmd):
try:
exec('d={}'.format(cmd))
return locals()['d']
except Exception as e:
return e
def main(vcp):
while True:
if vcp.any():
cmd=vcp.read_until(b'END').decode('utf-8')
data=run_cmd(cmd)
if data is not None:
if not isinstance(data, bytes):
data = bytes(data, encoding='utf8')
vcp.send(data)
pyb.delay(1)
main(USB_Port())
Re: Sending large array of numbers to host computer
Posted: Thu Sep 06, 2018 5:16 pm
by jickster
It’s easier to read your posts if you surround code with the code tags
Sent from my iPhone using Tapatalk Pro
Re: Sending large array of numbers to host computer
Posted: Fri Dec 31, 2021 8:09 am
by zaord
pythoncoder wrote: ↑Mon Sep 03, 2018 5:55 am
@Batman It's not clear from your post how you're doing this. I created the following script and transferred it to the Pyboard as rats10.py:
Code: Select all
import array, ujson
a = array.array('h', range(7500))
print(ujson.dumps(a))
At the shell I issued:
Code: Select all
pyboard.py --device /dev/pyboard -c "import rats10"
I got back the full array without issue.
Re distinguishing between data and other print statements, I assume you're running Python on the PC. I would attempt a json load in a try-except block. This should reject all but valid json data.
@pythoncoder, I can't get make the json.loads() working with array on the PC side. Do you have any solution for using json working with arrays ?
Re: Sending large array of numbers to host computer
Posted: Fri Dec 31, 2021 12:18 pm
by pythoncoder
I made a mistake here: JSON doesn't support arrays and it's a quirk that ujson appeared to work here. The following worked under the Unix build of MicroPython. However it's a bit of a hack - I'm not well-up on this stuff and hopefully someone will come up with a better way. Under CPython I believe the best way is using subprocess: feel free to Google
Here is the hack.
On Pyboard, rats10.py
Code: Select all
import array
a = array.array('h', range(7500))
print(str(a))
On the PC:
Code: Select all
import os
s = """
from array import array
a = """
with open("result.py", "w") as f:
f.write(s)
os.system('pyboard.py --device /dev/pyboard -c "import rats10" >> result.py')
from result import a
print(a[99]) # a is the array, retrieve an element
I'm sure there's a much more elegant way to do this on MicroPython...
Re: Sending large array of numbers to host computer
Posted: Tue May 24, 2022 3:35 pm
by teltonique21
pythoncoder wrote: ↑Fri Dec 31, 2021 12:18 pm
Code: Select all
import os
os.system('pyboard.py --device /dev/pyboard -c "import rats10" >> result.py')
from result import a
print(a[99]) # a is the array, retrieve an element
I'm sure there's a much more elegant way to do this on MicroPython...
Is there a way to get the standard output of the launched command? I see you use os.system and redirect to a file which you subsequently read for output. Is there a direct way to get the output just like in Python with subprocess.Popen ?