Unable to write numpy arrays to '.txt' or '.DAP' files on SD

The official PYBD running MicroPython, and its accessories.
Target audience: Users with a PYBD
Post Reply
kbrenner
Posts: 46
Joined: Mon Jan 20, 2020 8:05 pm

Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Post by kbrenner » Sun Jan 10, 2021 8:52 pm

Hi, I am trying to store data on the SD card on my PYBD-SF6W. I am running Micropython v1.13-150-gb7883ce74. I am able to write strings to '.txt' files no problem. I can also write dictionaries that contain numpy arrays to '.json' files without any issues. However, when I try to write numpy arrays directly to '.txt' or '.DAP' files, I receive fatal errors:
.txt error.PNG
.txt error.PNG (15.17 KiB) Viewed 4462 times
.DAP error.PNG
.DAP error.PNG (15.15 KiB) Viewed 4462 times
Does anyone have an idea as to why this might be happening? I did not have this issue on Micropython v1.12.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Post by jimmo » Mon Jan 11, 2021 1:04 am

kbrenner wrote:
Sun Jan 10, 2021 8:52 pm
Does anyone have an idea as to why this might be happening?
My understanding is that f.write(data) should attempt to call mp_get_buffer_raise on the ulab array type.

To isolate the problem, can you try doing

Code: Select all

import ulab as np
data = np.zeros((3,3))
buf = bytes(data)
print(buf)
That will at least check whether there's something going wrong with file i/o or with ulab's implementation of mp_get_buffer_raise.

kbrenner
Posts: 46
Joined: Mon Jan 20, 2020 8:05 pm

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Post by kbrenner » Mon Jan 11, 2021 1:56 am

jimmo wrote:
Mon Jan 11, 2021 1:04 am
kbrenner wrote:
Sun Jan 10, 2021 8:52 pm
Does anyone have an idea as to why this might be happening?
My understanding is that f.write(data) should attempt to call mp_get_buffer_raise on the ulab array type.

To isolate the problem, can you try doing

Code: Select all

import ulab as np
data = np.zeros((3,3))
buf = bytes(data)
print(buf)
That will at least check whether there's something going wrong with file i/o or with ulab's implementation of mp_get_buffer_raise.
Upon trying to that code, after typing buf = bytes(data), it stalled and then errored, and I wasn't even able to type print(buf):
buf error.PNG
buf error.PNG (13.18 KiB) Viewed 4446 times

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Post by jimmo » Mon Jan 11, 2021 2:58 am

kbrenner wrote:
Mon Jan 11, 2021 1:56 am
Upon trying to that code, after typing buf = bytes(data), it stalled and then errored, and I wasn't even able to type print(buf):
I think this might be worth raising at https://github.com/v923z/micropython-ulab

monquarter
Posts: 15
Joined: Sat Jan 11, 2020 2:31 pm

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Post by monquarter » Mon Jan 11, 2021 5:20 pm

As a workaround you can use the .flatten().tobytes() methods.

Code: Select all

a = np.zeros((1,1))
b = a.flatten().tobytes()
This won't give you an error, however it doesn't function properly. I think there may be a bug in the .tobytes() file as the lenght of the array is off by a factor of 8 (the size of float object).

User avatar
mathieu
Posts: 88
Joined: Fri Nov 10, 2017 9:57 pm

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Post by mathieu » Mon Jan 11, 2021 9:18 pm

monquarter wrote:
Mon Jan 11, 2021 5:20 pm
This won't give you an error, however it doesn't function properly. I think there may be a bug in the .tobytes() file as the lenght of the array is off by a factor of 8 (the size of float object).
You could try again, this bug was just fixed a couple of hours ago.

Post Reply