Page 1 of 1

Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Posted: Sun Jan 10, 2021 8:52 pm
by kbrenner
Hi, I am trying to store data on the SD card on my PYBD-SF6W. I am running Micropython v1.13-150-gb7883ce74. I am able to write strings to '.txt' files no problem. I can also write dictionaries that contain numpy arrays to '.json' files without any issues. However, when I try to write numpy arrays directly to '.txt' or '.DAP' files, I receive fatal errors:
.txt error.PNG
.txt error.PNG (15.17 KiB) Viewed 621 times
.DAP error.PNG
.DAP error.PNG (15.15 KiB) Viewed 621 times
Does anyone have an idea as to why this might be happening? I did not have this issue on Micropython v1.12.

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Posted: Mon Jan 11, 2021 1:04 am
by jimmo
kbrenner wrote:
Sun Jan 10, 2021 8:52 pm
Does anyone have an idea as to why this might be happening?
My understanding is that f.write(data) should attempt to call mp_get_buffer_raise on the ulab array type.

To isolate the problem, can you try doing

Code: Select all

import ulab as np
data = np.zeros((3,3))
buf = bytes(data)
print(buf)
That will at least check whether there's something going wrong with file i/o or with ulab's implementation of mp_get_buffer_raise.

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Posted: Mon Jan 11, 2021 1:56 am
by kbrenner
jimmo wrote:
Mon Jan 11, 2021 1:04 am
kbrenner wrote:
Sun Jan 10, 2021 8:52 pm
Does anyone have an idea as to why this might be happening?
My understanding is that f.write(data) should attempt to call mp_get_buffer_raise on the ulab array type.

To isolate the problem, can you try doing

Code: Select all

import ulab as np
data = np.zeros((3,3))
buf = bytes(data)
print(buf)
That will at least check whether there's something going wrong with file i/o or with ulab's implementation of mp_get_buffer_raise.
Upon trying to that code, after typing buf = bytes(data), it stalled and then errored, and I wasn't even able to type print(buf):
buf error.PNG
buf error.PNG (13.18 KiB) Viewed 605 times

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Posted: Mon Jan 11, 2021 2:58 am
by jimmo
kbrenner wrote:
Mon Jan 11, 2021 1:56 am
Upon trying to that code, after typing buf = bytes(data), it stalled and then errored, and I wasn't even able to type print(buf):
I think this might be worth raising at https://github.com/v923z/micropython-ulab

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Posted: Mon Jan 11, 2021 5:20 pm
by monquarter
As a workaround you can use the .flatten().tobytes() methods.

Code: Select all

a = np.zeros((1,1))
b = a.flatten().tobytes()
This won't give you an error, however it doesn't function properly. I think there may be a bug in the .tobytes() file as the lenght of the array is off by a factor of 8 (the size of float object).

Re: Unable to write numpy arrays to '.txt' or '.DAP' files on SD

Posted: Mon Jan 11, 2021 9:18 pm
by mathieu
monquarter wrote:
Mon Jan 11, 2021 5:20 pm
This won't give you an error, however it doesn't function properly. I think there may be a bug in the .tobytes() file as the lenght of the array is off by a factor of 8 (the size of float object).
You could try again, this bug was just fixed a couple of hours ago.