Binary file when there is no file system

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
Ephreal
Posts: 28
Joined: Tue Jan 15, 2019 1:41 pm

Binary file when there is no file system

Post by Ephreal » Tue Jan 15, 2019 1:49 pm

Hi

I'm using Micropython on a Nios 2 Softcore from Altera. I somehow need to load a image file into a Flash. I've tried to wrap this file into a Python string which I then could load from the processor it self. However due to how a string is processed in MicroPython this was not possible. There is generated a hash value for every string and the speed and size of this hash string is proportional with the size of the string length it self.

I do not have a files system on this Nios, all python modules are compile from the Memzip folder (no frozen modules though).

Is there a way to compile a binary file which can be imported or access in other ways through the REPL?

Regards
Ephreal

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Binary file when there is no file system

Post by dhylands » Tue Jan 15, 2019 8:55 pm

You can store the binary file on the file system if there is enough space.

Otherwise you might want to look at the font code that Peter Hinch (@pythoncoder) did over here:
https://github.com/peterhinch/micropython-font-to-py
which embeds binary font data into a python source file.

Ephreal
Posts: 28
Joined: Tue Jan 15, 2019 1:41 pm

Re: Binary file when there is no file system

Post by Ephreal » Mon Feb 25, 2019 4:21 pm

dhylands wrote:
Tue Jan 15, 2019 8:55 pm
You can store the binary file on the file system if there is enough space.
I guess you didn't read my entire post
dhylands wrote:
Tue Jan 15, 2019 8:55 pm
Otherwise you might want to look at the font code that Peter Hinch (@pythoncoder) did over here:
https://github.com/peterhinch/micropython-font-to-py
which embeds binary font data into a python source file.
I've tried this but somehow my 33mb file gets expanded to 200mb.

Regards

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Binary file when there is no file system

Post by dhylands » Mon Feb 25, 2019 6:51 pm

gets expanded to 200 mb of source file or 200 mb of binary data?

I'm assuming that you want that data to be stored in flash and not in RAM.

Since you're not using frozen files, then you could have it encoded as a C source file, and then you can use a memoryview to access different portions of the data.

Raw uncompressed data stored in a memzip should also be available via memoryview, but you'd need to figure out the address of the data.

The memzip_locate function will return a pointer and size of given file. You'd just need to make a python interface to that.

Both approaches would require writing some python interface code to expose your data to the python code.

Ephreal
Posts: 28
Joined: Tue Jan 15, 2019 1:41 pm

Re: Binary file when there is no file system

Post by Ephreal » Tue Feb 26, 2019 7:57 am

dhylands wrote:
Mon Feb 25, 2019 6:51 pm
gets expanded to 200 mb of source file or 200 mb of binary data?

I'm assuming that you want that data to be stored in flash and not in RAM.

Since you're not using frozen files, then you could have it encoded as a C source file, and then you can use a memoryview to access different portions of the data.

Raw uncompressed data stored in a memzip should also be available via memoryview, but you'd need to figure out the address of the data.

The memzip_locate function will return a pointer and size of given file. You'd just need to make a python interface to that.

Both approaches would require writing some python interface code to expose your data to the python code.
Sounds very interesting. My setup is that I am trying to make a Flash write of sorts where i can load the MicroPython code into my Nios and then load my binary file to flash from there.

My binary file is around 11MB but after I've added the python wrapper around it it's 33MB and when compiling the memzip-files.c reaches 209MB before the compiler tells me that it outside the range of my allocated memory

I was thinking, i have this in another post. To generate a C file with a function call which could return an array or pointer to me to the data. However I have no documentation of the MicroPython C API.

How does the memzip_location function work ?

regards

Post Reply