MemoryError when importing a custom module

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: MemoryError when importing a custom module

Post by pfalcon » Sat May 14, 2016 10:52 am

Llwy wrote:I tried to rebuild and reflash everything from scratch several times but I simply get "access denied" from the webREPL every time I try to type in the webREPL password (and yet the password is the right one as I type it just before at the initialisation step).
Apparently it's wrong, or you discovered a bug. Then it would be very important for the project that you investigated it further and reported details. There's an existing topic which covers this stuff: http://forum.micropython.org/viewtopic.php?f=16&t=1881
I am at my wits' (and skills) end on this matter so I guess I will not be able to do anything more than wait until next official release. I will then test it to see if I can load my module and continue to optimize it.
No, please don't give up. There's nothing complex to investigate and resolve these issues (or report back the details for us to look for solution). I'm actually very sad to hear such from a fellow Python programmer. Because we bring Python to microcontrollers to make it all easy, all thanks to Python. And yet people don't even want to look how it's done, despite it's very simple, high-level Python, and 5-minute look will give you an idea how to check you password issue, how to workaround it one-time, and how to avoid or resolve it: https://github.com/micropython/micropyt ... 66/scripts .
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

User avatar
jwissing
Posts: 29
Joined: Thu Jun 19, 2014 12:23 pm
Location: Germany

Re: MemoryError when importing a custom module

Post by jwissing » Sat May 14, 2016 4:25 pm

pfalcon wrote:
Llwy wrote:I tried to rebuild and reflash everything from scratch several times but I simply get "access denied" from the webREPL every time I try to type in the webREPL password (and yet the password is the right one as I type it just before at the initialisation step).
Apparently it's wrong, or you discovered a bug. Then it would be very important for the project that you investigated it further and reported details. There's an existing topic which covers this stuff: http://forum.micropython.org/viewtopic.php?f=16&t=1881
I am at my wits' (and skills) end on this matter so I guess I will not be able to do anything more than wait until next official release. I will then test it to see if I can load my module and continue to optimize it.
No, please don't give up. There's nothing complex to investigate and resolve these issues (or report back the details for us to look for solution). I'm actually very sad to hear such from a fellow Python programmer. Because we bring Python to microcontrollers to make it all easy, all thanks to Python. And yet people don't even want to look how it's done, despite it's very simple, high-level Python, and 5-minute look will give you an idea how to check you password issue, how to workaround it one-time, and how to avoid or resolve it: https://github.com/micropython/micropyt ... 66/scripts .
i can reproduce this behaviour when i rewrite port_config.py with micropython as password.
Stopping/starting webrepl and subsequent login attempt with that password gives access denied.

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: MemoryError when importing a custom module

Post by pfalcon » Sat May 14, 2016 4:37 pm

OMG guys, password is limited to 9 chars, I wouldn't even think there're so many lovers of long passwords ;-).
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: MemoryError when importing a custom module

Post by pfalcon » Mon May 16, 2016 10:22 pm

Test for max password length during setup was added to master.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

Llwy
Posts: 34
Joined: Thu Mar 10, 2016 7:43 am

Re: MemoryError when importing a custom module

Post by Llwy » Thu May 19, 2016 5:31 am

Thank you very much.
I am on a business trip this week so I will try this out when I get back home on the weekend.

Llwy

User avatar
rcolistete
Posts: 352
Joined: Thu Dec 31, 2015 3:12 pm
Location: Brazil
Contact:

Re: MemoryError when importing a custom module

Post by rcolistete » Sat May 21, 2016 9:53 am

Llwy wrote:Hello,
I am working on a module for the MFRC522 RFID reader. The module size is 11KB
I managed to copy the module on the micropython filesystem with the help of webrepl_cli.py.

When I try to import the module through the UART REPL, I get the following error:

Code: Select all

MemoryError: memory allocation failed, allocating 136 bytes
If I try to import a second time the module, no exception is raised but the module is clearly not loaded properly (the only property which is visible is __name__ and none of the defined classes are there).
A link to you module source code would be very useful for other Micropython users test the module on other boards (Pyboard v1.1, WiPy, etc).
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: MemoryError when importing a custom module

Post by pythoncoder » Sat May 21, 2016 10:09 am

@rcolistete This occurs when the compiler runs out of RAM: when you import a .py module the compiler running on the board converts the text file to bytecode, which is then run by the Python VM.

I've not tried this on the ESP but on the Pyboard there are two solutions. The first is to cross-compile the code to a .mpy file on the PC which can then be copied to the target and imported without invoking the on board compiler. See the mpy-cross directory in the source tree.

The second is to compile it as frozen bytecode which incorporates it into the firmware which also avoids compilation but also saves RAM on load. As I understand it only the first option is currently available on the ESP8266.
Peter Hinch
Index to my micropython libraries.

Llwy
Posts: 34
Joined: Thu Mar 10, 2016 7:43 am

Re: MemoryError when importing a custom module

Post by Llwy » Mon May 30, 2016 7:30 pm

Hello,
I have been experimenting with the extended heap size firmware and I still do not have enough memory to load the whole module.
Is there a way from the REPL to diagnose the available memory and the memory used by some module or another. I am kind of shooting in the dark currently by trying to reduce memory usage by making assumption but a real metric could help me in finding out whether I am even going into the right direction.

Thank you,

Llwy

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: MemoryError when importing a custom module

Post by Roberthh » Mon May 30, 2016 8:12 pm

Using cross-compiled code is not available too, at least officially. You can build your own version which supports it using this method: http://forum.micropython.org/viewtopic.php?f=16&t=1962
You have also build mpy-cross by a make command in micropython/mpy-cross.
You then would use mpy-cross to create a xxx.mpy file from a xxx.py file. This has to be loaded to your device and run the usual way.
Dont forget to remove the xxx.py file from the esp8266 otherwise this has preference.

User avatar
rcolistete
Posts: 352
Joined: Thu Dec 31, 2015 3:12 pm
Location: Brazil
Contact:

Re: MemoryError when importing a custom module

Post by rcolistete » Fri Jun 03, 2016 3:10 am

pythoncoder wrote:@rcolistete This occurs when the compiler runs out of RAM: when you import a .py module the compiler running on the board converts the text file to bytecode, which is then run by the Python VM.
Thanks. My suggestion was to compare different MicroPython boards with respect capability to load modules.
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).

Post Reply