out of memory

[gratefulfrog]

ok, I'm an idiot...

Fixed by:

Code: Select all

$ sudo apt-get install python-usb

[gratefulfrog]

And now, the results!!!

Remember, prior to the byte compiling and frozen code, I could not allocate an instance of the App at the REPL.

Now, with the byte compiled code loaded in flash:

Code: Select all

MicroPython v1.8.2-89-gb67eb20 on 2016-08-13; PYBv1.0 with STM32F405RG
Type "help()" for more information.
>>> import gc
>>> gc.collect()
>>> gc.mem_free()
102064
>>> from app import App
>>> a=App()
>>> gc.mem_free()
68448  # total cost import + allocation: 33.6KB
>>>

So, I can now finish the coding and try the system!!!

Thanks so much to all you guys!
Ciao,
Bob

[gratefulfrog]

Well, I'm back with yet another challenge related to the byte-compiling and importing from flash.

I am now facing an unexpected hindrance in my devt. cycle.

This is what happens:

• I have 20 or so .py files that I byte compile and load into flash. These files embed a dependency hierarchy.
• I would like to continue devt. on one of the files, so I put it on the sd-card.
• I change the sys.path to ['/sd', '/sd/lib', '/flash', '/flash/lib']
• Now, if I import the file, I get the sd-card version.
• However, if the file itself imports other files, then these cannot be found unless they appear on the sd-card as well.
• This means, that all the files need to be on the sd-card and then I run out of memory as before.

I don't understand why the import mechanism doesn't follow the sys.path and look for the modules on flash??

It works properly on linux:
file a.py is in ~/Desktop

Code: Select all

# file a.py
import b
def f(x):
    return b.g(x)

file b.py is in ~/.micropython/lib

Code: Select all

# file b.py
def g(x):
    print('I did g!')
    return x+1

Running micropython from ~/Desktop, it works fine:

Code: Select all

~/Desktop$ micropython 
Micro Python  on 2015-05-11; linux version
>>> import sys
>>> sys.path
['', '/home/bob/.micropython/lib', '/usr/lib/micropython']
>>> import a
>>> a.f(12)
I did g!
13
>>> 

But on the pyboard, with b.py compiled into flash, it says module b is not found.

Is this normal behavior or a bug?

Thanks for your thoughts!
Ciao,
Bob

[pythoncoder]

Perhaps I'm not following you here, but I thought you were using frozen bytecode. If you put foo.py into your frozen directory, compile the firmware, and load it you can issue

Code: Select all

import foo

from any of your modules and the frozen version will be found in preference to any other version regardless of the path. They don't live in /flash which refers to the specific filesystem on the built-in flash memory: they inhabit a region of flash memory outside of any explicit filesystem.

While developing I work as follows. I don't change the path. I implement "stable" modules as frozen bytecode, put the "development" modules on the SD card and ignore the built in flash. Obviously once you've got a stable working system you might choose to freeze everything except boot.py and main.py. These might live in flash so you could run without the SD card.

[gratefulfrog]

@pythoncoder

Thank you for your response.

Further exploration has lead me to understand the sys.path variable and to discover an error in my previous post.

I have done some further experimenting with these files:
Frozen a.py:

Code: Select all

# FROZEN file a.py
import b
def f(x):
    print('I am FROZEN in A!')
    return b.g(x)

Frozen b.py:

Code: Select all

# FROZEN file b.py
def g(x):
    print('I am FROZEN in B!')
    return x+1

SD-Card a.py

Code: Select all

# SD-CARD file a.py
import b
def f(x):
    print('I am A on the SD Card!')
    return b.g(x)

NOTE: b.py is NOT present on the SD-Card.

Now look at this sequence:

Code: Select all

>>> import a
>>> a.f(10)
I am FROZEN in A!
I am FROZEN in B!
11
>>>
PYB: sync filesystems
PYB: soft reboot
MicroPython v1.8.2-89-gb67eb20 on 2016-08-26; PYBv1.0 with STM32F405RG
Type "help()" for more information.
>>> import sys
>>> sys.path
['', '/flash', '/flash/lib', '/sd', '/sd/lib']
>>> sys.path[1] = sys.path[0]
>>> sys.path[0] = sys.path[3]
>>> sys.path
['/sd', '', '/flash/lib', '/sd', '/sd/lib']
>>> import a
>>> a.f(20)
I am A on the SD Card!
I am FROZEN in B!
21

I conclude that the sys.path '' (empty string) refers to the FROZEN code and the other elements refer to what one would expect.

That means one can put under-work files on the sd-card and use them in collaboration with the frozen code by appropriately setting the sys.path variable, which is exactly what I wanted!

This interaction has made my life a lot better!

Thanks for your help!
Ciao,
Bob

[pythoncoder]

Interesting and unexpected (by me). Yet os.listdir('') lists the contents of the SD card and always seems to yield the same result as os.listdir('/sd').

[dhylands]

The magic seems to be here:
https://github.com/micropython/micropyt ... #L114-L127
that loop is where it enumerates sys.path and if a path element has a non-zero length then it prepends the path element and a slash to the name of the module.

So if your path was ['', '/sd', '/sd/lib'] then it would look for 'foo', '/sd/foo', and '/sd/lib/foo'.

stat_dir_or_file then tries some variations (using as is (covers case of directory with __init__.py), adding .py, adding .mpy

and mp_import_stat_any checks for frozen modules.

So, yes - it looks like the empty string in sys.path determines where it searches for frozen modules.

I think it would be useful to have a way of determining what frozen files exist. It seems like it could be done as some type of virtual filesystem, but that feels way too heavy to me.

I think adding a function that just returns a tuple of names would be useful, especially from a debugging context.

[pythoncoder]

...I think adding a function that just returns a tuple of names would be useful, especially from a debugging context.

Great idea!