Unix Port - Import Paths

[semireg]

If I have my project organized with a 'src' and 'tests' directory at the same level, how do I import src/file.py from within tests/test-file.py ?

I understand that ~/.micropython/lib is searched... are other paths configurable? Can I pass the micropython binary an option or does it respect a env variable?

One way to solve it is to symlink using ln -s ../src src and then import with from src.file import *. Any other ideas?

[dhylands]

MicroPython uses sys.path to control which directories and the order that they are searched. The empty string represents frozen modules.

[semireg]

Ah, I read that... and for some reason assumed it was read-only

This works:

Code: Select all

sys.path.append('../src')
import file

But how could I easily run this for each test... are there are any published best practices for test rigs?

[dhylands]

I'll have to let somebody else answer how test suites do it.

In order to minimize fragmentation of the heap, I suspect you'll want to reinitialize between each test as well, which may mean only doing one test for each invocation of micropython, but I'm not sure.

It probably depends on exactly what you're trying to achieve.

[dhylands]

You could look at the tests/run-tests

It's a CPython script which invokes micropython for each test to run. These tests are primarily comparing the output of micropython with CPython to verify that they're the same. Your tests are probably trying to do something different.

But you could use a CPython test wrapper to invoke each test. The unix version of micropython takes a -c argument that allows python to be executed directly from the command line.

i.e.

micropython -c "print('hello')"

and you can use semicolons to separate multiple commands (to say adjust sys.path and do your import).

[stijn]

Code: Select all

sys.path.append('../src')
import file

But how could I easily run this for each test... are there are any published best practices for test rigs?

I'd advise against using relative directories as it forces the current working directory to be correct which is error prone. Use full paths instead, which simply always works, something like sys.path.append(os.path.join(os.path.dirname(__file__), '../src')).
But doing that manually for each test is messy. A nicer option is to set the MICROPYPATH environment variable which gets appended to sys.path. And as Dave suggests, depending on the type of tests you have you can use run-tests with the --keep-paths option which was added especially for running 'external' non-MicroPython-core tests.

So, practically, if you're using the unittest module run

Code: Select all

> export MICROPYPATH="/path/to/src"
> micropython /path/to/tests/test-file.py

For real-world usage where you have multiple tests you'll probably use ls and loop over /path/to/tests/*.py for example.

If you use output-based tests (which is very convenient for system/integration testing) use

Code: Select all

> export MICROPYPATH="/path/to/src"
> python run-tests --keep-path -d /path/to/tests

or, if you don't want to possible problems because you have MICROPYPATH set globally you can use a wrapper like

Code: Select all

#!/bin/sh
export MICROPYPATH="/path/to/src"
micropython $1

and then invoke it like

Code: Select all

export MICROPY_MICROPYTHON=/path/to/test-wrapper.sh
python run-tests -d /path/to/tests

[semireg]

Thank you, this helps a lot.

[semireg]

I'm having a difficult time getting this working satisfactorily.

With regards to:

Code: Select all

sys.path.append(os.path.join(os.path.dirname(__file__), '../src'))

It doesn't seem that os.path is implemented in micropython. Further, __file__ is a relative path anyways.

If I use:

Code: Select all

export MICROPYPATH="../src"

My sys.path is not appended, but overwritten to:

Code: Select all

['', '../src']

which means I lose the libs installed via upip:

Code: Select all

['', '/path/to/homedir/.micropython/lib', '/usr/lib/micropython']

If I use the "-c" route I face another issue. Here is a simple shell script to run the tests:

Code: Select all

#!/bin/bash

for TEST_FILE in $(ls test_*.py); do
  IMPORT_NAME="${TEST_FILE%.*}"
  echo "Testing $IMPORT_NAME"
  micropython -c "import sys; sys.path.append('../src'); import $IMPORT_NAME;"

  if [ $? -ne 0 ]; then
    echo "Failed test in $TEST_FILE"
    exit 1
  fi
done

The problem here is that __name__ is no longer __main__, so this test_empty.py:

Code: Select all

import unittest

print("file is:{}".format(__file__))
print("name is:{}".format(__name__))

class EmptyTestCase(unittest.TestCase):
    """Tests for stubbing."""

    def testTrue(self):
        self.assertTrue(True)

if __name__ == '__main__':
    unittest.main()

outputs:

Code: Select all

$ micropython -c "import sys; sys.path.append('../src'); import test_empty"
file is:test_empty.py
name is:test_empty

No test is run. See https://github.com/micropython/micropyt ... nittest.py.

vs...

Code: Select all

$ micropython test_empty.py 
file is:test_empty.py
name is:__main__
testTrue (EmptyTestCase) ... ok
Ran 1 tests

OK

Sure, this runs the test, but it doesn't have the correct sys.path. Hmmm... Any way to get the best of both worlds? My lack of experience with Python is showing

[stijn]

It doesn't seem that os.path is implemented in micropython. Further, __file__ is a relative path anyways.

My mistake, os.path is from micropython-lib and the version we use has os.path.realpath as well to get full paths

My sys.path is not appended, but overwritten to:

Yes but you can append the other paths need to MICROPYPATH?

[semireg]

Heh, yep. I woke up this morning and thought, doh... just add the other paths.

Code: Select all

#!/bin/bash

export CURDIR=$(pwd)
export MICROPYPATH="$HOME/.micropython/lib:/usr/lib/micropython:$CURDIR/../src"

for TEST_FILE in $(ls test_*.py); do
  IMPORT_NAME="${TEST_FILE%.*}"
  echo "Testing $IMPORT_NAME"
  micropython $TEST_FILE

  if [ $? -ne 0 ]; then
    echo "Failed test in $TEST_FILE"
    exit 1
  fi
done

echo "All tests passed."

All is well. Thank you.