Recommended CI Pipeline

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
benpnz
Posts: 2
Joined: Fri Jan 14, 2022 3:06 pm

Recommended CI Pipeline

Post by benpnz » Fri Jan 14, 2022 3:10 pm

Hi All,

I had some questions about the recommended CI pipeline for a micropython project in gitlab. I saw some discussion in a similar issue (viewtopic.php?f=2&t=7376&hilit=ci+pipeline) but it doesn't seem to answer this question completely.

- What is the recommended method of doing this? I could use pytest to test code that is compatible with standard python. However, my project makes use of uasyncio which is not available in standard python.

- Is there a micropython docker image that can be imported in to the CI pipeline? Or some recognized way of adding an emulator?

Please let me know if you have any questions and thanks in advance for any help.

Thanks,
Ben.

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Recommended CI Pipeline

Post by stijn » Fri Jan 14, 2022 5:27 pm

Not sure what you are asking exactly (depends on what your 'micropython project' is), but maybe this helps: in the MicroPython source code there's tools/ci.sh which does everything wrt building and testing. It's used by the github CI (see https://github.com/micropython/micropython/actions for output, and the .github/workflows directory for additional setup). And can be used locally as well.

And as mentioned in the link you found: tests/run-tests.py is definitely the way to go for running your own tests as well.

benpnz
Posts: 2
Joined: Fri Jan 14, 2022 3:06 pm

Re: Recommended CI Pipeline

Post by benpnz » Tue Jan 18, 2022 4:19 pm

Thanks for the response. My project is written in micropython and is basically going to be a task system using uasyncio to schedule tasks on a STM32F405 microcontroller to read from various sensors and write to actuators periodically. I know this isn't in massive detail but I am still starting out at the moment.

If I have understood the links you sent correctly it seems that 'tools/ci.sh' includes code to build the correct port of micropython for my architecture. So to confirm the process in a CI pipeline for a project like mine would be:

1) Build the micropython port required from source within the gitlab ci (or should the relevant binary just be added to the repo pre-built and used?).
2) Use this port to compile the micropython code.
2) Use this port to run the unittests (also in micropython).

Thanks again for your help and apologies if my questions are not clear, CI pipelines are not my strong point. Also if anyone could point me to an example gitlab-ci.yml file that would be amazing.

Thanks,
Ben.

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Recommended CI Pipeline

Post by stijn » Wed Jan 19, 2022 8:52 am

So to confirm the process in a CI pipeline for a project like mine would be
Yes that's it. It's also exactly what MicroPython does on Github Actions (and previously Travis).

However, there's no readily available STM32 emulator I think, so ci.sh doesn't run STM32 tests. In your case you'd build the unix port and use it to run all tests; if it runs on the unix port, it should run on the STM as well.

or should the relevant binary just be added to the repo pre-built and used?
I'm personally not a fan of adding such binaries to a repository. It's just much more interesting to have a script which creates the needed environment so you can use it locally and one the CI etc.

Also if anyone could point me to an example gitlab-ci.yml file that would be amazing.
All it has to do is the equivalent of

Code: Select all

    runs-on: ubuntu-20.04
    steps:
    - uses: actions/checkout@v2
    - name: Install packages
      run: source tools/ci.sh && ci_stm32_setup
    - name: Build
      run: source tools/ci.sh && ci_stm32_pyb_build
so shouldn't be too hard.

Post Reply