Keeping new port synchronised with mainline micropython

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
mvdw
Posts: 25
Joined: Tue May 26, 2015 11:57 pm
Location: Brisbane
Contact:

Keeping new port synchronised with mainline micropython

Post by mvdw » Thu Aug 19, 2021 7:41 am

I am looking to use a Realtek RTL8722 device in a project, and happily the device already has a custom micropython port. However, when I fired up the device and installed the supplied firmware, I noticed there's a couple of things missing, and also the micropython version is reasonably old.

The port has a github repo, which seems to have copied and pasted the original micropython code over, thus freezing it in time and losing any new features and bug fixes since the upstream code was copied across. Also, the way it's built is what I would call a little bit whacky, calling a local ARM toolchain (v6.3(!)) that is embedded in the repo, which replies somewhat on the build system being either linux (OK), or Cygwin (hmmm...). Also, the build instructions seem to deviate slightly from the way I've built micropython in the past for other devices (including my own custom port).

So, with that in mind, I consider I have two options:

1. Merge the two repos locally, being micropython upstream and the rtl8722 codebase, fix the build system, and keep the two synchronised by pulling my own local repo of micropython (maybe forking upstream).
2. Merge the two repos locally, fix the build system and submit the new port as a patch to micropython upstream.

Anyway, I guess the first steps are to merge the two repos, and then fix the build system. Given the current build is apparently complete, I don't expect that it will take too long, provided I can work out which files are which.

Given the way the rtl8722 repo is laid out, I'm not sure what files are from micropython upstream and which are local copies. The rtl8722 port seems to have moved the upstream micropython files into the port/rtl8722 directory.

So, is there any advice on how to determine which files are from upstream (and can be overwritten by newer ones), and which are completely new local files. Investigating the port/XXX directories doesn't fill me with a great deal of confidence as to which are which. I'd like to do this as efficiently as possible; perhaps I should base the layout on the STM port as that seems to be canon?

Anyway, once I've updated it and fixed the build, how would I go about getting it incorporated into mainline micropython? And should I inform Realtek/Ameba that this has been done, and if so, how?

If you'd like to check out the rtl8722 port codebase, the repo is at https://github.com/ambiot/ambd_micropython. Judging by the changelog, that repo has 2 extra releases over the repo found at https://github.com/xidameng/micropython_amebaD. The latter seems to be the repo created by user "xidameng", who asked a couple of questions on this forum about a year ago.

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Keeping new port synchronised with mainline micropython

Post by stijn » Thu Aug 19, 2021 4:41 pm

Easiest end result in my opinion is commit(s) which add the port in your fork, and rebasing those commits regularily on upstream micropython's master.

That last repository you mention actually started as a MicroPython fork (at commit 62f004b) and has commits on top of that so in principle you could squash all consecutive commits to one to inspect changes and/or rebase all that onto the latest master. However with the moving of the micropython directory to a subdirectory that doesn't really get you anywhere, plus I have the impression there are also commits which take changes from original mircopython and are added as new commits. And there's a lot of things, testing code etc which you probably don't want (or at least they're no good fit for an upstream patch). All of that combined makes it pretty hard to actually use git to do any merging for you, which is really too bad.

So from the looks of it what I'd do here is: fork micropython, create a branch, then start copying from ports/rtl8722 the bare minimum needed to be able to build a working binary (without extended hardware I/O etc so far, just to get the build system working). Record individual steps as individual commits (especially if you need to make changes to the core micropython code; these changes can go into separate PRs first). Then add features, again choosing logical commits. Finally if things are working and tested you can create a PR. However note there are some requirements to get a port added; should have a userbase larger than a couple of people, I think it is epected you maintain the port, etc. So perhaps better to start figuring that out as well.

Post Reply