One month's development report on MicroPython ESP8266 port

[pfalcon]

As this topic may be reached by external references, a quick recap. A month ago MicroPython developers finished a KickStarter project to develop a full-featured and well supported MicroPython port to ESP8266 WiFi SoC: https://www.kickstarter.com/projects/21 ... y-easy-iot . The kickstarter became a great success, with a number of stretch goals unlocked. Immediately after the campaign end, we proceeded to push finished code to public github repository (https://github.com/micropython/micropyt ... er/esp8266), while continuing work on unfinished parts and testing them via early-access program (participants are backers who contributed more than average to the success of the kickstarter).

Well, that's it folks - a month after the campaign end, majority of features whose early prototypes we showed you in the campaign videos are now on the Github, in pretty clean code, and pretty well tested by ourselves and early adopters. Among the features available are:

• Standard BSD socket API support
• Support for mounting filesystem in userspace (real filesystem, with long names and
  directories)
• Basic hardware support for SPI, I2C, 1-Wire, Neopixels
• Better API compliance with MicroPython standards
• Beginnings of documentation
• Regression testsuite

The current state of the repository is also on par with the latest early-access release, except for boot initialization setup (including mounting of filesystem on boot) - it is still work in progress and requires proper design and discussion (see e.g. https://github.com/micropython/micropython/issues/1741 , https://github.com/micropython/micropython/issues/1955) before being finalized.

Early-access backers, don't be sad though - we have more dogfood coming your way . Please expect a next early-access release for WebREPL testing.

To clarify, this release is not yet suitable for end users - the documentation is work in progress and misses many things, so you should be ready to consult the source code in case of questions, as being busy with fast-paced development, we are unable to provide end-user support for unfinished product. However, if you don't shy to
git log or poke around project with hundreds of files, feel welcomed to start hacking around (and if you're not used to git log around, feel free to use that as an excuse to start ). We have growing active community on the forum which already proved to be able to find solutions for many issues and paramount for wide-coverage
testing: http://forum.micropython.org/viewforum.php?f=16 .

Our development plans for next month:

• WebREPL
• SSL
• Documentation
• Finalizing other APIs
• If time permits, starting on first of stretch goals (that's MQTT as you guessed - please stay tuned for the official poll results as we collect the last votes coming in)

[deshipu]

Support for mounting filesystem in userspace (real filesystem, with long names and
directories)

So how do you mount the filesystem on the github images? I've seen this question asked several times already, and it was never answered. The docs have nothing on that either. There is a filesystem object in the os module, but it requires two parameters to create, and apparently just passing random values to it doesn't work. Looking at the source code doesn't help my untrained eye either. Some hints?

It would be nice to be able to test that.

[MaartenB]

Really excited to see this getting into github, and looking forward to play with this in the future.
I wanted to add a use case that you may not have considered, and may or may not be related to the two issued you mention.
CPython can be quite easily embedded in an application. Micropython (for esp, but maybe also others) could also be used that way. For instance, as a scripting language in ESPEasy for handling events (button push). In this case you may not want to expose the wifi module, but filesystem support would be useful I think. A boot.py wouldn't make much sense (also since there is no command line), although some initialization like site.py might be.

[trixam]

Hi

I would like to say thank you for the first month of coding, debugging, discussion,documentation and for a working paste mode in the REPL (!)

Hope's the file system and the doc for it comes quickly

Regards
Trixam

[kfricke]

So how do you mount the filesystem on the github images? I've seen this question asked several times already, and it was never answered. The docs have nothing on that either. There is a filesystem object in the os module, but it requires two parameters to create, and apparently just passing random values to it doesn't work. Looking at the source code doesn't help my untrained eye either. Some hints?

It would be nice to be able to test that.

I have had the same questions in my mind for several days now, but sadly not yet the time to ask. Also after reading the sources and checking the alpha firmware images i could not find a solution for this as well.
Maybe some functions should be introduced to check where a filesystem is mounted from?

[pfalcon]

I have had the same questions in my mind for several days now, but sadly not yet the time to ask. Also after reading the sources and checking the alpha firmware images i could not find a solution for this as well.
Maybe some functions should be introduced to check where a filesystem is mounted from?

Ok, let's try to find an answer together. @deshipu mentioned "There is a filesystem object in the os module". How could he possibly know? Well, just by looking at the source of "os" module: https://github.com/micropython/micropyt ... uos.c#L116 . Or even easier: by doing "dir(os)" after "import os", or just trying tab completion on "os.".

Ok we found out that there's "os.VfsFat", which is not exactly "os.mount" which is described for PyBoard but looks pretty close. Still, what to do next? Knowing that MicroPython is open-source project, there must be discussion about it somewhere on the project. The project is hosted on github, so let's search there: https://github.com/micropython/micropyt ... 3&q=VfsFat . What we immediately see is a hit for tests/extmod/vfs_fat_ramdisk.py which gives full usage example (and test!) for VfsFat. Don't stop there though, click on "Issues" tab on the left - you'll see the whole development history of this feature! It first was mentioned on Feb 6, and then finally implemented on Feb 13. The tickets linked provide extensive discussion and background behind introducing os.VfsFat as well as future plans regarding it.

Github is cool and provides wide and deep insight to any open-source project hosted on it, but do you *have* to use it to get that insight? Not really, it's just convenient shortcut. You could just grep the source code and would arrive at the same file tests/extmod/vfs_fat_ramdisk.py, which is, again, provides full usage example.

Guys, don't hesitate to search the source code and the project infrastructure (bugs, pull request, wiki, etc.)! There's wealth of information available which will get you along fair way. Also, to remind, what we discuss here is community developers' access to the early source code. If you're an end user, you don't need to do anything of the above, just kindly and patiently wait until the product is finished and released for your use.

[deshipu]

Thank you. I suppose I didn't try hard enough and/or didn't look in the right places.

[gschmott]

Ok, let's try to find an answer together. @deshipu mentioned "There is a filesystem object in the os module". How could he possibly know? Well, just by looking at the source of "os" module: https://github.com/micropython/micropyt ... uos.c#L116 . Or even easier: by doing "dir(os)" after "import os", or just trying tab completion on "os.".

Ok we found out that there's "os.VfsFat", which is not exactly "os.mount" which is described for PyBoard but looks pretty close. Still, what to do next? Knowing that MicroPython is open-source project, there must be discussion about it somewhere on the project. The project is hosted on github, so let's search there: https://github.com/micropython/micropyt ... 3&q=VfsFat . What we immediately see is a hit for tests/extmod/vfs_fat_ramdisk.py which gives full usage example (and test!) for VfsFat. Don't stop there though, click on "Issues" tab on the left - you'll see the whole development history of this feature! It first was mentioned on Feb 6, and then finally implemented on Feb 13. The tickets linked provide extensive discussion and background behind introducing os.VfsFat as well as future plans regarding it.

@pfalcon - thanks for the gentle introduction on how to utilize GitHub's search features. Unfortunately, I suspect I was not alone in being unaware of the general direction and roadmap for the VfsFat module. I only started to closely follow Micropython after the recent ESP8266 Kickstarter (to which I contributed) so it's not surprising I missed this. After reading the development thread things are clearer but what isn't entirely clear is whether there is an existing flash-based back-end for VfsFat to utilize. The example shows a RAM back-end but given the scarcity (and volatility) of RAM on the ESP8266 that isn't an optimal solution. Does one exist that we can already leverage or is the expectation that everyone develops one themselves using the primitive flash functions Espressif provides (in their C library) and integrating it into the ESP8266 variant of Micropython? I realize this port is still under heavy development so perhaps the plan is the provide this at some point. In this case I'll be patient and wait for further developments. If not, then although a suitable framework seems to be provided (e.g. VfsFat/FatFS), what currently exists in the public repository appears to fall short of what might be expected if a working (flash-based) back-end already existed. Have I missed something or or am I just being over-eager in my attempts to test/utilize this module using a persistent back-end?

[platforma]

Have I missed something or or am I just being over-eager in my attempts to test/utilize this module using a persistent back-end?

I think probably that :)
Things will get clearer with time and the documentation should reflect everything that has been worked on including the file system. So I'd prefer if Paul spent time writing the docs instead of forum posts!

[pythoncoder]

Has anyone had success with this test program? When I ran it, it formatted the filesystem on the ESP8266 flash (beware). It successfully wrote and read back the test file - but it was located in Flash, not on the ramdisk. It then issued an assertion failure (presumably because it hadn't written to the ramdisk).

I reduced the size of the ramdisk in case there were memory issues, but to no effect.