Differences between Micropython ports for ESP32 and ESP8266

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
mattyt
Posts: 122
Joined: Mon Jan 23, 2017 6:39 am

Re: Differences between Micropython ports for ESP32 and ESP8266

Post by mattyt » Sat Sep 08, 2018 2:10 pm

OutoftheBOTS_ wrote:
Sat Sep 08, 2018 12:01 am
The biggest problem I find with different platforms of MP is the machine module. It really breaks the mantra of MP of having a standard syntax across all platforms. IMO I think the PYB module should be fazed out in the favor of a machine module with standard syntax across all platforms.
As I understand that is the intent. pyb only exists for backwards compatibility and machine should be used in preference.

mattyt
Posts: 122
Joined: Mon Jan 23, 2017 6:39 am

Re: Differences between Micropython ports for ESP32 and ESP8266

Post by mattyt » Sat Sep 08, 2018 2:19 pm

OutoftheBOTS_ wrote:
Sat Sep 08, 2018 12:06 am
In my experience it is hard to have a standard library for NeoPixel as there is no such thing as a standard NeoPixel but rather so many clones with most of them having different timings and no matter how the seller advertised the IC on the LED it is not till u put them under the microscope and look at the IC that you can try ti identify it then set correct timings. On laboris port there is a method to set timing to suit the IC on the LED
True but there are commonly used timings that work on the vast majority of WS2812b and clones. Unfortunately the most common timing is not the default on the ESP32 - but it is on the ESP8266; passing 'timing=True' on the ESP32 Neopixel implementation matches the timing on the ESP8266.

The only reason I haven't gotten around to changing the default is that I would prefer to completely overhaul the Neopixel implementation on ESP32. My preference is to use RMT, like Loboris, but expose RMT as a MicroPython module. Then implement Neopixel as a Python module using the RMT module. This is different to the Loboris approach that keeps RMT as a C library. I think RMT is useful in its own right...

mattyt
Posts: 122
Joined: Mon Jan 23, 2017 6:39 am

Re: Differences between Micropython ports for ESP32 and ESP8266

Post by mattyt » Sat Sep 08, 2018 2:27 pm

OutoftheBOTS_ wrote:
Sat Sep 08, 2018 12:09 am
WebREPL comes to mind. I'm making custom ESP32-WROOM gizmos to run household tasks, I'm still not sure of a way for a remote wireless ESP32 embed to update microPython files without WebREPL
Most people have used FTP server to update files on a ESP32 wirelessly there is a number of FTP server around. It is my assumption that webrepl uses FTP to upload files.
Two points: 1) It's worth mentioning that WebREPL was implemented for the ESP32 in 1.9.4. 2) No, WebREPL does not use FTP for file transfers (it uses a simple, albeit proprietary transfer protocol).

HermannSW
Posts: 67
Joined: Wed Nov 01, 2017 7:46 am
Contact:

Re: Differences between Micropython ports for ESP32 and ESP8266

Post by HermannSW » Wed Oct 10, 2018 4:11 pm

mattyt wrote:
Sat Sep 08, 2018 2:27 pm
2) No, WebREPL does not use FTP for file transfers (it uses a simple, albeit proprietary transfer protocol).
Here are the details:
https://github.com/micropython/webrepl# ... e-transfer

I used loboris' MicroPython with ftp server before, but completely switched to using webrepl_cli.sh (like using scp) because WebREPL can be enabled on any MicroPython, no special builds needed. With WebREPL you can even do a remote shell (like ssh, see my signature), or let one MicroPython execute commands on another MicroPython.

User avatar
pythoncoder
Posts: 2991
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

The pyb module

Post by pythoncoder » Wed Oct 10, 2018 4:52 pm

mattyt wrote:
Sat Sep 08, 2018 2:10 pm
OutoftheBOTS_ wrote:
Sat Sep 08, 2018 12:01 am
...IMO I think the PYB module should be fazed out in the favor of a machine module with standard syntax across all platforms.
As I understand that is the intent. pyb only exists for backwards compatibility and machine should be used in preference.
It's not as simple as that. pyb supports a considerable amount of STM-specific functionality which couldn't (IMO) be ported to machine. So it's horses for courses: my approach is to use machine where possible for portability. But pyb is often essential. An example from something I'm working on: I2C slave mode is not available in machine.

You might also look at the complex STM specific timer/timer channel modes.
Peter Hinch

OutoftheBOTS_
Posts: 402
Joined: Mon Nov 20, 2017 10:18 am

Re: The pyb module

Post by OutoftheBOTS_ » Wed Oct 10, 2018 10:08 pm

pythoncoder wrote:
Wed Oct 10, 2018 4:52 pm
mattyt wrote:
Sat Sep 08, 2018 2:10 pm
OutoftheBOTS_ wrote:
Sat Sep 08, 2018 12:01 am
...IMO I think the PYB module should be fazed out in the favor of a machine module with standard syntax across all platforms.
As I understand that is the intent. pyb only exists for backwards compatibility and machine should be used in preference.
It's not as simple as that. pyb supports a considerable amount of STM-specific functionality which couldn't (IMO) be ported to machine. So it's horses for courses: my approach is to use machine where possible for portability. But pyb is often essential. An example from something I'm working on: I2C slave mode is not available in machine.

You might also look at the complex STM specific timer/timer channel modes.
Hmm this can make things become more messy as MP moves forward and becomes ported to more platforms and more hardware capabilities become exposed for MP to utilize. Super tricky trying to run a standard syntax on different hardware.

As a user of MP but not a contributor I would still prefer that the STM32 specific function was still ported to Machine module so that pyb module can be fazed out so that STM32 can be totally programmed just using machine module even though it means there will be methods in the machine module that can't be used by other platforms

mattyt
Posts: 122
Joined: Mon Jan 23, 2017 6:39 am

Re: The pyb module

Post by mattyt » Wed Oct 10, 2018 10:52 pm

pythoncoder wrote:
Wed Oct 10, 2018 4:52 pm
mattyt wrote:
Sat Sep 08, 2018 2:10 pm
As I understand that is the intent. pyb only exists for backwards compatibility and machine should be used in preference.
It's not as simple as that. pyb supports a considerable amount of STM-specific functionality which couldn't (IMO) be ported to machine. So it's horses for courses: my approach is to use machine where possible for portability. But pyb is often essential. An example from something I'm working on: I2C slave mode is not available in machine.

You might also look at the complex STM specific timer/timer channel modes.
You're absolutely right; my statement was oversimplified.

How about this for a suggestion? We provide a deprecation warning for anything in pyb where we should be using the equivalent feature from machine (with an intent to remove it over time). We could implement this with a debug print statement.

Then we need to figure out how to consistently manage port-specific features (ie everything left in pyb). There's an esp32 module for example; it seems that an stm32 module would be appropriate. The goal should be to prefer to put features in the generic modules and fallback to port-specific features only when there's no obvious way to generalise the api to the hardware.

The port-specific module may also be a good place to initially implement a feature with a goal to moving it out to a general module in the future as it matures. An example would be esp32.TouchPad - it's port-specific at the moment but could feasibly become a general module. Or the pyb.I2C slave mode.

There are some issues we'll need to resolve - for example how how to manage different capabilities of stm32 chips? And how to manage micros with hardware that is almost possible to generalise - but not quite. But I'd be curious to hear if you think this is an appropriate way forward...I can 'lobby' Damien if we have a reasonable proposal. :)

SpotlightKid
Posts: 377
Joined: Wed Apr 08, 2015 5:19 am

Re: The pyb module

Post by SpotlightKid » Thu Oct 11, 2018 4:33 am

mattyt wrote:
Wed Oct 10, 2018 10:52 pm
The port-specific module may also be a good place to initially implement a feature with a goal to moving it out to a general module in the future as it matures. An example would be esp32.TouchPad - it's port-specific at the moment but could feasibly become a general module. Or the pyb.I2C slave mode.
The I2C slave mode of pyb.I2C is built on-top of the STM32CubeMX library and the I2C and DMA hardware support of the ST chips. It can not be be directly ported to other architectures, not even those with also a Cortex-M.

User avatar
pythoncoder
Posts: 2991
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

The pyb module

Post by pythoncoder » Thu Oct 11, 2018 5:14 am

@mattyt I think the situation is quite simple. Try to code using machine. If you hit a roadblock where you need a port-specific feature, use the port-specific module and accept that your code is no longer portable.

I'm not sure deprecation warnings are practicable. Is instantiating pyb.Pin or pyb.Timer to be deprecated? The answer depends on how you plan to use it.

Unfortunately there is a persistent view that pyb is obsolescent; as far as I can see it is not. It is potentially one of a family of port-specific modules providing access to hardware features unique to that port. I don't really see a problem with this. Hardware features like esp32.TouchPad probably cannot be ported to STM: AFAICR the hardware simply does not exist. The timer channels on STM are completely platform-specific and you can be sure that other platforms will have their own incompatible realisations.

A generic machine module can support interfaces which are well-defined industry standards such as UART's, I2C, and SPI. You can produce abstractions for common pieces of hardware such as an RTC, but even there you'll find alligators - alarms, wakeups and suchlike. And, as @SpotlightKid suggested, a truly generic I2C has an issue. I think machine will always represent a lowest common denominator of functionality which all platforms can be expected to support.

As for lobbying Damien, this is your call. The subject was discussed ad nauseam on GitHub with no clear resolution. I suspect he's very busy, hopefully preparing Pyboard D for us :D
Peter Hinch

mattyt
Posts: 122
Joined: Mon Jan 23, 2017 6:39 am

Re: The pyb module

Post by mattyt » Sun Oct 14, 2018 10:56 am

SpotlightKid wrote:
Thu Oct 11, 2018 4:33 am
The I2C slave mode of pyb.I2C is built on-top of the STM32CubeMX library and the I2C and DMA hardware support of the ST chips. It can not be be directly ported to other architectures, not even those with also a Cortex-M.
Sure, and the ESP32 port is built on the ESP-IDF where there's support for I2C slave mode.

But the python API we present ought to be the same for both ports. That API should be in machine.I2C. The ESP32 could support the same MicroPython I2C slave API that's currently in pyb. It seems fairly generic.
pythoncoder wrote:
Thu Oct 11, 2018 5:14 am
@mattyt I think the situation is quite simple. Try to code using machine. If you hit a roadblock where you need a port-specific feature, use the port-specific module and accept that your code is no longer portable.
I agree, but shouldn't we put only truly port-specific features in the port-specific modules? Take a look at the pyb module; an awful lot of that module is not port specific and is duplicated in machine.

I've had many newcomers to MicroPython get confused over this. I've also ported a few libraries that used the pyb module instead of the generic alternatives - and worked perfectly fine on all ports when modified. I think we should be helping developers 'fall into the tar pit of success' :) by making it easy to write cross-port code.
pythoncoder wrote:
Thu Oct 11, 2018 5:14 am
I'm not sure deprecation warnings are practicable. Is instantiating pyb.Pin or pyb.Timer to be deprecated? The answer depends on how you plan to use it.
I would be happier if pyb.Pin was deprecated, yes. Or at least most of it. There's not much there that looks port-specific. You could still retain the pyb.Pin.board.X3 named-pins for example; but return an instance of machine.Pin.

I agree that Timers are currently fairly port specific because hardware support does differ fairly dramatically. So leave that port-specific, at least for now.
pythoncoder wrote:
Thu Oct 11, 2018 5:14 am
Unfortunately there is a persistent view that pyb is obsolescent; as far as I can see it is not. It is potentially one of a family of port-specific modules providing access to hardware features unique to that port. I don't really see a problem with this.
I guess I'm just making the case that some (a lot?) of pyb should be obsoleted - anything that has a generic equivalent. The port-specific features are all that should be in the port-specific modules.
pythoncoder wrote:
Thu Oct 11, 2018 5:14 am
Hardware features like esp32.TouchPad probably cannot be ported to STM: AFAICR the hardware simply does not exist. The timer channels on STM are completely platform-specific and you can be sure that other platforms will have their own incompatible realisations.
I'm not convinced. The esp32.TouchPad API is quite generic; it looks compatible with the SAMD range for example. It's basically just a value corresponding to the amount of capacitance detected on a pin.

If TouchPad was instantiated on a platform without cap touch features - like STM - then either the API shouldn't exist for that port or, better, an exception ("Capacitive touch not present for this port") should be raised.
pythoncoder wrote:
Thu Oct 11, 2018 5:14 am
A generic machine module can support interfaces which are well-defined industry standards such as UART's, I2C, and SPI. You can produce abstractions for common pieces of hardware such as an RTC, but even there you'll find alligators - alarms, wakeups and suchlike. And, as @SpotlightKid suggested, a truly generic I2C has an issue. I think machine will always represent a lowest common denominator of functionality which all platforms can be expected to support.
I think I feel the same way though I think my definition of what should be in machine is perhaps a little larger. Debating what should be in or out of machine is reasonable. But I don't see any good reason to leave methods in pyb that already have equivalents.

There are going to be differences between hardware - like you highlighted with RTC. There are a few ways we can handles this but carving out different port-specific API's when the majority - or the 80% - of features used is less than ideal since user code will be bound to a port.

In RTC for example we could define the common interface (including basic wakeup and alarm features) and throw exceptions on ports that don't have that support. Or, if that port has incompatible low-level API's then that functionality could go into the port-specific module - probably passing in the generic RTC instance as a parameter.
pythoncoder wrote:
Thu Oct 11, 2018 5:14 am
As for lobbying Damien, this is your call. The subject was discussed ad nauseam on GitHub with no clear resolution. I suspect he's very busy, hopefully preparing Pyboard D for us :D
I've seen some of the vast bikeshedding that happens on these kinds of tickets... :shock:

I don't expect Damien to do any of this work - I want PYBD as much as anyone! - but my reason for 'lobbying' is because I wouldn't want to raise a PR modifying pyb unless it had his blessing. :)

Post Reply