WAMP/AutobahnPython support in MicroPython

[danielm]

Autobahn.ws (http://autobahn.ws/) is multi-platform implementation of bi-directional realtime communication protocol called WAMP which I found useful for my projects in the past. I would like to see it working together with MicroPython as I am planning to use this platform for next iteration of my SLA/SLS 3D-printer controller.

I contacted Tavendo (creator of Autobahn) support and they replied that they are not currently planning to do the porting themselves, however if there will be only minor modifications requiried, they may find some time.

In this forum thread (https://groups.google.com/forum/#!searc ... YYkvF532kJ) following dependencies are listed for AutobahnPython:
----------------------------------------------------------------------------------------
The most important module from the standard lib (on Py3) is "asyncio".

Here is a list of all stdlib modules used in AutobahnPython:

abc
base64
binascii
collections
copy
datetime
future
hashlib
hmac
inspect
itertools
json
operator
os
pickle
platform
pprint
random
re
struct
sys
time
traceback
unittest
urlparse
urllib

The only 3rd party package requirement is "six".

None of above requires a C-extension to Python.
----------------------------------------------------------------------------------------

What is your opinion - would it be possible to make it work on MicroPython platform?

I cannot test it yet as I am waiting for my pyboard to be shipped.

Daniel

[SpotlightKid]

A lot of these modules are not available on MicroPython (yet) and porting them would be a considerable effort. Some of them have several dependencies themselves or would need high amounts of storage or memory, so IMHO it would be an effort unlikely to succeed.

This list of dependencies seems rather bloated, though, for a module simply providing support for two network protocols. Probably most of these are only needed for development or to support Python 2 and 3. But maybe it would be easier to write a new stripped-down implementation of these protocols for MicroPython. Or just look for less resource-intensive alternatives. MQTT, for example, should be very easy to implement, even on MicroPython for the pyboard, once a network socket implementation is in place. There is already a pure Python implementation for it (paho-mqtt), which one could probably use as a starting point.

[danielm]

Thanks a lot for your answer. I will take a look at MQTT and check if it is suitable for my use-case, it seems to be pure PubSub protocol. Basically I need to estabilish bi-directional realtime communication channel between browser-based UI and pyboard and I want to be able to remotely call functions on pyboard's side by pressing buttons in the UI and send updates from the pyboard in oposite direction.

Could you please elaborate more what do you mean by "once a network socket implementation is in place"? I was thinking that socket module is already implemented. For example it is mentioned in this Docs page: http://docs.micropython.org/en/latest/l ... twork.html .
Then there is usocket module, which has only two functions defined according to the Docs: http://docs.micropython.org/en/latest/l ... ocket.html .

[kfricke]

You can easily use the Mosquittto MQTT-Broker with it's Websockets support and the Paho Javscript library for this.

I am doing similar things with the packages from their Debian-Repository. The Moqsuitto packages in the official Debian repositories do not have websockets support yet. Downloads for other platforms should have the websocket support builtin since version 1.4

To be a little more precise on your use-case: Use one or multiple MQTT-topics to subscribe to on both sides (from the Javascript in your website and the MicroPython) and voilà... you do have your bi-directional real-time communication channel with the publish and subscribe mechanism.
In case you need guaranteed message delivery you can accomplish this using the QoS settings in a MQTT message.

My favorite MQTT feature is the message persistency on the broker using the "retain"-flag in messages.
There are plenty of real-world projects using MQTT, from sensor networks to home automation and even instant messaging solutions from Facebook and others.

[danielm]

In my current implementation I am using similar approach with WAMP:
- Crossbar.io as broker
- AutobahnPython on the device
- AutobahnJS on UI side

It is based on RPi so there is no problem with running AutobahnPython on it. However I want to move the development to MCU suitable for realtime tasks.

I think it should be easy to do RPC via MQTT's PubSub via message parsing.

Is there any security mechanism implemented in the MQTT which is performing client authentification before subscription to a specific topic? I want to prevent situation in which User B could control device of User A by subscribing to his topic.
Other solution which comes to my mind is to generate topic name long enough it would take long time to guess it by brute-force. I guess that the broker also has some protection implemented, for example in case the client is continuosly trying to subscribe to large number of topics.

Hope I will have pyboard in my hands on monday so I can start to play with MQTT client implementation for Python

[kfricke]

...
Is there any security mechanism implemented in the MQTT which is performing client authentification before subscription to a specific topic? I want to prevent situation in which User B could control device of User A by subscribing to his topic.
Other solution which comes to my mind is to generate topic name long enough it would take long time to guess it by brute-force. I guess that the broker also has some protection implemented, for example in case the client is continuosly trying to subscribe to large number of topics.

Sure. Besides the authentication you can define ACLs for different clients-ids or authenticated users. Security can be enhanced by using TLS for encryption of the network connection and authentication. This is of course not fully feasible on a MicroPython and MCU based device.

Hope I will have pyboard in my hands on monday so I can start to play with MQTT client implementation for Python

To play with MicroPython you can as well use the Unix port. I'd say that implementing such protocols initially on the Unix port would be far easier. You can later use it on a MCU based devices and optimize it further if necessary.

[SpotlightKid]

Security can be enhanced by using TLS for encryption of the network connection and authentication. This is of course not fully feasible on a MicroPython and MCU based device.

The MQTT client implementation of the Lua-based NodeMCU firmware for Esp8266 boards does TLS, AFAIK. I don't think it does client certificates, though.

[danielm]

SpotlightKid, I would like to ask a question regarding socket module status/availability you talked about in your first post in this thread.

Could you please elaborate more what do you mean by "once a network socket implementation is in place"? I was thinking that socket module is already implemented. For example it is mentioned in this Docs page: http://docs.micropython.org/en/latest/l ... twork.html .
Then there is usocket module, which has only two functions defined according to the Docs: http://docs.micropython.org/en/latest/l ... ocket.html .

Couly you please explain what is missing in current socket implementation and if there is any estimated time frame for future socket module development? Thanks.

[pfalcon]

danielm, you should understand that PyBoard doesn't have any ready-to-use networking hardware. So, networking capabilities would depend on additional modules attached. There're 2 modules currently supported, as described in the docs: WIZnet5K (Ethernet, http://docs.micropython.org/en/latest/l ... s-wiznet5k) and CC3K (WiFi, http://docs.micropython.org/en/latest/l ... class-cc3k). Socket functions available fully depend on a particular module, and otherwise are fully supported AFAIK.

MicroPython is still actively develops and its documentation is still work in progress, not everything is yet documented (for example, usocket module above documents top-level functions, but not methods of classes, like socket.socket). Networking support also sought to be expanded and streamlined, to support more networking types, protocols, modules (much of this goes beyond original PyBoard, for other boards). For example, soon it should be possible to support PyBoard networking using SLIP protocol over serial connection (i.e. without any special hardware).

[danielm]

Thank you for explanation, it is clear now.

For the initial prototype I will use pyboard + CC3K. Hopefully later something like CC3200 or ESP8266 but with 2x integrated 12-bit DACs becomes available and supported by MicroPython, including sockets. EMW3165 is rumored to have DAC integrated, however there is no detailed information in the datasheet except block diagram.

Btw is there any simple http client library I could use to perform simple HTTP GET and receive content of a file from the server or should I do it via manual constructing of HTTP GET TCP payload (e.g. "GET /getFile HTTP/1.0\r\n\r\n"). More specifically I will probably need to continuously append received chunks to a file on SD card. The file may have size in order of tens of MBs.