Smarthome firmware pysmartnode

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
kevinkk525
Posts: 156
Joined: Sat Feb 03, 2018 7:02 pm

Smarthome firmware pysmartnode

Post by kevinkk525 » Tue Jul 10, 2018 3:38 pm

I'd like to share my recently published "firmware" for easy smarthome deployment called pysmartnode, https://github.com/kevinkk525/pysmartnode. It's similar to ESPeasy in its usage target but a lot better as you can actually very easily create your own components in micropython and not C++. I have been working on this for over half a year now and hope that someone will find it useful.

The pysmartnode framework completely takes care of being connected to the mqtt broker, publishing basic device information, last will, sending logs over mqtt, optionally receive a configuration over mqtt and much more. It's configurable and lets you concentrate on programming your own scripts/modules/components.

On esp32 with psram you can just copy the files using ftp. Without psram you should at least generate bytecode first.
It's tested only on esp32 loboris fork but I don't see why it shouldn't run on the official esp32 firmware too.

Sadly I packed too many features into the core files with the last updates to version 3.8.1 and now it is only usable on esp8266 if you disable the filesystem. Have to fix that soon, sorry for that. (You disable the filesystem in the _boot.py)


With a configuration as simple as this in a file called "components.py" you load a component that publishes the free ram every 60s:

Code: Select all

COMPONENTS = {
    "_order": ["ram"],
    "ram": {
        "package": ".machine.ram",
        "component": "ram",
        "constructor_args": {
            "interval": 60
        }
    }
}
There are more standard components like a buzzer (with configurable sounds), led blinking, bell listener, temperature sensors, easyGPIO (to control all gpios by mqtt), a pms5003 air quality sensor and a few more which are all as easily configured as the example.
You can add your own sensors using a simple template and get it running in "no time".

I'd be glad if you try it and give me some feedback. Even if the feedback is like "this is crap, because ...". That would be helpful too.

kevinkk525
Posts: 156
Joined: Sat Feb 03, 2018 7:02 pm

Re: Smarthome firmware pysmartnode

Post by kevinkk525 » Wed Jul 18, 2018 11:52 am

Over one week without even a comment.. Guess the micropython world has not much need for something like this.. or is there some simple framework/firmware in micropython that I don't know of, which everyone uses? Or do you all write your own framework?

User avatar
pythoncoder
Posts: 2978
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Is it resilient?

Post by pythoncoder » Thu Jul 19, 2018 9:02 am

Have you tested the resilience of MQTT under conditions of poor WiFi communications? I found getting reliable operation quite difficult. To add to this you can never prove you've succeeded as there are so many ways a wireless link can be disrupted.

Some interesting tests are:
  • Run it for long periods (several days) logging long outages.
  • Run it while you slowly move the unit out of range of the AP, then gradually move it back into range.
  • Put the ESP and its power source into a Faraday cage then remove it (I used a microwave for this as you can see LED's inside).
I never achieved good resilience on ESP32, but the vendor OS has doubtless improved since I last looked at this. I plan to revisit my MQTT library once I get my hands on a sample of the new Pyboard.
Peter Hinch

kevinkk525
Posts: 156
Joined: Sat Feb 03, 2018 7:02 pm

Re: Smarthome firmware pysmartnode

Post by kevinkk525 » Thu Jul 19, 2018 10:41 am

Well, considering that I'm using your mqtt_as module, I'm quite confident that it is resilient.

It's a framework built on top of your mqtt_as module that simplifies communication, configuration and component addition.

SpotlightKid
Posts: 377
Joined: Wed Apr 08, 2015 5:19 am

Re: Smarthome firmware pysmartnode

Post by SpotlightKid » Thu Jul 19, 2018 11:26 am

I'm definitely interested in this project, I just didn't have time to check it out thoroughly yet. But I starred the Github project and told a friend about it, who's currently using ESPEasy.

kevinkk525
Posts: 156
Joined: Sat Feb 03, 2018 7:02 pm

Re: Smarthome firmware pysmartnode

Post by kevinkk525 » Thu Jul 19, 2018 11:30 am

Thank you for your feedback. It's nice to hear that there is some interest.

User avatar
pythoncoder
Posts: 2978
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Smarthome firmware pysmartnode

Post by pythoncoder » Fri Jul 20, 2018 4:53 am

Oops - sorry, I should have read the docs properly :oops:

I'm interested in this too, when I manage to find some time to try it.
Peter Hinch

kevinkk525
Posts: 156
Joined: Sat Feb 03, 2018 7:02 pm

Re: Smarthome firmware pysmartnode

Post by kevinkk525 » Fri Jul 20, 2018 8:29 am

That's ok :D It is listed under dependencies but it's shipped within the repository. (I used my "experimental" fork though, which is of course the same for esp8266 and esp32, just has some small modifications for esp32_LoBo).

The only thing that bothers me is that I'm having a hard time reducing the RAM demands to get it to fit onto and esp8266 with an enabled filesystem again. Although I have multiple options to reduce RAM, it still does not fit. But if you disable the filesystem I have still 5kB of RAM left while running these components: I2C, HTU21D, Buzzer, RAM publishing, Watchdog, LED notification. So there is still enough RAM for 3-5 additional components.

I tried importing modules that just execute one function and unloading them afterwards but it never gives me all the RAM back, only ~100-200B although the import took 1kB.

User avatar
pythoncoder
Posts: 2978
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Smarthome firmware pysmartnode

Post by pythoncoder » Mon Jul 23, 2018 3:18 am

Thinking about this some more, if you claim resilience on ESP32 I do think you should test it as I suggested above. This is for the following reasons:
  • In my docs for the underlying resilient MQTT library I stress that I did not achieve resilience on ESP32.
  • Your fork modifies my library, notably removing the esp32_pause() function. This may affect resilience.
  • There is an outstanding PR indicating that ESP32 firmware still has problems.
  • This issue still seems unresolved.
The firmware and the underlying vendor code may have improved since I tested, but this is unproven. There are still reasons to be dubious about the maturity of the ESP32 platform.
Peter Hinch

kevinkk525
Posts: 156
Joined: Sat Feb 03, 2018 7:02 pm

Re: Smarthome firmware pysmartnode

Post by kevinkk525 » Wed Jul 25, 2018 2:05 pm

You are right about that, it is at most as resilient as your library and the vendor code allows it to be.
I did not test if the changes in the vendor-code for the esp32 improved the resilience.

The changes I made are for loboris fork only and work very well so far but I did not do all resilience checks with this.
I never tested the official esp32 firmware as it was basically unusable at the time I started using loboris fork. So I did not try it after it got a lot of updates and stick to loboris port which has a lot more features.

Maybe I should put a warning into my docs about unknown resilience on esp32 as stated in the docs of mqtt_as and some open issues with vendor code.

Post Reply