Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.

Would you like to make good-intended, but backward-incompatible changes to WiFi setup?

Poll ended at Sun Jan 08, 2017 7:55 am

Yes, I think this change is good, even though it's backward-incompatible and will confuse some users. We'll get thru it.
36
100%
No, I don't think this change is sound technically or that making backward-incompatible changes of such scale is good.
0
No votes
 
Total votes: 36

Lysenko
Posts: 62
Joined: Wed Aug 17, 2016 1:21 pm

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by Lysenko » Wed Nov 09, 2016 11:26 am

dhylands wrote:My desktop persists the SSID/password, but it doesn't re-persist it every time it connects, only when the password actually changes.
How do you know? It might be rewriting the settings each time. It just wouldn't make any difference with a HD/SSD.

The issue is surely that Espressif choose to rewrite settings on state change by default and choose to fit persistent storage with a low write endurance. It isn't the job of a high level language to second guess those implementation decisions.

I'm not voting because option one breaks userspace (unacceptable in a patch) and option two leaves people who use a power hungry WiFi SoC as a battery powered sensor with an intractable problem.

If I were going to address this I would implement an optional esp2 or esp_ex module that surfaced the additional API necessary to avoid this.

User avatar
kfricke
Posts: 342
Joined: Mon May 05, 2014 9:13 am
Location: Germany

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by kfricke » Wed Nov 09, 2016 11:31 am

Paul did ask for "Explain why you do this, explain the issues with this approach (one would need to put AP password in plaintext in boot.py!), then explain what you do to address that." on the pull request. So here is my suggestion regarding this:

I do not care about storing plain text AP passwords on my sensor nodes. I simply do so.

Why? ... No one really care saving a WLAN pre-shared-key in plain text. Why should we do? The point is that the wireless network credential is a "shared key" (there is no secret described in the entire iEEE Wifi spec for WPA2 besides the temporary secrets shared across the wireless network devices).
Every Linux distribution I know does so (desktops usually maintain these in '/etc/NetworkManager/system-connections/<files>'). And I do guess that Espressif does this as well in their 16kB segment of flash.

Of course Linux distributions or other operating systems are multi-user environments with adequate security measures against accidental public visibility of the stored credentials. Some places are secured by strong cryptographic measurements (when only single users need secure storage of authenticating identities like SSH RSA-key pairs).
Local physical access always kills this security. So why could embedded devices be more secure regarding this

As a final note: We must not compare this to storing user credentials (e.g. authenticated access to a websocket based REPL).

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by pfalcon » Wed Nov 09, 2016 12:02 pm

deshipu wrote:@pfalcon we already know your opinion, thank you, you expressed it multiple times already in all those comments to pull requests. Can you please let the other forum members express their opinion now, as you suggested, so that we can learn where the community stands on this?
"We" - who? 3 users who have voted "yes" so far? As a ESP8266 Kickstarter project participant, I need to care about 1400 users. As a MicroPython maintainer, I need to care about thousands of users beyond that. This includes users who will never read beyond few initial lines of this thread. This includes users who will never read this thread at all. This includes users who will read this thread, cheerfully vote "yes", forget about it, and that submit "bugreports".

To get you guys motivated to care about thousands of users, not just your agenda, let's do it like this: poll is considered to be resolved when there're at least 1000 votes "yes", and less than 500 votes "no". Get the people voting, guys! (Spambots are disallowed, sorry.)
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

torwag
Posts: 220
Joined: Fri Dec 13, 2013 9:25 am

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by torwag » Wed Nov 09, 2016 12:05 pm

pfalcon wrote:
From a pure Micropython users point of view, I was first surprised to see that the wlan config is kind of persistent.
I said this in pull request, will repeat here again: How your desktop computer works? Do you re-enter AP password each time you power it on? How your phone works? You enter AP password each time you press power button? Then what you're actually surprised about? ESP8266 with MicroPython is a little personal computer, and works in a way comfortable to user, that's it.
That is very much true desktop computer, I agree, and it is true for embedded systems with an OS (lets say the RPi).
However, it is not what people expect from a microcontroller unit (at least if they have a electronic engineering background).
People expect that the uC goes to down to a very defined state whenever you hard reset it. They assume that all and everything from that point will follow the same path, which usually start with some init process. For that reasons, I had copied the wifi init stuff in my boot.py first, since I thought I have to initialise the wifi at boot. The same behaviour scheme you will find in many other microcontroller units even FPGAs and DSP require a dedicated initialisation step (e.g. load the bit-file from a flash memory into the FPGA).
Thus, your comparison might not be complete valid. What happens in the esp, is much more comparable to what happens in a PC during boot. Do you know any PC, which rewrites its bios chip several times during boot-up?

I can see why we have this behaviour. The esp was initially planned as a companion chip with a AT-based interface. People should and could expect that ones sending the credentials to the chip, it will be stored and available. That is, the chip inits itself independent of the state of the host controller and hence it can start to communicate with the wifi network "automagically".
Myself, as a developer, I'd prefer that ESP8266 credentials were not persistent. But as a user, I love that persistence feature very much, it's very practical! A lot of users is used to it, and a lot of those users won't read any of documentation updates, and there will be a blizzard of reports "My esp8266 doesn't remember my AP, help!!!111"
If this is such an heavy brain-breaking change, could we, for some time add a debug message to repl?
For the release before the change will happen:
"NOTE, IN THE FUTURE wifi credential storage is not persistent anymore, refer to http://www.micropython.org/XXXX for more details"
From the release that introduce that change:
"NOTE, wifi credential storage is NOT persistent anymore, refer to http://www.micropython.org/XXXX for more details"

That message could appear whenever the usually behaviour is expected. This is at least a method many command-line tools use to inform users that changes will happen / had been made and gives them time to adapt.

If we go with non-persistent as standard (also do not load from flash), the easiest solution would be to add two commands esp.write_wifi_flash() and esp.read_wifi_flash(), where no arguments for write would take whatever is set at that time, or optional allow to give the wifi credentials as argument (better names might be found).

Thus, the persistent behaviour like before would be then:

Code: Select all

import network
# enable station interface and connect to WiFi access point
nic = network.WLAN(network.STA_IF)
nic.active(True)
nic.connect(esp.read_wifi_flash())
which would nicely fit in boot.py

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by deshipu » Wed Nov 09, 2016 12:14 pm

pfalcon wrote: To get you guys motivated to care about thousands of users, not just your agenda, let's do it like this: poll is considered to be resolved when there're at least 1000 votes "yes", and less than 500 votes "no". Get the people voting, guys! (Spambots are disallowed, sorry.)
Oh, so now it's voting-driven development? I thought you asked for the poll to learn about what the community thinks, not to play games with it.

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by pfalcon » Wed Nov 09, 2016 12:16 pm

torwag wrote:it is not what people expect
I envy your affirmative assurance of knowing what people expect. I for one don't know, that's why I suggested to set up this poll. And I can only join in inviting you for a call in my previous message: you know what people want, just make them express it.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by pfalcon » Wed Nov 09, 2016 12:32 pm

Offtopic messages were removed.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

User avatar
kfricke
Posts: 342
Joined: Mon May 05, 2014 9:13 am
Location: Germany

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by kfricke » Wed Nov 09, 2016 12:35 pm

...respectfully deleted on-topic stuff (at least this is a community discussion)...

Here is my deal: Take your 500 votes and subtract the 90% of people you do scare away with your way of talking to them. So 25 is enough. :lol:

You want us to let the community discuss the topic and do simply play games with us. Please let us (the community) discuss here alone, thanks!

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by deshipu » Wed Nov 09, 2016 12:36 pm

pfalcon wrote:Offtopic messages were removed.
So it has come to that.

User avatar
kfricke
Posts: 342
Joined: Mon May 05, 2014 9:13 am
Location: Germany

Re: Avoid flash wearing by altering behavior originally introduced by Espressif (the inventor of the ESP8266)

Post by kfricke » Wed Nov 09, 2016 12:38 pm

deshipu wrote:
pfalcon wrote:Offtopic messages were removed.
So it has come to that.
I just got personal to him as well, so he was right with that to be honest.

Post Reply