Authentication
Authentication
I want to make server website to which multiple client micropython boards would log in. The functionality would be: post, fetch, register but I am concerned about the security. If I go the MySQL path I have to disclose the password in the code and leaves a big security hole. On the other end, I don't think I can find any other library/way that is usually thrown around when it comes to normal python like auth. How would you guys go about creating a secure authentication system like this?
Re: Authentication
Danger of an XY Problem here I think, Bokker. https://en.wikipedia.org/wiki/XY_problem
What are you actually trying to do?
My stream of consciousness went something like...
Why do they think it should be a Webserver, for MP boards to log into? Why not just a server (using sockets)? Why not use MQTT? What are they trying to do anyway?
Why do they think using SQL for the back end data would create a security problem of giving away passwords? What is the scenario for deploying these boards and how would any SQL password be disclosed? What are they trying to do anyway?
etc...
There is something you are actually trying to achieve. What is it?
What are you actually trying to do?
My stream of consciousness went something like...
Why do they think it should be a Webserver, for MP boards to log into? Why not just a server (using sockets)? Why not use MQTT? What are they trying to do anyway?
Why do they think using SQL for the back end data would create a security problem of giving away passwords? What is the scenario for deploying these boards and how would any SQL password be disclosed? What are they trying to do anyway?
etc...
There is something you are actually trying to achieve. What is it?
Re: Authentication
This is supposed to be an end-user product, the end user registers an account, attaches his esp32 to the online account. From this point the ESP32 fetches all the data from the server. As I am thinking about it maybe a basic API that would just deliver the data without accepting any data from the esp32 would work. All the settings are supposed to be done on the site and the ESP32 fetches these settings for each device.
I am not sure how would I go about identifying and attaching the esp32-s to the accounts but probably something as easy as a QR code, custom made for each device containing its HW ID or MAC address would work that would get scanned or enter on the website. With this setup, I would not need to accept any post requests from the ESP32.
Do you have any better idea how to solve this properly?
MQTT wont work because the devices are not on the same network.
I am not sure how would I go about identifying and attaching the esp32-s to the accounts but probably something as easy as a QR code, custom made for each device containing its HW ID or MAC address would work that would get scanned or enter on the website. With this setup, I would not need to accept any post requests from the ESP32.
Do you have any better idea how to solve this properly?
MQTT wont work because the devices are not on the same network.
Re: Authentication
This is not a limitation of MQTT as far as I know. It just needs an IP connection. Perhaps you are thinking of multicast? If you mean the origin network has general TCP/IP blocked to outside machines, and is limited to accessing web resources through a filtering proxy, then you can use MQTT over Websockets over port 80. MQTT isn't very suitable unless values are likely to change in real time.MQTT wont work because the devices are not on the same network
It is extremely hard to know how the ESP32 will identify itself without sending some kind of information, given that the user will be negotiating between their browser and your server in a completely different 'session'. I don't believe the MAC address will be visible to the remote server on a different LAN, and MAC addresses can easily be spoofed anyway.With this setup, I would not need to accept any post requests from the ESP32
Off the top of my head; If the end user has an account, then they ask via the website for a unique number to be generated to attach a new device. They can then switch to the ESP32's Access network type this with their Wifi SSID and password (and proxy information - not sure if support exists for this) in a browser page served by the ESP32 on its dedicated AP network. The ESP32 can then negotiate as a station with your local wifi network, and connect (over SSL, MQTT or whatever) to your server providing this number to prove it's account/device identity and retrieve the settings (perhaps JSON would be a useful format) and report the outcome to the user. For the ESP32 client to verify the server it would need to check some certificate information on the connection, or negotiate some kind of signed challenge proving it was your server. I am not sure what support there is for this.the end user registers an account, attaches his esp32 to the online account
I like RethinkDB as an expressive and comprehensible JSON-native database, but there are plenty if it's just a JSON text record which needs storing and you don't need special indexing or cross-referencing between JSON records.
What is the attack vector that you are concerned about? E.g. if someone was able to plain-text read my device settings by guessing its MAC address, would that be a problem? Are you concerned that people might be able to control the settings or manipulate them through a man-in-the-middle attack?
Re: Authentication
The product will only fetch public data, there is no reason for me to fear about people finding out someone else-s config or changing the config on the fly with a MITM attack.
About rethinkDB, I fired it up a container and it does look good but I just can't find any micropython "driver". Should I just fire plain JSON at the server?
About rethinkDB, I fired it up a container and it does look good but I just can't find any micropython "driver". Should I just fire plain JSON at the server?
Re: Authentication
RethinkDB is probably a red herring, although if you want to have comprehensible queries of cross-referenced data it's pretty nice. So far your case seems not to require it. I am also not suggesting you write code on the ESP32 to talk to a remote database, so a database driver is not necessary.
To be able to service the web-proxy-punching networked requests you just need to choose a back-end database that you like, with tables providing an index between unique IDs and fields containing JSON. Then you have e.g. http://flask.pocoo.org/ or any webserver technology you like exposing the 'API' you have designed to the ESP32 boards. If you can just use MAC addresses as IDs that makes life somewhat easier - same code and configuration on all boards, but it can still have a unique identifier to copy-paste between browser windows.
The webserver's job is providing some kind of UI for authenticated editing of settings, and turning uniquely identified HTTPS get requests from your ESP32 boards into JSON settings files retrieved from a database. Probably something like https://pythonspot.com/login-to-flask-app-with-twitter/ would be useful as it means people can use their pre-existing Google, Facebook, Twitter accounts to authenticate.
Equally there are other minimal frameworks you could start with... https://github.com/channelcat/sanic or https://falconframework.org/ with their respective authentication middleware.
However, it's still pretty hard to give a specific recommendation without a better idea what the 'settings' are, how likely they are to change and how devices should respond when that happens, which depends upon what the devices are. The workflow is important, since if you can make simplifications (e.g. require a manual restart of the ESP32 board to reload settings) then that reduces the complexity of the client behaviours and server technologies.
To be able to service the web-proxy-punching networked requests you just need to choose a back-end database that you like, with tables providing an index between unique IDs and fields containing JSON. Then you have e.g. http://flask.pocoo.org/ or any webserver technology you like exposing the 'API' you have designed to the ESP32 boards. If you can just use MAC addresses as IDs that makes life somewhat easier - same code and configuration on all boards, but it can still have a unique identifier to copy-paste between browser windows.
The webserver's job is providing some kind of UI for authenticated editing of settings, and turning uniquely identified HTTPS get requests from your ESP32 boards into JSON settings files retrieved from a database. Probably something like https://pythonspot.com/login-to-flask-app-with-twitter/ would be useful as it means people can use their pre-existing Google, Facebook, Twitter accounts to authenticate.
Equally there are other minimal frameworks you could start with... https://github.com/channelcat/sanic or https://falconframework.org/ with their respective authentication middleware.
However, it's still pretty hard to give a specific recommendation without a better idea what the 'settings' are, how likely they are to change and how devices should respond when that happens, which depends upon what the devices are. The workflow is important, since if you can make simplifications (e.g. require a manual restart of the ESP32 board to reload settings) then that reduces the complexity of the client behaviours and server technologies.
Re: Authentication
To be able to update the settings on the fly would be the best. The settings that would be transferred are really simple. Take it as a weather station device that will have changeable cities with weather reports. When it comes to this info I am not scared that someone will steal it.
I was thinking of making a JSON API site that will respond to URLs that contain the right mac addresses.
for example, the URL with 57-D6-CA-21-43-64 with base64 would be example.com/NTctRDYtQ0EtMjEtNDMtNjQ= I could have the app calculate this out from its own mac address and connect to the API and receive its own settings only. I would need to add the base64 encoded mac addresses manually probably to the API. And after the user registers an account the URL gets tied to the account. This way I could use one code across multiple devices. The API would have the mac addresses before the devices would even ship out.
The problem is that I want to add support for wifi settings. As this data is sensitive maybe I could add an extra layer of protection in the form of offline communication. Bluetooth instead of Wifi. The esp32 have a Bluetooth module. In theory, it would be better for the first time wifi setup/network changing to be done over Bluetooth. Bluetooth could be enabled and disabled with a button.
All this could be tied together with an APP that would backtrack to a website. Basic setup needs to be done with the app(bluetooth,wifi setup,account creation) and later it can be done from anywhere with just the website.
I was thinking of making a JSON API site that will respond to URLs that contain the right mac addresses.
for example, the URL with 57-D6-CA-21-43-64 with base64 would be example.com/NTctRDYtQ0EtMjEtNDMtNjQ= I could have the app calculate this out from its own mac address and connect to the API and receive its own settings only. I would need to add the base64 encoded mac addresses manually probably to the API. And after the user registers an account the URL gets tied to the account. This way I could use one code across multiple devices. The API would have the mac addresses before the devices would even ship out.
The problem is that I want to add support for wifi settings. As this data is sensitive maybe I could add an extra layer of protection in the form of offline communication. Bluetooth instead of Wifi. The esp32 have a Bluetooth module. In theory, it would be better for the first time wifi setup/network changing to be done over Bluetooth. Bluetooth could be enabled and disabled with a button.
All this could be tied together with an APP that would backtrack to a website. Basic setup needs to be done with the app(bluetooth,wifi setup,account creation) and later it can be done from anywhere with just the website.
Re: Authentication
Why is this information stored 'in the cloud' again? Why not store it on the device, and provide a web-based interface to manipulate it directly on the device?
Re: Authentication
I looked into that option too. I tried with a localhost webserver that would run on the device on one of the cores. But it just seemed to be too unstable. It needed multiple reboots and it had really poor performance. 5-10s waiting times on page loadings etc. I tried picoweb and MicroWebSrv. Beyond this I don't know how else would I store the data locally and have it easily editable. Maybe if i would be using just Bluetooth then I could ditch the cloud and have a app that would communicate with the device. I am not even sure if micropythons Bluetooth libs are up to date.
Re: Authentication
This isn't necessarily a fix, but a different way of handling your locally-served web UI which you could try...
https://github.com/ShrimpingIt/cockle/b ... er/boot.py
You can see some orienting information at...
https://github.com/micropython/micropyt ... -264750573
Basic idea, just serve the same GZipped, inlined Web UI from a single cached bundle at all times. Handle everything else as a GET request (to set values), or with client-side SPA logic (if you want to have more than one 'page' to your UI). Comment: my example boot.py script should be optimised to avoid the memory allocation on line 53, which I have wondered might be slowing down the serving of this page in my implementation.
https://github.com/ShrimpingIt/cockle/b ... er/boot.py
You can see some orienting information at...
https://github.com/micropython/micropyt ... -264750573
Basic idea, just serve the same GZipped, inlined Web UI from a single cached bundle at all times. Handle everything else as a GET request (to set values), or with client-side SPA logic (if you want to have more than one 'page' to your UI). Comment: my example boot.py script should be optimised to avoid the memory allocation on line 53, which I have wondered might be slowing down the serving of this page in my implementation.