Page 1 of 2

Urequests problem (multiple Post) with google docs/forms

Posted: Mon Sep 09, 2019 7:48 am
by MoustachedBird
Sorry guys, I'm a begginer in micropython, I wanted to know if it could be possible to convert this arduino code to micropython. This code is for posting data in google spreadsheets without using IFTTT (it requieres a public google script). I've taken it from a youtube example (https://www.youtube.com/watch?v=Fq1kgt_auns).

I'm not using the MLX90614, i have an analog sensor. I mean, i'm only interested on the communication part of the code. But i don't know if it could be possible to do http request directly with a public script using micropython. As a test i'd like to send a couple of integers...

Github repository:
https://github.com/embhobbb/esp8266/tre ... oogleSheet

Arduino code:

#include <ESP8266WiFi.h>
#include "HTTPSRedirect.h"
#include <Wire.h>
#include <Adafruit_MLX90614.h>

// Fill ssid and password with your network credentials
const char* ssid = "hidden";
const char* password = "qwerty12";

const char* host = "script.google.com";
const int httpsPort = 443;
const char *GScriptId = "AKfycbwhcnmC6cSE_P5MKice-T6GX8pfe4rFFNiKUnY9-vuwE9hwzCA";

// Write to Google Spreadsheet
String url = String("/macros/s/") + GScriptId + "/exec?tag=adc_A0&value=";

String payload = "";

HTTPSRedirect* client = nullptr;
// used to store the values of free stack and heap
// before the HTTPSRedirect object is instantiated
// so that they can be written to Google sheets
// upon instantiation

const int analog_ip = A0;
int inputVal = 0;
Adafruit_MLX90614 mlx = Adafruit_MLX90614();

void setup() {
Serial.begin(115200);
Serial.flush();

Serial.println();
Serial.print("Connecting to wifi: ");
Serial.println(ssid);
Wire.begin(D1,D2);
// flush() is needed to print the above (connecting...) message reliably,
// in case the wireless connection doesn't go through
Serial.flush();

WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());

// Use HTTPSRedirect class to create a new TLS connection
client = new HTTPSRedirect(httpsPort);
client->setPrintResponseBody(true);
client->setContentTypeHeader("application/json");

Serial.print("Connecting to ");
Serial.println(host);

// Try to connect for a maximum of 5 times
bool flag = false;
for (int i=0; i<5; i++){
int retval = client->connect(host, httpsPort);
if (retval == 1) {
flag = true;
break;
}
else
Serial.println("Connection failed. Retrying...");
}

if (!flag){
Serial.print("Could not connect to server: ");
Serial.println(host);
Serial.println("Exiting...");
return;
}

payload = "tag=aaaa&value=122";
client->POST(url, host, payload, false);
client->GET(url, host);
}

void loop() {
static int connect_count = 0;
static bool flag = false;


Serial.print("Ambient = ");
inputVal = mlx.readObjectTempC();
Serial.print(inputVal);
Serial.println("*C");

String myString = String(inputVal);
String FinalStringToSend;
FinalStringToSend = url + myString;


if (client != nullptr){
if (!client->connected()){
client->connect(host, httpsPort);
payload= "";
Serial.println("POST Data to Sheet");
// FinalStringToSend = url + myString;
Serial.println("POST url :" + FinalStringToSend);
client->POST(FinalStringToSend, host, payload);
}
}
else{
Serial.println(" >> Failed to POST data");
}
Serial.println("GET url :"+FinalStringToSend);
client->GET(FinalStringToSend, host);

delay(500);

}


I've already used IFTTT but at this point is not usefull for me, i need a sampling rate of more or equal to 100 Hz and as you know this exceeds the IFTTT's usage limit. I've tried making a RAM buffer, but i got a error saying that the buffer exceded the RAM size :shock: . As i told you, it could be possible to convert this code or do you have any suggestion of what could i do? Thank you :D

Re: Convert arduino code to esp8266 micropython (google sheets) or suggestion

Posted: Mon Sep 09, 2019 7:50 am
by pythoncoder
Converting Arduino code to MicroPython is not particularly difficult but you do need a reasonable knowledge of C/C++ and Python. The Adafruit way of accessing hardware is somewhat different but there is a driver for the MLX90614 so using that should be easy.

Re: Convert arduino code to esp8266 micropython (google sheets) or suggestion

Posted: Mon Sep 09, 2019 7:56 am
by MoustachedBird
Sorry, i forgot to say that i'm not using the MLX90614, i have an analog sensor. I mean, i'm only interested on the communication part of the code. But i don't know if it could be possible to do http request directly with a public script using micropython.

As a test i'd like to send a couple of integers...

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Posted: Mon Sep 09, 2019 9:17 am
by jimmo
Hi,

You might find some of the posts in this thread useful -- viewtopic.php?f=18&t=6492 -- I was able to make some values appear in a spreadsheet using a form submission.

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Posted: Mon Sep 09, 2019 8:39 pm
by MoustachedBird
jimmo wrote:
Mon Sep 09, 2019 9:17 am
Hi,

You might find some of the posts in this thread useful -- viewtopic.php?f=18&t=6492 -- I was able to make some values appear in a spreadsheet using a form submission.
Thank you for your time. Sorry, what do you mean with 'application/x-www-form-urlencoded' :shock:

>>> import urequests
>>> h = {'Content-Type': 'application/x-www-form-urlencoded' }
>>> form_url = 'https://docs.google.com/forms/d/e/<doc id>/formResponse'
>>> form_data = 'entry.714340709=micropython'
>>> r = urequests.post(form_url, data=form_data, headers=h)
>>> r.status_code
200

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Posted: Tue Sep 10, 2019 12:27 am
by jimmo
MoustachedBird wrote:
Mon Sep 09, 2019 8:39 pm
Thank you for your time. Sorry, what do you mean with 'application/x-www-form-urlencoded'
When you post data over HTTP you have a few options how to encode the data. urlencoded and json are two very common options. the example arduino code you posted actually uses urlencoded despite saying json.

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Posted: Tue Sep 10, 2019 4:34 am
by MoustachedBird
jimmo wrote:
Tue Sep 10, 2019 12:27 am
When you post data over HTTP you have a few options how to encode the data. urlencoded and json are two very common options. the example arduino code you posted actually uses urlencoded despite saying json.
Thank you so much, it works!!! But I have a last question, if i wanted to post data to more than one entry what would be neccesary to do? I've tried with this:

url = 'https://docs.google.com/forms/d/<MY_ID>/formResponse'
form_data = {'entry.61639300=1','entry.1495906291=2','entry.211888248=3'}
#user_agent = {'Content-Type': 'application/x-www-form-urlencoded'}
r = urequests.post(url, data=form_data, headers=user_agent)

But i've got this error:
TypeError: object with buffer protocol required

Also i've tried using useragent = {'Content-Type': 'application/json'} but i've got the same error.

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Posted: Tue Sep 10, 2019 6:00 am
by pythoncoder
The syntax

Code: Select all

form_data = {'entry.61639300=1','entry.1495906291=2','entry.211888248=3'}
causes form_data to be an instance of a Python set. It should be a string, as per your earlier example.

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Posted: Wed Sep 11, 2019 7:43 am
by MoustachedBird
pythoncoder wrote:
Tue Sep 10, 2019 6:00 am
The syntax

Code: Select all

form_data = {'entry.61639300=1','entry.1495906291=2','entry.211888248=3'}
causes form_data to be an instance of a Python set. It should be a string, as per your earlier example.
Thank you for your response. Sorry, i dont want to bother you , but now i want to do multiple post-requests, so i have edited the code, but i've got this error.

The code:

Code: Select all

    url = 'my_url'
    form_data = 'my.entry=my_data'
    user_agent = {'Content-Type': 'application/x-www-form-urlencoded'}
    while True:
    	response = urequests.post(url, data=form_data, headers=user_agent)
    	print("CONTENT:") 
    	print(response.json)
    	print("WAITING...") 
    	time.sleep_ms(500)
    	print("TRYING TO SEND...")
The serial monitor:
CONTENT:
<bound_method>
WAITING...
TRYING TO SEND...
Traceback (most recent call last):
File "main.py", line 50, in <module>
File "main.py", line 42, in main
File "urequests.py", line 115, in post
File "urequests.py", line 53, in request
OSError: -2
Again, i don't know if this is possible or maybe i have to increase the sleep time; when i increase it to 10 minutes it works, but for me it is too slow because of the sample rate. I don't get why the sleep time would be a problem since Google does receive the firts data and i would like to think that after that the esp8266 is free to send other. Thank you guys.

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Posted: Thu Sep 12, 2019 7:30 am
by pythoncoder
Is it possible that Google impose some form of throttling?