Urequests problem (multiple Post) with google docs/forms

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
MoustachedBird
Posts: 6
Joined: Mon Sep 09, 2019 7:12 am

Urequests problem (multiple Post) with google docs/forms

Post by MoustachedBird » Mon Sep 09, 2019 7:48 am

Sorry guys, I'm a begginer in micropython, I wanted to know if it could be possible to convert this arduino code to micropython. This code is for posting data in google spreadsheets without using IFTTT (it requieres a public google script). I've taken it from a youtube example (https://www.youtube.com/watch?v=Fq1kgt_auns).

I'm not using the MLX90614, i have an analog sensor. I mean, i'm only interested on the communication part of the code. But i don't know if it could be possible to do http request directly with a public script using micropython. As a test i'd like to send a couple of integers...

Github repository:
https://github.com/embhobbb/esp8266/tre ... oogleSheet

Arduino code:

#include <ESP8266WiFi.h>
#include "HTTPSRedirect.h"
#include <Wire.h>
#include <Adafruit_MLX90614.h>

// Fill ssid and password with your network credentials
const char* ssid = "hidden";
const char* password = "qwerty12";

const char* host = "script.google.com";
const int httpsPort = 443;
const char *GScriptId = "AKfycbwhcnmC6cSE_P5MKice-T6GX8pfe4rFFNiKUnY9-vuwE9hwzCA";

// Write to Google Spreadsheet
String url = String("/macros/s/") + GScriptId + "/exec?tag=adc_A0&value=";

String payload = "";

HTTPSRedirect* client = nullptr;
// used to store the values of free stack and heap
// before the HTTPSRedirect object is instantiated
// so that they can be written to Google sheets
// upon instantiation

const int analog_ip = A0;
int inputVal = 0;
Adafruit_MLX90614 mlx = Adafruit_MLX90614();

void setup() {
Serial.begin(115200);
Serial.flush();

Serial.println();
Serial.print("Connecting to wifi: ");
Serial.println(ssid);
Wire.begin(D1,D2);
// flush() is needed to print the above (connecting...) message reliably,
// in case the wireless connection doesn't go through
Serial.flush();

WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());

// Use HTTPSRedirect class to create a new TLS connection
client = new HTTPSRedirect(httpsPort);
client->setPrintResponseBody(true);
client->setContentTypeHeader("application/json");

Serial.print("Connecting to ");
Serial.println(host);

// Try to connect for a maximum of 5 times
bool flag = false;
for (int i=0; i<5; i++){
int retval = client->connect(host, httpsPort);
if (retval == 1) {
flag = true;
break;
}
else
Serial.println("Connection failed. Retrying...");
}

if (!flag){
Serial.print("Could not connect to server: ");
Serial.println(host);
Serial.println("Exiting...");
return;
}

payload = "tag=aaaa&value=122";
client->POST(url, host, payload, false);
client->GET(url, host);
}

void loop() {
static int connect_count = 0;
static bool flag = false;


Serial.print("Ambient = ");
inputVal = mlx.readObjectTempC();
Serial.print(inputVal);
Serial.println("*C");

String myString = String(inputVal);
String FinalStringToSend;
FinalStringToSend = url + myString;


if (client != nullptr){
if (!client->connected()){
client->connect(host, httpsPort);
payload= "";
Serial.println("POST Data to Sheet");
// FinalStringToSend = url + myString;
Serial.println("POST url :" + FinalStringToSend);
client->POST(FinalStringToSend, host, payload);
}
}
else{
Serial.println(" >> Failed to POST data");
}
Serial.println("GET url :"+FinalStringToSend);
client->GET(FinalStringToSend, host);

delay(500);

}


I've already used IFTTT but at this point is not usefull for me, i need a sampling rate of more or equal to 100 Hz and as you know this exceeds the IFTTT's usage limit. I've tried making a RAM buffer, but i got a error saying that the buffer exceded the RAM size :shock: . As i told you, it could be possible to convert this code or do you have any suggestion of what could i do? Thank you :D
Last edited by MoustachedBird on Sat Sep 14, 2019 4:02 am, edited 6 times in total.

Online
User avatar
pythoncoder
Posts: 3543
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Convert arduino code to esp8266 micropython (google sheets) or suggestion

Post by pythoncoder » Mon Sep 09, 2019 7:50 am

Converting Arduino code to MicroPython is not particularly difficult but you do need a reasonable knowledge of C/C++ and Python. The Adafruit way of accessing hardware is somewhat different but there is a driver for the MLX90614 so using that should be easy.
Peter Hinch

MoustachedBird
Posts: 6
Joined: Mon Sep 09, 2019 7:12 am

Re: Convert arduino code to esp8266 micropython (google sheets) or suggestion

Post by MoustachedBird » Mon Sep 09, 2019 7:56 am

Sorry, i forgot to say that i'm not using the MLX90614, i have an analog sensor. I mean, i'm only interested on the communication part of the code. But i don't know if it could be possible to do http request directly with a public script using micropython.

As a test i'd like to send a couple of integers...

User avatar
jimmo
Posts: 553
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Post by jimmo » Mon Sep 09, 2019 9:17 am

Hi,

You might find some of the posts in this thread useful -- viewtopic.php?f=18&t=6492 -- I was able to make some values appear in a spreadsheet using a form submission.

MoustachedBird
Posts: 6
Joined: Mon Sep 09, 2019 7:12 am

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Post by MoustachedBird » Mon Sep 09, 2019 8:39 pm

jimmo wrote:
Mon Sep 09, 2019 9:17 am
Hi,

You might find some of the posts in this thread useful -- viewtopic.php?f=18&t=6492 -- I was able to make some values appear in a spreadsheet using a form submission.
Thank you for your time. Sorry, what do you mean with 'application/x-www-form-urlencoded' :shock:

>>> import urequests
>>> h = {'Content-Type': 'application/x-www-form-urlencoded' }
>>> form_url = 'https://docs.google.com/forms/d/e/<doc id>/formResponse'
>>> form_data = 'entry.714340709=micropython'
>>> r = urequests.post(form_url, data=form_data, headers=h)
>>> r.status_code
200

User avatar
jimmo
Posts: 553
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Post by jimmo » Tue Sep 10, 2019 12:27 am

MoustachedBird wrote:
Mon Sep 09, 2019 8:39 pm
Thank you for your time. Sorry, what do you mean with 'application/x-www-form-urlencoded'
When you post data over HTTP you have a few options how to encode the data. urlencoded and json are two very common options. the example arduino code you posted actually uses urlencoded despite saying json.

MoustachedBird
Posts: 6
Joined: Mon Sep 09, 2019 7:12 am

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Post by MoustachedBird » Tue Sep 10, 2019 4:34 am

jimmo wrote:
Tue Sep 10, 2019 12:27 am
When you post data over HTTP you have a few options how to encode the data. urlencoded and json are two very common options. the example arduino code you posted actually uses urlencoded despite saying json.
Thank you so much, it works!!! But I have a last question, if i wanted to post data to more than one entry what would be neccesary to do? I've tried with this:

url = 'https://docs.google.com/forms/d/<MY_ID>/formResponse'
form_data = {'entry.61639300=1','entry.1495906291=2','entry.211888248=3'}
#user_agent = {'Content-Type': 'application/x-www-form-urlencoded'}
r = urequests.post(url, data=form_data, headers=user_agent)

But i've got this error:
TypeError: object with buffer protocol required

Also i've tried using useragent = {'Content-Type': 'application/json'} but i've got the same error.

Online
User avatar
pythoncoder
Posts: 3543
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Post by pythoncoder » Tue Sep 10, 2019 6:00 am

The syntax

Code: Select all

form_data = {'entry.61639300=1','entry.1495906291=2','entry.211888248=3'}
causes form_data to be an instance of a Python set. It should be a string, as per your earlier example.
Peter Hinch

MoustachedBird
Posts: 6
Joined: Mon Sep 09, 2019 7:12 am

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Post by MoustachedBird » Wed Sep 11, 2019 7:43 am

pythoncoder wrote:
Tue Sep 10, 2019 6:00 am
The syntax

Code: Select all

form_data = {'entry.61639300=1','entry.1495906291=2','entry.211888248=3'}
causes form_data to be an instance of a Python set. It should be a string, as per your earlier example.
Thank you for your response. Sorry, i dont want to bother you , but now i want to do multiple post-requests, so i have edited the code, but i've got this error.

The code:

Code: Select all

    url = 'my_url'
    form_data = 'my.entry=my_data'
    user_agent = {'Content-Type': 'application/x-www-form-urlencoded'}
    while True:
    	response = urequests.post(url, data=form_data, headers=user_agent)
    	print("CONTENT:") 
    	print(response.json)
    	print("WAITING...") 
    	time.sleep_ms(500)
    	print("TRYING TO SEND...")
The serial monitor:
CONTENT:
<bound_method>
WAITING...
TRYING TO SEND...
Traceback (most recent call last):
File "main.py", line 50, in <module>
File "main.py", line 42, in main
File "urequests.py", line 115, in post
File "urequests.py", line 53, in request
OSError: -2
Again, i don't know if this is possible or maybe i have to increase the sleep time; when i increase it to 10 minutes it works, but for me it is too slow because of the sample rate. I don't get why the sleep time would be a problem since Google does receive the firts data and i would like to think that after that the esp8266 is free to send other. Thank you guys.

Online
User avatar
pythoncoder
Posts: 3543
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Convert arduino code to esp8266 micropython (google sheets without IFTTT) or suggestion

Post by pythoncoder » Thu Sep 12, 2019 7:30 am

Is it possible that Google impose some form of throttling?
Peter Hinch

Post Reply