i try to implement an oauth2 workflow but i'm currently stuck quite at the beginning.
I load a website via https to receive a form where i can put user and password in when i load the website i run out of memory.
Sending the same request via postman returns a reponse with ~20KBytes (19kb body, 1 1kb header).
I hoped with ~100kb RAM i would be able to load the page but i am not getting past 3kb.
Do you have some suggestions/ideas for me?
I actually don't need the full page - i just need to extract from infos from hidden forms, but i have no idea to extract them on the fly. Therefore i wanted to download the full page and extract with some xml parser.
Code: Select all
<input type="hidden" name="SMAUTHREASON" value="">
<input type="hidden" name="target" value="">
So far i used several request.py libs, currently i am using: https://github.com/lucien2k/wipy-urllib ... equests.py
beause it reads the data in 1KB chunks in a loop. before i was using urequest.py from https://github.com/pfalcon/pycopy-lib/t ... est/urllib
Snippet from the loop after sending the request of the urequest.py
Code: Select all
s.write(request) # this sends the request
received_bytes = 0
while 1:
recv = s.read(1024)
received_bytes += 1024
if len(recv) == 0: break
self.text += recv.decode()
_LOGGER.debug('received bytes %s' % received_bytes)
memorystate.memStatus()
Code: Select all
DEBUG:urequests2:received bytes 1024
DEBUG:memorystate:free mem 43904 bytes
DEBUG:memorystate:used mem 75184 bytes
DEBUG:memorystate:gc.collect -> free mem 61280 bytes
DEBUG:memorystate:used mem 57824 bytes
DEBUG:urequests2:received bytes 2048
DEBUG:memorystate:free mem 57776 bytes
DEBUG:memorystate:used mem 61312 bytes
DEBUG:memorystate:gc.collect -> free mem 59216 bytes
DEBUG:memorystate:used mem 59888 bytes
DEBUG:urequests2:received bytes 3072
DEBUG:memorystate:free mem 54688 bytes
DEBUG:memorystate:used mem 64400 bytes
DEBUG:memorystate:gc.collect -> free mem 58192 bytes
DEBUG:memorystate:used mem 60912 bytes
Traceback (most recent call last):
File "main.py", line 23, in <module>
File "oauth.py", line 225, in request_initial_token
File "urequests2.py", line 156, in urlopen
File "urequests2.py", line 74, in __init__
MemoryError: memory allocation failed, allocating 4097 bytes