MemoryError what limit am I hitting?

[stephenmhall]

Hi everyone, I have MicroPython running on a Wemos D1 mini, The project is a 3D printer filament jam monitor. I am switching between ESPlorer and ampy in cmd to code and test run. I have the code running but seem to have have hit a wall where adding one more line causes the following error:-

"MemoryError: memory allocation failed, allocating 268 bytes"

I code quite a bit in Python but am new to the Micro version, I assume it's not a file length problem but a too many variables \ trying to be too clever kind of problem but my coding level is "know what to google" rather than a deep understanding of it's workings.

I posted here in case it is a limitation of the ESP8266 rather than MicroPython in general.

if it is helpful I will dump a copy of my code here, and you can all scoff at me.

[pythoncoder]

The ESP8266 has limited RAM and it is easy to hit that limit. If you post your code people may offer suggestions on making it more RAM efficient.

There is a provisional tutorial here http://hinch.me.uk/html/reference/constrained.html which covers some of the issues - note that this has not yet been reviewed so don't treat it as gospel.

[stephenmhall]

Thanks Coder, aye a little more browsing round the forum revealed the 22k or so limit on the ESP8266. Here is my code. don't judge me too harshly.

is there a code block view we can use posting to the forum?

[code]#!python3
import machine
import time
import socket

class Printer(object):
def __init__(self):
self.state = 0

class Filament(object):
def __init__(self, inputpin):
self.filamentCounter = machine.Pin(inputpin, machine.Pin.IN)
self.filamentCounter.irq(trigger=machine.Pin.IRQ_FALLING | machine.Pin.IRQ_RISING, handler=self.cb)
self.count = 0
self.moving = 0
self.stopped = 0
def cb(self, arg):
print("callback from ", arg)
self.count += 1

class PushButton(object):
def __init__(self, inputpin):
self.buttonpress = machine.Pin(inputpin, machine.Pin.IN)
self.buttonpress.irq(trigger=machine.Pin.IRQ_FALLING, handler=self.cb)
self.count = 0
self.currentTime = time.ticks_ms()
self.pressedTime = 0
def cb(self, arg):
print("Button pressed: ", arg)
self.currentTime = time.ticks_ms()
self.count += 1
if self.currentTime - self.pressedTime > 2000:
if pjob.state == 0:
pjob.state = 1
self.pressedTime = time.ticks_ms()
if pjob.state == 3:
pjob.state = 0
self.pressedTime = time.ticks_ms()
setLEDState(pjob.state)

fc1 = Filament(14) #D5
fc2 = Filament(12) #D6
btn1 = PushButton(13) #D7
pjob = Printer()
pinLEDr = machine.Pin(5, machine.Pin.OUT) #D1
pinLEDg = machine.Pin(4, machine.Pin.OUT) #D2
pinLEDb = machine.Pin(0, machine.Pin.OUT) #D3
pinBeeper = machine.Pin(15, machine.Pin.OUT) #D8
beeper = machine.PWM(pinBeeper, freq=300, duty=512)
beeper.deinit()
pinPause = machine.Pin(2, machine.Pin.OUT) #D4
pinPause.high()
tim = machine.Timer(-1)

def setLEDState(state):
if state == 0:
pinLEDr.high()
pinLEDg.high()
pinLEDb.high()
pinPause.high()
beeper.deinit()
startTimer()
if state == 1:
pinLEDr.low()
pinLEDg.low()
pinLEDb.high()
pinPause.high()
beeper.deinit()
startTimer()
if state == 2:
pinLEDr.low()
pinLEDg.high()
pinLEDb.low()
pinPause.high()
beeper.deinit()
startTimer()
if state == 3:
pinLEDr.high()
pinLEDg.low()
pinLEDb.low()
pinPause.low()
beeper.duty(512)

def checkCounters():
print("fc1 ", fc1.count," fc2 ",fc2.count)
if fc1.count != 0:
fc1.moving += 1
fc1.stopped = 0
fc1.count = 0
else:
fc1.moving = 0
fc1.stopped += 1
if fc2.count != 0:
fc2.moving += 1
fc2.stopped = 0
fc2.count = 0
else:
fc2.moving = 0
fc2.stopped += 1
if pjob.state == 1 and (fc1.moving >= 4 or fc2.moving >= 4):
pjob.state = 2
if pjob.state == 2 and (fc1.stopped >= 4 and fc2.stopped >= 4):
pjob.state = 3
setLEDState(pjob.state)

def do_connect():
import network
sta_if = network.WLAN(network.STA_IF)
if not sta_if.isconnected():
print('connecting to network...')
sta_if.active(True)
sta_if.connect('NCC-1701', '30KeirCircleWesthill')
while not sta_if.isconnected():
pass
print('network config:', sta_if.ifconfig())

do_connect()

def startTimer():
tim.init(period=2500, mode=machine.Timer.ONE_SHOT, callback=lambda t:checkCounters())

startTimer()

html = """<!DOCTYPE html>
<html>
<head> <title>Dual Filament Monitor</title>
<meta http-equiv="refresh" content="5" ></head>
<body> <h1>Dual Filament Monitor</h1>
<table border="2"> <tr><th>System</th><th>Condition</th></tr> %s </table>
</body>
</html>
"""

addr = socket.getaddrinfo('0.0.0.0', 80)[0][-1]
s = socket.socket()
s.bind(addr)
s.listen(1)
print('listening on', addr)

while True:
cl, addr = s.accept()
print('client connected from', addr)
cl_file = cl.makefile('rwb', 0)
statusText = ['Ready', 'Waiting', 'Running', 'Paused']
statusColours = ["#FFFF44","#4444FF","#00FF00","#FF0000"]
while True:
line = cl_file.readline()
if not line or line == b'\r\n':
break
if fc1.moving > 0:
fc1Text = ["MOVING", "#00FF00"]
else:
fc1Text = ["STOPPED", "#FF0000"]
if fc2.moving > 0:
fc2Text = ["MOVING", "#00FF00"]
else:
fc2Text = ["STOPPED", "#FF0000"]
rows = ['<tr><td>filament L</td><td bgcolor=%s>%s</td></tr>' % (fc1Text[1], fc1Text[0]),
'<tr><td>filament R</td><td bgcolor=%s>%s</td></tr>' % (fc2Text[1], fc2Text[0]),
'<tr><td>Job State</td><td bgcolor=%s>%s</td></tr>' % (statusColours[pjob.state], statusText[pjob.state])]
response = html % '\n'.join(rows)
cl.send(response)
cl.close()[/code]

[size=50]There is a code block, but a lot of bbcode it's disabled for users with less than a certain number of posts for anti-spam reasons. I added the code block to your post, it should update once you post a bit more! - platforma[/size]

[Roberthh]

It looks as if your code hits the memory limit during the compile phase. According to my experience & as a rule of thumb, the largest program that can be compiled is 10 lines per free kByte of RAM, so at 20k free RAM it's about 200 lines of code. You can raise the limit a little bit by splitting the code into several modules and doing a 'lazy' compile.
If you have the build environment set up, the you can place your code into the esp8266/modules directory. It will then be compiled at build time using mpy-cross and executed from flash. Then, the code will not consume RAM, and the code size can be much larger, but the test phase is slower. At least, mpy-cross will flag the syntax errors. You can also execute pre-compiled code from RAM, which avoids the on-board compile phase, but the standard build does not support that. I have a build available at https://github.com/robert-hh/Shared-Stuff, but you still have to compile mpy-cross.

[Llwy]

Hi Robert,
I have been using your frozen modules enabled firmware for more than a month and I am very happy with it.
It allows me to make programs without hitting the memory error at compile time.

Do you know if there are plans for your modifications to become part of the official firmware? Would there be any reason for it not to be adopted more universally?
With the memory limitations of the ESP8266, it seems to me that your mod is pretty much compulsory to make the platform usable...

Llwy

[Roberthh]

No Idea. I could make a pull request and see, what happens. But I guess it's not welcome because it increases the code size by 1444 bytes. And in a certain way, it's not strictly needed, since building a new image allows to include compiled python code in flash. The difference are the required tool chains. For building the ESP8266 image, you need the full esp tool chain, Creating mpy-cross can be done with the standard gcc tool set. And just uploading a .mpy file look easier than re-flashing the the whole image.

P.S.: My images are a little bit larger since the include upysh.py and my little on-board editor.

[mcauser]

I noticed something. When I paste large code blocks in REPL paste mode, if I use spaces it throws MemoryErrors, but if I switch spaces to tabs and try again, it works. Tabs use a lot less characters, so it falls under the memory limit.

Tabs vs spaces? PEP 8 recommends spaces.

[pythoncoder]

...Do you know if there are plans for your modifications to become part of the official firmware?...

I'm not sure about running .mpy files but frozen bytecode is supported by the current source. Grab the latest source from GitHub and build the cross compiler. Put the code to be frozen in esptool/modules and build with the tools from http://www.kaltpost.de/~wendlers/micropython/.

[stephenmhall]

Cheers Robert, I have ordered a neopixel led to replace the three led's and refactored the code for that, so far it is now running without a memory error. I will definitely look at getting the build environment setup, but that is way further down the rabbit hole than I am used to going with both micro-controllers and linux. Always good to expand ones knowledge.

Bring on the ESP32 with 512kB of ram instead of 160kB..

Stephen H

[ernitron]

The ESP8266 has limited RAM and it is easy to hit that limit. If you post your code people may offer suggestions on making it more RAM efficient.

There is a provisional tutorial here http://hinch.me.uk/html/reference/constrained.html which covers some of the issues - note that this has not yet been reviewed so don't treat it as gospel.

Wow... the tutorial just lists all of my coding errors!

However the example is pretty simple and I guess the internal compiler got exhausted when trying to allocate the long string variables for html. My solution was to move all html static text on files and reading them on-the-fly to ouptut the response. And of course having everything precompiled (only load the mpy files) it helped a lot.