RuntimeError: maximum recursion depth exceeded

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
Lornioiz
Posts: 36
Joined: Wed Aug 03, 2016 11:39 am
Location: Florence, Italy

RuntimeError: maximum recursion depth exceeded

Post by Lornioiz » Wed Jan 18, 2017 4:13 pm

Hello everyone,

I keep having this error while I'm not using a recursive function (meaning that is not using recursion a function that I wrote myself, mybe I'm using some fuction that is recursive and I don't know).

What my program does is as follow: every 150ms a timer calls a function. Everytime this function is called, one file is read and datas are extracted from it (data are subsequently used to light some leds). Every time the timer triggers, a different file from a pool of 99 is opened and processed. Once the 99th file has been read, it start reading backwards (98, 97 and so on).

Everything works great for a few minutes, then when I try to change some parameters (like deinit the timer or change file sequence) by the http server that control the application, the server http crashes. Funny thing is, the reading loop itself goes on forever (I can see that because the leds keep on turning on in the supposed order).

This is the traceback:

Code: Select all

  File "leolamp.py", line 121, in <lambda>
  File "lux_ll.py", line 104, in fairy
  File "lux_ll.py", line 126, in fairy_colora
  File "flashbdev.py", line 14, in readblocks
While the first two files are mine and don't use any kind of recursion (that I know of), I wonder what "flashbdev.py" is doing and why it (seem to) crash.

Anyone can help me?

Tnank you!

User avatar
pythoncoder
Posts: 3362
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: RuntimeError: maximum recursion depth exceeded

Post by pythoncoder » Wed Jan 18, 2017 4:28 pm

You'll find flashbdev.py in the source tree esp826/modules. It would seem to be the low level driver for the flash memory - my guess is that the readblocks method is being called recursively perhaps owing to re-entrant code. When you change parameters is there a situation where a timer callback executes before another has finished accessing a file?

I'm guessing wildly here -debugging this sort of thing without code is, er, challenging ;) But you could test for this by editing flashbdev.py to output debug data (this will involve rebuilding the firmware).
Peter Hinch

Lornioiz
Posts: 36
Joined: Wed Aug 03, 2016 11:39 am
Location: Florence, Italy

Re: RuntimeError: maximum recursion depth exceeded

Post by Lornioiz » Thu Jan 19, 2017 9:39 am

pythoncoder wrote:You'll find flashbdev.py in the source tree esp826/modules. It would seem to be the low level driver for the flash memory - my guess is that the readblocks method is being called recursively perhaps owing to re-entrant code. When you change parameters is there a situation where a timer callback executes before another has finished accessing a file?

I'm guessing wildly here -debugging this sort of thing without code is, er, challenging ;) But you could test for this by editing flashbdev.py to output debug data (this will involve rebuilding the firmware).

Thank you as always Pythoncoder!!
I tested the file access and elaboration and it should take about 60ms, therefore I thought a timer set to 150ms had a good margin. I will try to set a bigger interval and see what happens.

I noticed, however, that I don't get the error if the files are save directly to the root of the filesystem. I moved them to a subdirectory because with 300+ file in the root the filesystem got always corrupted after few hours. This corruption doen't seems to occur with few directory used as container... to bad it crashes!

User avatar
pythoncoder
Posts: 3362
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: RuntimeError: maximum recursion depth exceeded

Post by pythoncoder » Thu Jan 19, 2017 10:17 am

Accessing files in an interrupt handler is a bad idea because file access times are not deterministic. Further, code in interrupt handlers such as timer callbacks should be designed to execute fast. I suggest you read http://docs.micropython.org/en/latest/p ... rules.html.

I would probably avoid using a timer and just run a loop which achieves its timing like this:

Code: Select all

import utime
while True:
	tstart = utime.ticks_ms()
	# do all your stuff
	delta = 150 - utime.ticks_diff(utime.ticks_ms(), tstart)
	if delta > 0:
		utime.sleep_ms(delta)
Note I haven't actually tested this ;)
Peter Hinch

Lornioiz
Posts: 36
Joined: Wed Aug 03, 2016 11:39 am
Location: Florence, Italy

Re: RuntimeError: maximum recursion depth exceeded

Post by Lornioiz » Thu Jan 19, 2017 11:10 am

pythoncoder wrote: I would probably avoid using a timer and just run a loop which achieves its timing like this:

Code: Select all

import utime
while True:
	tstart = utime.ticks_ms()
	# do all your stuff
	delta = 150 - utime.ticks_diff(utime.ticks_ms(), tstart)
	if delta > 0:
		utime.sleep_ms(delta)
Note I haven't actually tested this ;)
Maybe is a stupid question, but since I'm running an http server to input commands, if I nest inside of it a new while loop how can I still have control over it? In other words, I can i get out the nested loop with a command sent from http?
just another thing: do you know why the filesystem gets corrupted so easily if there are a lot of file in the root? last time it worked for 15 minutes...

Thank you again!!

Lornioiz
Posts: 36
Joined: Wed Aug 03, 2016 11:39 am
Location: Florence, Italy

Re: RuntimeError: maximum recursion depth exceeded

Post by Lornioiz » Thu Jan 19, 2017 11:55 am

Lornioiz wrote: just another thing: do you know why the filesystem gets corrupted so easily if there are a lot of file in the root? last time it worked for 15 minutes...
I think i found the reason why the filesystem was getting corrupted so easily.
I have a couple of function that write on the filesystem, and I forgot to deactivate the timer that triggered the file read before calling them.
My guess is that there was some kind of overlapping of write/read.
Since I disabled the file saving functions the file system seems to be stable.
What I said above makes any sense to you?

Thanks

User avatar
Roberthh
Posts: 1367
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: RuntimeError: maximum recursion depth exceeded

Post by Roberthh » Thu Jan 19, 2017 12:25 pm

What I said above makes any sense to you?
It does. During my work on the ftp server I oberved that concurrent system calls, especially on the filesystem, brings trouble. You situation is similar. The firmware does not seem to be reentrant. The is a compiler switch in the FAT module, but as far as I recall, it is set to non-reentrant.

User avatar
pythoncoder
Posts: 3362
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: RuntimeError: maximum recursion depth exceeded

Post by pythoncoder » Thu Jan 19, 2017 1:54 pm

Lornioiz wrote:...What I said above makes any sense to you?...
I agree with the diagnosis.
I'd still be wary of file reads in an ISR. Rather than storing multiple files, have you considered storing a python object such as a dict or list in a single file using pickle or ujson? That way you can read the file once at the start and close it, accessing the various elements as required. This is much faster.
Peter Hinch

Lornioiz
Posts: 36
Joined: Wed Aug 03, 2016 11:39 am
Location: Florence, Italy

Re: RuntimeError: maximum recursion depth exceeded

Post by Lornioiz » Thu Jan 19, 2017 2:52 pm

pythoncoder wrote: I'd still be wary of file reads in an ISR. Rather than storing multiple files, have you considered storing a python object such as a dict or list in a single file using pickle or ujson? That way you can read the file once at the start and close it, accessing the various elements as required. This is much faster.
That was my first option, but unfortunately I should create 3 file of about 33kb each. It is 2400 rgb tuples (XXX,YYY,ZZZ) each file. So I discarded it on memory constraint basis. That's why I tried the road of multiple file to be read every 150ms.
I could split in two each file or decrese the amount of tuple by 50% but still I think it would be too much data for the memory available.

Oh, and I was wrong about the "recursion limit depth exceeded". I hit it no matter where I put the files... I'm trying to catch the exception but I have not been lucky so far (I understand is not the way to debug, but my programming knowledge is anything but vast).

Lornioiz
Posts: 36
Joined: Wed Aug 03, 2016 11:39 am
Location: Florence, Italy

Re: RuntimeError: maximum recursion depth exceeded

Post by Lornioiz » Thu Jan 19, 2017 2:55 pm

Roberthh wrote:
What I said above makes any sense to you?
It does. During my work on the ftp server I oberved that concurrent system calls, especially on the filesystem, brings trouble. You situation is similar. The firmware does not seem to be reentrant. The is a compiler switch in the FAT module, but as far as I recall, it is set to non-reentrant.
Would you please help me to understand? that means that once one operation (like writing a file) is interrupted by, let's say, a read operation, it cannot be resumed and therefore the filesistem gets corrupted?

Thanks!

Post Reply