Trying to write MicroPython debugger via sys.settrace

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
zamniah
Posts: 3
Joined: Tue Mar 17, 2020 2:31 pm

Trying to write MicroPython debugger via sys.settrace

Post by zamniah » Fri Mar 20, 2020 2:00 pm

I am attempting to write a debugger for MicroPython that would work on x86 unix port. Idea was to be able follow stack (this is possible and would allow breakpoints) and be able to inspect variables, along with an "eval shell".

The latter - inspecting variables and eval - doesn't work since there is no locals() and frame.f_locals.

According to docs: "MicroPython doesn’t maintain symbolic local environment, it is optimized to an array of slots. Thus, local variables can’t be accessed by a name."

So far I understand that these "slots" are different from __slots__, but can't find documentation on how this is implemented (I guess this is in source only?)

I am wondering if there would be some simple workaround, even if it would likely mean changing MicroPython's source.

Would you think that maybe creating GDB macros similar to "pyframe" and "pystack" that exist for Python would be a better approach?

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Trying to write MicroPython debugger via sys.settrace

Post by jimmo » Fri Mar 20, 2020 11:44 pm

zamniah wrote:
Fri Mar 20, 2020 2:00 pm
So far I understand that these "slots" are different from __slots__, but can't find documentation on how this is implemented (I guess this is in source only?)
Yes it's conceptually the same idea. The compiler just maps all local variables to indicies and allocates enough stack for that many slots. In the bytecode you these correspons to the load/store "fast" ops.

At runtime the variable names are gone (there is no locals dict).
zamniah wrote:
Fri Mar 20, 2020 2:00 pm
I am wondering if there would be some simple workaround, even if it would likely mean changing MicroPython's source.
I think rather than changing the behavior of the VM it might be possible to provide a reverse lookup of slot index to name (you'll have to find some way to put this information into the bytecode, similar to how the line number offsets are stored).

If you don't already know about it, adding three -v flags to the unix port dumps bytecode, very useful for stuff like this.

./micropython -v -v -v test.py
zamniah wrote:
Fri Mar 20, 2020 2:00 pm
Would you think that maybe creating GDB macros similar to "pyframe" and "pystack" that exist for Python would be a better approach?
This would be really cool (but still would have the same challenges for local vars)

zamniah
Posts: 3
Joined: Tue Mar 17, 2020 2:31 pm

Re: Trying to write MicroPython debugger via sys.settrace

Post by zamniah » Sat Mar 21, 2020 11:09 pm

Yes it's conceptually the same idea. The compiler just maps all local variables to indicies and allocates enough stack for that many slots. In the bytecode you these correspons to the load/store "fast" ops.
Is there a way inside micropython to at least get to this array of local variables, albeit nameless? I didn't find an obvious way at least.
./micropython -v -v -v test.py
This is quite nifty feature, thanks.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Trying to write MicroPython debugger via sys.settrace

Post by jimmo » Sun Mar 22, 2020 3:18 am

zamniah wrote:
Sat Mar 21, 2020 11:09 pm
Is there a way inside micropython to at least get to this array of local variables, albeit nameless? I didn't find an obvious way at least.
What do you mean by "inside micropython". From Python code... I'm not aware of a way to do so.

zamniah
Posts: 3
Joined: Tue Mar 17, 2020 2:31 pm

Re: Trying to write MicroPython debugger via sys.settrace

Post by zamniah » Tue Mar 24, 2020 4:07 pm

So after some digging in the code, the "fastn" in py/vm.c is the array of the local variables.

So far I figured out how to do "repr" on a local variables while inside the VM (on load/store), like:

Code: Select all

(gdb) call (mp_obj_str_t*)mp_builtin_repr(fastn[MP_BC_STORE_FAST_MULTI - (mp_int_t)ip[-1]])
$6 = (mp_obj_str_t *) 0x7fda7f5dd460
(gdb) p *$6
$7 = {
  base = {
    type = 0x5437a0 <mp_type_str>
  }, 
  hash = 26656, 
  len = 2, 
  data = 0x7fda7f5dd3e0 "14"
}
Though I can't figure out how to get called function's name in "mp_call_function_n_kw" for example. I can see that a function name is stored beforehand in STORE_NAME.

Maybe STORE_NAME could be used to also save names of local variables?

I've found there is "dict_locals" in mpstate.h, which is dict and looks like dict of locals suggested by the name (but I see the function's name is put there).

EDIT: another idea of mapping local var names to fastn slots: it seems they appear in the bytecode in order of use. Maybe user the ast module to parse code and match them?

User avatar
aivarannamaa
Posts: 171
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Re: Trying to write MicroPython debugger via sys.settrace

Post by aivarannamaa » Tue Jun 02, 2020 9:36 pm

I'm not good with C, so instead of looking at the code I simply ask -- would it be possible to create a native module which would expose this array of locals to MicroPython code (eg. the code running inside sys.settrace handler)?

It would be really nice if it was possible to build a firmware or Unix executable with both sys.settrace and some kind of locals support, because the former is not very useful without the latter.
Aivar Annamaa
https://thonny.org

Hisham
Posts: 1
Joined: Wed Jun 10, 2020 9:30 am

Re: Trying to write MicroPython debugger via sys.settrace

Post by Hisham » Wed Jun 10, 2020 9:33 am

I can already follow the current stack with my settrace code including lines executed which at least offer me possibility of making breakpoints.

Post Reply