Page 1 of 1

Trying to write MicroPython debugger via sys.settrace

Posted: Fri Mar 20, 2020 2:00 pm
by zamniah
I am attempting to write a debugger for MicroPython that would work on x86 unix port. Idea was to be able follow stack (this is possible and would allow breakpoints) and be able to inspect variables, along with an "eval shell".

The latter - inspecting variables and eval - doesn't work since there is no locals() and frame.f_locals.

According to docs: "MicroPython doesn’t maintain symbolic local environment, it is optimized to an array of slots. Thus, local variables can’t be accessed by a name."

So far I understand that these "slots" are different from __slots__, but can't find documentation on how this is implemented (I guess this is in source only?)

I am wondering if there would be some simple workaround, even if it would likely mean changing MicroPython's source.

Would you think that maybe creating GDB macros similar to "pyframe" and "pystack" that exist for Python would be a better approach?

Re: Trying to write MicroPython debugger via sys.settrace

Posted: Fri Mar 20, 2020 11:44 pm
by jimmo
zamniah wrote:
Fri Mar 20, 2020 2:00 pm
So far I understand that these "slots" are different from __slots__, but can't find documentation on how this is implemented (I guess this is in source only?)
Yes it's conceptually the same idea. The compiler just maps all local variables to indicies and allocates enough stack for that many slots. In the bytecode you these correspons to the load/store "fast" ops.

At runtime the variable names are gone (there is no locals dict).
zamniah wrote:
Fri Mar 20, 2020 2:00 pm
I am wondering if there would be some simple workaround, even if it would likely mean changing MicroPython's source.
I think rather than changing the behavior of the VM it might be possible to provide a reverse lookup of slot index to name (you'll have to find some way to put this information into the bytecode, similar to how the line number offsets are stored).

If you don't already know about it, adding three -v flags to the unix port dumps bytecode, very useful for stuff like this.

./micropython -v -v -v test.py
zamniah wrote:
Fri Mar 20, 2020 2:00 pm
Would you think that maybe creating GDB macros similar to "pyframe" and "pystack" that exist for Python would be a better approach?
This would be really cool (but still would have the same challenges for local vars)

Re: Trying to write MicroPython debugger via sys.settrace

Posted: Sat Mar 21, 2020 11:09 pm
by zamniah
Yes it's conceptually the same idea. The compiler just maps all local variables to indicies and allocates enough stack for that many slots. In the bytecode you these correspons to the load/store "fast" ops.
Is there a way inside micropython to at least get to this array of local variables, albeit nameless? I didn't find an obvious way at least.
./micropython -v -v -v test.py
This is quite nifty feature, thanks.

Re: Trying to write MicroPython debugger via sys.settrace

Posted: Sun Mar 22, 2020 3:18 am
by jimmo
zamniah wrote:
Sat Mar 21, 2020 11:09 pm
Is there a way inside micropython to at least get to this array of local variables, albeit nameless? I didn't find an obvious way at least.
What do you mean by "inside micropython". From Python code... I'm not aware of a way to do so.

Re: Trying to write MicroPython debugger via sys.settrace

Posted: Tue Mar 24, 2020 4:07 pm
by zamniah
So after some digging in the code, the "fastn" in py/vm.c is the array of the local variables.

So far I figured out how to do "repr" on a local variables while inside the VM (on load/store), like:

Code: Select all

(gdb) call (mp_obj_str_t*)mp_builtin_repr(fastn[MP_BC_STORE_FAST_MULTI - (mp_int_t)ip[-1]])
$6 = (mp_obj_str_t *) 0x7fda7f5dd460
(gdb) p *$6
$7 = {
  base = {
    type = 0x5437a0 <mp_type_str>
  }, 
  hash = 26656, 
  len = 2, 
  data = 0x7fda7f5dd3e0 "14"
}
Though I can't figure out how to get called function's name in "mp_call_function_n_kw" for example. I can see that a function name is stored beforehand in STORE_NAME.

Maybe STORE_NAME could be used to also save names of local variables?

I've found there is "dict_locals" in mpstate.h, which is dict and looks like dict of locals suggested by the name (but I see the function's name is put there).

EDIT: another idea of mapping local var names to fastn slots: it seems they appear in the bytecode in order of use. Maybe user the ast module to parse code and match them?