Slow qstr lookup in mp_obj_str_make_new

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
matejcik
Posts: 1
Joined: Thu Jun 03, 2021 1:24 pm

Slow qstr lookup in mp_obj_str_make_new

Post by matejcik » Thu Jun 03, 2021 1:40 pm

Our codebase is rather big and has ~10000 qstrs. This means that, since qstr_find_strn does a linear search, it takes a long time. And if the code is creating a lot of new strings via mp_obj_str_make_new, then qstr_find_strn could run a lot. In fact, at some point the hardware was spending ~33% of run-time just looking up qstrs.

I'm wondering, is it necessary to lookup qstrs in mp_obj_str_make_new at all? Presumably in some scenarios it could save memory by reusing the same interned string. But at least in our usecase, user-received strings are basically never interned (and ideally shouldn't be interned ever).

Maybe this is relevant for the compiler or for eval?

Could we remove the lookup completely? If not, would it make sense to make it configurable? I'll submit a PR if you think it is a good idea.

Or am I overlooking something?

Post Reply