Typesheds

[stijn]

Can you give a practical example i.e. real-life code you'd use, and the information returned? Just trying to understand why this should be part of MicroPython.

[aivarannamaa]

Can you give a practical example i.e. real-life code you'd use, and the information returned? Just trying to understand why this should be part of MicroPython.

That's how I see it used by Thonny IDE:

1. The user selects a device
2. Thonny enters the REPL and queries sys.stubs_info and receives something like ["micropython-built-ins==1.14", "micropython-some-lib==3.2.1"] (the first one representing the base modules and the second one a frozen extra module)
3. Thonny (in the background) creates a new private virtual environment on the local disk and does pip install micropython-built-ins==1.14 micropython-some-lib==3.2.1 in it. (Not required, if it already has such environment)
4. Thonny configures jedi to offer code completions from this environment.

All this will be invisible to the user.

When the user has installed some extra libraries to device's flash, then she will create a requirements.txt and Thonny combines this information with sys.stubs_info for preparing the code completion environment.

[stijn]

Ok I see what you mean now. The MicroPython version itself (and any modules in it) is already available from the REPL, the stub would have that same version. So then you just need versions of dependencies: have you considered using only that requirements.txt (maybe with a different name) for the rest? Possibly also having it list stubs by their actual names (i.e. not pip install micropython-some-lib==1.14 but pip install micropython-some-lib-stub==1.14, the name of the actual lib shoud be reserved for the lib since it can also by on PyPi). That could work without any changes needed to the core.

[aivarannamaa]

The MicroPython version itself (and any modules in it) is already available from the REPL, the stub would have that same version.

If I consider only certain builds of MicroPython (eg. official builds for PyBoard and ESP-s as published at micropython.org), then the current info does suffice, but there are other ports, added frozen modules etc.

I know that most people do use those official builds, and it's not a big deal if some methods are missing or extra on some boards, but it seems like the problem actually has a quite nice complete solution.

Most importantly, if the person making a port or compiling in some frozen modules needs to fill in a parameter about API stubs, then it promotes the idea, that the users expect typing information and documentation in a standard format and accessible from a standard location (even if null is allowed for this parameter).

I know that documentation is considered a burden by many, but you can generate rst files from pyi files and CircuitPython people are also generating pyi files from C files, so with a better approach it doesn't need more work than it currently does.

So far I've mentioned only code completion, but I guess many people would like to use MyPy in their MicroPython projects, and type information is required for this.

[stijn]

but there are other ports, added frozen modules etc.

Sure, but there are also user C modules, custom builds with other modules builtin, etc. You basically want version info on every module, on the import level. Limiting to frozen modules is going to get problematic soon (I think, not sure). I.e. if there's "import foo" you want to know "needs foo-stub-1.35". So just adding version information for frozen modules isn't going to cut it, it really needs to be possible for any type of module. So I'm suggesting starting with a separate file because that works for everything, it can work right now already, it doesn't require changes which might not ever be accepted because the increase code size, it works for any MicroPython version, also stripped down, etc. Sure if MicroPython would decide that something like CPython's __version__ can be added you could implement that later on (note that __version__ isn't even supported for all modules, i.e. CPython also refrained from baking version info into modules, instead pip/conda/.... all rely on external files for listing dependencies).

Anyway: for the rest I really like the idea of having a typeshed system. What would be even nicer, is having that fairly universal i.e. not just for Thonny. Have you considered that?
Another idea: requiring typesheds to be installable by pip might be somewhat annoying when developing/testing them? A way to say 'use /path/to/typeshed.py' as stub for this module could be handy.

[aivarannamaa]

So I'm suggesting starting with a separate file because that works for everything, it can work right now already, it doesn't require changes which might not ever be accepted because the increase code size, it works for any MicroPython version, also stripped down, etc.

You're right -- as the libraries on the flash need a separate specification anyway, I'll start with this.

Anyway: for the rest I really like the idea of having a typeshed system. What would be even nicer, is having that fairly universal i.e. not just for Thonny. Have you considered that?

I think the data can and should be made universal by packaging the stubs and uploading to PyPI and keeping it fresh. I don't consider myself the one to do this, though. I can be the thorn in someone's side, if required I hope that taking care of stubs will become the part of releasing a new MicroPython version or creating a variation of it.

If the data is readily available, then feeding it to a code completer is rather easy but it's not easy to find a common solution that fits everybody. For example, there is micropy-cli, which seems to be useful for VS Code users, but Thonny can't benefit much from it.

Another idea: requiring typesheds to be installable by pip might be somewhat annoying when developing/testing them? A way to say 'use /path/to/typeshed.py' as stub for this module could be handy.

I think when the user provides requirement.txt then the IDE can manage the environment automatically, so at the moment I don't see problems with it. But of course, annoying details may surface when I'm actually going to implement it...