How to add modules like ujson, upip and etc?
How to add modules like ujson, upip and etc?
Hello Everyone,
I am new to micropython world so advance apologies.
So basically I am an Intern in a company and they have ported basic support on Micropython to a board which they are developing right now. But they don't have modules like ujson, upip, urequest.
Can you please point me out to resources that I can use to port the above-mentioned modules.
Thanks in advance, any help will be appreciated.
I am new to micropython world so advance apologies.
So basically I am an Intern in a company and they have ported basic support on Micropython to a board which they are developing right now. But they don't have modules like ujson, upip, urequest.
Can you please point me out to resources that I can use to port the above-mentioned modules.
Thanks in advance, any help will be appreciated.
Re: How to add modules like ujson, upip and etc?
Modules get added to the firmware in different ways depending on whether they're written in C or Python...
ujson is written in C, it's usually part of the main firmware and is enabled with the MICROPY_PY_UJSON macro in mpconfigport.h (or mpconfigboard.h). This enables extmod/modujson.c and registers the module in py/objmodule.c
upip and urequests are written in Python and are usually "frozen" into the firmware at compile time via the manifest. See (for example) esp32/boards/manifest.py and esp32/boards/manifest_release.py If your port is an existing architecture (e.g. stm32, esp32, etc) then you can use the existing board config mechanism to add your own manifest. If you've done a complete port from scratch then you will need to add some stuff to your makefile (or build process) to run make-manifest.py and add the generated frozen_content.c into the compile.
Alternatively for the written-in-Python modules like upip and urequests you can just copy over the Python implementations to the filesystem. upip comes from microython/micropython/tools/upip.py and urequests from micropython/micropython-lib/urequests/urequests.py
Re: How to add modules like ujson, upip and etc?
Thank you for a detailed reply. I enabled MICROPY_PY_UJSON by adding following line to mpconfigport.h:jimmo wrote: ↑Thu Sep 17, 2020 7:28 amModules get added to the firmware in different ways depending on whether they're written in C or Python...
ujson is written in C, it's usually part of the main firmware and is enabled with the MICROPY_PY_UJSON macro in mpconfigport.h (or mpconfigboard.h). This enables extmod/modujson.c and registers the module in py/objmodule.c
upip and urequests are written in Python and are usually "frozen" into the firmware at compile time via the manifest. See (for example) esp32/boards/manifest.py and esp32/boards/manifest_release.py If your port is an existing architecture (e.g. stm32, esp32, etc) then you can use the existing board config mechanism to add your own manifest. If you've done a complete port from scratch then you will need to add some stuff to your makefile (or build process) to run make-manifest.py and add the generated frozen_content.c into the compile.
Alternatively for the written-in-Python modules like upip and urequests you can just copy over the Python implementations to the filesystem. upip comes from microython/micropython/tools/upip.py and urequests from micropython/micropython-lib/urequests/urequests.py
#define MICROPY_PY_UJSON (1)
After compiling ujson wasn't in the list of modules. Do I have to add modujson.c myself? In my extmod folder there are only machine_mem.o, machine_mem.P, machine_pulse.o, machine_pulse.P, utime_mphal.o and utime_mphal.P
Update:
Basically I wasn't doing make clean before compiling again. I was in the build folder (my bad).
So basically, I placed modujson.c to /source/extmod. Now when I make then compilation fails with the following error:
Code: Select all
undefined reference to `mp_module_ujson'
collect2: error: ld returned 1 exit status
make: *** [Makefile:110: micropython.elf] Error 1
Re: How to add modules like ujson, upip and etc?
Yeah, look at how (for example) the stm32 port pulls in extmod. (PY_O in your makefile should include the extmod files, via py.mk)
Re: How to add modules like ujson, upip and etc?
Will check and get back to you. Thank you very much for your support.
Re: How to add modules like ujson, upip and etc?
I have added https://github.com/micropython/micropyt ... modujson.c in my /extmod directory.
I have also updated py.mk file.
But when I make it throws me the following errors:
I have also updated py.mk file.
But when I make it throws me the following errors:
Code: Select all
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
make: Warning: File 'modhelloworld.c' has modification time 1563275 s in the future
GEN build/genhdr/qstr.i.last
GEN build/genhdr/qstr.split
GEN build/genhdr/qstrdefs.collected.h
QSTR not updated
CC ../py/../extmod/modujson.c
../py/../extmod/modujson.c: In function 'mod_ujson_load':
../py/../extmod/modujson.c:187:24: error: too few arguments to function 'mp_obj_new_str'
next = mp_obj_new_str(vstr.buf, vstr.len);
^~~~~~~~~~~~~~
In file included from ../../inc/py/objlist.h:29,
from ../py/../extmod/modujson.c:29:
../../inc/py/obj.h:642:10: note: declared here
mp_obj_t mp_obj_new_str(const char* data, size_t len, bool make_qstr_if_not_already);
^~~~~~~~~~~~~~
../py/../extmod/modujson.c:298:25: error: implicit declaration of function 'MP_ERROR_TEXT'; did you mean 'MP_ROM_INT'? [-Werror=implicit-function-declaration]
mp_raise_ValueError(MP_ERROR_TEXT("syntax error in JSON"));
^~~~~~~~~~~~~
MP_ROM_INT
../py/../extmod/modujson.c:298:25: error: passing argument 1 of 'mp_raise_ValueError' makes pointer from integer without a cast [-Werror=int-conversion]
mp_raise_ValueError(MP_ERROR_TEXT("syntax error in JSON"));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../py/../extmod/modujson.c:32:
../../inc/py/runtime.h:150:47: note: expected 'const char *' but argument is of type 'int'
NORETURN void mp_raise_ValueError(const char *msg);
~~~~~~~~~~~~^~~
cc1: all warnings being treated as errors
make: *** [../py/mkrules.mk:59: build/py/../extmod/modujson.o] Error 1
Re: How to add modules like ujson, upip and etc?
It sounds like maybe your fork is based on an older version of MicroPython (before we added string compression via the MP_ERROR_TEXT macro) but you've grabbed the very latest version of modujson.c ?
Re: How to add modules like ujson, upip and etc?
Can you point me out the commit version of which I should use modujson?
Thank you very much for your support.
Thank you very much for your support.
Re: How to add modules like ujson, upip and etc?
You should already have a modujson.c in your repository (well, unless you have a version which is over 6 years old), that's the one you have to use. It's a bit hard to give advice because we don't know what your source tree looks like - normally modujson.c is in the extmod directory but you say you put it there. Does this mean you're using a custom version where this file was deleted? Is the repository still in version control? What does 'git log -n1' give as output? In any case you have to figure out which version of MicroPython this is based on, then use the corresponding modujson.c.
Re: How to add modules like ujson, upip and etc?
Basically this a custom version fro RISCV based board (hifive1-revb). Yes, files were deleted. The repository is in version control and the directory structure is different when compared to official micropython's.