For pico port ntptime not standard?

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: For pico port ntptime not standard?

Post by jimmo » Tue Aug 23, 2022 5:48 am

Jibun no kage wrote:
Tue Aug 23, 2022 3:55 am
But I have not as yet figured out exactly how to remove select drivers right.
Anything that's written in C will be controlled by a flag (mpconfig.h -> mpconfigport.h -> mpconfigboard.h). If you're customising things you should be defining your own board with its own mpconfigboard.h (that can optionally include another board's config).

Some things might be controlled by Makefile options, although this is quite rare. You can override these variables in mpconfigboard.mk (or .cmake for rp2).

Anything that's written in Python will be included via a manifest. You can define your own manifest and not include the default manifest.

Some things are written in both C and Python (i.e. a low-level driver in C, a high-level driver in Python) in which case you need both mpconfigboard and manifest.

You should basically never have to modify a .mk or .cmake file to remove individual files. The files themselves have `#if FOO` around them that turns them into no-ops during compilation if the flag isn't set.

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: For pico port ntptime not standard?

Post by Roberthh » Tue Aug 23, 2022 6:07 am

Onewire is a two step configuration, since it uses a _onewire module, configured in/out per mpconfigxx.h files, and python drivers per manifest.py. For included python files, look also into the port's modules directory, because the usual manifest,py will include all scripts in that directory, for being backward compatible to the habit before manifest.py was established.

if you look into the drivers in extmod, you'll see that all configurable ones have a '#fdef xxxxx' bracket around it's core. So formally it exists and can be compiled, but does not any code. So you do not have to care about adding/removing to the object file list and linking.

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: For pico port ntptime not standard?

Post by Jibun no kage » Tue Aug 23, 2022 2:49 pm

Then I am missing something... because as a test/learning step I tried to exclude DHT support, as well as some other interfaces. And I hit the following...

Code: Select all

[  6%] Linking CXX executable firmware.elf
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/home/crosstoolng/micropython/drivers/dht/dht.c.obj: in function `dht_readinto':
dht.c:(.text.dht_readinto+0x6e): undefined reference to `machine_time_pulse_us'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: dht.c:(.text.dht_readinto+0x7e): undefined reference to `machine_time_pulse_us'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/machine_i2c.c.obj: in function `machine_i2c_transfer_single':
machine_i2c.c:(.text.machine_i2c_transfer_single+0x74): undefined reference to `mp_machine_soft_i2c_transfer'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: machine_i2c.c:(.text.machine_i2c_transfer_single+0xa4): undefined reference to `mp_machine_soft_i2c_type'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/machine_i2c.c.obj:(.rodata.machine_hw_i2c_type+0x38): undefined reference to `mp_machine_i2c_locals_dict'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/machine_i2c.c.obj:(.rodata.machine_i2c_p+0x14): undefined reference to `mp_machine_i2c_transfer_adaptor'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/machine_spi.c.obj:(.rodata.machine_spi_type+0x38): undefined reference to `mp_machine_spi_locals_dict'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/modmachine.c.obj:(.rodata.machine_module_globals_table+0x6c): undefined reference to `machine_time_pulse_us_obj'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/modmachine.c.obj:(.rodata.machine_module_globals_table+0x9c): undefined reference to `mp_machine_soft_i2c_type'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/modmachine.c.obj:(.rodata.machine_module_globals_table+0xb4): undefined reference to `machine_pwm_type'
/usr/lib/gcc/arm-none-eabi/8.3.1/../../../arm-none-eabi/bin/ld: CMakeFiles/firmware.dir/modmachine.c.obj:(.rodata.machine_module_globals_table+0xd4): undefined reference to `mp_machine_soft_spi_type'
collect2: error: ld returned 1 exit status
make[3]: *** [CMakeFiles/firmware.dir/build.make:5488: firmware.elf] Error 1
make[2]: *** [CMakeFiles/Makefile2:1644: CMakeFiles/firmware.dir/all] Error 2
make[1]: *** [Makefile:103: all] Error 2
make: *** [Makefile:27: all] Error 2
So I missed something somewhere or fat fingered something. Still learning. Keep a snapshot of the changes, so really easy to roll back and try again.

I had not thought about creating my own board definition, but as I am learning how all this is implemented, that is actually a good idea.

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: For pico port ntptime not standard?

Post by Jibun no kage » Tue Aug 23, 2022 3:00 pm

Oh, I just happened to notice where you guys are from, location information... Been to Europe many times, more than I can recall, most summers, since I was 4... to Munich, German grandparents. But have traveled all over Europe as I got older, even in East Berlin, the FDR (if that rings a bell?) when it existed. Have not been down under as yet, but it is on the bucket list, as is to see Moscow if I get the chance. :)

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: For pico port ntptime not standard?

Post by Jibun no kage » Tue Aug 23, 2022 5:56 pm

Ok, I did get the drivers and frozen modules excluded from the standard pico build image. For example, neopixel, dht, onewire excluded. Was not sure why ds18x20 disappeared at first, then I recalled it is 1-wire sensor, and since I removed such from mpconfigport.h, it can't exist, so all good so far. However, when I tried to create a custom board definition hit a bump, see below.

//-----
// Custom PICO Image Test...
// Remove Select Drivers And Frozen Modules...

$ cp ~/micropython/ports/rp2/mpconfigport.h mpconfigport.h.original
$ nano ~/micropython/ports/rp2/mpconfigport.h
#define MICROPY_PY_ONEWIRE (0)

$ cp ~/micropython/ports/rp2/boards/manifest.py ~/micropython/ports/rp2/boards/manifest.py.original
$ nano ~/micropython/ports/rp2/boards/manifest.py
#freeze("$(MPY_DIR)/drivers/onewire")
#freeze("$(MPY_DIR)/drivers/dht", "dht.py")
#include("$(MPY_DIR)/drivers/neopixel/manifest.py")

$ cd ~/micropython/ports/rp2
$ make BOARD=PICO

MicroPython v1.19.1-315-g13dceaa4e-dirty on 2022-08-23; Raspberry Pi Pico with RP2040
Type "help()" for more information.
>>> import micropython
>>> dir(micropython)
['__class__', '__name__', 'const', 'alloc_emergency_exception_buf', 'heap_lock', 'heap_unlock', 'kbd_intr', 'mem_info', 'opt_level', 'qstr_info', 'schedule', 'stack_use']
>>> help('modules')
__main__ math ubinascii uos
_boot micropython ucollections urandom
_boot_fat rp2 ucryptolib ure
_rp2 uarray uctypes uselect
_thread uasyncio/__init__ uerrno ustruct
_uasyncio uasyncio/core uhashlib usys
builtins uasyncio/event uheapq utime
cmath uasyncio/funcs uio uzlib
framebuf uasyncio/lock ujson
gc uasyncio/stream umachine
Plus any modules on the filesystem

// Latest Nightly Buiid...
// v1.19.1-315-g13dceaa4e (2022-08-23) .uf2
MicroPython v1.19.1 on 2022-08-23; Raspberry Pi Pico with RP2040
Type "help()" for more information.
>>> import micropython
>>> dir(micropython)
['__class__', '__name__', 'const', 'alloc_emergency_exception_buf', 'heap_lock', 'heap_unlock', 'kbd_intr', 'mem_info', 'opt_level', 'qstr_info', 'schedule', 'stack_use']
>>> help('modules')
__main__ framebuf uasyncio/funcs ujson
_boot gc uasyncio/lock umachine
_boot_fat math uasyncio/stream uos
_onewire micropython ubinascii urandom
_rp2 neopixel ucollections ure
_thread onewire ucryptolib uselect
_uasyncio rp2 uctypes ustruct
builtins uarray uerrno usys
cmath uasyncio/__init__ uhashlib utime
dht uasyncio/core uheapq uzlib
ds18x20 uasyncio/event uio
Plus any modules on the filesystem

//-----
// Custom PICO Board Test...
// Remove Select Drivers And Frozen Modules...
// Add Custom Board Definition...

$ cp -ar ~/micropython/ports/rp2/board/PICO ~/micropython/ports/rp2/board/PICO_DD
$ cd ~/micropython/ports/rp2/board/PICO_DD
$ nano board.json
"id": "rp2-pico-dd",

$ make BOARD=PICO_DD
[ -e build-PICO_DD/Makefile ] || cmake -S . -B build-PICO_DD -DPICO_BUILD_DOCS=0 -DMICROPY_BOARD=PICO_DD
PICO_SDK_PATH is /home/crosstoolng/micropython/lib/pico-sdk
PICO platform is rp2040.
Build type is MinSizeRel
PICO target board is pico_dd.
CMake Error at /home/crosstoolng/micropython/lib/pico-sdk/src/boards/generic_board.cmake:22 (message):
Unable to find definition of board 'pico_dd' (specified by PICO_BOARD):
Looked for pico_dd.h in /home/crosstoolng/micropython/lib/pico-sdk/src/boards/include/boards (additional paths specified by PICO_BOARD_HEADER_DIRS)
Looked for pico_dd.cmake in /home/crosstoolng/micropython/lib/pico-sdk/src/boards (additional paths specified by PICO_BOARD_CMAKE_DIRS)
Call Stack (most recent call first):
/home/crosstoolng/micropython/lib/pico-sdk/src/board_setup.cmake:28 (include)
/home/crosstoolng/micropython/lib/pico-sdk/src/CMakeLists.txt:15 (include)
-- Configuring incomplete, errors occurred!
See also "/home/crosstoolng/micropython/ports/rp2/build-PICO_DD/CMakeFiles/CMakeOutput.log".
See also "/home/crosstoolng/micropython/ports/rp2/build-PICO_DD/CMakeFiles/CMakeError.log".
make: *** [Makefile:26: all] Error 1

I copied the header file for my custom board definition...
cp ~/micropython/lib/pico-sdk/src/boards/include/boards/pico.h ~/micropython/lib/pico-sdk/src/boards/include/boards/pico-dd.h

But still got the error that the cmake file could not find it? The cmake should accept what it receives from what I can tell looking at the cmake file its self. So not sure what is going on here.

I thought it would use the generic by default... "include(boards/generic_board.cmake)"

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: For pico port ntptime not standard?

Post by Jibun no kage » Tue Aug 23, 2022 6:33 pm

And, I have not figure out how to remove the framebuf module? Only reference I can find to it is in extmod and it has no reference to it in any of the manifest files I have seen? So guess the question is, how to remove it? I would have thought when I removed the displays, it would be excluded but apparently not.

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: For pico port ntptime not standard?

Post by Roberthh » Tue Aug 23, 2022 6:43 pm

Do not enable it in mpconfigport.h or disable it deliberately with:

#define MICROPY_PY_FRAMEBUF (0)

in mpconfigport.h. It helps after major changes to the configuration to run

make clean

before starting a new build.

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: For pico port ntptime not standard?

Post by Jibun no kage » Tue Aug 23, 2022 9:42 pm

Yes, unless you clean up, ugly issues.

Odd, MICROPY_PY_FRAMEBUF does not exist in rp2 mpconfigport.h

I do find it in other ports...

/home/crosstoolng/micropython/extmod/modframebuf.c:#if MICROPY_PY_FRAMEBUF
/home/crosstoolng/micropython/extmod/modframebuf.c:#endif // MICROPY_PY_FRAMEBUF
/home/crosstoolng/micropython/examples/natmod/framebuf/framebuf.c:#define MICROPY_PY_FRAMEBUF (1)
/home/crosstoolng/micropython/py/mpconfig.h:#ifndef MICROPY_PY_FRAMEBUF
/home/crosstoolng/micropython/py/mpconfig.h:#define MICROPY_PY_FRAMEBUF (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
/home/crosstoolng/micropython/ports/esp8266/boards/GENERIC_512K/mpconfigboard.h:#define MICROPY_PY_FRAMEBUF (0)
/home/crosstoolng/micropython/ports/mimxrt/mpconfigport.h:#define MICROPY_PY_FRAMEBUF (1)
/home/crosstoolng/micropython/ports/stm32/boards/NUCLEO_F091RC/mpconfigboard.h:#define MICROPY_PY_FRAMEBUF (0)
/home/crosstoolng/micropython/ports/stm32/boards/NUCLEO_G474RE/mpconfigboard.h:#define MICROPY_PY_FRAMEBUF (0)
/home/crosstoolng/micropython/ports/stm32/boards/B_L072Z_LRWAN1/mpconfigboard.h:#define MICROPY_PY_FRAMEBUF (0)
/home/crosstoolng/micropython/ports/stm32/boards/NUCLEO_L073RZ/mpconfigboard.h:#define MICROPY_PY_FRAMEBUF (0)
/home/crosstoolng/micropython/ports/stm32/boards/NUCLEO_WL55/mpconfigboard.h:#define MICROPY_PY_FRAMEBUF (0)
/home/crosstoolng/micropython/ports/unix/variants/standard/mpconfigvariant.h:#define MICROPY_PY_FRAMEBUF (0)
/home/crosstoolng/micropython/ports/webassembly/mpconfigport.h:#define MICROPY_PY_FRAMEBUF (1)

But is it there in the nightly Pico image...

icroPython v1.19.1 on 2022-08-23; Raspberry Pi Pico with RP2040

Type "help()" for more information.
>>> import micropython
>>> help('modules')
__main__ framebuf uasyncio/funcs ujson
_boot gc uasyncio/lock umachine
_boot_fat math uasyncio/stream uos
_onewire micropython ubinascii urandom
_rp2 neopixel ucollections ure
_thread onewire ucryptolib uselect
_uasyncio rp2 uctypes ustruct
builtins uarray uerrno usys
cmath uasyncio/__init__ uhashlib utime
dht uasyncio/core uheapq uzlib
ds18x20 uasyncio/event uio
Plus any modules on the filesystem
>>>

Could it for Pico be sourced from the 'py' tree...

/home/crosstoolng/micropython/py/mpconfig.h:#define MICROPY_PY_FRAMEBUF (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: For pico port ntptime not standard?

Post by Jibun no kage » Tue Aug 23, 2022 10:05 pm

Oh! If I explicitly add it as 0 (disabled) in the mpconfigport.h file in the rp2 tree, or in the board configuration mpconfigboard.h file that will override the anything in the py tree, right?

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: For pico port ntptime not standard?

Post by Jibun no kage » Wed Aug 24, 2022 12:51 am

Ok... Have custom Pico and Pico-W images that load, and work as far as I have tested thus far. These custom images don't have the various drivers and modules I wanted to excluded, so that is success!

This was all learning/practice for creating a really lean ESP01 1MB firmware image. I will create a different thread for that effort for any questions I have, ah, I should have a few. :)

Thanks guys, this has been a lot of fun learning how to do it, if a bit frustrating here and there. But I remember a professor of mine at University saying, "It is not until you get a bit flustered, that you really learn anything. After all, if you figure it out right away, you really already knew how to do it, right?" I already hated that comment when I heard it. LOL

Post Reply