Downloading, Building and Deploying the Firmware

Discussion and questions about boards that can run MicroPython but don't have a dedicated forum.
Target audience: Everyone interested in running MicroPython on other hardware.
User avatar
SureshVakati
Posts: 42
Joined: Fri Feb 24, 2017 3:52 pm

Downloading, Building and Deploying the Firmware

Post by SureshVakati » Fri Feb 24, 2017 4:59 pm

Hello Everyone,
I am new to micropython. I am using STM32F405 in my project. I have deployed micropython into it and wrote some modules. I am including all my modules with in the firmware (Frozen). Now I am trying to add an OTA feature to my project, Can someone help me how can I do it for STM32f4xx? I have cellular modem attached to my processor, so I can download a new firmware from my server using FTP. My question is where should I place that downloaded firmware and how to do update and what format do I have to download (.hex, .dfu or .bin)?

Moderator Note: Please create new topics and don't hijack another topic.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Downloading, Building and Deploying the Firmware

Post by dhylands » Fri Feb 24, 2017 7:52 pm

The stmhal doesn't really have a mechanism in place for doing OTA, other than using the bootloader over one of the bootloader supported methods (USB, serial).

The filesystem also isn't big enough to store the firmware image on, so you'd probably need to use an sdcard and also write a bootloader that can update the firmware.

User avatar
SureshVakati
Posts: 42
Joined: Fri Feb 24, 2017 3:52 pm

Re: Downloading, Building and Deploying the Firmware

Post by SureshVakati » Fri Feb 24, 2017 11:35 pm

Thanks Dave,

I have an SD card connected to my board. My friend has bootloader written in C for STM32F405 which updates his application code Over the Air, Can I use that along with micropython? What is the procedure to do that? How can I change the memory location of Micropython firmware?

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Downloading, Building and Deploying the Firmware

Post by dhylands » Sat Feb 25, 2017 12:16 am

In your board definition file, (stmhal/boards/BOARDNAME/mpconfigboard.mk) the variable LD_FILE is used to specify the linker script file.

You can provide your own linker script file instead. Note that other parts of the code assume that the filesystem is located from the 16K mark through to the 128K mark. So if the bootloader fits in 16K then you should be able to just modify the linker script.

User avatar
SureshVakati
Posts: 42
Joined: Fri Feb 24, 2017 3:52 pm

Re: Downloading, Building and Deploying the Firmware

Post by SureshVakati » Sat Feb 25, 2017 1:49 am

Yeah, he mentioned his bootloader just resides with in 16KB.

Code: Select all

//****************************************************************************
// 
// Boot Loader resides at 0x08000000..0x08003FFF (16 KB)
// Application resides at 0x08020000..0x0803FFFF (128 KB)

He is configuring so many settings related to the board in his bootloader script (6000 lines of C code), if I use his settings, isn't it gonna conflict with micropython?

"So if the bootloader fits in 16K then you should be able to just modify the linker script." I have no idea what to change here :(

Code: Select all

/*
    GNU linker script for STM32F405
*/

/* Specify the memory areas */
MEMORY
{
    FLASH (rx)      : ORIGIN = 0x08000000, LENGTH = 0x100000 /* entire flash, 1 MiB */
    FLASH_ISR (rx)  : ORIGIN = 0x08000000, LENGTH = 0x004000 /* sector 0, 16 KiB */
    FLASH_TEXT (rx) : ORIGIN = 0x08020000, LENGTH = 0x080000 /* sectors 5,6,7,8, 4*128KiB = 512 KiB (could increase it more) */
    CCMRAM (xrw)    : ORIGIN = 0x10000000, LENGTH = 0x010000 /* 64 KiB */
    RAM (xrw)       : ORIGIN = 0x20000000, LENGTH = 0x020000 /* 128 KiB */
}

ENTRY(Reset_Handler)

/* produce a link error if there is not this amount of RAM for these sections */
_minimum_stack_size = 2K;
_minimum_heap_size = 16K;
 
/* Define tho top end of the stack.  The stack is full descending so begins just
   above last byte of RAM.  Note that EABI requires the stack to be 8-byte
   aligned for a call. */
_estack = ORIGIN(RAM) + LENGTH(RAM);

/* RAM extents for the garbage collector */
_ram_end = ORIGIN(RAM) + LENGTH(RAM);
_heap_end = 0x2001c000; /* tunable */

/* define output sections */
SECTIONS
{
    /* The startup code goes first into FLASH */
    .isr_vector :
    {
        . = ALIGN(4);
        KEEP(*(.isr_vector)) /* Startup code */

        /* This first flash block is 16K annd the isr vectors only take up
           about 400 bytes. So we pull in a couple of object files to pad it
           out. */

        . = ALIGN(4);
        */ff.o(.text*)
        */stm32f4xx_hal_sd.o(.text*)

        . = ALIGN(4);
    } >FLASH_ISR
   
    /* The program code and other data goes into FLASH */
    .text :
    {
        . = ALIGN(4);
        *(.text*)          /* .text* sections (code) */
        *(.rodata*)        /* .rodata* sections (constants, strings, etc.) */
    /*  *(.glue_7)   */    /* glue arm to thumb code */
    /*  *(.glue_7t)  */    /* glue thumb to arm code */

        . = ALIGN(4);
        _etext = .;        /* define a global symbol at end of code */
    } >FLASH_TEXT
     
    /*
    .ARM.extab :
    {
        *(.ARM.extab* .gnu.linkonce.armextab.*)
    } >FLASH

    .ARM :
    {
        __exidx_start = .;
        *(.ARM.exidx*)
        __exidx_end = .;
    } >FLASH
    */
        
    /* used by the startup to initialize data */
    _sidata = LOADADDR(.data);
        
    /* This is the initialized data section
    The program executes knowing that the data is in the RAM
    but the loader puts the initial values in the FLASH (inidata).
    It is one task of the startup to copy the initial values from FLASH to RAM. */
    .data :
    {
        . = ALIGN(4);
        _sdata = .;        /* create a global symbol at data start; used by startup code in order to initialise the .data section in RAM */
        _ram_start = .;    /* create a global symbol at ram start for garbage collector */
        *(.data*)          /* .data* sections */

        . = ALIGN(4);
        _edata = .;        /* define a global symbol at data end; used by startup code in order to initialise the .data section in RAM */
    } >RAM AT> FLASH_TEXT
            
    /* Uninitialized data section */
    .bss :
    {
        . = ALIGN(4);
        _sbss = .;         /* define a global symbol at bss start; used by startup code */
        *(.bss*)
        *(COMMON)

        . = ALIGN(4);
        _ebss = .;         /* define a global symbol at bss end; used by startup code and GC */
    } >RAM

    /* this is to define the start of the heap, and make sure we have a minimum size */
    .heap :
    {
        . = ALIGN(4);
	PROVIDE ( end = . );
	PROVIDE ( _end = . );
        _heap_start = .;    /* define a global symbol at heap start */
        . = . + _minimum_heap_size;
    } >RAM

    /* this just checks there is enough RAM for the stack */
    .stack :
    {
        . = ALIGN(4);
        . = . + _minimum_stack_size;
        . = ALIGN(4);
    } >RAM

    /* Remove information from the standard libraries */
    /*
    /DISCARD/ :
    {
        libc.a ( * )
        libm.a ( * )
        libgcc.a ( * )
    }
    */

    .ARM.attributes 0 : { *(.ARM.attributes) }
}
I haven't done all these things before, If I understood you correctly, I can compile his code and place it with in that 16k memory and use his bootloader scripts? All I am trying to do is update micropython firmware through his bootloader script.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Downloading, Building and Deploying the Firmware

Post by dhylands » Sat Feb 25, 2017 4:21 am

The bootloader would be in the first 16K. The next 112K would be the file system, and micropython would need to start at 0x08020000.

You basically need to merge stm32f405.ld and common.ld into your own .ld file, and change:

Code: Select all

    /* The startup code goes first into FLASH */
    .isr_vector :
    {
        . = ALIGN(4);
....
        . = ALIGN(4);
    } >FLASH_ISR
and change FLASH_ISR to FLASH_TEXT. Theoretically, that's all that needs to change.

User avatar
SureshVakati
Posts: 42
Joined: Fri Feb 24, 2017 3:52 pm

Re: Downloading, Building and Deploying the Firmware

Post by SureshVakati » Sat Feb 25, 2017 8:16 pm

I followed what you said above.
1.Combined STM32f405.ld and Common.ld into my.ld, here is the entire file after >Flash_ISR to >Flash_TEXT and changed LD_FILE=my.ld from mpconfigboard.mk file
2. I rebuilt the micropython firmware and loaded into the board, while loading I still see memory start point is 0x8000000

Code: Select all

/*
    GNU linker script for STM32F405
*/

/* Specify the memory areas */
MEMORY
{
    FLASH (rx)      : ORIGIN = 0x08000000, LENGTH = 0x100000 /* entire flash, 1 MiB */
    FLASH_ISR (rx)  : ORIGIN = 0x08000000, LENGTH = 0x004000 /* sector 0, 16 KiB */
    FLASH_TEXT (rx) : ORIGIN = 0x08020000, LENGTH = 0x080000 /* sectors 5,6,7,8, 4*128KiB = 512 KiB (could increase it more) */
    CCMRAM (xrw)    : ORIGIN = 0x10000000, LENGTH = 0x010000 /* 64 KiB */
    RAM (xrw)       : ORIGIN = 0x20000000, LENGTH = 0x020000 /* 128 KiB */
}

ENTRY(Reset_Handler)

/* produce a link error if there is not this amount of RAM for these sections */
_minimum_stack_size = 2K;
_minimum_heap_size = 16K;
 
/* Define tho top end of the stack.  The stack is full descending so begins just
   above last byte of RAM.  Note that EABI requires the stack to be 8-byte
   aligned for a call. */
_estack = ORIGIN(RAM) + LENGTH(RAM);

/* RAM extents for the garbage collector */
_ram_end = ORIGIN(RAM) + LENGTH(RAM);
_heap_end = 0x2001c000; /* tunable */

/* define output sections */
SECTIONS
{
    /* The startup code goes first into FLASH */
    .isr_vector :
    {
        . = ALIGN(4);
        KEEP(*(.isr_vector)) /* Startup code */

        /* This first flash block is 16K annd the isr vectors only take up
           about 400 bytes. So we pull in a couple of object files to pad it
           out. */

        . = ALIGN(4);
        */ff.o(.text*)
        */stm32f4xx_hal_sd.o(.text*)

        . = ALIGN(4);
    } >FLASH_TEXT
   
    /* The program code and other data goes into FLASH */
    .text :
    {
        . = ALIGN(4);
        *(.text*)          /* .text* sections (code) */
        *(.rodata*)        /* .rodata* sections (constants, strings, etc.) */
    /*  *(.glue_7)   */    /* glue arm to thumb code */
    /*  *(.glue_7t)  */    /* glue thumb to arm code */

        . = ALIGN(4);
        _etext = .;        /* define a global symbol at end of code */
    } >FLASH_TEXT
     
    /*
    .ARM.extab :
    {
        *(.ARM.extab* .gnu.linkonce.armextab.*)
    } >FLASH

    .ARM :
    {
        __exidx_start = .;
        *(.ARM.exidx*)
        __exidx_end = .;
    } >FLASH
    */
       
    /* used by the startup to initialize data */
    _sidata = LOADADDR(.data);
       
    /* This is the initialized data section
    The program executes knowing that the data is in the RAM
    but the loader puts the initial values in the FLASH (inidata).
    It is one task of the startup to copy the initial values from FLASH to RAM. */
    .data :
    {
        . = ALIGN(4);
        _sdata = .;        /* create a global symbol at data start; used by startup code in order to initialise the .data section in RAM */
        _ram_start = .;    /* create a global symbol at ram start for garbage collector */
        *(.data*)          /* .data* sections */

        . = ALIGN(4);
        _edata = .;        /* define a global symbol at data end; used by startup code in order to initialise the .data section in RAM */
    } >RAM AT> FLASH_TEXT
           
    /* Uninitialized data section */
    .bss :
    {
        . = ALIGN(4);
        _sbss = .;         /* define a global symbol at bss start; used by startup code */
        *(.bss*)
        *(COMMON)

        . = ALIGN(4);
        _ebss = .;         /* define a global symbol at bss end; used by startup code and GC */
    } >RAM

    /* this is to define the start of the heap, and make sure we have a minimum size */
    .heap :
    {
        . = ALIGN(4);
   PROVIDE ( end = . );
   PROVIDE ( _end = . );
        _heap_start = .;    /* define a global symbol at heap start */
        . = . + _minimum_heap_size;
    } >RAM

    /* this just checks there is enough RAM for the stack */
    .stack :
    {
        . = ALIGN(4);
        . = . + _minimum_stack_size;
        . = ALIGN(4);
    } >RAM

    /* Remove information from the standard libraries */
    /*
    /DISCARD/ :
    {
        libc.a ( * )
        libm.a ( * )
        libgcc.a ( * )
    }
    */

    .ARM.attributes 0 : { *(.ARM.attributes) }
}
bootloader says application is corrupted. When I look at memory ranges, micropython.hex still shows range from 0x800000.

Here is the bootloader code where it checks application is valid or not. Any inputs?

Code: Select all

if (CRC32(0xFFFFFFFF, Size, (void *)APPLICATION_BASE) != 0)
  {
#ifndef SILENT
    puts("Application Corrupted");
#endif
    return(0);
  }
 
  return(1);
}

Code: Select all

// Generic CRC 32 bit implementation for buffer/stream validation
 
DWORD CRC32(DWORD Crc, DWORD Size, BYTE *Buffer)
{
  while(Size--)
  {
#if 0
  int i;
 
    Crc = Crc ^ (DWORD)*Buffer++;
 
    for(i=0; i<8; i++)
      if (Crc & 1)
        Crc = (Crc >> 1) ^ 0xEDB88320;
      else
        Crc = (Crc >> 1);
#else
    static const DWORD CrcTable[] = {
      0x00000000,0x1DB71064,0x3B6E20C8,0x26D930AC,0x76DC4190,0x6B6B51F4,0x4DB26158,0x5005713C,
      0xEDB88320,0xF00F9344,0xD6D6A3E8,0xCB61B38C,0x9B64C2B0,0x86D3D2D4,0xA00AE278,0xBDBDF21C };
 
    Crc = Crc ^ (DWORD)*Buffer++;
 
    Crc = (Crc >> 4) ^ CrcTable[Crc & 0x0F];
 
    Crc = (Crc >> 4) ^ CrcTable[Crc & 0x0F];
#endif
  }
 
  return(Crc);
} 
Attachments
Bootloader =_Rnge.PNG
This is the bootloader range.
Bootloader =_Rnge.PNG (13.49 KiB) Viewed 6985 times
STM32 link.PNG
Micropython still shows the firmware range is from 0x8000000
STM32 link.PNG (8.3 KiB) Viewed 6986 times

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Downloading, Building and Deploying the Firmware

Post by dhylands » Sat Feb 25, 2017 9:49 pm

Here's the steps I did:

Code: Select all

cd stmhal/boards
mkdir CUSTOM
cp PYBV10/* CUSTOM
I then edited mpconfigboard.mk to look like:

Code: Select all

MCU_SERIES = f4
CMSIS_MCU = STM32F405xx
AF_FILE = boards/stm32f405_af.csv
LD_FILE = boards/CUSTOM/custom.ld
and created CUSTOM/custom.ld to be the combination of stm32f405.ld and common.ld with, changing .isr_vector to point to FLASH_TEXT instead of FLASH_ISR.

I then cd'd back into the stmhal directory and did:

Code: Select all

make BOARD=CUSTOM
and when it finished, I ran:

Code: Select all

672 >readelf -l build-CUSTOM/firmware.elf 

Elf file type is EXEC (Executable file)
Entry point 0x804f839
There are 4 program headers, starting at offset 52

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x008000 0x08020000 0x08020000 0x4e5d8 0x4e5d8 R E 0x8000
  LOAD           0x058000 0x20000000 0x0806e5d8 0x00160 0x026a8 RW  0x8000
  LOAD           0x05a6a8 0x200026a8 0x0806e738 0x00000 0x04000 RW  0x8000
  LOAD           0x05e6a8 0x200066a8 0x0806e738 0x00000 0x00800 RW  0x8000

 Section to Segment mapping:
  Segment Sections...
   00     .isr_vector .text 
   01     .data .bss 
   02     .heap 
   03     .stack 
So it seems to be working the way I would expect.

User avatar
SureshVakati
Posts: 42
Joined: Fri Feb 24, 2017 3:52 pm

Re: Downloading, Building and Deploying the Firmware

Post by SureshVakati » Sat Feb 25, 2017 10:14 pm

I was selecting the wrong file from previous build. I deleted the build folder and re created, now it is showing 0x8020000 but bootloader is still showing application is corrupted. I need to work on bootloader. What should I except, when bootloader points to application memory area? Micropython creates file system and run boot.py and main.py?

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Downloading, Building and Deploying the Firmware

Post by dhylands » Sun Feb 26, 2017 1:36 am

Not being familiar with your friends bootloader, I'm not sure how or why it would consider that the application is corrupted.

When MicroPython runs for the first time, it should create the filesystem and bootloader.

Does the bootloader update the VTOR registor to point to the new irq vector table location?

Post Reply