graphing the memory map

[Neon22]

The .map file shows how much memory is used in the RAM and ROM.
It is generated when a build is compiled.
This is useful to see how much memory space a build uses and especially useful to see how much space is left over for a specific board implementation. (Perhaps not so useful for Unix.)

I am writing a parser and viewer to visualize these memory issues.
I have several ".map" files but I would like more from diff people and boards to see the variations and make my parser more robust.
Please zip up your ".map" files and send them to me if you want to help me make this work.

Here is my initial sample output for a single sample of a Teensy, stmhal, and unix build (64bit).
Graphing coming soon. Also ability to see all functions in map.

FYI:
.text is where program instructions are stored (FLASH)
.data is where initialised static variables are stored (SRAM)
.bss is where uninitialised data is stored

Code: Select all

For: teensy_micropython_dh_01.map
Found 6 sections. 7801 lines read
 ['Memory Configuration', 'Discarded input sections', 'Linker script and memory map', 'Allocating common symbols', 'OUTPUT', 'Preamble']
Reading:
Parsing:
 Parsing Blocks
 Parsing Common Symbols
 Parsing Linker and Mem map
Memory stats
.text               start=0x00000000 length=0x2e824  (0k,186kB)
.glue_7             start=0x0002e824 length=0x0      (186k,0B)
.glue_7t            start=0x0002e824 length=0x0      (186k,0B)
.vfp11_veneer       start=0x0002e824 length=0x0      (186k,0B)
.v4_bx              start=0x0002e824 length=0x0      (186k,0B)
.iplt               start=0x0002e824 length=0x0      (186k,0B)
.rel                start=0x0002e824 length=0x0      (186k,0B)
.ARM                start=0x0002e824 length=0x0      (186k,0B)
.usbdescriptortable start=0x1fff8000 length=0xa0     (524256k,160B)
.dmabuffers         start=0x1fff80a0 length=0x0      (524256k,0B)
.usbbuffers         start=0x1fff80a0 length=0x360    (524256k,864B)
.data               start=0x1fff8400 length=0x4dc    (524257k,1kB)
.igot               start=0x1fff88dc length=0x0      (524258k,0B)
.noinit             start=None       length=None     (0k,0)
.bss                start=0x1fff88dc length=0x38c    (524258k,908B)
.heap               start=0x1fff8c68 length=0x4000   (524259k,16kB)
.stack              start=0x1fffcc68 length=0x800    (524275k,2kB)

Code: Select all

For: unix_micropython_dh_01.map
Found 7 sections. 8393 lines read
 ['Memory Configuration', 'Discarded input sections', 'Cross Reference Table', 'Linker script and memory map', 'Allocating common symbols', 'OUTPUT', 'Preamble']
Reading:
Parsing:
 Parsing Blocks
 Parsing Common Symbols
 Parsing Linker and Mem map
 Parsing Cross refs
Memory stats
.interp           start=0x0000000000400238 length=0x1c     (4096k,28B)
.note             start=0x0000000000400254 length=0x20     (4096k,32B)
.note             start=0x0000000000400274 length=0x24     (4096k,36B)
.hash             start=None       length=None     (0k,0)
.gnu              start=0x0000000000400298 length=0xac     (4096k,172B)
.dynsym           start=0x0000000000400348 length=0xc48    (4096k,3kB)
.dynstr           start=0x0000000000400f90 length=0x502    (4099k,1kB)
.gnu              start=0x0000000000401492 length=0x106    (4101k,262B)
.gnu              start=0x0000000000401598 length=0x0      (4101k,0B)
.gnu              start=0x0000000000401598 length=0xa0     (4101k,160B)
.rela             start=0x0000000000401638 length=0x150    (4101k,336B)
.rela             start=0x0000000000401788 length=0xa38    (4101k,2kB)
.init             start=0x00000000004021c0 length=0x1a     (4104k,26B)
.plt              start=0x00000000004021e0 length=0x6e0    (4104k,1kB)
.text             start=0x00000000004028c0 length=0x297a0  (4106k,165kB)
.fini             start=0x000000000042c060 length=0x9      (4272k,9B)
.rodata           start=0x000000000042c080 length=0x13850  (4272k,78kB)
.rodata1          start=None       length=None     (0k,0)
.eh_frame_hdr     start=0x000000000043f8d0 length=0x271c   (4350k,9kB)
.eh_frame         start=0x0000000000441ff0 length=0xc1bc   (4359k,48kB)
.gcc_except_table start=None       length=None     (0k,0)
.exception_ranges start=0x000000000044e1ac length=         (4408k,)
.eh_frame         start=None       length=None     (0k,0)
.gcc_except_table start=None       length=None     (0k,0)
.exception_ranges start=None       length=None     (0k,0)
.tdata            start=None       length=None     (0k,0)
.tbss             start=None       length=None     (0k,0)
.preinit_array    start=0x000000000064edcc length=0x0      (6459k,0B)
.init_array       start=0x000000000064edd0 length=0x8      (6459k,8B)
.fini_array       start=0x000000000064edd8 length=0x8      (6459k,8B)
.ctors            start=None       length=None     (0k,0)
.dtors            start=None       length=None     (0k,0)
.jcr              start=0x000000000064ede0 length=0x8      (6459k,8B)
.data             start=None       length=None     (0k,0)
.dynamic          start=0x000000000064ede8 length=0x210    (6459k,528B)
.got              start=0x000000000064eff8 length=0x8      (6459k,8B)
.got              start=0x000000000064f000 length=0x380    (6460k,896B)
.data             start=0x000000000064f380 length=0x18     (6460k,24B)
.tm_clone_table   start=0x000000000064f398 length=0x0      (6460k,0B)
.data1            start=0x000000000064f398 length=         (6460k,)
.bss              start=0x000000000064f3a0 length=0x600    (6460k,1kB)
.lbss             start=0x000000000064f9a0 length=         (6462k,)
.lrodata          start=None       length=None     (0k,0)
.ldata            start=0x0000000000a4f9a0 length=0x0      (10558k,0B)
.stab             start=None       length=None     (0k,0)
.stabstr          start=None       length=None     (0k,0)
.stab             start=None       length=None     (0k,0)
.stab             start=None       length=None     (0k,0)
.stab             start=None       length=None     (0k,0)
.stab             start=None       length=None     (0k,0)
.comment          start=0x0000000000000000 length=0x24     (0k,36B)
.debug            start=None       length=None     (0k,0)
.line             start=None       length=None     (0k,0)
.debug_srcinfo    start=None       length=None     (0k,0)
.debug_sfnames    start=None       length=None     (0k,0)
.debug_aranges    start=None       length=None     (0k,0)
.debug_pubnames   start=None       length=None     (0k,0)
.debug_info       start=None       length=None     (0k,0)
.debug_abbrev     start=None       length=None     (0k,0)
.debug_line       start=None       length=None     (0k,0)
.debug_frame      start=None       length=None     (0k,0)
.debug_str        start=None       length=None     (0k,0)
.debug_loc        start=None       length=None     (0k,0)
.debug_macinfo    start=None       length=None     (0k,0)
.debug_weaknames  start=None       length=None     (0k,0)
.debug_funcnames  start=None       length=None     (0k,0)
.debug_typenames  start=None       length=None     (0k,0)
.debug_varnames   start=None       length=None     (0k,0)
.debug_pubtypes   start=None       length=None     (0k,0)
.debug_ranges     start=None       length=None     (0k,0)
.debug_macro      start=None       length=None     (0k,0)
.gnu              start=None       length=None     (0k,0)
/DISCARD/         start=None       length=None     (0k,0)

Code: Select all

For: stmhal_firmware_dh_01.map
Found 7 sections. 15591 lines read
 ['Memory Configuration', 'Discarded input sections', 'Cross Reference Table', 'Linker script and memory map', 'Allocating common symbols', 'OUTPUT', 'Preamble']
Reading:
Parsing:
 Parsing Blocks
 Parsing Common Symbols
 Parsing Linker and Mem map
 Parsing Cross refs
Memory stats
.isr_vector   start=0x08000000 length=0x3520   (131072k,13kB)
.text         start=0x08020000 length=0x3d418  (131200k,245kB)
.glue_7       start=0x0805d418 length=0x0      (131445k,0B)
.glue_7t      start=0x0805d418 length=0x0      (131445k,0B)
.vfp11_veneer start=0x0805d418 length=0x0      (131445k,0B)
.v4_bx        start=0x0805d418 length=0x0      (131445k,0B)
.iplt         start=0x0805d418 length=0x0      (131445k,0B)
.rel          start=0x0805d418 length=0x0      (131445k,0B)
.data         start=0x20000000 length=0x60     (524288k,96B)
.igot         start=0x20000060 length=0x0      (524288k,0B)
.bss          start=0x20000060 length=0x278c   (524288k,9kB)
.heap         start=0x200027ec length=0x4000   (524297k,16kB)
.stack        start=0x200067ec length=0x800    (524313k,2kB)
.ARM          start=0x00000000 length=0x37     (0k,55B)

[stijn]

I don't currently have access to any file sharing solution, care to PM me an email adress if you're interested in msvc map files?

[pfalcon]

Neon22, thanks for proceeding with this!

[Neon22]

Using D3 for graphing. What a cool javascript library.
I'm going to be out of touch for almost two weeks.
Here's progress:
- parsing one format of .map file
- not yet parsing the msvc version but not expecting any problems with it. (thanks @stijn for samples)
- extracting all top level regions and parsing all symbols but not yet exporting them meaningfully.
- initial D3 result is attached.
Planning on making several visualizations of the data including stacked linear graph and node tree as well as these pie(donut) graphs.
They will all be interactive (courtesy D3) so will also be showing differences between them.


Still need some (non msvc) pyboard samples.

[stijn]

Looking good!!
Is everything up to the top (gui layer) written in Python? If so I imagine it would be nice to try out different display libs, IPython/matplotlib etc. Or maybe that's what you are doing already?

[Neon22]

Its in two parts - one is python code (of course) which is parsing the .map files and generating a csv (maybe json - we will see) file.
This is then embedded in an html page (to avoid sec XSS issues) and displayed in any browser (but IE is NG as usual).

The display technology is D3 a javascript data mapping and graphing toolkit/framework.
The advantage is that the results can be interactive rather than static and its easily viewable in a browser without huge code overhead on any OS. And people are clearly developing on several platforms.

So I have used http://d3js.org/instead of matplotlib etc for this reason.

[pfalcon]

So, is the code available somewhere to look/play with?

[Neon22]

Ressurecting ancient thread.
My code now online in repo here:
- https://github.com/Neon22/micropython-examine-mem-maps

I finally got D3 to play well. Thanks Firebug.
Code is still not in very good shape but now works for memory sizes (no symbols yet). See repo readme for more details
To play:
- download the files.
- Edit read_maps_v1.py at the end to include names of all .map files you woudl like to examine
- Edit same file at the head - CATS variable to show which memory regions you would like to see
Run read_maps_v1.py it wil generate debug output and create mappings.csv
Open D3-memmaps-v1.html to view. It wil load the csv.
You may have to allow file:// temporarily.

Clicking on one of the thumbnail charts will focus and refresh the main piechart. The majority of the code in the html is about having the transitions between maps work well.

Next steps are outlined in the Readme.
- need to improve read_maps.
- add entire symbol table and navigation to html.

[abidi]

hello,

I tested the python script,it works very well however it does not support all format of map file.
shoud i modify in the script ?
Thanks

[pfalcon]

I missed that the code was posted eventually. While hacking on esp8266 last summer, I put up a quick script myself to do some comparisons on maps, which I probably "lost" by now too. Well, we may need such tool working on esp8266 port now, as well as on other ports.