uPython Memory Manager

[stijn]

- no idea if 'and' etc is in some standard. Possibly, but now you ended up with a mix of things like 'and', '&=', etc which does not make everything more readable
- any project which allows easily replacing malloc/free should do fine, but I don't really know any from the top of my head. Mayabe there's something you are using already?
- macro/inline: I consider the latter more 'C', more readable (esp. multiline), much safer (type safety for one, const correctness, but also check what happens if you'd call Pool_filled_set(pool, ++index)). 'class method in C' makes me think of a function taking a pointer to a struct as first argument e.g. `bool Pool_available( const Pool* )` but maybe that's just me. Though what is witnessed in issue 293 is indeed rather unfortunate, I'd still say you're better of first checking if it really is a problem for you as well.

[cloudformdesign]

small updates:
- switched to using &, &&, ^, !, etc instead of associated macros (as requested)
- created "tinymem.h" and "*.c" with documentaiton in the .h file. Check it out!
- I will be integrating http://locklessinc.com/downloads/t-test1.c into tinymem as an inital "standard unit test." If anyone knows of any other tests, please let me know!
- stjin: I don't have any long term projects that can use this as I've mostly programmed C in microcontrollers, and there are obvious reasons I have never done dynamic memory allocation with them!

I'm thinking some kind of lightweight network monitoring tool would be best. Something I can throw known signals at and get a known result. Let me know if you think of anything.

Also, I definitely have uses for inline functions. I will have to explore and test them more, but they look pretty interesting and easy to use. I am concerned from the previous thread, as they seem to take more flash -- that is something I cannot accept for a "tiny" memory manager!

[cloudformdesign]

What is the best way to have configurable options?

I have never had much luck with just using #ifndef statements because defines don't have to be global. The best way I have seen is to use something like tinymem_config.h file that the user has to create (or copy from the standard project) and include in their Makefile. Micropython does something like this with it's platform configurations -- is this the recommended method?

[thorwald]

- Tests tests tests: I have finished the basic unit tests, but many many more need to be written. In particular, I would like to find a pre-made test suite for memory managers. Does anybody know of any? I would like to test all relevant aspects: performance, reliability, etc. If you know of anything that one

I haven't yet found full test suites, but this paper:
http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf

Recommends testing using a trace of what memory allocation does in real programs.

It might also have some interesting info with regards to the fragmentation issue.

Anyway, i'll keep looking.

[thorwald]

- Tests tests tests:

Here's a memory allocated, with some testing code inside, not a lot but something:

http://www.gii.upv.es/tlsf/

[DuaneKaufman]

Hello,

I know I am note well-versed in this type of thing, but I just read about a Python implementation _inside_ of GRUB (that is, without OS support) from Intel, called BITS:

http://lwn.net/Articles/641244/

I wonder how they manage memory, and if there is anything to learn from their Project?

Duane

[dhylands]

I believe that CPython (which is what they say is running) uses reference counted memory objects, ultimately obtained from malloc.

It would use the same garbage collection as regular python. grub probably even has a malloc implementation. They said no OS, but they didn't say no C runtime.

And because its running on a PC, they have loads of RAM and aren't really concerned about being memory constrained.

[cloudformdesign]

Just a quick update I have made a major breakthrough that required a complete rewrite of the library.

New features:
- 34 bits per pointer, down from 50
- defrag cam happen without sorting, meaning it is simpler and much faster
- better implementation overall, with a better philosophy of testing thanks to a bunch of sources and help.
- freed arrays can no longer "run out", uses the free data itself to store a doubly linked list of freed values
- freeing and allocating data is SUPER fast, probably less than 20 clocks or so in most cases, and never more than like 200

Edit:
It has now been actually pushed, take a look. It is still NOT READY for use, but it passes the extensive test that I have given it
to up to 200 loops (haven't tried more)

Also, I based it off of the fantastic library https://github.com/dimonomid/umm_malloc ... m_malloc.c , check that one
out too!