Suggested standard approach to font handling

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
User avatar
dhylands
Posts: 2109
Joined: Mon Jan 06, 2014 6:08 pm
Location: Shuswap, BC, Canada
Contact:

Re: Suggested standard approach to font handling

Postby dhylands » Tue Jan 03, 2017 4:00 pm

I'm pretty sure that the Pin dicts and arrays are not pulled into RAM. This would require that the dicts and arrays be created statically using C declarations rather than code (since anything created with python code will necessarily have to be from RAM).

v923z
Posts: 15
Joined: Mon Dec 28, 2015 6:19 pm

Re: Suggested standard approach to font handling

Postby v923z » Tue Jan 03, 2017 9:33 pm

pythoncoder wrote:
v923z wrote:I was wondering, how much memory overhead a dictionary adds...
That's an interesting technique: I didn't realise convert could do that. How are you dealing with the issue of alignment?


I am not sure I see what you mean: if it is the vertical alignment of fonts (i.e., the baseline of characters), then the bitmap produced by convert is automatically aligned.

pythoncoder wrote:My code aims to produce, given a nominal font size, a set of bitmaps with the vertical alignment pre-computed. This is for performance reasons: the aim is that rendering should be fast and simple. The alternative approach is to store the metrics with each glyph and figure out the positioning at render time. While this is probably OK for small displays, it may be too slow for large ones capable of displaying substantial amounts of text.


I believe, what I have is pretty similar to yours: each character is in a field of fixed height, and its position inside that field is stored in the bytearray itself, so no extra computation is required at run time. I don't think it would make too much sense to chop off the white spaces below and above the characters, and store the size of that as extra. If you look at the output of draw_string in the notebook, you can see that the characters are properly aligned.

pythoncoder wrote:While I appreciate the flexibility of using a dict, again there may be a performance issue: a dict lookup is likely to be slower than handling two levels of indirection, although I haven't tested this.


This is absolutely true, though I would mention that the rendering itself is orders of magnitude slower than the lookup, so it doesn't really matter.

pythoncoder wrote:As for RAM use, this can be fixed with frozen bytecode. My solution produces bytes objects which are immutable. This means that they can be accessed in place, consuming almost no RAM. A dict is mutable, so even if you freeze the bytecode, the runtime will pull the dict into RAM (because Python allows you to change its contents).


I have figured that the dictionary is rather expensive. If I take 32-pt FreeSans fonts for 80 standard characters, I consume something like 30 kB of RAM, while the actual data is only 7.3kB, and the frozen bytecode takes up something like 20 kB.
One has to add, though, that here I am talking about the costs of carrying the whole class, which contains the dictionary, plus the height function. However, the class can't probably be so expensive. I don't think that there is too much difference between

Code: Select all

def height():
    return 32
   
font = {'1': b'\x00....'}


and

Code: Select all

class Font(object):
    def __init__(self):
        self.font = {'1': b'\x00....'}

    def height():
        return 32


while the first form becomes somewhat problematic, if one has to use two kinds of fonts at the same time: you'll never know whose height the function height() returns...

User avatar
pythoncoder
Posts: 1263
Joined: Fri Jul 18, 2014 8:01 am

Re: Suggested standard approach to font handling

Postby pythoncoder » Wed Jan 04, 2017 10:06 am

v923z wrote:...if it is the vertical alignment of fonts (i.e., the baseline of characters), then the bitmap produced by convert is automatically aligned.
Apologies, I hadn't appreciated that. In any event, the issue of how you get from a font to a bitmap is secondary. The important issue as far as I'm concerned is how you present the data on a device with limited resources.

v923z wrote:I believe, what I have is pretty similar to yours: each character is in a field of fixed height, and its position inside that field is stored in the bytearray itself, so no extra computation is required at run time. I don't think it would make too much sense to chop off the white spaces below and above the characters, and store the size of that as extra. If you look at the output of draw_string in the notebook, you can see that the characters are properly aligned.
Agreed.

v923z wrote:This is absolutely true, though I would mention that the rendering itself is orders of magnitude slower than the lookup, so it doesn't really matter.
Agreed.

v923z wrote:I have figured that the dictionary is rather expensive. If I take 32-pt FreeSans fonts for 80 standard characters, I consume something like 30 kB of RAM, while the actual data is only 7.3kB, and the frozen bytecode takes up something like 20 kB.
One has to add, though, that here I am talking about the costs of carrying the whole class, which contains the dictionary, plus the height function. However, the class can't probably be so expensive. I don't think that there is too much difference between

Code: Select all

def height():
    return 32
   
font = {'1': b'\x00....'}


and

Code: Select all

class Font(object):
    def __init__(self):
        self.font = {'1': b'\x00....'}

    def height():
        return 32


while the first form becomes somewhat problematic, if one has to use two kinds of fonts at the same time: you'll never know whose height the function height() returns...
The first form works if you store one font per module. Python modules are in many ways similar to classes. Assume font1.py and font2.py use my approach. The following will work:

Code: Select all

import font1, font2
h1 = font1.height()
h2= font2.height()


The key technical problem is this. Bitmapped fonts comprise substantial amounts of constant data. MicroPython targets such as the ESP8266 have very limited amounts of free RAM. On such hardware this data is best stored in Flash, so a solution aiming to be a standard should offer this as an option.

MicroPython offers just one* efficient* way to do this: as precompiled Python source files containing bytes objects stored as frozen bytecode. The incremental cost of importing a font stored in that way is (from memory) about 150 bytes.

*As I understand it.
* You can store them in a random access file but glyph-wise retrieval is slow.
Peter Hinch

v923z
Posts: 15
Joined: Mon Dec 28, 2015 6:19 pm

Re: Suggested standard approach to font handling

Postby v923z » Wed Jan 04, 2017 3:39 pm

pythoncoder wrote:Python modules are in many ways similar to classes. Assume font1.py and font2.py use my approach. The following will work:

Code: Select all

import font1, font2
h1 = font1.height()
h2= font2.height()



Indeed. I totally overlooked this. :oops:

pythoncoder wrote:The key technical problem is this. Bitmapped fonts comprise substantial amounts of constant data. MicroPython targets such as the ESP8266 have very limited amounts of free RAM. On such hardware this data is best stored in Flash, so a solution aiming to be a standard should offer this as an option.

MicroPython offers just one* efficient* way to do this: as precompiled Python source files containing bytes objects stored as frozen bytecode. The incremental cost of importing a font stored in that way is (from memory) about 150 bytes.


Well, you have pretty much convinced me. As I said above, according to a couple of tests that I ran here, loading the font by means of a dictionary is insanely expensive.

User avatar
pythoncoder
Posts: 1263
Joined: Fri Jul 18, 2014 8:01 am

font_to_py.py now supports extended ASCII

Postby pythoncoder » Fri Jan 13, 2017 7:54 am

In response to a suggestion from @Roberthh and comments here and on GitHub I've updated this utility. It now offers command line arguments to define the range of characters to include in a font file. It defaults to standard ASCII (32-126) but you can specify an arbitrary set of consecutive characters in the range 0-255. You can also specify the character to be output if the user attempts to display a character not in the set (default "?"). Docs here https://github.com/peterhinch/micropython-font-to-py.git.

Since the Python font module includes the code to retrieve the glyph, no changes to application code are required.

The module has two new functions enabling the character range to be retrieved.
Peter Hinch


Return to “Development of MicroPython”

Who is online

Users browsing this forum: No registered users and 2 guests