Displays with driver subclassed from framebuf

Discuss development of drivers for external hardware and components, such as LCD screens, sensors, motor drivers, etc.
Target audience: Users and developers of drivers.
User avatar
pythoncoder
Posts: 3121
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Displays with driver subclassed from framebuf

Post by pythoncoder » Mon Aug 13, 2018 9:49 am

The framebuf class is able to work with multiple colours and the official SSD1306 driver is subclassed from framebuf. Are any colour displays available with drivers written in this way?
Peter Hinch

User avatar
deshipu
Posts: 1334
Joined: Thu May 28, 2015 5:54 pm

Re: Seeking display with driver subclassed from framebuf

Post by deshipu » Mon Aug 13, 2018 11:50 am

It would use up way too much RAM to be practical. If you look at the common 320x240x16 display, it comes up to ~150kB just for the framebuffer. It's much better to generate the pixel data dynamically as you are sending it to the display.

User avatar
pythoncoder
Posts: 3121
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Seeking display with driver subclassed from framebuf

Post by pythoncoder » Mon Aug 13, 2018 1:03 pm

In which case I'm puzzled at the effort you put into ensuring that the framebuf module could cope with multi-colour operation. Has this never been put to use?
Peter Hinch

User avatar
deshipu
Posts: 1334
Joined: Thu May 28, 2015 5:54 pm

Re: Seeking display with driver subclassed from framebuf

Post by deshipu » Mon Aug 13, 2018 5:45 pm

My hope was to use framebuf as a general purpose object for creating, storing and manipulating image data, similar to PyGame's Surface objects, which then in turn could be used as building blocks for more advanced display drivers and frameworks. For example, you could use it to store images of buttons or other UI elements, that you would send to the display without having to keep a copy of the whole memory in your microcontroller. Unfortunately the resistance I encountered made it impractical to continue, and I stopped before I even got half the functionality I needed.

User avatar
dhylands
Posts: 2905
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Seeking display with driver subclassed from framebuf

Post by dhylands » Mon Aug 13, 2018 7:52 pm

Why don't we just allocate the framebuffer with some number of "header" bytes which could be used to store the i2c commands?

User avatar
deshipu
Posts: 1334
Joined: Thu May 28, 2015 5:54 pm

Re: Seeking display with driver subclassed from framebuf

Post by deshipu » Mon Aug 13, 2018 8:30 pm

dhylands wrote:
Mon Aug 13, 2018 7:52 pm
Why don't we just allocate the framebuffer with some number of "header" bytes which could be used to store the i2c commands?
I'm not sure I follow. Care to elaborate a little?

OutoftheBOTS_
Posts: 450
Joined: Mon Nov 20, 2017 10:18 am

Re: Seeking display with driver subclassed from framebuf

Post by OutoftheBOTS_ » Mon Aug 13, 2018 9:05 pm

@Deshipu
My hope was to use framebuf as a general purpose object for creating, storing and manipulating image data, similar to PyGame's Surface objects, which then in turn could be used as building blocks for more advanced display drivers and frameworks. For example, you could use it to store images of buttons or other UI elements, that you would send to the display without having to keep a copy of the whole memory in your microcontroller. Unfortunately the resistance I encountered made it impractical to continue, and I stopped before I even got half the functionality I needed.
I would be very interested in this for my ESP projects as the ESP has very low pin count so normally don't have spare pins for UI buttons and 2.4" SPI screen with touch is pretty cheap(have seen as cheap as $5US).

I originally started developing the idea on the ESP8266 where I used the flash to store the background image and foreground UI buttons (as BMP files) to be able to place the foreground UI buttons then be able to restore the background image from the BMP file but now use ESP32 with psRAM so I can load the whole background image and foreground buttons into memory. I did stop development of this idea because Laboris was talking about implementing it in C and his programs are always a lot more efficient than anything I write, I also have been quite side tracked from Micro-Python atm in trying to teach myself C on a STM32 processors.

User avatar
dhylands
Posts: 2905
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Seeking display with driver subclassed from framebuf

Post by dhylands » Mon Aug 13, 2018 9:15 pm

deshipu wrote:
Mon Aug 13, 2018 8:30 pm
dhylands wrote:
Mon Aug 13, 2018 7:52 pm
Why don't we just allocate the framebuffer with some number of "header" bytes which could be used to store the i2c commands?
I'm not sure I follow. Care to elaborate a little?
Lets say you want a 128x64 pixel (1 bit per pixel) frame buffer. For the actual frame buffer, you need to allocate 128x64/8 = 1024 bytes. Let's say you need up to 4 bytes of commands to prepend when sending on the I2C bus. This means that you'd allocate 1028 bytes when actually allocating the frame buffer. The frame buffer would know that it should skip the first 4 bytes and use the last 1024 bytes as the actual frame buffer.

You can then manipulate the 4 header bytes, and do the I2C write starting in the middle of the header. If you needed 1 byte command then you'd put the command at the last byte of the header and then the command and the frame buffer are contiguous.

User avatar
deshipu
Posts: 1334
Joined: Thu May 28, 2015 5:54 pm

Re: Seeking display with driver subclassed from framebuf

Post by deshipu » Mon Aug 13, 2018 11:39 pm

dhylands wrote:
Mon Aug 13, 2018 9:15 pm
deshipu wrote:
Mon Aug 13, 2018 8:30 pm
dhylands wrote:
Mon Aug 13, 2018 7:52 pm
Why don't we just allocate the framebuffer with some number of "header" bytes which could be used to store the i2c commands?
I'm not sure I follow. Care to elaborate a little?
Lets say you want a 128x64 pixel (1 bit per pixel) frame buffer. For the actual frame buffer, you need to allocate 128x64/8 = 1024 bytes. Let's say you need up to 4 bytes of commands to prepend when sending on the I2C bus. This means that you'd allocate 1028 bytes when actually allocating the frame buffer. The frame buffer would know that it should skip the first 4 bytes and use the last 1024 bytes as the actual frame buffer.

You can then manipulate the 4 header bytes, and do the I2C write starting in the middle of the header. If you needed 1 byte command then you'd put the command at the last byte of the header and then the command and the frame buffer are contiguous.
But that doesn't solve the problem of not having enough memory to handle the color displays in any way.

I was actually planning to eventually have something like a "send over SPI" method on them, so that you could send all the setup commands first, then send the pixel data in the correct format in an efficient way. This way you could have a 4-bit framebuf with a palette, but still send 16-bit pixel information to your display, saving 4x the memory. The whole idea is to separate your internal representation of the image from the format that your display requires, doing the conversion on the fly while sending, so that you can use a more efficient representation.

Your suggestion pretty much ties framebuf to I2C-based displays and forces you to use the same representation as the display does.

User avatar
mcauser
Posts: 328
Joined: Mon Jun 15, 2015 8:03 am

Re: Seeking display with driver subclassed from framebuf

Post by mcauser » Tue Aug 14, 2018 3:11 am

The framebuffer does not also have to be the exact same dimensions as the display's memory.
eg. a 64x48 display using the SSD1306 (128x64), where you initialise the display to only write to a portion of the memory.
Side note: Clever use of the hardware scroll functions gives you a bonus 2nd offscreen hardware buffer.
Some of the epaper displays are like this too - where you write to offscreen "pixels" then toggle between the buffers.

My understanding is there are 2 problems here. How much memory is required to store the entire buffer (for a 1-1 copy) and how to efficiently send the buffer over the common buses I2C, SPI and UART.

For drivers such as the SSD1306, you can but do not need to send the entire frame buffer in each write. You can use the column and page addressing commands to specify a "window" of the hardware buffer to write to, then chunk up the framebuffer into multiple writes. To extract the window of pixels from the framebuffer in the correct order, it needs to also understand the addressing mode, horizontal, vertical etc.

We have different configurations of the framebuffer (vertical/horizontal, msb/lsb first) to allow the pixel data to be stored in the same format as the display memory, so the entire buffer can be written directly without modification. This makes writes fast with a side effect of more complicated gets/sets. If we use the addressing commands to chunk up the writes, we lose the performance benefit of making the framebuffer use the exact same pixel layout, as we'd need build a new window buffer anyway. Which means, with a get-window translation layer between the framebuffer and display buffer, we could store the framebuffer in any pixel layout (or depth - RGB565/666).

Would a chunked write solve the problem of allocating 1028 bytes on top of the 1024 bytes? Maybe sending them in pages of 128x8 pixels (128 bytes)?

Post Reply