SPI start stop latency

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
OutoftheBOTS_
Posts: 847
Joined: Mon Nov 20, 2017 10:18 am

SPI start stop latency

Post by OutoftheBOTS_ » Wed Dec 06, 2017 12:31 am

I search and couldn't find anything on this topic.

I am writing a library for a SPI screen and this involves busing larger amounts of data via SPI to the screen.

I have noticed if I have a large amount of data and I send it in 1 large chunck it goes very fast but if I send the same amount of data in lots of little then it is very very slow.

e.g if I have a file (150kb big) on flash that has the image and needs to be sent to the screen RAM via SPI. If I have a program that loops reading 1 byte then sending that byte then repeating it will take maybe 20secs to move all the data but if I have then same program but change it from reading 1 byte then sending to reading 1kb then sending then it takes under a sec. The larger the chunks the faster it is to move the same data from flash to screen via SPI.

I put a scope on the clock line of the SPI bus to see what's happening and ran this code

Code: Select all

data = b'x00'
while True : hspi.write(data)
what I saw on the scope is the number of pulses needed to send the byte of data then a very very long space before the set of pulses.

For me to become better educated and write better programs I like to understand why it is so.

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: SPI start stop latency

Post by deshipu » Wed Dec 06, 2017 2:44 pm

When you send the whole buffer all at once what happens internally is that MicroPython calls an internal C function, which does all the checks on the data types, sets up the SPI peripheral and all the buffers it needs, converts all the values it needs from Python types to C types, and then in a very tight loop iterates over the bytes in the buffer and sends them to the SPI peripheral. This is fast.

When you do a loop in Python and send byte by byte, everything I described above happens for every byte. This has to be slow.

OutoftheBOTS_
Posts: 847
Joined: Mon Nov 20, 2017 10:18 am

Re: SPI start stop latency

Post by OutoftheBOTS_ » Wed Dec 06, 2017 9:20 pm

Thanks for your response.

Micro-Python hspi.write(data) can only take byte array as a parameter, is it needed that Micro-Python converts all the values it needs from Python types to C types?

This leads on to 2 more questions.

Q1.
Python is a higher level language with more checks than C. Is it possible that if I was to write the program in C that I could omit performing all these checks and basically just dump prepared data from either RAM or Flash to SPI in smaller chunks but get speeds closer to if I did it in bigger chunks??

Q2.
Python only has 1 type of integer, when I say write the code "var_A = 10" does it only assign 1 byte of RAM to this var_A or does it automatically assign 4 bytes of RAM as a standard int?? I ask this question because RAM on the ESP8266 is of a premium and 4 bytes is 4 times the amount of 1 byte and whether I should look at packing data like this "var_A = ustruct.pack ('B', 10)" if I know that the values for var_A will never be over 255??

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: SPI start stop latency

Post by pythoncoder » Sat Dec 09, 2017 8:23 am

Regarding the first question you need to consider how much you'd gain over sending the data in large chunks. I think you'd find the gains might be small. Consider pre-allocating a buffer for a chunk at the start of your program using a bytearray. This avoids the overhead of allocating memory every time you want to send a chunk. If you experiment with the buffer size you'll probably find that a law of diminishing returns will apply: a buffer larger than (say) 1KB won't deliver much improvement in performance because the Python overhead is almost certainly independent of buffer size.

Regarding integers MicroPython does use a 32 bit word for storing integers unless they become too large to fit in 32 bits. There are ways to store bytes. These are standard Python techniques - see the bytearray and bytes objects in the Python 3 docs. The Python language provides the tools to pack data into individual bits and bitfields but this is going to extremes if the goal is merely to save RAM.

I suggest you read http://docs.micropython.org/en/latest/p ... ained.html for a detailed explanation of RAM saving techniques.
Peter Hinch
Index to my micropython libraries.

OutoftheBOTS_
Posts: 847
Joined: Mon Nov 20, 2017 10:18 am

Re: SPI start stop latency

Post by OutoftheBOTS_ » Sun Dec 10, 2017 8:38 am

Thanks for your response.

Yes there is definitely the law of demising returns. I certainly find that chunks as big as 1kb are far far far faster than doing 1kb as 1000 bytes. Once I get too much above 1kb you notice it is slightly faster but not heaps (diminishing returns).
The Python language provides the tools to pack data into individual bits and bitfields but this is going to extremes if the goal is merely to save RAM.
Yes some of it might be going a bit over board but I am more just experimenting to see what's worth while and what's heaps of work for little return.

I am finding the limiting factor of what I can do on the ESP8266 is the available ram. After micropython is loaded into memory there is only 22kb of memory left, then you have to load the program then what's left is what is what the program can use for allocation.

I am playing with graphics so RAM can be used up super fast. I am just writing libraries atm so they will then sit ontop of the main program so can't afford to be too hungry for memory.

I am mainly trying to learn atm, in the end I have ordered a ESP32 Wrover with 4mb RAM so won't be too tight for RAM when it gets here.

I have only been programing for a year now and only ever heard of Python less than a year ago so I am still very much in the learning stage. I was learning programming with my 13yr old son and we were programing in Microsoft Small Basic and he wanted to make a Rubik's cube solving robot and I needed to make a program to read the colors on the faces and discovered OpenCV with python. The program that solves the cube and turns the motors is written in Small basic but the computer vision is written in python you can see the finished project here https://www.youtube.com/watch?v=qcF5PFXZC3o

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: SPI start stop latency

Post by pythoncoder » Mon Dec 11, 2017 6:20 am

OutoftheBOTS_ wrote:
Sun Dec 10, 2017 8:38 am
...I am finding the limiting factor of what I can do on the ESP8266 is the available ram. After micropython is loaded into memory there is only 22kb of memory left, then you have to load the program then what's left is what is what the program can use for allocation. ...
If you issue <ctrl>D at the REPL and issue micropython.mem_info() you should see something around 36KB free. The solution to writing larger programs on the ESP8266 is to use frozen bytecode. See http://docs.micropython.org/en/latest/e ... ained.html in particular the section on Flash Memory.
Peter Hinch
Index to my micropython libraries.

OutoftheBOTS_
Posts: 847
Joined: Mon Nov 20, 2017 10:18 am

Re: SPI start stop latency

Post by OutoftheBOTS_ » Mon Dec 11, 2017 7:42 pm

I seem to get more like 22kb free but I am also using webrepl

Thanks for that I haven't yet played with frozen byte codes or read the docs about it but it is on my list.

I will google frozen byte code and learn how to use it. I amuse that I will need a Linux machine for compiling.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: SPI start stop latency

Post by pythoncoder » Thu Dec 14, 2017 2:10 pm

OutoftheBOTS_ wrote:
Mon Dec 11, 2017 7:42 pm
I seem to get more like 22kb free but I am also using webrepl...
That explains it.
Peter Hinch
Index to my micropython libraries.

Post Reply