Page 1 of 1

Read a binary file with a clock

Posted: Fri May 01, 2020 12:20 pm
by fredb
Hi, everybody,

I start in this forum with a problem, but I would have preferred to come up with a solution...
I spent a whole week looking for a solution before posting the following :

I have a project to read the bytes of a binary file and present them on a port (port A) of Pyboard_D_SF6W.
I have to do this every 100µs and read a datafile in binary mode.
At first, i want to measure the reading time of one byte of the file on an oscilloscope, so i wrote this code :

Code: Select all

from pyb import Pin, Timer

strobe = Pin(Pin.cpu.A0, Pin.OUT_PP)

# Timer 10KHz, period=100µs
def tim1_isr(timer):
    global flag

tim1=Timer(1, freq=10000)

i=300000  # File of 30 sec (300 Kb)

fi = open("/flash/data.bin", "rb")
while i>0:
    if flag==1:
        octet =  # read next byte



#---- END OF SRC ----------------------------
I measured the time of the strobe signal staying at a high level: 17µs every 100µs, I thought it was good for the next development but when I analyzed the whole reading, I found a glitch in the curve of the oscilloscope.

About every second, the strobe signal stays at a high level for 230µs and it was no good for me...

I tried this code without reading a file, and the problem doesn't appear. So it looks like it comes from reading the file.
It looks like a pause created by reading a block of data or resetting a buffer. Do you think this problem is solvable?

Thank you for your help.

Re: Read a bynary file with a clock

Posted: Fri May 01, 2020 6:10 pm
by pythoncoder
The .read() method is a blocking method. It usually returns fast, but every time a sector is read it can block for a long time: I have observed 4ms but this will depend on the storage medium (Flash, SD card or other). File I/O is not fast. Options are to buffer data in RAM or, if the data is constant, it can be stored in Flash as a frozen bytes instance.

Re: Read a bynary file with a clock

Posted: Fri May 01, 2020 9:49 pm
by OutoftheBOTS_
Why do you want to read a byte from a file during this timed loop??

Re: Read a bynary file with a clock

Posted: Sat May 02, 2020 6:11 am
by Roberthh
Can you pre-buffer the content of the file? What is the size of the file?

Re: Read a bynary file with a clock

Posted: Sat May 02, 2020 2:00 pm
by fredb
Thank you for your comments.
I also noticed a latency of 3.8ms every 1.2 seconds when playing my file.
Finally, I will test a solution with a 16MB external flash memory accessible by SPI (S25FL128S).
The size of the file I need to read is 10MB maximum, so I will read it from the sd card and copy it to the SPI flash. Then I read the SPI flash memory at a speed of 10000 bytes per second. I hope there will be no latency when the memory changes sectors.
I'm playing with the limits of MicroPython but it's interesting...

Re: Read a bynary file with a clock

Posted: Fri May 08, 2020 12:10 pm
by fredb
Some news : it works !
First, the 10MB binary file is read and bytes are copied to the S25FL128S flash memory by page of 512 bytes in a single command for each page. This copy takes about 16 seconds.
The next step is the reading of the flash memory. It takes 76µs for each byte.
There is no latency when the flash changes sectors of 256 kB.
It should be noted that I had to run the SPI used to program the flash memory at its maximum clock speed: 18MHz.

Re: Read a bynary file with a clock

Posted: Fri May 08, 2020 4:42 pm
by Jurjen
Hi, would it be possible for you to publish the code? Thanks!

Re: Read a binary file with a clock

Posted: Wed May 27, 2020 1:32 pm
by fredb
Here is the source code to read memory à 10KHz. I can't publish the entire source code because it doesn't belong to me.

Code: Select all

Read Flash memory (S25FL128S) at 10KHz with external interrupt on PB1 (external clock)
The memory already contains data

from pyb import Pin, SPI
import machine

#----- SPI to read S25FL128S
spi = SPI(2, SPI.MASTER, baudrate=18000000, polarity=0, phase=0, bits=8, firstbit=SPI.MSB)
cs = Pin(Pin.cpu.B12, Pin.OUT_PP) # C4

#----- Resets the chip
rst = Pin(Pin.cpu.C5, Pin.OUT_PP) # A10

#----- External clock Interrupt 10kHz -------------
ocxo = Pin(Pin.cpu.B1, Pin.IN, Pin.PULL_UP)

def clk_isr(param):
    global clkflag

#----- Strobe line to validate the data put on PORTA
strobe = Pin(Pin.cpu.B0, Pin.OUT_PP)

# PA0..PA7 bits
machine.mem16[stm.GPIOA+stm.GPIO_MODER]=0x5555  # PA0..PA7 in output (0b01) 
machine.mem16[stm.GPIOA+stm.GPIO_OTYPER]=0x0000  # PA0..PA7 in Push-Pull (0b0) 
machine.mem16[stm.GPIOA+stm.GPIO_OSPEEDR]=0xFFFF  # PA0..PA7 in high speed (100MHz)
machine.mem8[stm.GPIOA+stm.GPIO_ODR]=0x00 # PA0..PA7 to 0

#------ S25FL128S command codes
RDID = 0x9F
RES = 0xAB
RDSR1 = 0x05
READ = 0x03
PP = 0x02
WREN = 0x06
WRDI = 0x04

#--- First byte reading
cmd = bytearray(4)
rec = bytearray(1)
cmd[0] = READ
cmd[1] = 0x00
cmd[2] = 0x00
cmd[3] = 0x00

#----- Variables init
mem_addr = 0
mem_size = 10485760 

#----- Enable clock interrupt
ocxo.irq(trigger=Pin.IRQ_FALLING, handler=clk_isr)

#----- Main loop
while mem_addr<mem_size:
    if clkflag==1:
        machine.mem8[stm.GPIOA+stm.GPIO_ODR]=rec[0]  # Write data on PortA
        strobe.high()  # Validate data
        #  Increments the address in the memory command
        cmd[3] += 1
        if cmd[3]==0:
            cmd[2] += 1
            if cmd[2]==0:
                cmd[1] += 1
        mem_addr += 1  #  increments the address counter
        # Memory reading operation on SPI
        clkflag=0  # Interrupt flag reset

#----- Disable clock interrupt