os(uos) usage for mount

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
codyhanks
Posts: 17
Joined: Thu Nov 30, 2017 7:30 pm

os(uos) usage for mount

Post by codyhanks » Fri Jun 15, 2018 1:05 am

I am having trouble with using uos for mounting a spi device.

Code: Select all

class flash():
	def __init__(self,port,cs):
        	self._port = port
        	self._cs= cs
        	self._cs.high()
        	self._block_cnt=31
        	self._block_size=4096
        	self._baseoffsett=0x0
        	self.command(clsr, 0)
        	self.command(wren, 0)
        	
	def readblocks(self,block_num,buf):
        	cmd=bytearray(1)
        	cmd[0] = fourread
        	cmd.extend(self.block_to_bytes(self._baseoffsett+block_num *self._block_size))
        	#print('read cmd {}'.format(list(cmd)))
        	self._cs.low()
        	time.sleep_us(1)
       		self._port.send(cmd)
        	self._port.readinto(buf)
        	self._cs.high()
        
        def writeblocks(self,block_num,buf):
        	pagesize = 256
        	cmd=bytearray(5)
        	cmd[0]=fourp4e
        	cmdwrite=bytearray(5)
        	cmdwrite[0] = fourpp
        	#for each block erase block and write again 
        	nblocks, err = divmod(len(buf), self._block_size)
        	mv = memoryview(buf)
        	offset=0
        	curblock=0
        	while(curblock < nblocks):
           		#erase this block
            		cmd[1:4]=self.block_to_bytes(self._baseoffsett+(block_num*self._block_size)+(curblock*self._block_size))[0:3]
            		self.command(wren, 0)
            		self.command(cmd, 0)
            		#print('cmd_clear {}'.format(list(cmd)))
            		#wait for write complete 
            		self.command(rdsr1, 1)
            		while ((self._buffer[0]&0x01) ==0x01):
                		#print('clear in progress')
               			self.command(rdsr1, 1) 
           	#write this block 
            	for x in range(self._block_size/pagesize):
                	#print('x {} start point {}'.format(x,curblock*self._block_size+(x*pagesize)))
               		cmdwrite[1:4]=self.block_to_bytes(self._baseoffsett+(block_num*self._block_size)+(curblock*self._block_size)+(x*pagesize))[0:3]
                	#print('cmdwrite = {}'.format(list(cmdwrite)))
                	self.command(wren, 0)
                	time.sleep_us(1)
                	self._cs.low()
                	time.sleep_us(1)
                	self._port.send(cmdwrite)
                	startpoint=(curblock*self._block_size)+(x*pagesize)
                	tempbuff = buf[startpoint:startpoint+pagesize]
                	print(tempbuff)
                	self._port.send(buf[startpoint:startpoint+pagesize])
                	self._cs.high()
               	 	time.sleep_us(1)
                	self.command(rdsr1, 1)
                	while ((self._buffer[0]&0x01) ==0x01):
                   		print('Write in progress')
                    		self.command(rdsr1, 1)
                	#print('Done')
            	curblock+=1
            	
        def ioctl(self,op,arg):
        	if op==4:
            		return self._block_size
       		if op==5:
           		return self._block_cnt
        
    	    def count(self):
       		return self._block_cnt
  
This is the response I am getting

>>> uos.VfsFat.mkfs(_mtr.flash_ic)
>>> vfs=uos.VfsFat(_mtr.flash_ic)
>>> uos.mount(vfs,'/test')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [Errno 19] ENODEV


I have verified the block read and write are working I am sure i am missing something small and simple But cannot seem to see it.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: os(uos) usage for mount

Post by pythoncoder » Fri Jun 15, 2018 6:13 am

It's hard to debug code without access to the hardware: the ENODEV error may suggest a hardware issue. I can't see anything obviously wrong but you might like to look at my ferroelectric RAM driver which is a pretty minimal example of a mountable device driver.

Your post prompted me to update the test program framtest_pb.py to use VFS mounting. You'll see it uses similar code to yours except that mine assumes an already existing filesystem. It works correctly.
Peter Hinch
Index to my micropython libraries.

Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: os(uos) usage for mount

Post by Damien » Fri Jun 15, 2018 1:48 pm

If you define the ioctl method (which is good, you should do this), then don't define the count method because count will override ioctl.

codyhanks
Posts: 17
Joined: Thu Nov 30, 2017 7:30 pm

Re: os(uos) usage for mount

Post by codyhanks » Fri Jun 15, 2018 10:55 pm

Thanks for the help after digging into this for a while the mkfs doesn't seem to be actually making any changes to the sectors on the drive.
I am wondering if it has to do with the line.

line 5144 in oofatfs/ff.c

Code: Select all

 if (sz_vol < 50) return FR_MKFS_ABORTED;   /* Check if volume size is >=50s */


but not sure where, the issue might be that its not actually creating the fatfs on the drive.

Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: os(uos) usage for mount

Post by Damien » Sat Jun 16, 2018 1:15 am

The FAT filesystem does need a lot of blocks to create a blank filesystem, that's why there is this minimum limit of 50 blocks. You can't really get around this, simply due to the FAT format. This number (50) was already decreased from a much larger number in the original FAT driver.

Another limitation is that on pyboard (stm32 port) currently you must have a 512 byte block size. It looks like you are using 4096 as your block size. The block size corresponds to the amount of RAM the FATFS driver needs for caching purposes, so it's currently limited to 512 on pyboard to reduce RAM usage. esp8266 uses 4096 to be compatible with the page erase size on the esp8266.

It would be possible to allow bigger block sizes on pyboard but that requires some improvements to the code base to make it work efficiently.

FAT is not really the best filesystem for embedded devices because of the above limitations. We are currently looking to add littlfs support https://github.com/micropython/micropython/pull/3847 to get around these limitations: littlfs only needs a few blocks at minimum for the filesystem meta data, and the erase-page size can be large without affecting RAM usage.

codyhanks
Posts: 17
Joined: Thu Nov 30, 2017 7:30 pm

Re: os(uos) usage for mount

Post by codyhanks » Sat Jun 16, 2018 10:13 am

Thanks that helps with another question I was having as to whether the uos uses the CCRAM on STM or not.

codyhanks
Posts: 17
Joined: Thu Nov 30, 2017 7:30 pm

Re: os(uos) usage for mount

Post by codyhanks » Mon Jun 25, 2018 6:30 pm

So I modified the driver. I have tested with a blockread and block write, it will read an write the given data correctly I do assume that it will not read/write blocks over 131,072 bytes at a time. uos mount is still failing as well as pyb.mount.


page size is 512

>>> import uos
>>> uos.VfsFat.mkfs(flash_ic)
sect out 0
>>> vfs = uos.VfsFat(flash_ic)
>>> uos.mount(vfs,'/test')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [Errno 19] ENODEV

read blocks/write blocks works correctly.


>>> flash_ic.writeblocks(65280,ain)
sect in 0
sect in 1
sect out 0
>>> ain=bytearray(2048)
>>> flash_ic.readblocks(65024,ain)
sect in 0
sect out 1
sect in 1


Code: Select all

class flash():
    _sector_cache=-1
    _sector_dirty=False
    _status_rg=bytearray(10)
    _buffer=bytearray(256)
    
    _sect_size=65536
    _sect_cnt=510
    _sect_offset=0x20000
     
    _page_size=512
    _page_cnt=65280
    
    _port = None
    _cs = None
    
    
    def __init__(self,port,cs):
        self._port = port
        self._cs= cs
        self._cs.high()
        self._mv_status_rg=memoryview(self._status_rg)
        self._mv_buffer=memoryview(self._buffer)
        self.command(clsr, 0)
        
        
    def get_id(self):
        self.command(rdid, 3)
    
    
    def command(self,cmd,bytes):
        self._cs.low()
        time.sleep_us(1)
        self._port.send(cmd)
        if(bytes>0):
            self._port.readinto(self._mv_status_rg[0:bytes])
        self._cs.high()
        
    def wait_write(self):
        self.command(rdsr1, 1)
        while ((self._mv_status_rg[0]&0x01) ==0x01):
            #print('clear in progress')
            self.command(rdsr1, 1)
    
    def ioctl(self,op,arg):
        if op==3:
            if(self._sector_dirty==True):
                self.sectorout(self._sector_cache)
        if op==4:
            return self._page_size
        if op==5:
            return self._page_cnt
    
    def readblocks(self,address,buf):
        ba=uctypes.bytearray_at(0x10000000,65536)
        sect = (address >>16)
        page = (address &0xffff)
        if(sect != self._sector_cache):
            self.sectorin(sect)
        for x in range(page,min(65536,page+len(buf))):
            buf[x-page]=ba[x]
        remaining = (page+len(buf))-65535
        start = (len(buf)-remaining)
        if(remaining >65536):
            raise
        if(remaining >0):
            sect +=1
            self.sectorin(sect)
            for x in range(remaining):
                buf[start+x]=ba[x]
        
            
    def writeblocks(self,address,buf):
        ba=uctypes.bytearray_at(0x10000000,65536)
        sect = (address >>16)
        page = (address &0xffff)
        if(sect!=self._sector_cache):
            self.sectorin(sect)
        for x in range(page,min(65536,page+len(buf))):
            ba[x]=buf[x-page]
        self._sector_dirty=True
        remaining = (page+len(buf))-65535
        start = (len(buf)-remaining)
        if(remaining >65536):
            raise
        if(remaining >0):
            sect +=1
            self.sectorin(sect)
            for x in range(remaining):
                ba[x]=buf[start+x]
            self._sector_dirty=True
        
        
    
    
    def sectorin(self,sector):
        print('sect in {}'.format(sector))
        ba=uctypes.bytearray_at(0x10000000,65536)
        if (sector != self._sector_cache) and (self._sector_cache != -1):
            if self._sector_dirty:
                self.sectorout(self._sector_cache)
                
        for x in range(self._sect_size/(self._page_size/2)):
            cmd=bytearray(1)
            cmd[0] = fourread
            cmd.extend(self.block_to_bytes(int(self._sect_offset+(sector *self._sect_size)+(x*(self._page_size/2)))))
            self._cs.low()
            time.sleep_us(1)
            self._port.send(cmd)
            #print('read cmd {}'.format(list(cmd)))
            self._port.recv(self._mv_buffer)
            #print('buffer {}'.format(bytearray(self._mv_buffer)))
            ba[x*(int(self._page_size/2)):(x+1)*(int(self._page_size/2))]=self._mv_buffer
            #print('ba {}'.format(bytearray(ba[x*256:(x+1)*256])))
            self._cs.high()
            time.sleep_us(1)
        self._sector_cache=sector
        self._sector_dirty=False
        
                   
    def sectorout(self,sector):
        print('sect out {}'.format(sector))
        ba=uctypes.bytearray_at(0x10000000,65536)
        cmd=bytearray(1)
        cmd[0]=fourse
        cmd.extend(self.block_to_bytes(self._sect_offset+(sector *self._sect_size)))
        
        #print('erase command {}'.format(list(cmd)))
        self.command(wren, 0)
        self.command(cmd, 0)
        self.command(rdsr1, 1)
        while ((self._mv_status_rg[0]&0x01) ==0x01):
            #print('clear in progress')
            self.command(rdsr1, 1)
        for x in range(self._sect_size/(self._page_size/2)):
            cmdwrite=bytearray(1)
            cmdwrite[0] = fourpp
            cmdwrite.extend(self.block_to_bytes(int(self._sect_offset+(sector *self._sect_size)+(x*(int(self._page_size/2))))))
            self.command(wren, 0)
            time.sleep_us(1)
            self._cs.low()
            time.sleep_us(1)
            self._port.send(cmdwrite)
            #print('cmd write {}'.format(list(cmdwrite)))
            self._mv_buffer = ba[x*(int(self._page_size/2)):(x+1)*(int(self._page_size/2))]
            #print('buffer len {}, buf {}'.format(len(self._mv_buffer),self._mv_buffer))
            self._port.send(self._mv_buffer)
            self._cs.high()
            time.sleep_us(1)
            self.command(rdsr1, 1)
            while ((self._mv_status_rg[0]&0x01) ==0x01):
                #print('Write in progress')
                self.command(rdsr1, 1)
            
    def block_to_bytes(self,block_num):
        ba=bytearray(4)
        ba[3]=block_num&0xff
        ba[2]=(block_num>>8)&0xff
        ba[1]=(block_num>>16)&0xff
        ba[0]=(block_num>>24)&0xff
        return ba    


User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: os(uos) usage for mount

Post by pythoncoder » Tue Jun 26, 2018 8:01 am

Are you sure your code takes account of the fact that the address arg to readblocks and writeblocks is a block address? I'd expect to see a << 9 or similar somewhere to take account of the 512 byte logical block size.

Aplogies if I'm missing something - I only had time for a quick look.
Peter Hinch
Index to my micropython libraries.

codyhanks
Posts: 17
Joined: Thu Nov 30, 2017 7:30 pm

Re: os(uos) usage for mount

Post by codyhanks » Wed Jun 27, 2018 11:57 pm

That did the trick, I didn't realize that the readblocks was a block address not a full byte address aligned to the pages. Once I shifted to the full byte address it works.

Thanks, just missed that on the details thanks.

Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: os(uos) usage for mount

Post by Damien » Thu Jun 28, 2018 3:28 am

Good that you got it working.

I've updated the docs in https://github.com/micropython/micropyt ... 11674e3476 to make it clearer that the block number is an index, not an address.

Post Reply