Can be stored data inside non volative memory ?

[Damien]

@pythoncoder very nice!

I wonder if you can do a similar thing using uctypes and memoryview to point to the raw backup memory and make a buffer object? Then you could save and restore basic data structures using json.loads and json.dumps.

Also note that there are backup registers that you can use without any set up. But they are less flexible than the backup ram.

[pythoncoder]

@Damien

make a buffer object

That is a nice idea. Experimentation suggests that, thanks to the developers of uctypes, it's trivial. Using my above class

Code: Select all

>>> b = BkpRAM()
>>> for x in range(1024):
...  b[x] = 0
... 
>>> 
>>> import uctypes
>>> a =uctypes.bytearray_at(b.BKPSRAM, 4096)
>>> a[0]
0
>>> a[0] = 1
>>> b[0]
1
>>> a[4] = 99
>>> b[1]
99
>>> 

there are backup registers

In my ongoing attempts to write up low power operation of the Pyboard https://github.com/peterhinch/micropyth ... opower.git I've given an indication how these can be used.

[BOB63]

pythoncoder and bcd ,
thank you so much for all the details provided !
It's really a great school for me.

[BOB63]

Hi ,
I've implement the class suggested by pythoncoder into my program and testing storing int value It work perfectly.
Now I'm in the situation where I receive from a serial interface a string like this :

Code: Select all

inBuffer="$$$,z1,Y,22,30,23,45,Y,N,N,N,N,Y,Y,1144" 

that I split into an array using ',' as separator .

Now I'm in trouble when inside a loop I try to store in memory each single element of this array because some of them are strings and some of them are numbers .
In my mind I was expecting to save $$$ , considering that I'm using a 32 bit memory , 00363636 if convert each character as int or as 0x00242424 in Hex , but in case of z1 I start to be a little bit confused if I don't use the hex value of each character.

I was looking if there is a smart way to code /decode strings and numbers so to be store in 32 bit memory location in one shoot.
May be that must I use in such way a conversion as utf-32, having not idea at all from which point start ... ?:shock:
May be that must I convert to hex each character in the element array ?
I don't se impossible implement this with my poor knowledge.
Or may be that there is a smart python instruction that do the right job in one shoot ?

Any idea ?

[dhylands]

There are a couple of ways of converting '$$$' into 0x00242424 is to use struct. If you know that the $$$ is 3 bytes, then you can append a 00 to make it 4 and then do:

Code: Select all

>>> import ustruct as struct
>>> x = struct.unpack('<i', b'$$$' + b'\x00')
>>> hex(x[0])
'0x242424'

This assumes that your string is a bytestring: b'$$$' and not a unicode string '$$$'.

You could also do:

Code: Select all

>>> str = '$$$'
>>> x = ord(str[0]) + (ord(str[1]) << 8) + (ord(str[2]) << 16)
>>> hex(x)
'0x242424'

If you were using a byte string, then you don't need the ord:

Code: Select all

>>> bstr = b'$$$'
>>> x = bstr[0] + (bstr[1] << 8) + (bstr[2] << 16)
>>> hex(x)
'0x242424'

I'm not sure what you want to convert z1 into.

[pythoncoder]

@BOB63 The other option is to address the memory as a bytearray rather than an array of integers: see my response above to @Damien's suggestion. This avoids the need for conversion. You could copy the string to RAM and retrieve it verbatim. Assuming your string is a bytestring.

[dbc]

When this thread originally came up, I was deep into a project using struct.pack/unpack. That planted the seeds of an idea to riff off of @pythoncoder's very nice backup SRAM ideas with an automagic allocator and formatter based around struct.pack() and/or uctypes.struct().

Tonight I had some time sitting in a coffee shop waiting for my daugther's event to be over, so I spent a few minutes putting together this little backup SRAM manager.

You can allocate/access the SRAM in two ways:

1) Define a list of struct.pack() formats, and use that to create an instance of BackupSRAM which can then be accessed as a dictionary of tuples.

2) Define a uctypes.struct() instance which maps onto a region of backup SRAM.

You can intermix the two in any order. The actual layout depends on the order that you allocate your structures, and no layout information is saved in the SRAM, so if you re-arrange the order of your allocation statements (or any part of the definitions) and expect to get the same stuff back, you're gonna have a bad time.

Thanks to pythoncoder for doing the heavy lifting of datasheet research. The code is not heavily tested -- I submit my almond-milk-mocha-no-whip fueled hackery for your amusement.

Code: Select all

import pyb
import stm
import uctypes
import struct

class BackupSRAM():
    LOC = 0x40024000
    SIZE = 4096
    END = LOC+SIZE
    _next_available = LOC
    stm.mem32[stm.RCC + stm.RCC_APB1ENR] |= 0x10000000 # PWREN bit
    stm.mem32[stm.PWR + stm.PWR_CR] |= 0x100 # Set the DBP bit in the PWR power control register
    stm.mem32[stm.RCC +stm.RCC_AHB1ENR]|= 0x40000 # enable BKPSRAMEN
    stm.mem32[stm.PWR + stm.PWR_CSR] |= 0x200 # BRE backup register enable bit
    def __init__(self, format_list):
        self._dir = {}
        for name, fmt in format_list:
            sz = struct.calcsize(fmt)
            self._dir[name] = self._alot(sz), fmt, sz
    @classmethod
    def _alot(cls, size):
        t = cls._next_available
        if t + size > cls.END:
            raise ValueError
        cls._next_available += size
        return t
    @classmethod
    def as_ctype(cls, layout):
        nv_struct = uctypes.struct(cls._next_available, layout)
        sz = uctypes.sizeof(nv_struct)
        cls._alot(sz)
        return nv_struct # Give an instance of uctypes.Struct instead of BackupSRAM
    def __getitem__(self, index):
        loc,fmt,sz = self._dir[index]
        buf = uctypes.bytearray_at(loc,sz)
        return struct.unpack(fmt, buf)
    def __setitem__(self, index, value):
        loc,fmt,sz = self._dir[index]
        buf = uctypes.bytearray_at(loc,sz)
        t = struct.pack(fmt, *value)
        # Wouldn't it be nice to have struct.pack_into(...)?  Oh, well. 
        for i in range(sz):
            buf[i] = t[i]

bkram = BackupSRAM([('a','HH'),('b','IIBI')])
print (bkram._dir)
bkram['a'] = 22,33
print (bkram['a'])

bkram['b'] = (5,6,7,8)

ct = BackupSRAM.as_ctype({'a':uctypes.UINT32 | 0})
ct.a = 99


ct2 = BackupSRAM.as_ctype({'b':uctypes.UINT32 | 0, 'c':uctypes.UINT16 | 4})
ct2.b = 101
ct2.c = 139

bkram2 = BackupSRAM([('d','I')])
bkram2['d'] = 200,

print (bkram['a'], bkram['b'])
print (bkram2['d'])
print (ct2.b, ct2.c)
print (ct.a)

[pythoncoder]

Neat! Efficient and flexible. There is also the approach, alluded to by @Damien, of using JSON. This can convert a Python object of arbitrary complexity to a string which can then be stored in the bytearray. This is likely to be slower than your approach, but it does avoid having to use struct format specifiers. This advantage is offset by the fact that the saved string length could vary at runtime, whereas in your case the data size is evident at design time. For anyone interested, given the class:

Code: Select all

class BkpRAM(object):
    BKPSRAM = 0x40024000
    def __init__(self):
      stm.mem32[stm.RCC + stm.RCC_APB1ENR] |= 0x10000000 # PWREN bit
      stm.mem32[stm.PWR + stm.PWR_CR] |= 0x100 # Set the DBP bit in the PWR power control register
      stm.mem32[stm.RCC +stm.RCC_AHB1ENR]|= 0x40000 # enable BKPSRAMEN
      stm.mem32[stm.PWR + stm.PWR_CSR] |= 0x200 # BRE backup register enable bit
    def __getitem__(self, idx):
        assert idx >= 0 and idx <= 0x3ff, "Index must be between 0 and 1023"
        return stm.mem32[self.BKPSRAM + idx * 4]
    def __setitem__(self, idx, val):
        assert idx >= 0 and idx <= 0x3ff, "Index must be between 0 and 1023"
        stm.mem32[self.BKPSRAM + idx * 4] = val
    def get_bytearray(self):
        return uctypes.bytearray_at(self.BKPSRAM, 4096)

you can then do this:

Code: Select all

bram = BkpRAM()
ba = bram.get_bytearray()
a = {'rats':77, 'dogs':99,'elephants':9, 'zoo':100} # could be a much more complicated object
z = json.dumps(a).encode('utf8')
bram[0] = len(z)
ba[4: 4+len(z)] = z # Copy into backup RAM and go into standby

On recovery from standby:

Code: Select all

bram = BkpRAM()
ba = bram.get_bytearray()
a = json.loads(bytes(ba[4:4+bram[0]]).decode("utf-8")) # retrieve the object

The same should also be possible using pickle.loads() and pickle.dumps() rather than JSON though I haven't tried it yet.

[BOB63]

Hi pythoncoder,
I've tried to implement your solution using JSON on my project ( receive some data by xbee via rs323 and write into the mcu eeprom).
In short , from host using a Processing program i create a string with a structure required to be used with your example above , and i send using two xBee modules to the mpython board.
This is the string sent :
{'header':'$$$','enable_a':'Y','hrs_a':11,'mins_a':00,'hre_a':12,'mine_a':15,'lu_a':'Y','ma_a':'Y','me_a':'Y','gi_a':'Y','ve_a':'Y','sa_a':'Y','do_a':'Y','vsonda_a':0,'enable_b':'Y','hrs_b':13,'mins_b':18,'hre_b':18,'mine_b':45,'lu_b':'Y','ma_b':'Y','me_b':'Y','gi_b':'Y','ve_b':'Y','sa_b':'Y','do_b':'Y','vsonda_b':0,'set_a':2000,'set_b':2000,'yyyy_h':2015,'mm_h':12,'gg_h':12,'wd_h':7,'hr_h':23,'min_h':45,'flg_a':'N','flg_b':'N','footer':'###','x':0}

This is the code that I use to receive the data :

Code: Select all

while test==0:
      inBuffer_chr=""
      if uart.any(): 
         inBuffer=uart.readline()
                        
         if inBuffer!="":  
            uart.write(inBuffer+"\n")   #echo dato ricevuto
            print("Received data : ")  
            print(inBuffer)  

 
         if inBuffer_chr=="connecting":  
            print("Connesso")
            send_setting1()
            s=""
            x=0
         else:
            ba = bram.get_bytearray()

            z = json.dumps(inBuffer).encode('utf8') 

            print("Data after Json.dump :")
            print(z)

            bram[0] = len(z)
            ba[4: 4+len(z)] = z
            
            bt = json.loads(bytes(ba[4:4+bram[0]]).decode("utf-8")) # retrieve the object
            k=bt['enable_a']   ## <--- Error line
            print(k)

As you can see from the below screen shoot of PuTTY I receive the right data , but after the JSON.DUMP in the result there are included some "\" that were not sent , and when i try to address one of the elements I got the error.

To prove if the string sent was in such way not created correctly I've pasted the string into your example above and works ok.
Any idea what coul be wrong in the code ?

Cattura.JPG (61.1 KiB) Viewed 7613 times

[dhylands]

bt is a string (from json.loads). Trying to index into bt using a string 'enable_a' isn't valid.