Can be stored data inside non volative memory ?

[BOB63]

Hi dhylands,
why using the same approach in this case it works ?

Code: Select all

class BkpRAM(object):
    BKPSRAM = 0x40024000
    def __init__(self):
      stm.mem32[stm.RCC + stm.RCC_APB1ENR] |= 0x10000000 # PWREN bit
      stm.mem32[stm.PWR + stm.PWR_CR] |= 0x100 # Set the DBP bit in the PWR power control register
      stm.mem32[stm.RCC +stm.RCC_AHB1ENR]|= 0x40000 # enable BKPSRAMEN
      stm.mem32[stm.PWR + stm.PWR_CSR] |= 0x200 # BRE backup register enable bit
    def __getitem__(self, idx):
        assert idx >= 0 and idx <= 0x3ff, "Index must be between 0 and 1023"
        return stm.mem32[self.BKPSRAM + idx * 4]
    def __setitem__(self, idx, val):
        assert idx >= 0 and idx <= 0x3ff, "Index must be between 0 and 1023"
        stm.mem32[self.BKPSRAM + idx * 4] = val
    def get_bytearray(self):
        return uctypes.bytearray_at(self.BKPSRAM, 4096)


bram = BkpRAM()
ba = bram.get_bytearray()

a={'header':'$$$','enable_a':'N','hrs_a':0,'mins_a':0,'hre_a':0,'mine_a':0,'lu_a':'N','ma_a':'N','me_a':'N','gi_a':'N','ve_a':'N','sa_a':'N','do_a':'N','vsonda_a':0,'enable_b':'N','hrs_b':0,'mins_b':0,'hre_b':0,'mine_b':0,'lu_b':'N','ma_b':'N','me_b':'N','gi_b':'N','ve_b':'N','sa_b':'N','do_b':'N','vsonda_b':0,'set_a':2000,'set_b':2000,'yyyy_h':2015,'mm_h':12,'gg_h':12,'wd_h':0,'hr_h':22,'min_h':16,'flg_a':'N','flg_b':'N','footer':'###','x':0}
z = json.dumps(a).encode('utf8')

print(z)
bram[0] = len(z)

ba[4: 4+len(z)] = z # Copy into backup RAM and go into standby

bram = BkpRAM()
ba = bram.get_bytearray()
b = json.loads(bytes(ba[4:4+bram[0]]).decode("utf-8")) # retrieve the object
k=b['header']
print(k)

[dhylands]

Bah - no I was wrong. json.loads returns a dictionary. That's what I get for answering questions using my phone while waiting to board a plane.

I'll take a more detailed look tomorrow...

[pythoncoder]

Why use JSON to store a bytes object? The purpose of JSON or Pickle is to convert arbitrary objects into strings to enable them to be stored on byte oriented devices - this is a process known as serialisation. You already have a bytes object: there is no need to serialise it. So I'd do something like this:

Code: Select all

bram[0] = len(inBuffer)
ba[4: 4+len(inBuffer)] = inBuffer
# To retrieve
bt = bytes(ba[4:4+bram[0]])
 

Note I haven't tested this. I appreciate this doesn't answer the question of why your code doesn't work: I don't honestly know.

[pythoncoder]

To add to my above post I've done the following tests under cPython.

Code: Select all

>>> a = bytes(x for x in range(256))
>>> z = json.dumps(a).encode('utf8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.4/json/encoder.py", line 192, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.4/json/encoder.py", line 250, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.4/json/encoder.py", line 173, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff' is not JSON serializable

So it would seem that MicroPython is failing to throw an exception here. Interestingly Pickle works:

Code: Select all

>>> z = pickle.dumps(a)
>>> x = pickle.loads(z)
>>> for t in zip(a,x):
...  if t[0] != t[1]:
...   print(t[0],t[1])
... 
>>> 

In other words the unpickled bytes object is identical to the original. I repeated this test successfully on the Pyboard. Conclusions:

• JSON can't serialise arbitrary bytes objects (Pickle can).
• Cpython throws an exception if JSON fails. MicroPython does not. I will raise an issue.
• For this and other reasons I'll change the example in the micropower document in favour of Pickle.
• My above message stands: in @BOB63's instance there is no need to serialise data from a UART. However he has exposed a genuine issue. The point of object serialisation (in my view) is that it should work with arbitrary objects. In this respect JSON fails and Pickle succeeeds.

[BOB63]

pythoncoder,
Thanks again for your efforts to find and answer to my issues.
The reason why I'm interested to use this method on data received from UART is that I've never used before the dictionary structures and I find it very smart and easy to use. For this reason I would like receive the data , store in memory and put into a dictionary to be used in other tasks of my program.
Looking forward to see your new solutions.
Thanks. Roberto.

[pythoncoder]

OK, you have two choices: put the data from the UART into a dictionary and store the dictionary in backup RAM, or store the raw bytes from the UART in backup RAM directly, putting it in the dictionary when you restore.

I prefer the second approach but either will work.

I've updated my write up on micropower operation to use pickle instead of JSON. As you've discovered some objects can't be serialised using JSON; MicroPython is less helpful than cPython in that it doesn't raise an exception if you try to serialise an object it can't handle. In my brief testing Pickle is able to handle arbitrary bytes and bytearray objects whether stand-alone or stored in a structure like a dictionary. It's also (in my opinion) more "Pythonic". The principal merit of JSON is compatibility with other languages or applications which, though relevant if storing objects in a file is irrelevant in our applications with data stored in RAM. You'll need to get the pickle code from https://github.com/micropython/micropython-lib

So, if you want to adopt the first approach and store a dictionary, have a look at my updated write-up. Your code should work with pickle

[BOB63]

Yes I'm using the first method. With picke it works as expected and I recover data using the dictionary keys.
Great!
Thanks as usual . Roberto

[BOB63]

Well,at the end I've been able to complete my project and all work fine as expected. Now looking to the code I think that could be improved,because as is said before, my intention was use dictionary to send/receive data between pyBOARD and PC application storing the dictionary into EEPROM to be restored if required. My expectation was restore the data from EEPROM and use in any part of program without the need to associate each value to a global variable . What happen with my code is that inside a function " restore_data() " I restore the dictionary from the EEPROM but if I want use the values outside the function I need define global variable to which associate the data from the dictionary, otherwise outside the "restore_data" function I can't recover any output directly from the dictionary.I've also tried inside the function to return as output the dictionary but doesn't works.Have data already stored into a dictionary and need store the same data into variables don't seems to be so smart to me
Where I'm wrong and how I could write a better code?
Thanks Roberto.

[pythoncoder]

I think you need to post some code if we're to figure out why you can't access the dictionary. On the face of it returning the object should work.

[BOB63]

Hi,pythoncoder,
while I was posting the code I've discovered my mistake !!
I was using the wrong instruction print(saved_data('footer')) instead than the right one print(saved_data['footer']).
This produce the error : TypeError: 'dict' object is not callable
I've interpreted the error like the object was not accessible and lost some hours around this stupid error !