C data structure to python dictionary

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
learnerforlife
Posts: 13
Joined: Mon Sep 17, 2018 5:38 am

C data structure to python dictionary

Post by learnerforlife » Wed Sep 26, 2018 6:09 am

Hi, Can anyone tell if there is a way to map a data structure from C to a dictionary or maybe a JSON object in python. In my case specifically the data structure is a linked list containing three variables - name, data type and a union containing the data. I want to be able to pass this list to python as a dictionary or any other way if possible and access the data by using the names as the keys. My first thought was maybe it could be done with uctypes.structs but since it defines members using offset from a starting memory address (correct me if I have misunderstood), I'm not sure how to do it.

OutoftheBOTS_
Posts: 627
Joined: Mon Nov 20, 2017 10:18 am

Re: C data structure to python dictionary

Post by OutoftheBOTS_ » Wed Sep 26, 2018 6:28 am

you can unpack C types in python using the struct.unpack method see https://docs.python.org/3/library/struct.html

User avatar
pythoncoder
Posts: 3503
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: C data structure to python dictionary

Post by pythoncoder » Wed Sep 26, 2018 12:05 pm

There is also uctypes.
Peter Hinch

learnerforlife
Posts: 13
Joined: Mon Sep 17, 2018 5:38 am

Re: C data structure to python dictionary

Post by learnerforlife » Wed Sep 26, 2018 12:13 pm

Are you suggesting that I pass the memory address of the linked list head to python, read the bytearray at the address, map it into a ustruct, get the second node's address from the ustruct and repeat the previous steps till there is no more items in list and build a dictionary out of those structs.
If its correct I have two questions:
1. Is there a more elegant way to do this than what I just described?
2. I don't want to make the copy of those variables in the c struct inside python, I would rather prefer to refer to the same memory address for saving some memory but the method I just described doesn't meet this requirement.

shawwwn
Posts: 16
Joined: Tue Feb 06, 2018 5:22 am

Re: C data structure to python dictionary

Post by shawwwn » Wed Sep 26, 2018 1:21 pm

Yes, the elegant way is in @pythoncoder's mention.
I haven't look into uctypes' implementation, but I'm sure it just involve memory mapping instead of copying(as its name suggested ;) ).
You can refer to this module about how to use uctypes.

To avoid copy when slicing an array, you may also find memoryview useful.

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

C data structure to python dictionary

Post by jickster » Wed Sep 26, 2018 2:11 pm

Are you going to be using the C API in part?

Show us the struct definition

Sent from my iPhone using Tapatalk Pro

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: C data structure to python dictionary

Post by jickster » Wed Sep 26, 2018 6:03 pm

OutoftheBOTS_ wrote:
Wed Sep 26, 2018 6:28 am
you can unpack C types in python using the struct.unpack method see https://docs.python.org/3/library/struct.html
Is he doing this in C or Python?

OutoftheBOTS_
Posts: 627
Joined: Mon Nov 20, 2017 10:18 am

Re: C data structure to python dictionary

Post by OutoftheBOTS_ » Wed Sep 26, 2018 10:40 pm

jickster wrote:
Wed Sep 26, 2018 6:03 pm
OutoftheBOTS_ wrote:
Wed Sep 26, 2018 6:28 am
you can unpack C types in python using the struct.unpack method see https://docs.python.org/3/library/struct.html
Is he doing this in C or Python?
I am having a little trouble understanding exactly what the OP wants to do but to me it appears he has some data that is in C data types and he wants to access it in python. I have many times had it where I am receiving from and external source (SPI, I2C, UART) C type data that I need to use in my Python script. I unpack that C type data in to Python type data using ustruct.unpack(). Later in the thread the OP expresses that he doesn't want to make a copy of the data but using unpack will certainly make a copy.

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: C data structure to python dictionary

Post by jickster » Wed Sep 26, 2018 10:45 pm

If the data is in C then he needs to use the C API to expose the data in Python.

Right?


Sent from my iPhone using Tapatalk Pro

learnerforlife
Posts: 13
Joined: Mon Sep 17, 2018 5:38 am

Re: C data structure to python dictionary

Post by learnerforlife » Thu Sep 27, 2018 4:34 am

Hi sorry if I was not clear, I am not a very experienced programmer, let me explain in detail.
We have a device which acts like a IoT gateway. we already have all the data acquisition part done in C. The controllers that we are querying the data from are mainly communicating over Modbus protocol. So after we have queried the data, it is passed through a decoding logic which reads from the byte arrays received from Modbus controllers and interprets them and returns a linked list which contains parameter names and their values.
ex. {name:voltage, value:250}->{name:curent, value:10}->{...}...
And this list could be of variable length.

Now we want to expose this functionality for writing the actual business logic in python to a third party who may not know C programming at all. So I need this data to be available in python in an easily accessible way (by easy I mean he shouldn't have to worry about where its coming from or how to map it in python) say like a dictionary, so the person writing logic could do something like device['voltage'] and he can get the voltage value.

If it was a single struct variable I could easily map it but what I am having problem understanding is how to do it for a list, which is also going to be variable in length.
Also following is the structure I am using in C.

typedef union var_data
{
int8_t int8_val;
uint8_t uint8_val;
int16_t int16_val;
uint16_t uint16_val;
int32_t int32_val;
uint32_t uint32_val;
long long_val;
float float_val;
char *string_val;

} var_data_t;


typedef struct data_list
{

char *prop_name;
var_data_t data;
struct data_list *next;
uint8_t data_type;
} data_list_t;

Post Reply