upy C module generator

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
jgpeiro
Posts: 2
Joined: Wed Aug 05, 2015 8:27 am

upy C module generator

Post by jgpeiro » Sat Dec 21, 2019 7:47 pm

After some work I here is an experimental upy C module generator, useful for beginners who want compile new modules in C.
This generator takes a python api interface as an input and generates complete C template that can be compiled with the kernel.
The input can be any python function or class definition and the output is a set of structures and definitions ready to fill with user code.

Example:
Input:

Code: Select all

def foo():
	pass
Output:

Code: Select all

/****************************************************************************
 * File:	modfoo.c
 * Brief:		
  * Author:		
 ****************************************************************************/
/****************************************************************************
 * Private Includes.
 ****************************************************************************/
#include	"py/obj.h"
#include	"py/runtime.h"

/****************************************************************************
 * Private Defines.
 ****************************************************************************/
#define	MODULE_FOO_ENABLED	(1)
MP_REGISTER_MODULE(MP_QSTR_foo, foo_module, MODULE_FOO_ENABLED );

/****************************************************************************
 * Private Types.
 ****************************************************************************/

/****************************************************************************
 * Private Functions Declarations.
 ****************************************************************************/
STATIC mp_obj_t foo_foo();

/****************************************************************************
 * Private Variables Declarations.
 ****************************************************************************/
STATIC const mp_obj_fun_builtin_fixed_t foo_foo_obj;
const mp_map_elem_t foo_globals_table[];
STATIC const mp_obj_dict_t foo_globals;
const mp_obj_module_t foo_module;

/****************************************************************************
 * Private Variables Definitions.
 ****************************************************************************/
STATIC MP_DEFINE_CONST_FUN_OBJ_0( foo_foo_obj, foo_foo );
const mp_map_elem_t foo_globals_table[] = 
{
    { MP_OBJ_NEW_QSTR( MP_QSTR___name__ ), MP_OBJ_NEW_QSTR( MP_QSTR_foo ) },
    { MP_OBJ_NEW_QSTR( MP_QSTR_foo ), (mp_obj_t)&foo_foo_obj },
};
const mp_obj_module_t foo_module = 
{
    .base = { &mp_type_module },
    .globals = (mp_obj_dict_t*)&foo_globals,
};

/****************************************************************************
 * Public Variables Definitions.
 ****************************************************************************/
STATIC MP_DEFINE_CONST_DICT( foo_globals, foo_globals_table );

/****************************************************************************
 * Private Functions Definitions.
 ****************************************************************************/
STATIC mp_obj_t foo_foo()
{
    // ... 
    return mp_const_none;
}


/****************************************************************************
 * Public Functions Definitions.
 ****************************************************************************/


The generator is divided in three blocks.
- C template generator
- Python ast parser
- Module generator

The C generator is a tool that can be used to generate C code from python scritps.
Example:

Code: Select all

fb = FileBuilder( "fl_name" )
fb.addFunction( isPublic=True, varType="void", name="foo", params="", body="    return;" )
print( fb.getHeader() )
print( fb.getSource() )

Code: Select all

/****************************************************************************
 * File:	fl_name.h
 * Brief:		
 * Author:		
 ****************************************************************************/
#ifndef	_FL_NAME_H_
#define	_FL_NAME_H_
/****************************************************************************
 * Public Includes.
 ****************************************************************************/

/****************************************************************************
 * Public Defines.
 ****************************************************************************/

/****************************************************************************
 * Public Types.
 ****************************************************************************/

/****************************************************************************
 * Public Variables.
 ****************************************************************************/

/****************************************************************************
 * Public Functions.
 ****************************************************************************/
void foo();

#endif	//#ifndef	_FL_NAME_H_

Code: Select all

/****************************************************************************
 * File:	fl_name.c
 * Brief:		
 * Author:		
 ****************************************************************************/
/****************************************************************************
 * Private Includes.
 ****************************************************************************/
#include	"fl_name.h"

/****************************************************************************
 * Private Defines.
 ****************************************************************************/

/****************************************************************************
 * Private Types.
 ****************************************************************************/

/****************************************************************************
 * Private Functions Declarations.
 ****************************************************************************/

/****************************************************************************
 * Private Variables Declarations.
 ****************************************************************************/

/****************************************************************************
 * Private Variables Definitions.
 ****************************************************************************/

/****************************************************************************
 * Public Variables Definitions.
 ****************************************************************************/

/****************************************************************************
 * Private Functions Definitions.
 ****************************************************************************/

/****************************************************************************
 * Public Functions Definitions.
 ****************************************************************************/
void foo()
{
    return;
}
Here is the implementation of the C generator.

Code: Select all

(
    PUBLIC_STD_INCLUDES,
    PUBLIC_INCLUDES,
    PUBLIC_STD_INCLUDES,
    PUBLIC_INCLUDES,
    PUBLIC_DEFINES,
    PUBLIC_TYPES_DECLARATION,
    PUBLIC_TYPES_DEFINITION,
    PUBLIC_TYPEDEFS_DECLARATION,
    PUBLIC_TYPEDEFS_DEFINITION,
    PUBLIC_VARIABLES_DECLARATION,
    PUBLIC_FUNCTIONS_DECLARATION,
    
    PRIVATE_STD_INCLUDES,
    PRIVATE_INCLUDES,
    PRIVATE_DEFINES,
    PRIVATE_TYPES_DECLARATION,
    PRIVATE_TYPES_DEFINITION,
    PRIVATE_TYPEDEFS_DECLARATION,
    PRIVATE_TYPEDEFS_DEFINITION,
    PRIVATE_FUNCTIONS_DECLARATION,
    PRIVATE_VARIABLES_DECLARATION,
    PRIVATE_VARIABLES_DEFINITION,
    PUBLIC_VARIABLES_DEFINITION,
    PRIVATE_FUNCTIONS_DEFINITION,
    PUBLIC_FUNCTIONS_DEFINITION 
) = range(24)

class FileBuilder:
    def __init__( self, name = "", author = "", brief = "" ):
        self.comment = ""
        self.strings = []
        
        self.commentStart = "/****************************************************************************\n"
        self.commentEnd =   " ****************************************************************************/\n"
        
        self.name = name
        self.author = author
        self.brief = brief
        for i in range(24):
            self.strings.append("")
        
        self.addInclude( False, False, self.name + ".h" )

    def addComment( self, comment ):
        self.comment = "/*" + comment + "*/\n"

    def addInclude(self, isPublic, isStandard, fileName ):
        if (isPublic):
            if (isStandard):
                self.strings[PUBLIC_STD_INCLUDES] += self.comment + "#include\t<" + fileName + ">\n"
            else:
                self.strings[PUBLIC_INCLUDES] += self.comment + "#include\t\"" + fileName + "\"\n"
        else:
            if (isStandard):
                self.strings[PRIVATE_STD_INCLUDES] += self.comment + "#include\t<" + fileName + ">\n"
            else:
                self.strings[PRIVATE_INCLUDES] += self.comment + "#include\t\"" + fileName + "\"\n"
        self.comment = ""

    def addDefine( self, isPublic, identifier, replacement ):
        if (isPublic):
            self.strings[PUBLIC_DEFINES] += self.comment + "#define\t" + identifier + "\t(" + replacement + ")\n"
        else:
            self.strings[PRIVATE_DEFINES] += self.comment + "#define\t" + identifier + "\t(" + replacement + ")\n"
        self.comment = ""

    def addType( self, isPublic, varType, name, value ):
        if (isPublic):
            self.strings[PUBLIC_TYPES_DECLARATION] += varType + " " + name + ";\n"
            self.strings[PUBLIC_TYPES_DEFINITION] += self.comment + varType + " " + name + "\n{\n" + value + "\n};\n"
        else:
            self.strings[PRIVATE_TYPES_DECLARATION] += varType + " " + name + ";\n"
            self.strings[PRIVATE_TYPES_DEFINITION] += self.comment + varType + " " + name + "\n{\n" + value + "\n};\n"
        self.comment = ""

    def addTypedef( self, isPublic, varType, name ):
        if (isPublic):
            self.strings[PUBLIC_TYPEDEFS_DECLARATION] += "typedef " + varType + " " + name + ";\n"
            self.strings[PUBLIC_TYPEDEFS_DEFINITION] += self.comment + "typedef " + varType + " " + name + ";\n"
        else:
            self.strings[PRIVATE_TYPEDEFS_DECLARATION] += "typedef " + varType + " " + name + ";\n"
            self.strings[PRIVATE_TYPEDEFS_DEFINITION] += self.comment + "typedef " + varType + " " + name + ";\n"
        self.comment = "";

    def addVariable( self, isPublic, varType, name, value ):
        if (isPublic):
            self.strings[PUBLIC_VARIABLES_DECLARATION] += varType + " " + name + ";\n"
            self.strings[PUBLIC_VARIABLES_DEFINITION] += self.comment + varType + " " + name + " = " + value + ";\n"
        else:
            self.strings[PRIVATE_VARIABLES_DECLARATION] += varType + " " + name + ";\n"
            self.strings[PRIVATE_VARIABLES_DEFINITION] += self.comment + varType + " " + name + " = " + value + ";\n"
        self.comment = ""

    def addFunction( self, isPublic, varType, name, params, body ):
        if (isPublic):
            self.strings[PUBLIC_FUNCTIONS_DECLARATION] += varType + " " + name + "(" + params + ")" + ";\n"
            self.strings[PUBLIC_FUNCTIONS_DEFINITION] += self.comment + varType + " " + name + "(" + params + ")\n{\n" + body + "\n}\n\n"
        else:
            self.strings[PRIVATE_FUNCTIONS_DECLARATION] += "static " + varType + " " + name + "(" + params + ")" + ";\n"
            self.strings[PRIVATE_FUNCTIONS_DEFINITION] += self.comment + "static " + varType + " " + name + "(" + params + ")\n{\n" + body + "\n}\n\n"
        self.comment = ""

    def getHeader( self ):
        header = ""
        header += self.commentStart
        header += " * File:\t" + self.name + ".h\n"
        header += " * Brief:\t\t" + self.brief + "\n"
        header += " * Author:\t\t" + self.author + "\n"
        header += self.commentEnd
        header += "#ifndef\t_" + self.name.upper() + "_H_\n"
        header += "#define\t_" + self.name.upper() + "_H_"
        header += "\n"
        header += self.commentStart + " * Public Includes.\n" + self.commentEnd
        header += self.strings[PUBLIC_STD_INCLUDES]
        header += self.strings[PUBLIC_INCLUDES]
        header += "\n"
        header += self.commentStart + " * Public Defines.\n" + self.commentEnd
        header += self.strings[PUBLIC_DEFINES]
        header += "\n"
        header += self.commentStart + " * Public Types.\n" + self.commentEnd
        #header += self.strings[PUBLIC_TYPEDEFS_DECLARATION]
        header += self.strings[PUBLIC_TYPEDEFS_DEFINITION]
        #header += self.strings[PUBLIC_TYPES_DECLARATION]
        header += self.strings[PUBLIC_TYPES_DEFINITION]
        header += "\n"
        header += self.commentStart + " * Public Variables.\n" + self.commentEnd
        header += self.strings[PUBLIC_VARIABLES_DECLARATION]
        header += "\n"
        header += self.commentStart + " * Public Functions.\n" + self.commentEnd
        header += self.strings[PUBLIC_FUNCTIONS_DECLARATION]
        header += "\n"
        header += "#endif\t//#ifndef\t_" + self.name.upper() + "_H_"
        return header

    def getSource( self ):
        source = ""
        source += self.commentStart
        source += " * File:\t" + self.name + ".c\n"
        source += " * Brief:\t\t" + self.brief + "\n"
        source += " * Author:\t\t" + self.author + "\n"
        source += self.commentEnd
        source += self.commentStart + " * Private Includes.\n" + self.commentEnd
        source += self.strings[PRIVATE_STD_INCLUDES]
        source += self.strings[PRIVATE_INCLUDES]
        source += "\n"
        source += self.commentStart + " * Private Defines.\n" + self.commentEnd
        source += self.strings[PRIVATE_DEFINES]
        source += "\n"
        source += self.commentStart + " * Private Types.\n" + self.commentEnd
        #source += self.strings[PRIVATE_TYPEDEFS_DECLARATION]
        source += self.strings[PRIVATE_TYPEDEFS_DEFINITION]
        #source += self.strings[PRIVATE_TYPES_DECLARATION]
        source += self.strings[PRIVATE_TYPES_DEFINITION]
        source += "\n"
        source += self.commentStart + " * Private Functions Declarations.\n" + self.commentEnd
        source += self.strings[PRIVATE_FUNCTIONS_DECLARATION]
        source += "\n"
        source += self.commentStart + " * Private Variables Declarations.\n" + self.commentEnd
        source += self.strings[PRIVATE_VARIABLES_DECLARATION]
        source += "\n"
        source += self.commentStart + " * Private Variables Definitions.\n" + self.commentEnd
        source += self.strings[PRIVATE_VARIABLES_DEFINITION]
        source += "\n"
        source += self.commentStart + " * Public Variables Definitions.\n" + self.commentEnd
        source += self.strings[PUBLIC_VARIABLES_DEFINITION]
        source += "\n"
        source += self.commentStart + " * Private Functions Definitions.\n" + self.commentEnd
        source += self.strings[PRIVATE_FUNCTIONS_DEFINITION]
        source += "\n"
        source += self.commentStart + " * Public Functions Definitions.\n" + self.commentEnd
        source += self.strings[PUBLIC_FUNCTIONS_DEFINITION]
        source += "\n"
        return source
The python ast parser. This block uses python ast built-in module and a simple helper class that stores the tree.
Example:

Code: Select all

source_code = """
def foo():
    pass
"""

visitor = Visitor( 0 )
tree = ast.parse( source_code )
visitor.visit( tree )

print( visitor.static_attrs )
print( visitor.FunctionDefs )
print( visitor.ClassDefs )

Code: Select all

[]
[{'type': '', 'name': 'foo', 'args': []}]
[]
Here is the code:

Code: Select all

import ast

class Visitor( ast.NodeVisitor ):
    def __init__( self, dbg=0 ):
        print( "dbg", dbg )
        self.dbg = dbg
        self.ClassDefs = []
        self.FunctionDefs = []
        self.args = []
        self.static_attrs = []
        self.self_attrs = []
        self.iden = 0

    def generic_visit( self, node ):
        if( self.dbg == 2 ):
            print( " "*self.iden, type(node) )
        
        self.iden += 4
        ast.NodeVisitor.generic_visit( self, node )
        self.iden -= 4
        
    def visit_ClassDef(self, node):
        if( self.dbg >= 1 ):
            print( " "*self.iden, "class", node.name )

        tmp1 = [_ for _ in self.FunctionDefs]
        tmp2 = [_ for _ in self.static_attrs]
        self.FunctionDefs = []
        self.static_attrs = []
        
        self.iden += 4
        ast.NodeVisitor.generic_visit( self, node )
        self.iden -= 4

        self.ClassDefs += [ { "name":node.name, "static_attrs":self.static_attrs, "self_attrs":self.self_attrs, "functions":self.FunctionDefs } ]
        self.FunctionDefs = tmp1
        self.static_attrs = tmp2
        self.self_attrs = []
    
    def visit_FunctionDef(self, node):
        if( self.dbg >= 1 ):
            print( " "*self.iden, "def", node.name )
        
        self.iden += 4
        ast.NodeVisitor.generic_visit( self, node )
        self.iden -= 4
        self.FunctionDefs += [ { "type":"", "name":node.name, "args":self.args } ]
        self.args = []
        
    def visit_arg(self, node):
        if( self.dbg >= 1 ):
            print( " "*self.iden, "arg", node.arg )
        
        self.args += [ { "type":"", "name":node.arg } ]
        self.iden += 4
        ast.NodeVisitor.generic_visit( self, node )
        self.iden -= 4

    def visit_Name( self, node ):
        if( self.dbg >= 1 ):
            print( " "*self.iden, "name", node.id )
        
        if( node.id != "self" ):
            self.static_attrs += [ { "type":"", "name":node.id } ]
        
        self.iden += 4
        ast.NodeVisitor.generic_visit( self, node )
        self.iden -= 4
        
    def visit_Attribute( self, node ):
        if( self.dbg >= 1 ):
            print( " "*self.iden, "attr", node.attr )
        
        self.self_attrs += [ { "type":"", "name":node.attr } ]
        self.iden += 4
        ast.NodeVisitor.generic_visit( self, node )
        self.iden -= 4

Code: Select all

class Module:
    def __init__( self, name, attrs, functions, classes ):
        self.name = name
        self.attrs = attrs
        self.functions = functions
        self.classes = classes
    
    def show( self ):
        print( "name", module.name )
        
        for attr in module.attrs:
            print( "    ", attr["type"], attr["name"] )
        #print( "" )

        for function in module.functions:
            print( "    ", function["type"], function["name"] )
            for arg in function["args"]:
                print( "    ", "    ", arg["type"], arg["name"] )
            #print( "" )
        #print( "" )

        for cls in module.classes:
            print( "    ",cls["name"] )

            print( "    ","    ", "static_attrs" )
            for attr in cls["static_attrs"]:
                print( "    ","    ", "    ", attr["type"], attr["name"] )
            #print( "" )

            print( "    ","    ", "self_attrs" )
            for attr in cls["self_attrs"]:
                print( "    ","    ", "    ", attr["type"], attr["name"] )
            #print( "" )

            print( "    ", "    ", "functions" )
            for function in cls["functions"]:
                print( "    ","    ", "    ", function["type"], function["name"] )
                for arg in function["args"]:
                    print( "    ","    ", "    ", "    ", arg["type"], arg["name"] )
            #    print( "" )
            #print( "" )
The third block does the main job. It takes the tree from the ast and uses the C module generator to build our C code. The result is a functional dummy that can be integrated and compiled with the micropython kernel (tested on v1.11).
The code:

Code: Select all

def build_class_foreing_type( fb, cls ):
    isPublic = False
    varType = "struct"
    name = "s" + cls["name"].capitalize()
    value = ""
    #value  = "{\n"
    value += "    mp_obj_base_t base;\n"
    value += "    // ..."
    #value += "}\n"
    fb.addType( isPublic, varType, name, value )
    
    isPublic = False
    varType = "struct" + " s" + cls["name"].capitalize()
    name = "t" + cls["name"].capitalize()
    fb.addTypedef( isPublic, varType, name )
    
def build_class_make_new( fb, cls ):
    assert( cls["functions"][0]["name"] == "__init__" )
    args = cls["functions"][0]["args"]
    
    isPublic = False
    varType = "mp_obj_t"
    name = cls["name"] + "_make_new"
    params = " const mp_obj_type_t *type_in, size_t n_args, size_t n_kw, const mp_obj_t *args "

    body  = ""
    body += "    mp_arg_check_num( n_args, n_kw, " + str(len(args)) + ", " + str(len(args)) + ", false );\n"
    body += "    t" + cls["name"].capitalize() + " *self = m_new_obj( t" + cls["name"].capitalize() + " );\n"
    body += "    self->base.type = &" + cls["name"] + "_type;\n"
    #body += "    // args = " + str( [ arg["type"] + " " + arg["name"] for arg in args[1:]] ) + "\n"
    body += "    // ...\n"
    body += "    return MP_OBJ_FROM_PTR( self );"

    fb.addFunction( isPublic, varType, name, params, body )

def build_class_print( fb, cls ):
    args = cls["functions"][0]["args"]
    
    isPublic = False
    varType = "void"
    name = cls["name"] + "_print"
    params = " const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind "
    
    body  = ""
    body += "    (void)kind;\n"
    body += "    t" + cls["name"].capitalize() + " *self = MP_OBJ_TO_PTR( self_in );\n"
    body += "    mp_printf( print, \"" + cls["name"] + "=%x\\n\", (int)self );"

    fb.addFunction( isPublic, varType, name, params, body )

def build_class_function_reference( fb, cls, function ):
    string  = "static MP_DEFINE_CONST_FUN_OBJ_" + str( len( function["args"] ) )
    string += "( " + cls["name"] + "_" + function["name"] + "_obj, " + cls["name"] + "_" + function["name"] + " );\n"
    fb.strings[PRIVATE_VARIABLES_DEFINITION] += string  
    string = "static const mp_obj_fun_builtin_fixed_t " + cls["name"] + "_" + function["name"] + "_obj;\n"
    fb.strings[PRIVATE_VARIABLES_DECLARATION] += string

def build_class_function( fb, cls, function ):
    isPublic = False
    varType = "mp_obj_t"
    name = cls["name"] + "_" + function["name"]
    params = ""
    for i, arg in enumerate( function["args"] ):
        if( i > 0 ):
            params += ", "
        params += "mp_obj_t " + arg["name"] + "_obj"
    
    body  = ""
    body += "    t" + cls["name"].capitalize() + " *self = MP_OBJ_TO_PTR( self_obj );\n"
    body += "    (void)self;\n"
    body += "    // ... \n"
    body += "    return mp_const_none;" # + " // " + function["type"]

    fb.addFunction( isPublic, varType, name, params, body )
    
    build_class_function_reference( fb, cls, function )

def build_class_locals_table_reference( fb, cls ):
    string  = "static MP_DEFINE_CONST_DICT"
    string += "( " + cls["name"] + "_locals_dict, " + cls["name"] + "_locals_dict_table );\n"
    fb.strings[PUBLIC_VARIABLES_DEFINITION] += string
    string = "static const mp_obj_dict_t " + cls["name"] + "_locals_dict;\n"
    fb.strings[PRIVATE_VARIABLES_DECLARATION] += string
    
def build_class_locals_table( fb, cls ):
    
    isPublic = False
    varType = "const mp_rom_map_elem_t"
    name = cls["name"] + "_locals_dict_table[]"
    
    value = "\n{\n"
    for function in cls["functions"][1:]:
        value += "    { MP_ROM_QSTR( MP_QSTR_" + function["name"] + " ),"
        value += " MP_ROM_PTR( &" + cls["name"] + "_" + function["name"] + "_obj) },\n"
    value += "}"
    
    fb.addVariable( isPublic, varType, name, value )
    
    build_class_locals_table_reference( fb, cls )

def build_class_type( fb, cls ):
    isPublic = False
    varType = "const mp_obj_type_t"
    name = cls["name"] + "_type"
    
    value  = "\n{\n"
    value += "    { &mp_type_type },\n"
    value += "    .name = MP_QSTR_" + cls["name"] + ",\n"
    value += "    .make_new = " + cls["name"] + "_make_new,\n"
    value += "    .print = " + cls["name"] + "_print,\n"
    value += "    .locals_dict = (mp_obj_dict_t*)&" + cls["name"] + "_locals_dict,\n"
    value += "}"
    
    fb.addVariable( isPublic, varType, name, value )

def build_module_globals_table_reference( fb, module ):
    string  = "static MP_DEFINE_CONST_DICT"
    string += "( " + module.name + "_globals, " + module.name + "_globals_table );\n"
    fb.strings[PUBLIC_VARIABLES_DEFINITION] += string
    string = "static const mp_obj_dict_t " + module.name + "_globals;\n"
    fb.strings[PRIVATE_VARIABLES_DECLARATION] += string

def build_module_globals_table( fb, module ):
    isPublic = False
    varType = "const mp_map_elem_t"
    name = module.name + "_globals_table[]"
    
    value  = "\n{\n"
    value += "    { MP_OBJ_NEW_QSTR( MP_QSTR___name__ ),"
    value += " MP_OBJ_NEW_QSTR( MP_QSTR_" + module.name + " ) },\n"
    for attr in module.attrs:
        value += "    // { MP_OBJ_NEW_QSTR( MP_QSTR_" + attr["name"] + " ),"
        value += " (mp_obj_t)&" + module.name + "_" +attr["name"] + "_obj },\n"
    for function in module.functions:
        value += "    { MP_OBJ_NEW_QSTR( MP_QSTR_" + function["name"] + " ),"
        value += " (mp_obj_t)&" + module.name + "_" + function["name"] + "_obj },\n"
    for cls in module.classes:
        value += "    { MP_OBJ_NEW_QSTR( MP_QSTR_" + cls["name"] + " ),"
        value += " (mp_obj_t)&" + cls["name"] + "_type },\n"
    value += "}"
    
    fb.addVariable( isPublic, varType, name, value )
    build_module_globals_table_reference( fb, module )

def build_module_object_reference( fb, module ):
    string  = "MP_REGISTER_MODULE"
    string += "(MP_QSTR_" + module.name + ", " + module.name + "_module,"
    string += " MODULE_" + module.name.upper() + "_ENABLED );\n"
    fb.strings[PRIVATE_DEFINES] += string

def build_module_object( fb, module ):
    isPublic = False
    varType = "const mp_obj_module_t"
    name = module.name + "_module"
    
    value  = "\n{\n"
    value += "    .base = { &mp_type_module },\n"
    value += "    .globals = (mp_obj_dict_t*)&" + module.name + "_globals,\n"
    value += "}"
    
    fb.addVariable( isPublic, varType, name, value )
    build_module_object_reference( fb, module )

def build_module_function_reference( fb, module, function ):
    string  = "static MP_DEFINE_CONST_FUN_OBJ_" + str( len( function["args"] ) )
    string += "( " + module.name + "_" + function["name"] + "_obj, " + module.name + "_" + function["name"] + " );\n"
    fb.strings[PRIVATE_VARIABLES_DEFINITION] += string
    string = "static const mp_obj_fun_builtin_fixed_t " + module.name + "_" + function["name"] + "_obj;\n"
    fb.strings[PRIVATE_VARIABLES_DECLARATION] += string

def build_module_function( fb, module, function ):
    isPublic = False
    varType = "mp_obj_t"
    name = module.name + "_" + function["name"]
    params = ""
    for i, arg in enumerate( function["args"] ):
        if( i > 0 ):
            params += ", "
        params += "mp_obj_t " + arg["name"] + "_obj"
    
    body  = ""
    body += "    // ... \n"
    body += "    return mp_const_none;" # + " // " + function["type"]

    fb.addFunction( isPublic, varType, name, params, body )
    
    build_module_function_reference( fb, module, function )
    return

def build_integration_instructions( fb, module ):
    comment = "\n\n"
    comment += "integration steps:\n"
    comment += "1.\n"
    comment += "file: mpconfigport.h\n"
    comment += "line: // extra built in modules to add to the list\n"
    comment += "add : extern const struct _mp_obj_module_t " + module.name + "_module;\n"
    comment += "2.\n"
    comment += "file: mpconfigport.h\n"
    comment += "line: // #define MICROPY_PORT_BUILTIN_MODULES\n"
    comment += "add : { MP_ROM_QSTR(MP_QSTR_" + module.name + "), (mp_obj_t)&" + module.name + "_module }, \n"
    comment += "3.\n"
    comment += "file: makefile.mk\n"
    comment += "line: // SRC_C =  \n"
    comment += "add : " + "mod" + module.name + ".c \n"
    fb.brief = comment

def build_module_includes( fb, module ):
    fb.addInclude( False, False, "py/obj.h" )
    fb.addInclude( False, False, "py/runtime.h" )
    
def build_module_enable( fb, module ):
    isPublic = False 
    identifier = "MODULE_" + module.name.upper() + "_ENABLED"
    replacement = "1"
    fb.addDefine( isPublic, identifier, replacement )

def build_class_include( fb, cls ):
    fb.strings[PRIVATE_INCLUDES] += "// #include\t\"" + cls["name"]+".h" + "\"\n"
    
def build_module( module ):
    fb = FileBuilder( "mod" + module.name  )
    
    build_integration_instructions( fb, module )
    build_module_includes( fb, module )
    build_module_enable( fb, module )
    
    for cls in module.classes:
        build_class_include( fb, cls )
        build_class_foreing_type( fb, cls )
        build_class_make_new( fb, cls )
        build_class_print( fb, cls )
        for function in cls["functions"][1:]:
            build_class_function( fb, cls, function )
        build_class_locals_table( fb, cls )
        build_class_type( fb, cls )

    for function in module.functions:
        build_module_function( fb, module, function )
    
    build_module_globals_table( fb, module )
    build_module_object( fb, module )
    
    c_header = fb.getHeader()
    c_source = fb.getSource().replace( "static", "STATIC" )
    
    return c_header, c_source

Finally a complete example:
Input:

Code: Select all

source_code = """
class Device:
    def __init__( self ):
        pass
    
    def read_from( self, buf, timeout ):
        pass
    
    def write_into( self, buf, timeout ):
        pass
"""

visitor = Visitor()
tree = ast.parse( source_code )
visitor.visit( tree )

module = Module( "device", visitor.static_attrs, visitor.FunctionDefs, visitor.ClassDefs )
c_header, c_source = build_module( module )
print( c_header )
print( c_source )
Output:

Code: Select all

/****************************************************************************
 * File:	moddevice.h
 * Brief:		

integration steps:
1.
file: mpconfigport.h
line: // extra built in modules to add to the list
add : extern const struct _mp_obj_module_t device_module;
2.
file: mpconfigport.h
line: // #define MICROPY_PORT_BUILTIN_MODULES
add : { MP_ROM_QSTR(MP_QSTR_device), (mp_obj_t)&device_module }, 
3.
file: makefile.mk
line: // SRC_C =  
add : moddevice.c 

 * Author:		
 ****************************************************************************/
#ifndef	_MODDEVICE_H_
#define	_MODDEVICE_H_
/****************************************************************************
 * Public Includes.
 ****************************************************************************/

/****************************************************************************
 * Public Defines.
 ****************************************************************************/

/****************************************************************************
 * Public Types.
 ****************************************************************************/

/****************************************************************************
 * Public Variables.
 ****************************************************************************/

/****************************************************************************
 * Public Functions.
 ****************************************************************************/

#endif	//#ifndef	_MODDEVICE_H_

Code: Select all

/****************************************************************************
 * File:	moddevice.c
 * Brief:		

integration steps:
1.
file: mpconfigport.h
line: // extra built in modules to add to the list
add : extern const struct _mp_obj_module_t device_module;
2.
file: mpconfigport.h
line: // #define MICROPY_PORT_BUILTIN_MODULES
add : { MP_ROM_QSTR(MP_QSTR_device), (mp_obj_t)&device_module }, 
3.
file: makefile.mk
line: // SRC_C =  
add : moddevice.c 

 * Author:		
 ****************************************************************************/
/****************************************************************************
 * Private Includes.
 ****************************************************************************/
#include	"moddevice.h"
#include	"py/obj.h"
#include	"py/runtime.h"
// #include	"Device.h"

/****************************************************************************
 * Private Defines.
 ****************************************************************************/
#define	MODULE_DEVICE_ENABLED	(1)
MP_REGISTER_MODULE(MP_QSTR_device, device_module, MODULE_DEVICE_ENABLED );

/****************************************************************************
 * Private Types.
 ****************************************************************************/
typedef struct sDevice tDevice;
struct sDevice
{
    mp_obj_base_t base;
    // ...
};

/****************************************************************************
 * Private Functions Declarations.
 ****************************************************************************/
STATIC mp_obj_t Device_make_new( const mp_obj_type_t *type_in, size_t n_args, size_t n_kw, const mp_obj_t *args );
STATIC void Device_print( const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind );
STATIC mp_obj_t Device_read_from(mp_obj_t self_obj, mp_obj_t buf_obj, mp_obj_t timeout_obj);
STATIC mp_obj_t Device_write_into(mp_obj_t self_obj, mp_obj_t buf_obj, mp_obj_t timeout_obj);

/****************************************************************************
 * Private Variables Declarations.
 ****************************************************************************/
STATIC const mp_obj_fun_builtin_fixed_t Device_read_from_obj;
STATIC const mp_obj_fun_builtin_fixed_t Device_write_into_obj;
const mp_rom_map_elem_t Device_locals_dict_table[];
STATIC const mp_obj_dict_t Device_locals_dict;
const mp_obj_type_t Device_type;
const mp_map_elem_t device_globals_table[];
STATIC const mp_obj_dict_t device_globals;
const mp_obj_module_t device_module;

/****************************************************************************
 * Private Variables Definitions.
 ****************************************************************************/
STATIC MP_DEFINE_CONST_FUN_OBJ_3( Device_read_from_obj, Device_read_from );
STATIC MP_DEFINE_CONST_FUN_OBJ_3( Device_write_into_obj, Device_write_into );
const mp_rom_map_elem_t Device_locals_dict_table[] = 
{
    { MP_ROM_QSTR( MP_QSTR_read_from ), MP_ROM_PTR( &Device_read_from_obj) },
    { MP_ROM_QSTR( MP_QSTR_write_into ), MP_ROM_PTR( &Device_write_into_obj) },
};
const mp_obj_type_t Device_type = 
{
    { &mp_type_type },
    .name = MP_QSTR_Device,
    .make_new = Device_make_new,
    .print = Device_print,
    .locals_dict = (mp_obj_dict_t*)&Device_locals_dict,
};
const mp_map_elem_t device_globals_table[] = 
{
    { MP_OBJ_NEW_QSTR( MP_QSTR___name__ ), MP_OBJ_NEW_QSTR( MP_QSTR_device ) },
    { MP_OBJ_NEW_QSTR( MP_QSTR_Device ), (mp_obj_t)&Device_type },
};
const mp_obj_module_t device_module = 
{
    .base = { &mp_type_module },
    .globals = (mp_obj_dict_t*)&device_globals,
};

/****************************************************************************
 * Public Variables Definitions.
 ****************************************************************************/
STATIC MP_DEFINE_CONST_DICT( Device_locals_dict, Device_locals_dict_table );
STATIC MP_DEFINE_CONST_DICT( device_globals, device_globals_table );

/****************************************************************************
 * Private Functions Definitions.
 ****************************************************************************/
STATIC mp_obj_t Device_make_new( const mp_obj_type_t *type_in, size_t n_args, size_t n_kw, const mp_obj_t *args )
{
    mp_arg_check_num( n_args, n_kw, 1, 1, false );
    tDevice *self = m_new_obj( tDevice );
    self->base.type = &Device_type;
    // ...
    return MP_OBJ_FROM_PTR( self );
}

STATIC void Device_print( const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind )
{
    (void)kind;
    tDevice *self = MP_OBJ_TO_PTR( self_in );
    mp_printf( print, "Device=%x\n", (int)self );
}

STATIC mp_obj_t Device_read_from(mp_obj_t self_obj, mp_obj_t buf_obj, mp_obj_t timeout_obj)
{
    tDevice *self = MP_OBJ_TO_PTR( self_obj );
    (void)self;
    // ... 
    return mp_const_none;
}

STATIC mp_obj_t Device_write_into(mp_obj_t self_obj, mp_obj_t buf_obj, mp_obj_t timeout_obj)
{
    tDevice *self = MP_OBJ_TO_PTR( self_obj );
    (void)self;
    // ... 
    return mp_const_none;
}


/****************************************************************************
 * Public Functions Definitions.
 ****************************************************************************/


ihornehrutsa
Posts: 9
Joined: Sat Oct 26, 2019 8:38 pm

Re: upy C module generator

Post by ihornehrutsa » Thu Oct 15, 2020 6:54 am

Could you publish your code on GitHub?
Thanks.

User avatar
pythoncoder
Posts: 4571
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: upy C module generator

Post by pythoncoder » Thu Oct 15, 2020 9:46 am

Peter Hinch

ihornehrutsa
Posts: 9
Joined: Sat Oct 26, 2019 8:38 pm

Re: upy C module generator

Post by ihornehrutsa » Thu Oct 15, 2020 1:29 pm

Thanks, Peter.

I try to contribute to
https://github.com/prusnak/micropython-extmod-generator
and
https://github.com/pazzarpj/micropython-ustubby
to make a fully functional tool for developing MicroPython external C modules.

module.py -> GENERATOR -> module.c -> COMPILER -> firmware.bin -> MicroPython MCU board -> import module

Post Reply