incomplete import

[iconoclastica]

Having worked with python for years, I am now trying my luck with micro-python on an ESP32 lite module. I want the code I am having trouble with now to run both in ython 3.4 on the pc as well as on the ESP board (i.e. Lolin32 lite). To overcome the last differences, I tried to mimic a #IFDEF/#ELSE construction known from c-like programming languages.

On the pc, this runs well. On the board, however, importing the module makes available only some of the units, classes, and functions. I think it varies per session which members are available or not, but I cannot confirm this yet for sure. The module I am importing is listed at the end of this question.

After importing he unit in Python on the pc and running

for v in [m for m in sorted(dir(platformSpecific)) if not m.startswith('_')]:
print("%-20s : %s" % (v, getattr(platformSpecific, v)))

I get this result, which is what I expect:

Code: Select all

    AutoNumber           : <enum 'AutoNumber'>
    EMBEDDED             : False
    Enum                 : <enum 'Enum'>
    getDeltaTime         : <function getDeltaTime at 0x02782F60>
    getTicks_ms          : <function getTicks_ms at 0x02732588>
    initializestatic     : <function initializestatic at 0x02793930>
    json                 : <module 'json' from 'C:\\Program Files (x86)\\python\\Anaconda3\\lib\\json\\__init__.py'>
    machine              : isoetes
    os                   : <module 'os' from 'C:\\Program Files (x86)\\python\\Anaconda3\\lib\\os.py'>
    platform             : <module 'platform' from 'C:\\Program Files (x86)\\python\\Anaconda3\\lib\\platform.py'>
    programmingLanguage  : cpython
    sockets              : <module 'socket' from 'C:\\Program Files (x86)\\python\\Anaconda3\\lib\\socket.py'>
    sys                  : <module 'sys' (built-in)>
    sysname              : nt
    time                 : <module 'time' (built-in)>

The same in micro-python results in:

Code: Select all

    AutoNumber           : <class 'AutoNumber'>
    EMBEDDED             : True
    SortOfEnum           : <class 'SortOfEnum'>
    json                 : <module 'ujson'>
    machine              : ESP32 module with ESP32
    os                   : <module 'uos'>
    programmingLanguage  : micropython
    sockets              : <module 'usocket'>
    sys                  : <module 'sys'>
    sysname              : esp32
    time                 : <module 'utime'>

Especially a set of two 'get'-functions and a decorator are missing. This is not consistent though.

For a test, I forced the EMBEDDED variable to true (and made a few minor changes to avoid non-existing units and functions). Then the get-functions are exported as expected:

Code: Select all

    AutoNumber           : <class 'platformSpecific.SortOfEnum'>
    EMBEDDED             : True
    SortOfEnum           : <class 'platformSpecific.SortOfEnum'>
    getDeltaTime         : <function getDeltaTime at 0x027C2150>
    getTicks_ms          : <function getTicks_ms at 0x02595150>
    initializestatic     : <function initializestatic at 0x027D77C8>
    json                 : <module 'json' from 'C:\\Program Files (x86)\\python\\Anaconda3\\lib\\json\\__init__.py'>
    machine              : N/A
    os                   : <module 'os' from 'C:\\Program Files (x86)\\python\\Anaconda3\\lib\\os.py'>
    programmingLanguage  : cpython
    sockets              : <module 'socket' from 'C:\\Program Files (x86)\\python\\Anaconda3\\lib\\socket.py'>
    sys                  : <module 'sys' (built-in)>
    sysname              : N/A
    time                 : <module 'time' (built-in)>

The first time I noticed something wrong, the classes were missing, or SortOfEnum was available but AutoNumber was not. Now I check again, trying to be sure and not make a fool of myself again, everything that should be there, is there.
What is going on here?


Follows the code of the module. I have tried this as file in the root-directory and as __init__.py in a folder of lib:

Code: Select all

    import sys
    import os
    
    
    programmingLanguage = sys.implementation.name  
    EMBEDDED            =(programmingLanguage == "micropython")
    
    if not EMBEDDED:
        import platform
        import time
        from enum import Enum
        import socket as sockets    # @UnusedImport
        import json                 # @UnusedImport
    
        sysname = os.name
        machine = platform.node()
    
        
        def getTicks_ms():
            return 1000 * time.time()
        
        def getDeltaTime(t2, t1):
            return t2-t1
    
    
        def initializestatic(cls):
            # no need to call cls.initclass()
            return cls
        
        class AutoNumber(Enum):
            def __new__(cls, doc):
                value = len(cls.__members__) 
                obj = object.__new__(cls)
                obj._value_ = value
                obj.doc = doc
                return obj
            
    
          
    else:
        import utime    as time     # @UnresolvedImport @UnusedImport @Reimport
        import usocket  as sockets  # @UnresolvedImport @UnusedImport @Reimport
        import ujson    as json     # @UnusedImport     @UnusedImport @Reimport
    
        sysname, _, _, _, machine = os.uname()  # @UndefinedVariable
        
    
        def getTicks_ms():
            return time.ticks_ms()
        
        def getDeltaTime(t2, t1):
            return time.ticks_diff(t2, t1)
        
        
        def initializestatic(cls):
            cls.initclass()
            return cls
    
    
        class SortOfEnum():
            
            class _EnumElement:  ### (= SimpleNamespace)
                def __init__(self, **kwargs):
                    self.__dict__.update(kwargs)      
                         
                def __str__(self):
                    keys = sorted(self.__dict__)
                    items = ("{}={!r}".format(k, self.__dict__[k]) for k in keys)
                    return "{}({})".format(type(self).__name__, ", ".join(items))
            
                def __eq__(self, other):
                    return self.__dict__ == other.__dict__
        
            def __new__(cls, value):
                return cls.index[value]
            
            @classmethod
            def initclass(cls):
                items = [(cmd, getattr(cls, cmd)) for cmd in dir(cls) if not cmd.startswith('_')]
                cls.index = []
                for item in enumerate(items):
                    item = cls._EnumElement(value=item[0], name=item[1][0], doc=item[1][1])
                    if item.name != "initclass":
        #                 print(item)
                        setattr(cls, item.name, item)
                        cls.index.append(item)
                        
                        
        AutoNumber = SortOfEnum

[dhylands]

I'm not sure why you might be having a problem, but I thought that I'd point out that the non-embedded code may still be compiled and included even though it isn't executed.

I find its better to put the embedded functionality and host functionality in different files and then just import the appropriate files.

[iconoclastica]

I'm not sure why you might be having a problem, but I thought that I'd point out that the non-embedded code may still be compiled and included even though it isn't executed.

I find its better to put the embedded functionality and host functionality in different files and then just import the appropriate files.

Hmm.. that might be helpful. Although it will lead to more maintenance load. For the moment I will follow your advice until I get a better feeling for the ideosyncracies of micro-python. I wonder if my issue has something to do with http://docs.micropython.org/en/latest/g ... -as-loaded, though that can't explain why objects from the middle of the code are missing.

Other strange things show up too. Members such as __name__ and __dict__ are unavailable within class definitions. I don't see this mentioned in the differences-doc? What more such surprises I should prepare for? (Already I notice incompatibilities in socket.)

Wim

[pythoncoder]

I notice you're using __new__(). Note that the differences warns that this may not work. There are also caveats regarding object names and __dict__. In general it's worth remembering that the introspection features of CPython are only partially supported.

Regarding objects like socket it's important to be aware that MicroPython is designed to be used on extremely resource-constrained devices like ESP8266. Inevitably these classes are subsets of the full CPython ones.

[iconoclastica]

Thanks for the hints, Peter. I have it working now, after removing the upeasy image and flashing the esp32 1.10, without too many compromises. Only the absence of sys.exec_info() causes me to ignore more exceptions than I would in

Code: Select all

while self.runServer:
                socket.settimeout(self.timeout)
                try:
                    conn, addr = socket.accept()
                except OSError:
                    # sys.exc_info() does not exist on micro-python
                    # error = sys.exc_info() # exception context
                    # if error[0] == sockets.timeout:
                    continue
                    # else:
                    #     raise

But there were several annoyances, such as sockets that don't have __exit__ implemented so the with-construction fires an exception, and functions such as to_bytes() that don't follow the specs (keyword argument not accepted). The replacement SortOfEnum works a charm, though, even if I will remove the doc on the options.

Thus I now have non-blocking P2P communication between my mycroscope and pc without the need for a USB-cable on a set-up that is too crowded already. I don't know yet where I will meet the memory limits of the system.

-- Wim