The threading question...

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
Post Reply
securigy
Posts: 7
Joined: Wed Aug 21, 2019 12:37 am

The threading question...

Post by securigy » Sat Nov 30, 2019 1:32 am

I am developing with Pycom MicroPython, but I believe that all I wrote is quite compatible with other MicroPythons dialects...
Basically I created an object from a class and in the constructor of that object I spawned a thread that reads analog data from a sencsor in a tight while-loop. The main thread has also a while-loop that supposed to read the data from the spawned thread every 5 seconds...
The problem is that I see that the spawned thread is running but the main thread is not...not sure if it has to do with 'acquire' and 'release' of a flag that supposed to sync the data generation and reading (my main thread).
I posted this question on pycom forum but got no relevant responses...
Is anybody can tell me why the main thread that is not running?

In the main.py I call the constructor that starts the thread:

Code: Select all

ECG = DFRobotECG('P18', 460)

idx = 0;
while(idx < 3):

        print("Main.py: Started loop iteration at local time: {}".format(utime.localtime()))
    
        utime.sleep(2)

        try:

            message = ECG.GetData()
            print("message: {} json string: {}".format(i+1, message))

            data = '{"k": "%s", "d": "%s", "t": "%s"}' % (DEVICE_KEY, message, TOPIC)
            lteSocket.send(bytes(data, 'ascii'))
            utime.sleep(0.5)

            print("==== LTE Socket data sent")
            result = lteSocket.recv(8).decode()
            print("==== LTE Socket result received: {}".format(result))
 
            idx = idx + 1
            print("i={}".format(i))               

            #time.sleep(5) # let the ECG thread read for 10 sec

        except Exception as e:
            print("Exception from while loop: {}".format(e.args[0]))
       
except Exception as e2:
    print("Exception from main code: %s" % e2.args[0])
    #pass # do nothing on error
The above class is the one below. When I run main the thread is running in a pretty tight loop - I actually happy that I get reading every 4-5 millisecond since it will produce the right accuracy for the graph and its interpretation (EKG/ECG).
BUT:
1. the main thread is not running. It is not even getting to the 1st print statement in a while loop: "print("Main.py: Started loop...". Any clues why?
2. After a few minutes the thread throws exception about memory allocation...Why?

Code: Select all


class DFRobotECG:

    history = [(0,0)] 

    def __init__(self, analogInputPin, voltref):

        print("### Entered DFRobot constructor")

        self.exitFlag = False

        self.adc = ADC(bits=12)
        print("adc: {}".format(self.adc))

        # Output Vref of P22
        self.adc.vref_to_pin('P22')
        print("set vref to pin 22")

        # Set calibration - see note above
        self.adc.vref(voltref)#1100)
        print("set adc ref value to 460")

        # Analogog Input Pin on Expansion Board 3.1 should be P13, P14, P17, P18 (G4, G5, G30, G31)
        self.adc_c = self.adc.channel(pin=analogInputPin, attn=ADC.ATTN_11DB) # •	Output Voltage: 0 - 3.3V
        print("got adc channel on P18")

        # allocate locks - initially unlocked
        self.exitFlagLock = _thread.allocate_lock()
        self.ReadLock = _thread.allocate_lock()

        _thread.start_new_thread(self.ReadInput, ())

        print("### Exited DFRobotECG constructor")


    def SetExitFlag(self, exit_flag):
    
        self.exitFlagLock.acquire() 
    
        self.exitFlag = exit_flag
    
        self.exitFlagLock.release()
    
    def GetExitFlag(self):
    
        with self.exitFlagLock: 
    
            return self.exitFlag

    
    def GetData(self):
    
        self.ReadLock.acquire()
    
        self.jsonMsg = ujson.dumps(self.history)
    
        del self.history[:] # empty the array
    
        self.ReadLock.release()
    
        return self.jsonMsg


    def ReadInput(self):
    
        self.start = time.ticks_ms()

        while (True):
        
            if (self.GetExitFlag() == False):
    
                try:
    
                    self.ReadLock.acquire() 
    
                    # get the voltage value
    
                    self.v = self.adc_c.voltage()
    
                    # get the current ticks
                     self.diff = utime.ticks_diff(time.ticks_ms(), self.start)
    
                    # make json payload
    
                    tup = (self.diff, self.v)
    
                    print("Voltage: {}".format(tup))
    
                    self.history.append(tup)
    
                    self.ReadLock.release()
    
                    utime.sleep_ms(1)
    
                except Exception as e:
    
                    print("ReadInput loop exception: {}".format(e.args[0]))
    
        print('Exiting ReadInput...')

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: The threading question...

Post by pythoncoder » Sat Nov 30, 2019 7:12 am

I know this doesn't answer your question but I would strongly recommend using uasyncio instead of threading. The reasons may be found in this doc.

tl;dr Threading is less efficient in terms of RAM usage. Worse, programs using pre-emptive multi tasking are notoriously hard to debug.
Peter Hinch
Index to my micropython libraries.

securigy
Posts: 7
Joined: Wed Aug 21, 2019 12:37 am

Re: The threading question...

Post by securigy » Sat Nov 30, 2019 7:35 am

Appreciate your insight...However:
1. I have a bigger fish to fry in terms of getting the whole concept to work, then I might look at efficacy, etc.
2. At this point based on my previous looks at the coop async library I found it not really intuitive and difficult to grasp - I am used to the same paradigm as I had in C/C++/C#, and therefore I will leave this to the time when the whole system will work: calibrating and reading sensors, delivering data over CAT-M1 and visualizing it and more importantly building wannabe-commercial product (circuitry, PCB, etc.). For a non-EE guy it is a big fish; I hope you get the drift... :roll:

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: The threading question...

Post by jimmo » Sun Dec 01, 2019 11:02 am

I will need to spend some more time reading your code, but I did notice that both threads have sleeps in them, which might not work the way you expect. See https://github.com/micropython/micropyt ... -558865337 for some discussion about this (it's about ESP32, but the point generally is that sleep is a busy loop and not a thread yield).

I'll second Peter's suggestion to use Python's asyncio -- if you've looked at it more than about 18 months ago, it's improved dramatically (both documentation and API). Also be aware that MicroPython's uasyncio is about to get a bit of an update in https://github.com/micropython/micropython/pull/5332

Also useful to investigate this further would be know which PyCom board are you using? The CC3200 port is quite different to the ESP32 one, especially when it comes to threading.

securigy
Posts: 7
Joined: Wed Aug 21, 2019 12:37 am

Re: The threading question...

Post by securigy » Sun Dec 01, 2019 9:20 pm

I resolved the problem - it was something stupid I did in my main thread - artificially suspending the main thread indefinitely...
I will definitely give a look at async library later. I am not out of the woods yet. Right now I have a while loop in the main thread that sleeps every 2 seconds. Meanwhile the sensor thread generates data in a tight loop so I get a reading every 3-4 msec. This is great because it is ECG and it will have great resolution. Having ECG graph with 10 msec resolution is the most crude reading I can have. BTW, I put another sleep in the sensor thread in order to slow it down but only for debugging purposes - otherwise the console is flooded with messages I was printing and I could not see the entire history. This sleep in the sensor thread will eventually go away.

My device is Pycom's FiPy and therefore my MicroPython is even more limited than the one you have on ESP32. It has _thread library but not threading library, so I don't have events, etc...How would you suspend main thread otherwise than sleep ?

Right now I have another problem that I am puzzled about. I send the data over CAT-M1 to Hologram network. I do it the while loop in the main.py by using sockets. CAT-M1 can handle up to 300-375 Kbps which I am not even close to, but when I try to read back the response I get nothing and recv request times out (time out that I set on the socket). I can see the data that I sent on the Hologram but get no response back...

Does anybody know if there is a response from the socket that was used to send data to Hologram?
That is, if I do the following, should I expect any response in result = lteSocket.recv(1024)
The reason I am asking is that the code does not run beyond this point as it waits for response and I wonder if I should expect any response at all... Interestingly enough this code worked with another (environmental) sensor...

Here is the part of the main:

Code: Select all


            message = ECG.GetData()
            print("message: {} json string: {}".format(idx+1, message))

            data = '{"k": "%s", "d": "%s", "t": "%s"}' % (DEVICE_KEY, message, TOPIC)
            print("data: {}".format(data))
            lteSocket.send(bytes(data, 'ascii'))
            print("==== LTE Socket data sent")
            utime.sleep(2)

            try:
                print('Reading response from socket...')
                result = lteSocket.recv(1024) #.decode()
            if (len(result) == 0):
                print("==== LTE Socket: Received no response...")
            else:
                print("==== LTE Socket: Recieved result: {}".format(result))
EDIT:
From this example in documentation:
https://docs.pycom.io/tutorials/lte/cat-m1/

I read that:
*When using the expansion board and the FiPy together, the RTS/CTS jumpers MUST be removed as those pins are being used by the LTE radio. Keeping those jumpers in place will lead to erratic operation and higher current consumption specially while in deepsleep.*

So is anybody can point out what and where do I need to remove on Expansion Board 3.1 ?

Post Reply