Page 11 of 11

Re: uasyncio - asyncio-like cooperative multitasking framework for uPy

Posted: Wed Dec 26, 2018 9:47 pm
by pfalcon
This thread is now back to the "announcements of new uasyncio releases" mode. Please ask technical questions in separate threads, properly titled - after you searched the forum for existing discussions. Please ask non-technical questions in dedicated threads - after you read the existing material on the matter.

Re: uasyncio - asyncio-like cooperative multitasking framework for uPy

Posted: Thu Dec 27, 2018 5:31 am
by kevinkk525
pfalcon wrote:
Wed Dec 26, 2018 9:31 pm
make the default upip install of micropython-asyncio work with mainline MicroPython
mattyt, are you sure you knock on the right door? Did you read the post above at all? Did you read the links in it? There's a patch which needs to be merged to "make the default upip install of micropython-asyncio work with mainline MicroPython". You should implore to the maintainer of the mainline. And heck, the mainline has fallen into subpar maintenance, as discussed by the links above. I'm doing all what I can to continue development and improve the situation with the mainline too (raising questions, offering help with the maintenance, submitting patches). What are you doing to improve the situation?
At least we are not doing anything to make the situation worse. Putting a version on pypi that can't work on the mainline micropython (yet) is a very unhealthy and hostile move, further scattering the micropython environment and harming the community.
You can't demand mainline developers to implement the patch that you want to see by leveraging that the typical newbie to micropython will try to use your new uasyncio module from pypi which won't work on his mainline firmware build. So he gets frustrated and leaves micropython in the worst case. So why do that? It does not matter what you think about mainline, it is still the entry point to micropython.
What exactly are you doing that improves the micropython environment with this move?
You could have published your new version as micropython-uasyncio-dev and everyone would have been happy.

Re: uasyncio - asyncio-like cooperative multitasking framework for uPy

Posted: Tue Jan 08, 2019 10:58 am
by mattyt
pfalcon wrote:
Wed Dec 26, 2018 9:31 pm
mattyt, are you sure you knock on the right door? Did you read the post above at all? Did you read the links in it?
Yes, I had read the entire sordid mess. I'm well aware that I'm "knocking on the door" of the person who just willfully broke mainline MicroPython. Again.

I'm going to have to take another group of fledgling MicroPython programmers and explain to them how to work around the issue you've created. I fail to understand how you think this is good for the community? Particularly since it would have taken little effort to avoid (by renaming the 'development' package of asyncio as discussed above).
pfalcon wrote:
Wed Dec 26, 2018 9:31 pm
There's a patch which needs to be merged to "make the default upip install of micropython-asyncio work with mainline MicroPython". You should implore to the maintainer of the mainline.
Interesting point of view. I read that PR and Damien raised a reasonable objection to the change. Your responses since weren't particularly compelling.

Shouldn't the onus be on the person raising the PR to make a convincing case to integrate a change?
pfalcon wrote:
Wed Dec 26, 2018 9:31 pm
And heck, the mainline has fallen into subpar maintenance, as discussed by the links above.
Just because the project isn't moving at the pace - or in the direction - that you want doesn't mean it's 'fallen into subpar maintenance'.
pfalcon wrote:
Wed Dec 26, 2018 9:31 pm
I'm doing all what I can to continue development and improve the situation with the mainline too (raising questions, offering help with the maintenance, submitting patches).
I'm honestly and genuinely grateful for your contributions. Seriously; thanks Paul!
pfalcon wrote:
Wed Dec 26, 2018 9:31 pm
What are you doing to improve the situation?
What does that have to do with anything? I do what I can. I apologise if you feel it's not enough. I would dearly love to contribute more.

In any case, I wasn't going to respond because doing so is exhausting and feels futile. You've made up your mind and seem unlikely to change. The only purpose of my response is that I think you should be aware that at least I feel that you've not acted in the best interest of this community with this decision. But c'est la vie.

Re: uasyncio - asyncio-like cooperative multitasking framework for uPy

Posted: Sat Feb 02, 2019 8:33 am
by pfalcon
A major evolutionary step for uasyncio - a long-standing bug with cancellation of uasyncio.sleep() call was fixed. That bug didn't allow to write reliable networking applications (with timeouts, etc.), and rendered uasyncio practically useless for applications outside its initial domain (which was again, writing networking servers). Now uasyncio becomes a truly generic async scheduling framework. This is released as uasyncio.core 2.1 .

The history of solving this bug is "truly dramatic" - the work started in the framework of ESP8266 kickstarter campaign, one of the stretch goal of which was bringing uasyncio to ESP8266 and let it shine there. Cancellation support was at all implemented then, with a known issue of not being able to cancel uasyncio.sleep(). Solving that issue required considerable architectural refactors to uasyncio and its support modules in MicroPython core. All in all, by December 2017, I had a solution for uasyncio.sleep() problem too. But out of sudden, the owner of project, and my companion in the kickstarter, rose against the next batch of uasyncio-related improvements in MicroPython, and I had my commit access removed. While I continues work in my fork, there was an outcry of "don't break our loved bugs with the mainline", so related patches sat for a year in my backlog. Last December I brought up issue of the need of maintenance of mainline again, with just as disappointing results. So, the decision to go and fix the pending issues and continue active development in Pycopy, my fork of MicroPython, solidified. Even then a couple of "well-wishers" from the Cult of Unfixed Bug started a sh%tstorm in a teacup, as can be beheld in the messages above.

So, all in all, bugs are being fixed and intended to be fixed further. The active development of MicroPython happens in the Pycopy fork, linked above. Per the previous announcements, it is required for the latest versions of uasyncio. There're more bugs to fix and features to add, stay tuned.

Re: uasyncio - asyncio-like cooperative multitasking framework for uPy

Posted: Sat Mar 02, 2019 8:38 pm
by eltomek
For posterity, as I've spend several days investigating this case, maybe this is going to save somebody else's time.

While running my picoweb server and some mqtt reporting to Thingspeak executed within uasyncio loop on my Wemos D1 mini (ESP8266) after some time I was getting a vague

Code: Select all

ValueError: generator already executing
error message reported for

Code: Select all

await asyncio.sleep(delay_secs)
line of code of the coro definition.

More context:
Error message:

Code: Select all

Traceback (most recent call last):
  File "main.py", line 233, in <module>
  File "picoweb/__init__.py", line 298, in run
  File "uasyncio/core.py", line 154, in run_forever
  File "uasyncio/core.py", line 109, in run_forever
  File "main.py", line 223, in mem
ValueError: generator already executing
where "main.py", line 223, in mem is:

Code: Select all

await asyncio.sleep(delay_secs)
"main.py", line 233, in is:

Code: Select all

loop.create_task(mem(1))
As it was happening after dozens of minutes but in the same loop of iteration every time I thought it might be related to memory.
Quick check on gc.mem_free() every loop iteration and it turned out the the heap was going down to as low as ~10kB while the script encountered an interruption with the above error message.

Now, with gc.collect() executed every Thingspeak upload iteration of the loop the script really runs forever ;-)
Alternatively you can set the gc.threshold(2000) at the top of your script to make sure gc does the collection with 2kB advance, 2000 is an empirical value confirmed to work for my Wemos D1 mini board.

Anyway, it's interesting why does the uasyncio return this error message and why it fails allocating the memory.

Interestingly, as a follow up, I checked the gc.threshold(-1) to disable allocation threshold and then heap goes to as low as 400 bytes even not causing the issue described above. So, not sure if I nailed this problem right unfortunately.