Simple terminal (VT100) editor widget (and other widgets)

[Roberthh]

@mbirth
Hello, and thanks for the visit. Sorry for the confusion, but I think you see complications where there aren't any.

The only file you need is pye.py. It contains the full code of the text editor based on pfalcons Edit widgets. You can run pye.py on any platforn with python3 and a vt100/xterm console, like linux or OS X like any other text editor (e.g. nano, gedit). You would simple issue the command:

Code: Select all

python3 pye.py filename

Micropython works too, if you install micropython-lib (https://github.com/micropython/micropython-lib). pye.py runs on pyboard too. A command sequence would be:

Code: Select all

from pye install *
pye("filename")  or pye(list_of_strings)

The file pye.py contains special sections to deal with the I/O interfaces of linux, PyBoard and WiPy. To separate them, they start with:

Code: Select all

if sys.platform == "PyBoard".

or the like. Without that, python would raise an error, when the code is executed on the other platforms. The file pye.py contains C-Preprocessor statements, which allow variants of the file for specific platforms, like.

Code: Select all

cpp -D LINUX pye.py | grep -v "^# .*$" >pex.py
cpp -D PYBOARD pye.py | grep -v "^# .*$" >peb.py
cpp -D BASIC -D WIPY -D DEFINES pye.py | sed "s/#.*$//" | sed "/^$/d" >wipye.py

The removes the sections for the other platforms, but still keep the "if sys.platform == ..." statements, which do not hurt. The last example is mandatory for WiPy due its restricted memory size. WiPy would not parse the full pye.py code. The -D DEFINES replaces the symbolic key names by numeric constants, and the -D BASIC excludes some of the features. I have seen other ways to achieve that, but for me it works. The shebang file consist of a single line as the first line in the python script:

Code: Select all

#!/usr/bin/env python3

which tells the linux bash or csh to use python3 to run pye.py. It's not included by default since cpp cannot deall with it.

Best regard

[andrew]

@Roberthh I'd love to try your editor on my WiPy but it seems to run out of memory even when freshly booted. Any ideas?

Code: Select all

Login succeeded!
Type "help()" for more information.

>>> import gc
>>> gc.mem_alloc()
2160
>>> gc.mem_free()
55232
>>> from wipye import pye
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError: memory allocation failed, allocating 176 bytes
>>>

Andrew

[Roberthh]

Hello Andrew, memory-wise it's during loading at the limit. What I see is after Ctrl-D:

Code: Select all

PYB: soft reboot
MicroPython v1.5-1-ge954604 on 2015-10-21; WiPy with CC3200
Type "help()" for more information.
>>> import gc
>>> gc.mem_alloc()
1264
>>> gc.mem_free()
55072
>>> from wipye import *
>>> gc.mem_free()
42336
>>> 

It's only critical during loading. After that, I could load & edit the file wipye.py itself, which is like 450 lines.
Just for an attempt, try deleting some code. In the function handle_edit_key, there are sequences like elif key == 0x... Each of them handles a single function. 0x09 is Tab, 0x15 is backtab, 0x18, 0x04 and 0x16 are the buffer functions, and so on... For trial, you can remove any of them. You can also delete the functions scroll_up() and scroll_down() and any calls to them. They are only called once each in wipye. That makes scrolling slower, but saves a few hundred bytes.
I know that the recent Version of MicroPython works better, since sometime in the last two weeks, the memory allocation stuff was improved. And be sure that you use the version of wipye.py generated by the strip.sh script.
Regards, Robert

[Roberthh]

@andrew
I handcrafted a smaller version of wipye, just as a trial. It's attached. It has like 70 lines less of code and consumes 3k less in memory. Almost all functions are still there. It seems, that the size of the source code is more the limiting factor than the parsed memory footprint. I do not know the internals of the language parsing, which makes size optimization kind of a blind flight. What I found interesting is that each function adds about 400 bytes in memory. I might reduce the number of small & rarely called functions.
Robert

[andrew]

@andrew
I handcrafted a smaller version of wipye, just as a trial. It's attached. It has like 70 lines less of code and consumes 3k less in memory. Almost all functions are still there. It seems, that the size of the source code is more the limiting factor than the parsed memory footprint. I do not know the internals of the language parsing, which makes size optimization kind of a blind light. What I found interesting is that each function adds about 400 bytes in memory. I might reduce the number of small & rarely called functions.
Robert

Thanks for that but for some reason it's still not working for me despite having more memory available than you show.

Code: Select all

>>> dir()
['__name__']
>>> import gc
>>> gc.collect()
>>> gc.mem_alloc()
1424
>>> gc.mem_free()
55968
>>> gc.collect()
>>> from wipye_small import pye
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError: memory allocation failed, allocating 1462 bytes
>>> dir()
['gc', '__name__']
>>>

Andrew

[mbirth]

on my WiPy but it seems to run out of memory even when freshly booted. Any ideas?

For me it loaded fine after I rewrote my main.py to use lazy imports and did a gc.collect() before trying to import wipye. mem_free was around 52-53k.

However, I was not able to do anything once it was loaded. It didn't register my key presses (was connected via miniterm.py from Linux).

[andrew]

on my WiPy but it seems to run out of memory even when freshly booted. Any ideas?

For me it loaded fine after I rewrote my main.py to use lazy imports and did a gc.collect() before trying to import wipye. mem_free was around 52-53k.

However, I was not able to do anything once it was loaded. It didn't register my key presses (was connected via miniterm.py from Linux).

You'll have to forgive me as I'm still new to Python. What are lazy imports?

Andrew

[mbirth]

What are lazy imports?

It's loading your libraries when they're needed instead of on top the .py file. So instead of:

Code: Select all

import os

def whereami():
    print(os.getcwd())

You'd do it like this:

Code: Select all

def whereami():
    import os
    print(os.getcwd())

This way, os is first imported when it's needed instead of being imported right away.

But according to your available memory readings, this shouldn't be a problem in your case.

[Roberthh]

@mbirth
Hello all. Thanks for trying.
On WiPy and PyBoard, it's not a standalone program. If you want to edit something, you have to input:

Code: Select all

pye("filenname")

at the REPL prompt. The sequence I use is:

Code: Select all

from wipye import *
pye("myfile")

Instead of a filename, you may also supply a list of strings. I'm using Telnet on Linux and Putty or Teraterm on Windows for a telnet connection. If you want to use the UART connection, you have to wite:

Code: Select all

pye("filename", device=1, baud=xxxx)  # UART0 on WiPY, UART 1 on PyBoard, or
pye("filename", device=2, baud =xxxx)  # UART1 on WiPY, UART 2 on PyBoard

The default speed is 115200 Baud
@andrew
I do not know why it does not load. Did you upgrade WiPy to the latest version of the firmware? On the other hand, I have seen difficulties in the memory handler of micropythin with the Linux port, which I did not expect. I'm still trying to replicate that.
Regards

[andrew]

@andrew
I do not know why it does not load. Did you upgrade WiPy to the latest version of the firmware? On the other hand, I have seen difficulties in the memory handler of micropythin with the Linux port, which I did not expect. I'm still trying to replicate that.
Regards

Ah, I hadn't thought of upgrading the firmware as until yesterday my WiPy appeared to be stuck permanently in safe boot mode precluding the option. So I've just upgraded and yes, the wipye module imported and I was able to execute pye('wipye.py'). Yay! Thanks for this - it should make quick edits to scripts a lot easier.

Andrew