ESP8266 UART Driver (AT command based communication)

[kfricke]

I did start to implement a driver to communicate with an ESP8266 board using AT commands over UART. This is an alternative approach to the port of MicroPython to this board and uses the default firmware on the ESP8266. This is usually the stock firmware on those devices when you purchase them. Esspressif does publish documentation and updates to this commands.

This driver can be useful and is reasonable in some reasons:

• It can use a MicroPython implementation on a mightier platform than the ESP which has more features than possible on the ESP8266 itself.
• Can be used on a Raspberry Pi or PC with the MicroPython Unix port or may even be ported to CPython.
• Makes it possible to use multiple of these modules.

So far the motivation for this driver, now to it problems... I have been successfully accessing several WIFI commands and things start to work quite well so far. No communication commands are implemented so far. Most notably i see problems in the ESP AT command firmware: It has tons of bugs.
My favorite bug currently is that the AT command to connect to a WIFI does always fail. The reason for this seems to be that the SSID must be 10 characters long
Another one is the WIFI scan command. It does not return all WIFIs and also prints rubbish on the serial console. The ESP8266 platform is completely event driven and the callback methods of the WIFI scan seem to write to the console without synchronization. Multiple scan results are being written "into each other".

Sure the ESP AT firmware receives updates. I am already using the current stable revision (0.21 from the 0.9.5 SDK) and there also is a beta of the next firmware. But does anyone of you have similar experience with this modules? Does it make sense to continue with this approach when such bugs are in my way? Maybe it makes sense to use the MicroPython port approach to implement the needed AT commands. This is possible but would mean even more pressure and work on this port, which i wanted to avoid in the first place

[Damien]

I also started writing an AT driver months ago but found lots of bugs and then had better things to do

My main issue was the lack of synchronisation. Especially I didn't see how you could rate limit the data coming in: it seems that any available data just gets spewed onto the UART bus.

It might be a good idea to use uPy running on the ESP to implement a sane wifi slave module. Could use JSON for data transfer over the UART with sequence ids and proper rate limiting for a read.

I don't think it would require much to get the uPy port to a state where it can be used for this purpose. Don't need IO or proper sockets, just basic wifi access.

[kfricke]

Ok, good to know that i am not the only one with this experience.

Regardless the annoying bugs this is really a fun little project. So I will continue to implement a driver using this interface. Partial motivation is that it will hopefully mature down the road. The changes in the beta firmware (0.9.6-b1) might fix some of the current bugs at least.
Biggest motivation is that i can hack something hopefully useful in pure python

What exactly do you mean with rate limit in this context? I think that message integrity (what i did mean with synchronized output to console) and sequence ids are quite essential, but alone should handle the situation pretty well.

On the ESP port side of this i did try to implement sending an UDP packet and did give up with the memory allocation of that stupid C struct (espconn) in the ESP SDK. Shame on me and my C-disability
If someone could throw this into the ESP port, it would be gorgeous!

I will implement this driver in a way which makes it transparent to the MicroPython client what it is communicating with on the UART side. Let it be a buggy ESP AT firmware or a cool MicroPython JSON one.

Of course JSON requests and responses would be a nice way to communicate. There is also a part in the ESP-IoT-SDK which allows registration of AT commands. Don't know if this would be easier to start with.

[kfricke]

Just as a short update on my progress, here is a link to my current development state of this "driver": https://github.com/kfricke/micropython-esp8266uart

I am using the AT firmware version 0.21 from the SDK version 0.9.5. I will split it up from different (unreasonable) classes into modules (WIFI, IP and so on) soon. Also needed is test coverage.

Currently implemented is a small subset:

• Test AT interface
• Soft-Reset
• WIFI
  • Get WIFI mode
  • Set WIFI mode (station, access point or both)
  • Connect
  • Disconnect
  • Get access point info
  • List all access points (buggy on ESP8266 side of the AT interface)
  • List access point information (iirc, broken atm)
  • Set access point configuration (creates a WIFI network)
  • Get current access point configuration
• IP networking
  • Get connection status
  • Start connection (only UDP client and heavily untested!)
  • Send (only UDP client and heavily untested!)
  • Ping

[kfricke]

Besides code clean up, better (print-)debugging, more consistency parsing and general stability, more WIFI commands have been implemented:

• List IPs of stations connected to access point
• Set DHCP config (oddities are buried in this AT command )
• Set automatically connect to AP in station mode
• Set and get station IP
• Set and get access point IP

Some simple test cases have been implemented.

There is still some work ahead to cover all networking commands and reach basic usability!

[Damien]

By rate limiting I mean how to deal with the data coming in from a large webpage? Say you do a GET request for something that is 200k and you just want to parse the data looking for a special string of characters. You don't need 200k RAM for that, as long as the data comes in small chunks you can process them one after the other. But this will only work if you can tell the esp that you are not yet ready for the next chunk. Such feature definitely exists at the TCP/IP layer but I don't know if the esp exposes it in the AT command set.

[kfricke]

There is the AT command "AT+ CIPMODE – Set transfer mode" which knows an unvarnished mode for data transmission. This seems to reduce the send speed of data to 2048 bytes every 20 ms. At least the documentation for the AT command "AT+CIPSEND – Send data" suggests this.
Receiving data from the network using the AT command "+IPD – Receive network data" does not mention this at all.
The 2048 / 20ms "limit" does exceed the default baudrate of 115200 (note to self: implement switching the baudrate command)

Since i have not had time to implement this part of the API I can not tell anything about this. Sending and receiving larger chunks of data will become interesting. There is a 2048 bytes limit for sending (AT-command) packets and nothing is mentioned regarding size for receiving packets. Lets see how this works soon.

Should become interesting to see if one can use slow baudrates to reach the intended rate limit. But i guess not

The C API has hold and unhold methods for TCP connections, which might be something like rate limiting on the network.This is just a guess, tbh.

[juergen]

Hi,
I think you wrote an interisting module, but I couldn't manage to get it running errorfree:
Traceback (most recent call last):
File "main.py", line 9, in <module>
File "esp8266uart.py", line 236, in test
File "esp8266uart.py", line 232, in _execute_command
File "esp8266uart.py", line 141, in _send_command
IndexError: list index out of range
I couldn't figure out why this happens, but actually I've choosen a slightly different approach to communicate with the ESP8266:

Code: Select all

from pyb import UART
import pyb

def commCycle(command): #This controls one command exection cycle.
    uart.write(command + '\r\n')
    myLine = ''
    while  not  "OK" in myLine:
        while not uart.any():
            pass
        myLine = uart.readline()
        debuguart.write(myLine)
        lcd.write(myLine)

uart = UART(6, 9600,  timeout=1000,  timeout_char=3000,  read_buf_len=8192) #This UART talks to ESP8266
# the read buffer of the micropy seems to be quite limited. If you don't empty it fast enough, it will overrun.
debuguart = UART(3, 9600) #This UART sends (only) to a serial - USB converter which is connected to a PC with a terminal runnning (miniterm.py)

lcd = pyb.LCD('X')
lcd.light(True)
lcd.write("I'm running 'main.py'\n")

debuguart.write('Starting ---------------------\r\n')
commCycle("AT")
commCycle("AT+GMR")#Version
commCycle("AT+CWLAP")#List WLANs
lcd.light(False)

This works. Especially "AT+CWLAP" takes time, so my first attempts with different reading methods failed.
As you can see, I send the answers from ESP8266 to the LCD and to a second computer (running miniterm.py) via USB serial converter for debugging. If you do not want to replicate this setting, comment it out.
Caution: my hardware setting only works, if the ESP8266 is separately supplied with 3.3 V.
I tried to feed it from the pyboard, but lcd and ESP8266 and micro SD card together can't be handled by the pyboard.

[wagnerc4]

This can be usefull:

The object "SERVER" has been improved with the "select.poll" library to make it asynchronous

Code: Select all

# esp8266.py
from select import poll
from pyb import UART, delay

class SERVER():
  def __init__(self, response):
    self.response = response
    self.uart = UART(1, 115200)
    self.uart.write('AT\r\n')
    print(self.uart.readall().decode('utf-8'))
    self.uart.write('AT+CIPMUX=1\r\n')
    print(self.uart.readall().decode('utf-8'))
    self.uart.write('AT+CIPSERVER=1,1336\r\n')
    print(self.uart.readall().decode('utf-8'))
    self.poller = poll()
    self.poller.register(self.uart, 1)
  def send_rs(self, rq):
    cid = rq[rq.find('+IPD,') + len('+IPD,')]
    rs = self.response(rq)
    self.uart.write('AT+CIPSEND=%s,%s\r\n' % (cid, len(rs)))
    delay(300)
    self.uart.write('%s\r\n' % rs)
    delay(300)
    for i in range(10):
      if self.uart.any():
        msg = self.uart.readall().decode('utf-8')
        if 'SEND OK' in msg: break
      delay(100)
    self.uart.write('AT+CIPCLOSE=%s\r\n' % cid)
    delay(300)
  def process_rq(self):
    ev = self.poller.poll(1000)
    if len(ev)>0 and self.uart.any():
      rq = self.uart.readall().decode('utf-8')
      if '+IPD,' in rq: self.send_rs(rq)
  def __del__(self):
    poller.unregister(self.uart)
    uart.deinit()

this is a example for use the above code:

Code: Select all

# main
from re import search
from esp8266 import SERVER

html = """
<html>
<head>
  <link rel="icon" href="#" />
  <meta name="viewport" content="initial-scale=1" />
</head>
<input id="y" type="range" min="-5" max="5" step="1" value="%s"
       style="margin:50px 50px 0 0; transform:rotate(270deg);"
       onchange="update()" />
<input id="x" type="range" min="-5" max="5" step="1" value="%s"
       style="margin-left:150px;"
       onchange="update()" />
<script>
  function update() {
    window.location.href = '?' + document.getElementById('y').value +
                           '&' + document.getElementById('x').value;
  }
</script>
</html>
"""

last_x = last_y = 0

def response(rq):
  global last_x, last_y
  x = search(r'\&-?\d+', rq)
  y = search(r'\?-?\d+', rq)
  last_x = x.group(0)[1:] if x else last_x
  last_y = y.group(0)[1:] if y else last_y
  return html % (last_y, last_x)

server = SERVER(response)
while True: server.process_rq()
del server

I took the code from here:
https://hackaday.io/project/6877-microp ... uped-robot
https://github.com/guyz/pyesp8266

[jamonda]

This code does not work with my pyboard v1.1