no blocking scan function

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
Post Reply
lbayo
Posts: 24
Joined: Tue Mar 16, 2021 2:10 pm

no blocking scan function

Post by lbayo » Sat Jul 03, 2021 7:39 am

Hi guys,

I have a program that works correctly (finally) using the async functions but ... there is a process that uses the scan () function to determine the RSSI of the WiFi network (here, we have a lot of nets) and the execution of this function (which is not multitask) locks the other processes for 2 seconds aprox.
Is there an equivalent to scan () function that can be used in asynchronous mode?
The process is

Code: Select all

async def get_rssi():
    global rssi
    s = network.WLAN()
    ssid = config['ssid'].encode('UTF8')
    
    while True:
        try:
            tx=time_ns()
            rssi = [x[3] for x in s.scan() if x[0] == ssid][0]
            print ("rssi time:%12.3f mS   rssi:%4d dB"%((time_ns()-tx)/1E6, rssi)); 
        except:
            rssi = -199
        await asyncio.sleep(10)

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: no blocking scan function

Post by jimmo » Tue Jul 06, 2021 7:15 am

lbayo wrote:
Sat Jul 03, 2021 7:39 am
Is there an equivalent to scan () function that can be used in asynchronous mode?
Unfortunately no. But this is a reasonably frequently requested feature.

marcidy
Posts: 133
Joined: Sat Dec 12, 2020 11:07 pm

Re: no blocking scan function

Post by marcidy » Wed Jul 07, 2021 6:17 am

It looks feasible to make scanning non-blocking, can someone knowledgeable please take a look at the reasonableness of below and potentially address the questions? It's just a sketch of the solution, I'll work out the details in a PR.

tl;dr:

The main question i have here is if it's ok to delay collecting the scan results until the user calls the function, since the results will sit in memory if the user never triggers the collection. A clean-up can be triggered based on other state changes, (e.g. if a connection is attempted), though this would need documentation at least. There's a few other questions below as well.

The scan example from esp-idf, it's non-blocking, and uses the same event handling the port uses.

from esp-idf/examples/wifi/scan/main/scan.c:
... event handler

Code: Select all

static bool scan_done = false;
static void display_scan_result(void);

static void event_handler(void* arg, esp_event_base_t event_base,
                                int32_t event_id, void* event_data)
{
    if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_SCAN_DONE) {
        scan_done = true;
    }
}
... retrieve scan result (similar enough to the blocking case, here to be thorough)

Code: Select all

/* Initialise a wifi_ap_record_t, get it populated and display scanned data */
static void display_scan_result(void)
{
    uint16_t number = DEFAULT_SCAN_LIST_SIZE;
    wifi_ap_record_t ap_info[DEFAULT_SCAN_LIST_SIZE];
    uint16_t ap_count = 0;
    memset(ap_info, 0, sizeof(ap_info));
    ESP_ERROR_CHECK(esp_wifi_scan_get_ap_records(&number, ap_info));
    ESP_ERROR_CHECK(esp_wifi_scan_get_ap_num(&ap_count));
    ESP_LOGI(TAG, "Total APs scanned = %u", ap_count);
    for (int i = 0; (i < DEFAULT_SCAN_LIST_SIZE) && (i < ap_count); i++) {
        ESP_LOGI(TAG, "SSID \t\t%s", ap_info[i].ssid);
        ESP_LOGI(TAG, "RSSI \t\t%d", ap_info[i].rssi);
        print_auth_mode(ap_info[i].authmode);
        if (ap_info[i].authmode != WIFI_AUTH_WEP) {
            print_cipher_type(ap_info[i].pairwise_cipher, ap_info[i].group_cipher);
        }
        ESP_LOGI(TAG, "Channel \t\t%d\n", ap_info[i].primary);
    }
}

adding handling of WIFI_EVENT_SCAN_DONE in modnetwork.c:event_handler could do it, with some open questions on managing the scan result and checking the scanning status.

Could add a STAT_SCANNING and STAT_SCANDONE to the enum.

Code: Select all

enum {
    STAT_IDLE       = 1000,
    STAT_CONNECTING = 1001,
    STAT_GOT_IP     = 1010,
};
and update status accordingly.

Would we rely on the user triggering the collection of scan results into a list? e.g. WLAN.get_scan_results() or something? I'm not sure if there is any better way to manage the results. It doesn't seem right to me to append them onto the interface instance, but there's a question about what to do with the results hanging around in memory.

The main question i have here is if it's ok to delay this until the user calls the function since the results will sit in memory if the user never triggers it. A clean-up can be triggered based on other state changes perhaps (e.g. if a connection is attempted).

A second question is if the behavior should be kept backwards compatible. Seems relatively easy to pass a "block" parameter to WLAN.scan if so, defaulting to True?

Notes from the docs:
The scanning triggered by esp_wifi_start_scan() will not be effective until connection between ESP32 and the AP is established. If ESP32 is scanning and connecting at the same time, ESP32 will abort scanning and return a warning message and error number ESP_ERR_WIFI_STATE. If you want to do reconnection after ESP32 received disconnect event, remember to add the maximum retry time, otherwise the called scan will not work. This is especially true when the AP doesn’t exist, and you still try reconnection after ESP32 received disconnect event with the reason code WIFI_REASON_NO_AP_FOUND.
Pretty sure this (and a few other lines) imply scanning and connecting are mutually exclusive activities, with calls to connect taking precedence. e.g. a micropython call to connect could stop the scan and silently clear out the scan results without losing too much compared to using the IDF directly. I'll test these assumptions to be sure.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: no blocking scan function

Post by jimmo » Thu Jul 08, 2021 7:55 am

I had assumed that the most useful part of a non-blocking scan would be to return results incrementally while the scan ran in the background. But it seems (from my reading of the IDF code) that this isn't possible.

Anyway, sounds like you just want a non-blocking scan that you can retrieve the results later. This seems possible and yeah I agree with all your comments.
marcidy wrote:
Wed Jul 07, 2021 6:17 am
Would we rely on the user triggering the collection of scan results into a list? e.g. WLAN.get_scan_results() or something? I'm not sure if there is any better way to manage the results. It doesn't seem right to me to append them onto the interface instance, but there's a question about what to do with the results hanging around in memory.
This is a tricky question and one that is difficult to manage in ESP32 generally... the issue is that the callback happens on a different task, so it will not be able to allocate on the MicroPython heap.

It seems reasonable to me though that the results should hang around (in IDF-managed memory, i.e. until you call esp_wifi_scan_get_ap_records)...
marcidy wrote:
Wed Jul 07, 2021 6:17 am
A second question is if the behavior should be kept backwards compatible.
Definitely, this is a requirement.

marcidy
Posts: 133
Joined: Sat Dec 12, 2020 11:07 pm

Re: no blocking scan function

Post by marcidy » Thu Jul 08, 2021 3:08 pm

jimmo wrote:
Thu Jul 08, 2021 7:55 am
I had assumed that the most useful part of a non-blocking scan would be to return results incrementally while the scan ran in the background. But it seems (from my reading of the IDF code) that this isn't possible.
I agree, I'll look into it further anyways. maybe there's some traffic on the idf's github/forums on it.
This is a tricky question and one that is difficult to manage in ESP32 generally... the issue is that the callback happens on a different task, so it will not be able to allocate on the MicroPython heap.
Ah, didn't think about that, thanks. I live in an area with lot's of APs so I have some good real world test cases for a large AP list.

I'll work through this and hopefully get a PR up soon, thanks for the feedback

marcidy
Posts: 133
Joined: Sat Dec 12, 2020 11:07 pm

Re: no blocking scan function

Post by marcidy » Sat Jul 10, 2021 11:41 pm


Post Reply