BLE Scan forever stops after 7+ Hours

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

BLE Scan forever stops after 7+ Hours

Post by devnull » Sun Mar 29, 2020 8:52 am

Just discovered that the BLE scan forever appears to fail silently (without error) and stops discovering after about 7 hours.

Code: Select all

def irq(event,adv):
  if adv[0] != 0 or len(adv[4]) != 22 or adv[4][4:7] != b'\x02 \xaa': return
  else:
    print(adv)
    schedule(found,adv)

bt.gap_scan(0,10000, 10000)

{'cb:d0:c5:0c:27:bd': [25.8, 53.7, 100, -72, 25868205], 'c5:ac:70:b4:d1:20': [25.8, 52.1, 94, -61, 25866256], 'cd:d7:51:15:0f:7e': [27.6, 52.2, 100, 85, 25865825]}
Above you can see the scan call and the logging of data collected from 3 beacon devices. The final element in each list is the ticks_ms when the event last occured.

25866256 is the largest time tick_ms value / 1000 / 60 / 60 = 7.185 hours, that is when it stopped discovering, there appears to be no errors or exceptions, it just seems to stop !

The logging of the known devices continues in a loop, however the values never change, and logging the IRQ_SCAN_RESULT interrupt event just stops.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by jimmo » Mon Mar 30, 2020 1:11 am

Hi,

This sounds very likely to be https://github.com/espressif/esp-idf/issues/4196 (I reported it to Espressif in October last year)

It was supposedly fixed recently (but not available in any released IDF version). I haven't had a chance to investigate. If you wanted to test this out, you could cherrypick those two commits mentioned in https://github.com/espressif/esp-idf/is ... -598099312 onto the v4.0 tag of your esp-idf and modify the hash in esp32/Makefile.

i.e. in your esp-idf repo:

Code: Select all

git fetch origin
git checkout -b scan-fix v4.0
git cherry-pick 48bd2d74b4209bbad7c29d2d856810ee43a1a92c
git cherry-pick 74c7db3f49e8605d7ec3736a3ba508a9f8d3fd9a
git submodule update --init --recursive
git rev-parse HEAD
then update esp32/Makefile to use the hash that was printed out in the last command.

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by devnull » Tue Mar 31, 2020 9:20 am

Thanks Jimmo;

But this failed during cherry pick:

Code: Select all

# git fetch origin
remote: Enumerating objects: 636, done.
remote: Counting objects: 100% (636/636), done.
remote: Total 884 (delta 636), reused 636 (delta 636), pack-reused 248
Receiving objects: 100% (884/884), 716.96 KiB | 449.00 KiB/s, done.
Resolving deltas: 100% (677/677), completed with 300 local objects.
From https://github.com/espressif/esp-idf
   e2e4cd1a7..4fe04f115  master       -> origin/master
   80180f470..94e2981d1  release/v3.1 -> origin/release/v3.1
   08caf05cd..c604f980d  release/v3.2 -> origin/release/v3.2
   c44a9f9a4..6c9c253ed  release/v3.3 -> origin/release/v3.3
   fb14ab08c..57a5a486f  release/v4.0 -> origin/release/v4.0
 * [new tag]             v4.1-beta1   -> v4.1-beta1

# git checkout -b scan-fix v4.0
warning: unable to rmdir 'components/aws_iot/aws-iot-device-sdk-embedded-C': Directory not empty
warning: unable to rmdir 'components/bt/lib': Directory not empty
warning: unable to rmdir 'components/esp32/lib': Directory not empty
warning: unable to rmdir 'components/micro-ecc/micro-ecc': Directory not empty
warning: unable to rmdir 'components/nimble/nimble': Directory not empty
Checking out files: 100% (4618/4618), done.
M	components/coap/libcoap
M	components/lwip/lwip
M	components/mqtt/esp-mqtt
Previous HEAD position was 143d26aa4 Merge branch 'bugfix/doc_package_versions_v3.3' into 'release/v3.3'
Switched to a new branch 'scan-fix'

# git cherry-pick 48bd2d74b4209bbad7c29d2d856810ee43a1a92c
error: could not apply 48bd2d74b... NimBLE: Add Host based privacy (RPA) feature support
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'
No idea what that error means !

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by devnull » Tue Mar 31, 2020 9:27 am

Just tried this on a fresh clone and get the same error:

Code: Select all

git clone https://github.com/espressif/esp-idf.git esp-idf.b6
cd esp-idf.b6
git fetch origin
git checkout -b scan-fix v4.0
# git cherry-pick 48bd2d74b4209bbad7c29d2d856810ee43a1a92c
error: could not apply 48bd2d74b... NimBLE: Add Host based privacy (RPA) feature support
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by devnull » Tue Mar 31, 2020 10:16 am

Can this be worked-around by stopping & restarting the scan periodically ?

Just trying this out, not sure if it will solve the problem:

Code: Select all

def irq(event,adv):
  if event == _IRQ_SCAN_RESULT:
    if adv[0] != 0 or len(adv[4]) != 22 or adv[4][4:7] != b'\x02 \xaa': return
    else:
      if _DEBUG: print(adv)
      schedule(found,adv)
  
  elif event == _IRQ_SCAN_COMPLETE:
    schedule(restart,event)  

def start(ms=60000):
  bton();
  bt.irq(handler=irq)
  bt.gap_scan(ms,10000, 10000)

def restart(event):
  stop()
  pass
  start()

def stop():
  bt.gap_scan(None)
  btoff()

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by jimmo » Tue Mar 31, 2020 1:03 pm

It's a git conflict error, resolve it the usual way. Because the conflicts are with the submodules, you just have to go and fetch them and checkout them to hte right revision.

But I'm not really sure it's worth the effort -- I have no idea if these patches will build on top of v4.0.

I'll ask them if they're willing to backport to v4.0. I don't really have time right now to update MicroPython to IDF v4.1-beta1.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by jimmo » Tue Mar 31, 2020 11:47 pm

Oh... they did backport it to v4.0 but they just re-used the same tag. Argh. This cursed chip.

Anyway. To test this out. In your esp-idf repo,

Code: Select all

git fetch origin
git checkout origin/release/v4.0
git reset --hard HEAD
git submodule update --init --recursive
git rev-parse HEAD
The last line should be a3f3c7bdc3e81d88eba076c31cd92bca9fadd02c

Update esp32/Makefile and set ESPIDF_SUPHASH_V4 to that, then "make clean" and "make". This compiles for me, but I haven't had a chance to test it yet.

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by devnull » Thu Apr 02, 2020 12:04 am

Thanks Jimmo;

Did that and flashed the firmware last night, first test showed that it failed even earlier than previously, but it was too late last night to investigate, so will try again today.

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by devnull » Fri Apr 03, 2020 12:08 am

This appears to solve the problem, it's been running for 1 day now and is still discovering.

Thanks so much

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: BLE Scan forever stops after 7+ Hours

Post by jimmo » Fri Apr 03, 2020 1:07 am

Great. Were you able to figure out why it failed the first time but works now?

Post Reply