Question about the dht module (and maybe some more)

[MostlyHarmless]

Disclaimer: I am relatively new to the ESP32 and Micropython. I am familiar with microcontrollers in general (did some hobby tasks with MicroChip PICs in C) and know my way around circuits in general.

I am trying to learn my way around the Micropython code base the way I usually do it in a new environment. Grab some random gadget from the box, wire it up and try getting it to work. Then study the code to learn how it works (or why it doesn't work the way I thought it should).

So today I am playing with a DHT22 and immediately was struck by the huge run time of measure(). 273ms to query something that responds in a protocol that needs 100us per bit on average and only sends 5 bytes? That looks wrong. No offense, but it just looks wrong.

Turns out that I can easily reduce that "blocking" response time by removing the initial "pull high" delay of 250ms and making sure the port is left in "dht_high" state on exit of function dht_readinto(). A call to measure() now only takes 23ms (yupp, 273 - 250 = 23) and it doesn't seem to hurt. So what is the purpose of that initial 250ms delay in high state? It can't be protecting against polling the DHT22/11 too fast, because that would require 2 seconds cooloff time (1 second in the DHT11 case). I can only come up with one reason, which is that without an initially enforced high state for some time there is a chance for a failed communication on first try. But that should not warrant to waste a quarter second on every measure() call forever.

Hardware arrangements seldom change during runtime on embedded systems, so leaving the port in high state until the next call to measure() seems safe to me.

Either I am missing something or there is some room to improve performance here.


Regards, Jan

[jimmo]

This seems to be very common in most of the DHT22/AM2303 drivers. People seem to say stuff like "The sensors claim to not like being read more than every 250 ms." but I can't find any definitive documentation or explanation for this. I'm guessing that there are a bunch of different variants and some really do take this long to wake from sleep. Or perhaps this has somehow cargo-culted its way through an entire family tree of DHT drivers.

I agree though that the current situation definitely isn't ideal -- even in the case where you read the sensor only very occasionally (i.e. once a minute) it's still not good that it has to block for 250ms. But maybe in that case you do want to let the device go to sleep. So the API might need to change...

[MostlyHarmless]

People seem to say stuff like "The sensors claim to not like being read more than every 250 ms." but I can't find any definitive documentation or explanation for this.

The documentation I found says that the accuracy goes down if polled faster than every 2 seconds for the DHT22. I don't know what's inside of the white cage, but humidity sensors are often based on capacitors. If that is the case here then that makes perfectly sense.

I agree though that the current situation definitely isn't ideal -- even in the case where you read the sensor only very occasionally (i.e. once a minute) it's still not good that it has to block for 250ms. But maybe in that case you do want to let the device go to sleep. So the API might need to change...

I dislike API changes that break backwards compatibility. There is an opportunity here to do this backwards compatible though. My proposal is to make DHTBase.measure(self, delay=250) and leave the GPIO pin in high state after the call. That way existing applications should not see a change in behavior while new ones can customize the delay. Reading the code again, the line should be in high state anyway once the DHT is done sending. I'll verify that though.

There is one other thing in that code that caught my attention. That is simple integer arithmetic done on ticks. Which boils down to the ports actually not even exposing ticks_diff() functionality on the C level. There is only a ticks_diff() method dealing with mp_obj_t arguments. Doing simple int arithmetic on ticks means there is a potential wrap around problem here.

Where I come from (big beefy database servers) stuff as time sensitive as this is usually thrown into macros or inline definitions to even avoid prefetch queue problems caused by function calls. However, both methods bloat the code, so I rather ask for your guidance on this as well.


Thanks, Jan

[MostlyHarmless]

Reading the code again, the line should be in high state anyway once the DHT is done sending. I'll verify that though.

Verified. I removed the lines that do

Code: Select all

mp_hal_pin_od_high_dht(pin);
mp_hal_delay_ms(250);

then ran

Code: Select all

while True:
    try:
        d.measure()
        print("Temp.: {0:5.1f}".format(d.temperature()))
    except Exception as e:
        print(str(e))
    time.sleep(0.5)

and this is the result:

ScreenImg_002.png (31.42 KiB) Viewed 6028 times

I know, only resting for 0.5 seconds isn't right, but this was for analyzing the digital signal, not to test sensor accuracy.

The lower half of the screen is a zoom in to the upper half's black background portion. So the line is "high" until the ESP32 pulls it "low" for exactly 18ms, after which the DHT22 sends its response (using less than 5ms). Then the line stays "high" again until the next interval.

Thinking of that ... I actually need to design this better. The improved (still backwards compatible) API should be able to perform both, the initial delay (250ms default) and the start pulse (18ms) by sleeping via uasyncio's sleep_ms().

Oh well, back to the drawing board it is.


Regards, Jan


Edit: uploading image for permanent link

[MostlyHarmless]

... The improved (still backwards compatible) API should be able to perform both, the initial delay (250ms default) and the start pulse (18ms) by sleeping via uasyncio's sleep_ms().

Did some more testing/hacking with uasyncio and it turns out that handling the 18ms "start pulse" delay async isn't really worth the effort ATM. uasyncio needs about 6..8ms to switch tasks, so trying to do so twice within that 18ms window only makes it almost certain that the DHT module doesn't recognize the start pulse at all because it gets too long.

Other than that the results look good. With this commit in place

https://github.com/wieck/micropython/co ... fcf38ebcc3

I can run something like

Code: Select all

# ----
# main.py for test3 - DHT stuff
# ----
import machine
import dht
import utime
import uasyncio as asyncio

async def every_20():
    wait_until = utime.ticks_add(utime.ticks_ms(), 20)
    while True:
        wait_ms = utime.ticks_diff(wait_until, utime.ticks_ms())
        if wait_ms > 0:
            await asyncio.sleep_ms(wait_ms)
        diff = utime.ticks_diff(utime.ticks_ms(), wait_until)
        wait_until = utime.ticks_add(wait_until, 20)
        if abs(diff) > 5:
            print("every_20(): diff was {0:3d}ms".format(diff))

async def print_temp(d):
    wait_until = utime.ticks_add(utime.ticks_ms(), 2000)
    while True:
        wait_ms = utime.ticks_diff(wait_until, utime.ticks_ms())
        if wait_ms > 0:
            print("waiting for {0:4d}ms".format(wait_ms))
            await asyncio.sleep_ms(wait_ms)
        wait_until = utime.ticks_add(wait_until, 2000)

        start = utime.ticks_ms()
        try:
            d.measure()
        except Exception as ex:
            print(str(ex))
        end = utime.ticks_ms()
        print("Temp: {0:5.1f} delay: {1:3d}ms".format(d.temperature(), utime.ticks_diff(end, start)))

d = dht.DHT22(machine.Pin(4), delay = 0)

loop = asyncio.get_event_loop()
loop.create_task(every_20())
loop.create_task(print_temp(d))
loop.run_forever()

The output of that code looks like

Code: Select all

Temp:  21.4 delay:  22ms
waiting for 1970ms
every_20(): diff was  17ms
Temp:  21.4 delay:  22ms
waiting for 1971ms
every_20(): diff was  16ms

So the call to DHT22.measurement() now takes about 22ms (down from 270+). This makes sense as the total communication from the beginning of the 18ms start pulse to the DHT being done with transmitting looks like 22.2ms on my oscilloscope. With two task switches this causes a delay in the every_20() task of about 17ms when it cannot be scheduled on time while the DHT22.measurement() call is running. Note that the every_20() task was explicitly designed to expose this problem.

This branch is not ready to merge yet as it totally lacks documentation updates. I would like to get feedback before doing those.


Regards, Jan

[uCTRL]

DHT22 data sheet.

https://lastminuteengineers.com/datashe ... asheet.pdf

Specified sampling rate (0.5Hz or 2 seconds).

I guess you can test and confirm actual sampling rate, by observing the rate of change in readings?

[MostlyHarmless]

Specified sampling rate (0.5Hz or 2 seconds).

I guess you can test and confirm actual sampling rate, by observing the rate of change in readings?

Did as suggested and am as clueless as before what that sampling rate is really supposed to mean.

Test setup was as before now toggling the interval of reading the sensor every 20 seconds between 2000ms and 500ms. Then heating it up with a hair dryer and letting it cool off again several times. The interval at which the sensor is queried seems to have no effect on the result. When left alone the reading is the exact same to 0.1 of both values at both polling intervals. When heating it up rapidly the values jump (as expected) in intervals as short as 500ms (not really expected).

What I do get are occasional ETIMEDOUT errors when using a polling interval below 470ms. Seems the DHT doesn't recognize the 18ms "low" start pulse if the line wasn't "high" for at least 450ms (at least for the specimen of DHT22 I got ... YMMV).

Based on all that my proposal for the doc changes is to suggest not to call .measure() faster than every X ms when used in an eventloop or a uasyncio coro. To error on the safe side, X should be the sampling rate from the data sheet (2000ms for DHT22, 1000ms for DHT11). Individual sensors may work faster, but doing so can lead to occasional ETIMEDOUT errors.


Regards, Jan

[MostlyHarmless]

Expecting a couple 8266s in the mail tomorrow, so at least I will be able to test these changes with two different chips.

[MostlyHarmless]

Expecting a couple 8266s in the mail tomorrow, so at least I will be able to test these changes with two different chips.

Either I am doing something stupid, the modules I got today are crap or the ESP8266 port is somehow broken in the OPEN_DRAIN department.

I can't get the dht module to work with an ESP8266 in open drain mode. Granted, I only tested GPIO4 and GPIO5, but since the datasheet says all GPIO can be used with open drain, I don't expect testing more ports will change anything.

ports/esp8266 seems to try to enable open drain like the Espressif stuff does it for I2C ports. But that doesn't seem to work. I can see on the O-Scope that the DHT22 tries to pull the line low, but it can only drop the voltage by something like 0.4V. Not enough to trigger anything.

If I hack the code in drivers/dht/dht.c to toggle the pin between IN and OUT modes (instead of using OPEN_DRAIN), it all works beautifully and the O-Scope shows exactly what one would expect.

Are these ESP8266 modules I got broken, or is there a bug in the esp8266 port?


Regards, Jan

[MostlyHarmless]

This whole thing seems to be an old issue.

What I don't understand is that this commit seems to have fixed it in some cases. But that code is enabling the pin and driving it actively high. If I remove that very macro and let the dht driver fall back to the regular mp_hal_pin_od_high() macro and thus letting the pin float high, it all works "for me".

There are different products out there all labeled DHT22. Some come with a small breakout board that has a pull-up resistor (usually in the 4K7 to 10K range) and a capacitor on them, some are just the bare sensor. This might be a case where one size doesn't fit all and we need different DHT modes in the driver.


Regards, Jan