random offtopic posts

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
jms
Posts: 108
Joined: Thu May 05, 2016 8:29 pm
Contact:

random offtopic posts

Post by jms » Mon Nov 14, 2016 10:33 am

It's a very very sad state of affairs indeed when a device capable of transferring the odd gigabyte without keeling over is considered in anyway satisfactory. No wonder the banks are in the state they are.

Perhaps I'm getting too old for this.

(Supplying through USB is absolutely fine but you must have a few microfarads on the board. Saying it's present but at the other end of a cable shows lack of understanding.)

Jon

Lysenko
Posts: 62
Joined: Wed Aug 17, 2016 1:21 pm

Re: Downloading 1GB of data with MicroPython ESP8266 port

Post by Lysenko » Mon Nov 14, 2016 11:48 am

jms wrote:It's a very very sad state of affairs indeed when a device capable of transferring the odd gigabyte without keeling over is considered in anyway satisfactory. No wonder the banks are in the state they are.
That's a common reaction from electronic engineers (in my experience) and explains both why I2C was designed without the concept of a timeout and why they find networks (especially wireless) so frustrating.

With a network you have to assume that communication is a statistical probability rather than an electrical certainty and design on the basis that failure is part of normal function. Investing time in trying to reduce the failure frequency is often a misallocation of effort because the failure condition will still arise so it is often more productive to invest in mitigation and recovery rather than prevention.

One sensor network I've worked on is specified as being able to maintain 3sigma accuracy with 50% of nodes offline and 90% of communication packets disrupted.

jms
Posts: 108
Joined: Thu May 05, 2016 8:29 pm
Contact:

Re: Downloading 1GB of data with MicroPython ESP8266 port

Post by jms » Mon Nov 14, 2016 12:08 pm

The ESP consumes about 80mA and gets a little warm as expected and no problem at all even over a long cable and lots of hubs as long as there is a little local supply decoupling. If something is hot rather than warm there's something very wrong.

(Most modern electronics for the most part works perfectly well when it's hot enough to burn skin though reliability decreases.)

There's a difference between packet loss (acceptable for those of use that used and still use thick and thin Ethernet) as well as Wi-Fi for its own reasons and software that keels over for no reason anybody really understands. I accept packet loss and even link failure but not unexplained software behaviour. A new project has had me dig out my thick ethernet transceivers.

I2C timing is purely a product of master clocking (more messy if multi0ple masters) and clock stretching. Many devices do have timeouts. One product my code drives has dozens of I2C switches and multiplexers and also a load of resets for them and code that attempts to recover from lockups. No matter what the main board does you can (hot) plug 48 SFPs and each of those contains at least an EEPROM. Took me weeks to get that solid.

Jon

Lysenko
Posts: 62
Joined: Wed Aug 17, 2016 1:21 pm

Re: Downloading 1GB of data with MicroPython ESP8266 port

Post by Lysenko » Mon Nov 14, 2016 1:05 pm

jms wrote: There's a difference between packet loss (acceptable for those of use that used and still use thick and thin Ethernet) as well as Wi-Fi for its own reasons and software that keels over for no reason anybody really understands. I accept packet loss and even link failure but not unexplained software behaviour.
I've seen cases where with 2000 deployed units you get a couple of failures (probably SW, but maybe silicon or something else) every 24 hours. Reproducing that on a bench would take years - and it might not turn out to be SW anyway.

Then there's the tank gun instrumentation that failed (sponaneous reboot) when a particular sort of shell was fired. For various reasons the EM environment there was highly complex, partly classified and possibly unshieldable (if the issue turned out to be magnetic).

That's why I treat processors much the same as networks. I expect the WDT (etc) to periodically kick in for no apparent reason and devote time to optimising the reboot process and state persistence rather than trying to avoid the failure condition completely. With something like MP there is heap fragmentation to consider so I would trap out the processor from time to time on general principle anyway.

My point about I2C was that Phillips didn't design timeout into the protocol because they worked on the assumption that if something fails it is acceptable to just lock up the system and declare "somebody else's problem". A TCP engineer would never think like that. As you say, I2C often has timeout today, but that's largely because people spotted the design error and then things like SMBus formalised a way of dealing with it.

jms
Posts: 108
Joined: Thu May 05, 2016 8:29 pm
Contact:

Re: Downloading 1GB of data with MicroPython ESP8266 port

Post by jms » Mon Nov 14, 2016 3:54 pm

Hate to spoil things but TCP itself has no timeout unless you enable keepalives.

SMBus is more about not paying royalties and maybe cranking up the speed a little. As is TWSI and PMBus. Once you've checked the voltage and frequency and (unlikely) 10-bit addressing they're all the same. Very few devices use the error checking facilities.

Lysenko
Posts: 62
Joined: Wed Aug 17, 2016 1:21 pm

Re: Downloading 1GB of data with MicroPython ESP8266 port

Post by Lysenko » Mon Nov 14, 2016 4:37 pm

jms wrote:Hate to spoil things but TCP itself has no timeout unless you enable keepalives.
Yes it does. There would be no way to orchestrate packet re-transmission otherwise.

http://www.pcvr.nl/tcpip/tcp_time.htm

Lysenko
Posts: 62
Joined: Wed Aug 17, 2016 1:21 pm

random offtopic posts

Post by Lysenko » Mon Nov 14, 2016 4:40 pm

jms wrote: SMBus is more about not paying royalties and maybe cranking up the speed a little. As is TWSI and PMBus. Once you've checked the voltage and frequency and (unlikely) 10-bit addressing they're all the same. Very few devices use the error checking facilities.
SMBus is slower or equal to I2C, it doesn't improve speed, only robustness:

https://www.maximintegrated.com/en/app- ... mvp/id/476

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Downloading 1GB of data with MicroPython ESP8266 port

Post by pythoncoder » Tue Nov 15, 2016 9:57 am

Lysenko wrote:That's a common reaction from electronic engineers (in my experience) and explains both why I2C was designed without the concept of a timeout and why they find networks (especially wireless) so frustrating.
This particular electronics engineer is accustomed to designing radio links where unreliability has to be accounted for. After a lengthy and frustrating experience with MQTT I concluded that writing robust network code is a quite separate specialist skill which, alas, I don't possess.
Peter Hinch
Index to my micropython libraries.

jms
Posts: 108
Joined: Thu May 05, 2016 8:29 pm
Contact:

Re: random offtopic posts

Post by jms » Tue Nov 15, 2016 10:58 am

Yes of course TCP has transmission timeouts. I though it was to do with inactivity. My mistake. Yesterday was one of those days. I locked myself out due tue a keyring error later on.

Anyway the problem with network code is that 1) it is tricky and often misunderstood anyway and 2) This device's network stack is somewhat different to a heavyweight OS and not too well documented 3) The hardware possibly has its own issues

Lysenko
Posts: 62
Joined: Wed Aug 17, 2016 1:21 pm

Re: Downloading 1GB of data with MicroPython ESP8266 port

Post by Lysenko » Tue Nov 15, 2016 12:06 pm

pythoncoder wrote:This particular electronics engineer is accustomed to designing radio links where unreliability has to be accounted for. After a lengthy and frustrating experience with MQTT I concluded that writing robust network code is a quite separate specialist skill which, alas, I don't possess.
I was mostly thinking of my colleague at the next desk at the time ;)

Network code is very similar to wireless really. You transmit a packet and wait for an acknowledgement from the intended recipient(s), always remembering that the recipient might not recieve what you sent and might not (be able to) acknowledge it even if they do. TCP "connections" are just an illusion generated by the TCP stack implementing state, timeout and retries for you and blocking sockets with no timeout and no pre-emptive multi-threading are therefore highly questionable.

Your MQTT problems were nothing to do with your skill level IMO, they were to do with running a connection based SCADA protocol on top of an incompletely implemented socket stack. There is nothing you could have done besides fully implement setsockopt (in C) or use a connectionless protocol like CoAP (in Python). Personally I think that the latter is a better tool for sensor nets anyway.

Post Reply