Page 1 of 2

random offtopic posts

Posted: Mon Nov 14, 2016 10:33 am
by jms
It's a very very sad state of affairs indeed when a device capable of transferring the odd gigabyte without keeling over is considered in anyway satisfactory. No wonder the banks are in the state they are.

Perhaps I'm getting too old for this.

(Supplying through USB is absolutely fine but you must have a few microfarads on the board. Saying it's present but at the other end of a cable shows lack of understanding.)

Jon

Re: Downloading 1GB of data with MicroPython ESP8266 port

Posted: Mon Nov 14, 2016 11:48 am
by Lysenko
jms wrote:It's a very very sad state of affairs indeed when a device capable of transferring the odd gigabyte without keeling over is considered in anyway satisfactory. No wonder the banks are in the state they are.
That's a common reaction from electronic engineers (in my experience) and explains both why I2C was designed without the concept of a timeout and why they find networks (especially wireless) so frustrating.

With a network you have to assume that communication is a statistical probability rather than an electrical certainty and design on the basis that failure is part of normal function. Investing time in trying to reduce the failure frequency is often a misallocation of effort because the failure condition will still arise so it is often more productive to invest in mitigation and recovery rather than prevention.

One sensor network I've worked on is specified as being able to maintain 3sigma accuracy with 50% of nodes offline and 90% of communication packets disrupted.

Re: Downloading 1GB of data with MicroPython ESP8266 port

Posted: Mon Nov 14, 2016 12:08 pm
by jms
The ESP consumes about 80mA and gets a little warm as expected and no problem at all even over a long cable and lots of hubs as long as there is a little local supply decoupling. If something is hot rather than warm there's something very wrong.

(Most modern electronics for the most part works perfectly well when it's hot enough to burn skin though reliability decreases.)

There's a difference between packet loss (acceptable for those of use that used and still use thick and thin Ethernet) as well as Wi-Fi for its own reasons and software that keels over for no reason anybody really understands. I accept packet loss and even link failure but not unexplained software behaviour. A new project has had me dig out my thick ethernet transceivers.

I2C timing is purely a product of master clocking (more messy if multi0ple masters) and clock stretching. Many devices do have timeouts. One product my code drives has dozens of I2C switches and multiplexers and also a load of resets for them and code that attempts to recover from lockups. No matter what the main board does you can (hot) plug 48 SFPs and each of those contains at least an EEPROM. Took me weeks to get that solid.

Jon

Re: Downloading 1GB of data with MicroPython ESP8266 port

Posted: Mon Nov 14, 2016 1:05 pm
by Lysenko
jms wrote: There's a difference between packet loss (acceptable for those of use that used and still use thick and thin Ethernet) as well as Wi-Fi for its own reasons and software that keels over for no reason anybody really understands. I accept packet loss and even link failure but not unexplained software behaviour.
I've seen cases where with 2000 deployed units you get a couple of failures (probably SW, but maybe silicon or something else) every 24 hours. Reproducing that on a bench would take years - and it might not turn out to be SW anyway.

Then there's the tank gun instrumentation that failed (sponaneous reboot) when a particular sort of shell was fired. For various reasons the EM environment there was highly complex, partly classified and possibly unshieldable (if the issue turned out to be magnetic).

That's why I treat processors much the same as networks. I expect the WDT (etc) to periodically kick in for no apparent reason and devote time to optimising the reboot process and state persistence rather than trying to avoid the failure condition completely. With something like MP there is heap fragmentation to consider so I would trap out the processor from time to time on general principle anyway.

My point about I2C was that Phillips didn't design timeout into the protocol because they worked on the assumption that if something fails it is acceptable to just lock up the system and declare "somebody else's problem". A TCP engineer would never think like that. As you say, I2C often has timeout today, but that's largely because people spotted the design error and then things like SMBus formalised a way of dealing with it.

Re: Downloading 1GB of data with MicroPython ESP8266 port

Posted: Mon Nov 14, 2016 3:54 pm
by jms
Hate to spoil things but TCP itself has no timeout unless you enable keepalives.

SMBus is more about not paying royalties and maybe cranking up the speed a little. As is TWSI and PMBus. Once you've checked the voltage and frequency and (unlikely) 10-bit addressing they're all the same. Very few devices use the error checking facilities.

Re: Downloading 1GB of data with MicroPython ESP8266 port

Posted: Mon Nov 14, 2016 4:37 pm
by Lysenko
jms wrote:Hate to spoil things but TCP itself has no timeout unless you enable keepalives.
Yes it does. There would be no way to orchestrate packet re-transmission otherwise.

http://www.pcvr.nl/tcpip/tcp_time.htm

random offtopic posts

Posted: Mon Nov 14, 2016 4:40 pm
by Lysenko
jms wrote: SMBus is more about not paying royalties and maybe cranking up the speed a little. As is TWSI and PMBus. Once you've checked the voltage and frequency and (unlikely) 10-bit addressing they're all the same. Very few devices use the error checking facilities.
SMBus is slower or equal to I2C, it doesn't improve speed, only robustness:

https://www.maximintegrated.com/en/app- ... mvp/id/476

Re: Downloading 1GB of data with MicroPython ESP8266 port

Posted: Tue Nov 15, 2016 9:57 am
by pythoncoder
Lysenko wrote:That's a common reaction from electronic engineers (in my experience) and explains both why I2C was designed without the concept of a timeout and why they find networks (especially wireless) so frustrating.
This particular electronics engineer is accustomed to designing radio links where unreliability has to be accounted for. After a lengthy and frustrating experience with MQTT I concluded that writing robust network code is a quite separate specialist skill which, alas, I don't possess.

Re: random offtopic posts

Posted: Tue Nov 15, 2016 10:58 am
by jms
Yes of course TCP has transmission timeouts. I though it was to do with inactivity. My mistake. Yesterday was one of those days. I locked myself out due tue a keyring error later on.

Anyway the problem with network code is that 1) it is tricky and often misunderstood anyway and 2) This device's network stack is somewhat different to a heavyweight OS and not too well documented 3) The hardware possibly has its own issues

Re: Downloading 1GB of data with MicroPython ESP8266 port

Posted: Tue Nov 15, 2016 12:06 pm
by Lysenko
pythoncoder wrote:This particular electronics engineer is accustomed to designing radio links where unreliability has to be accounted for. After a lengthy and frustrating experience with MQTT I concluded that writing robust network code is a quite separate specialist skill which, alas, I don't possess.
I was mostly thinking of my colleague at the next desk at the time ;)

Network code is very similar to wireless really. You transmit a packet and wait for an acknowledgement from the intended recipient(s), always remembering that the recipient might not recieve what you sent and might not (be able to) acknowledge it even if they do. TCP "connections" are just an illusion generated by the TCP stack implementing state, timeout and retries for you and blocking sockets with no timeout and no pre-emptive multi-threading are therefore highly questionable.

Your MQTT problems were nothing to do with your skill level IMO, they were to do with running a connection based SCADA protocol on top of an incompletely implemented socket stack. There is nothing you could have done besides fully implement setsockopt (in C) or use a connectionless protocol like CoAP (in Python). Personally I think that the latter is a better tool for sensor nets anyway.