Page 4 of 4
Re: Call for testsuite running results
Posted: Sat Apr 16, 2016 9:17 am
by pfalcon
gojimmypi wrote:
If my reply here is inappropriate, well I won't be offended if my message gets deleted. I do think it is related to the testsuite. I always think that if I have a problem, others might also. If you disagree, well... sorry. my bad.
No, you're welcome to. Just if you don't get a reply, please don't think there's a problem with testsuite which nobody knows how to resolve
. Always by default err on your side, and then it may be hard for somebody else to say what's wrong on your system, while you have it at your fingertips to investigate further.
Re: Call for testsuite running results
Posted: Sat Apr 16, 2016 9:25 am
by pfalcon
One thing to remember is that eps8266's "OS" may output debug information concurrently with a running test, in which case test will "fail" (few folks on this thread reported results which can be attributed to this). The provided instructions try to minimize that by asking to run testsuite after erasing flash (so the esp8266 doesn't try to connect to anything), but that's still not 100% guarantee. To get full guarantee, build specifically for testsuite with "make UART_OS=1". But I wouldn't recommend running such binary all the other time, because if something goes wrong later, you won't have useful debugging output to analyze/report.
So yep, testing is hard, and a particular issue is filtering out "false positives".