Page 1 of 1

Problem with Newline if i sending textfiles with udp-socket

Posted: Sun May 17, 2020 12:10 pm
by Stefan72
Hello,

i am new with MicroPython and i would like to convert my Script written in LUA to Micropython.

With this i would like to send a Text-File from my own Windows-Application to the ESP8266 and write them down to the Flash.

I open a socket and waiting for incomming Data

Code: Select all

s_udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s_udp.bind(("", 4005))

while True:
    data, addr = s_udp.recvfrom(1024)

    if Debug:
        print("(Debug): received:", data, "from:", addr)
        
        
 ... do something with data ......       
My Problem is, it looks like that Micropython is changing the 2 Newline-Bytes from original $0d$0a to \r\n ! If i write this to Flash then it is not identical with that File that i am sending.

This is my Testfile

Code: Select all

Hello,

this is a testfile to show my Problem with Newline !

Best Regards
This is what i get in data

Code: Select all

(Debug): received: Hello,\r\n\r\nthis is a testfile to show my Problem with Newline !\r\n\r\nBest Regards\r\n' from ('192.168.1.10', 53963
What should i do to receive the File correctly ? With my Lua-Script it does !


Best Regards

Stefan

Re: Problem with Newline if i sending textfiles with udp-socket

Posted: Sun May 17, 2020 5:16 pm
by Christian Walther
Are you sure you quoted that debug output exactly? The (lack of) quotes and parentheses look suspicious.

What I get is something like

Code: Select all

(Debug): received: b'Hello\r\nWorld' from: ('192.168.0.101', 58309)
Note the b, which indicates that this is a bytes, not a string. print() prints those as their repr(). If you want to print the contents as a string, you need to convert it first: str(data, 'utf-8'). For saving to a file though, bytes are exactly what you need.

In other words, in all likelihood everything is correct with what you receive. Try printing len(data) too and you’ll see that those \r and \n are just one byte each.

Re: Problem with Newline if i sending textfiles with udp-socket

Posted: Sun May 17, 2020 8:10 pm
by SpotlightKid
Also, make sure that you open the file to write the data to in binary mode:

Code: Select all

with open("myfile.bin", "wb") as fobj:
    fobj.write(data)

Re: Problem with Newline if i sending textfiles with udp-socket

Posted: Sun May 17, 2020 10:02 pm
by Stefan72
@Christian

You are absolutly right. I removed the 'b from the Output because i was thinking it has nothing to do with my problem. I didn't know what it means. :oops:

@SpotlightKid

I am opening the File in binary-mode

TempFile = open('sysupdate.tmp', 'wb')

i am writing with

TempFile.write(buffer)

buffer is the parameter of a function that writes the received File in chunks of 1024 bytes. I call it from the main-routine with

functionname(data)

after all chunks are received i close the file.

TempFile.close()

My Problem is, if i download the written file sysupdate.tmp from the ESP8266 it looks like the same as the Print-Output

Code: Select all

Hello,\r\n\r\nthis is a testfile to show my Problem with Newline !\r\n\r\nBest Regards\r\n
There is not the original $0d$0a in the File rather \r\n ! What i am doing wrong ?


I hope you can understand my problem, my english is not perfect.

Best Regards

Stefan

Re: Problem with Newline if i sending textfiles with udp-socket

Posted: Mon May 18, 2020 5:46 am
by dhylands
b’\x0d’ is the same as b’\r’ and b’\x0a’ is the same as b’\n’

Micropython (and python) always use the shortest representation when printing.

Re: Problem with Newline if i sending textfiles with udp-socket

Posted: Mon May 18, 2020 7:16 am
by Stefan72
Hello dhylands,

i know what you say. And #13#10 is the same as $0d$0a and the same as \r\n !

But if i write the received Data to a File, which came with $0d$0a from the original source file, micropython change this to \r\n\.

If i open the source file with a Texteditor it looks like....

Code: Select all

Hello,

this is a testfile to show my Problem with Newline !

Best Regards

With a Hex-Editor i see the $0d$0a at the End of every Line

If i open the received File from the ESP8266 with a Texteditor it looks like ....

Code: Select all

Hello,\r\n\r\nthis is a testfile to show my Problem with Newline !\r\n\r\nBest Regards\r\n
This is not the same File anymore ! But why ?

If i do this with nodemcu and my Lua-Script it works like it should. Sendfile and Receivedfile are identical.


Best Regards

Stefan