Problem with Newline if i sending textfiles with udp-socket

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
Post Reply
Stefan72
Posts: 3
Joined: Sun May 17, 2020 11:43 am

Problem with Newline if i sending textfiles with udp-socket

Post by Stefan72 » Sun May 17, 2020 12:10 pm

Hello,

i am new with MicroPython and i would like to convert my Script written in LUA to Micropython.

With this i would like to send a Text-File from my own Windows-Application to the ESP8266 and write them down to the Flash.

I open a socket and waiting for incomming Data

Code: Select all

s_udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s_udp.bind(("", 4005))

while True:
    data, addr = s_udp.recvfrom(1024)

    if Debug:
        print("(Debug): received:", data, "from:", addr)
        
        
 ... do something with data ......       
My Problem is, it looks like that Micropython is changing the 2 Newline-Bytes from original $0d$0a to \r\n ! If i write this to Flash then it is not identical with that File that i am sending.

This is my Testfile

Code: Select all

Hello,

this is a testfile to show my Problem with Newline !

Best Regards
This is what i get in data

Code: Select all

(Debug): received: Hello,\r\n\r\nthis is a testfile to show my Problem with Newline !\r\n\r\nBest Regards\r\n' from ('192.168.1.10', 53963
What should i do to receive the File correctly ? With my Lua-Script it does !


Best Regards

Stefan

Christian Walther
Posts: 109
Joined: Fri Aug 19, 2016 11:55 am

Re: Problem with Newline if i sending textfiles with udp-socket

Post by Christian Walther » Sun May 17, 2020 5:16 pm

Are you sure you quoted that debug output exactly? The (lack of) quotes and parentheses look suspicious.

What I get is something like

Code: Select all

(Debug): received: b'Hello\r\nWorld' from: ('192.168.0.101', 58309)
Note the b, which indicates that this is a bytes, not a string. print() prints those as their repr(). If you want to print the contents as a string, you need to convert it first: str(data, 'utf-8'). For saving to a file though, bytes are exactly what you need.

In other words, in all likelihood everything is correct with what you receive. Try printing len(data) too and you’ll see that those \r and \n are just one byte each.

SpotlightKid
Posts: 416
Joined: Wed Apr 08, 2015 5:19 am

Re: Problem with Newline if i sending textfiles with udp-socket

Post by SpotlightKid » Sun May 17, 2020 8:10 pm

Also, make sure that you open the file to write the data to in binary mode:

Code: Select all

with open("myfile.bin", "wb") as fobj:
    fobj.write(data)

Stefan72
Posts: 3
Joined: Sun May 17, 2020 11:43 am

Re: Problem with Newline if i sending textfiles with udp-socket

Post by Stefan72 » Sun May 17, 2020 10:02 pm

@Christian

You are absolutly right. I removed the 'b from the Output because i was thinking it has nothing to do with my problem. I didn't know what it means. :oops:

@SpotlightKid

I am opening the File in binary-mode

TempFile = open('sysupdate.tmp', 'wb')

i am writing with

TempFile.write(buffer)

buffer is the parameter of a function that writes the received File in chunks of 1024 bytes. I call it from the main-routine with

functionname(data)

after all chunks are received i close the file.

TempFile.close()

My Problem is, if i download the written file sysupdate.tmp from the ESP8266 it looks like the same as the Print-Output

Code: Select all

Hello,\r\n\r\nthis is a testfile to show my Problem with Newline !\r\n\r\nBest Regards\r\n
There is not the original $0d$0a in the File rather \r\n ! What i am doing wrong ?


I hope you can understand my problem, my english is not perfect.

Best Regards

Stefan

User avatar
dhylands
Posts: 3385
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Problem with Newline if i sending textfiles with udp-socket

Post by dhylands » Mon May 18, 2020 5:46 am

b’\x0d’ is the same as b’\r’ and b’\x0a’ is the same as b’\n’

Micropython (and python) always use the shortest representation when printing.

Stefan72
Posts: 3
Joined: Sun May 17, 2020 11:43 am

Re: Problem with Newline if i sending textfiles with udp-socket

Post by Stefan72 » Mon May 18, 2020 7:16 am

Hello dhylands,

i know what you say. And #13#10 is the same as $0d$0a and the same as \r\n !

But if i write the received Data to a File, which came with $0d$0a from the original source file, micropython change this to \r\n\.

If i open the source file with a Texteditor it looks like....

Code: Select all

Hello,

this is a testfile to show my Problem with Newline !

Best Regards

With a Hex-Editor i see the $0d$0a at the End of every Line

If i open the received File from the ESP8266 with a Texteditor it looks like ....

Code: Select all

Hello,\r\n\r\nthis is a testfile to show my Problem with Newline !\r\n\r\nBest Regards\r\n
This is not the same File anymore ! But why ?

If i do this with nodemcu and my Lua-Script it works like it should. Sendfile and Receivedfile are identical.


Best Regards

Stefan

Post Reply