How to get a binary string into a bytearray?

RP2040 based microcontroller boards running MicroPython.
Target audience: MicroPython users with an RP2040 boards.
This does not include conventional Linux-based Raspberry Pi boards.
Post Reply
User avatar
Pip
Posts: 11
Joined: Mon Jul 12, 2021 6:37 pm
Contact:

How to get a binary string into a bytearray?

Post by Pip » Mon Jul 19, 2021 5:53 pm

Platform:
  • Raspberry Pi-Pico connected to Windows 10 64 bit
  • Coding in MicroPython 3.7.9 (32-bit) via Thonny 3.3.3

I am trying to get a binary string of 0s & 1s into a byte array.
If I convert them into a character first it’s okay, that is until the values go over 127d (8 bits) then I get this:

Code: Select all

num8Bits = ['01111110',
            '11000011',
            '11000011',
            '11000011',            
            '01111110',
            '01111110',
            '11000011',
            '11000011',
            '11000011',
            '11000011',
            '01111110']

y=''
for x in num8Bits:
    y += chr(int(x,2))

mybytearray = bytearray(y) 
print (mybytearray)

Expected >>bytearray(b'\x7e\xc3\xc3\xc3\x7e\x7e\xc3\xc3\xc3\xc3\x7e')
Result   >>bytearray(b'~\xc3\x83\xc3\x83\xc3\x83~~\xc3\x83\xc3\x83\xc3\x83\xc3\x83~')  

I have also tried building the string as a hex set of values as opposed to characters, these assemble as bytes such as 0x7e0xc30xc30xc30x7e…. and so on
But if I try the recommend method of something like bytes.fromhex(‘ 7ec3c3c37e’) I get : File AttributeError: type object 'bytes' has no attribute 'fromhex'

Is this a limitation of MicroPython over python I wonder?

Any pointers would be most helpful.

Cheers
Last edited by Pip on Mon Jul 19, 2021 11:21 pm, edited 1 time in total.

danjperron
Posts: 51
Joined: Thu Dec 27, 2018 11:38 pm
Location: Québec, Canada

Re: How to get a binary string into a bytearray?

Post by danjperron » Mon Jul 19, 2021 7:49 pm

Hi Pip,

your expected answer is wrong!

from your source array
num8Bits = ['01111110',
'11000011', ...

you can't have 0x7e,0x7e,... it should be 0x7e,0xc3 and that is what you have.

B.T.W. the '~' tidle is 0x7e. You have a string of byte array. this show a string. Everything not ASCII will be show with the 0xVV but if it is an ASCII character (>0x1f and < 0x80) it will be the character.

Maybe just try

[ hex(int(i,2)) for i in num8Bits] ==> this is not a bytearray but a visual string of the array in hex.
the byte array is bytearray([ int(i,2) for i in num8Bits])

User avatar
Pip
Posts: 11
Joined: Mon Jul 12, 2021 6:37 pm
Contact:

Re: How to get a binary string into a bytearray?

Post by Pip » Mon Jul 19, 2021 11:37 pm

Hi Dan
Thanks for your reply.

Yep you are right, that was a typo in my original post, which I have now corrected/edited for those who follow this thread.

Yes I understand that MicroPython will print a valid character such as x7e as a ~, but I still can’t understand the result from passing the string into a bytearray. Note that the bytearray has 18 bytes in it, not the 11 that were passed to it from the string.
danjperron wrote:
Mon Jul 19, 2021 7:49 pm
Maybe just try

[ hex(int(i,2)) for i in num8Bits] ==> this is not a bytearray but a visual string of the array in hex.
the byte array is bytearray([ int(i,2) for i in num8Bits])
Using hex(int(i,2)) yields results >> 0x7e0xc30xc30xc30x7e0x7e0xc30xc30xc30xc30x7e

Which is correct and as expected, and something I had already tried, but couldn't get it to into a bytearray. I did try to replace the ”0x” with a ” \x” in the string, but the back slash tripped up the replace syntax, even when escaping the backslash with "\\x" which passed the two backslashes into the string rather than the one: :? As in:

Code: Select all

num8Bits = ['01111110',
            '11000011',
            '11000011',
            '11000011',            
            '01111110',
            '01111110',
            '11000011',
            '11000011',
            '11000011',
            '11000011',
            '01111110']

z=''
for x in num8Bits:
    z += hex(int(x,2))
z=z.replace("0x","\\x")
print (z)
mybytearray = bytearray(z)
print (mybytearray)
Giving an output of:
>>x7e\xc3\xc3\xc3\x7e\x7e\xc3\xc3\xc3\xc3\x7e
>>bytearray(b'\\x7e\\xc3\\xc3\\xc3\\x7e\\x7e\\xc3\\xc3\\xc3\\xc3\\x7e')

User avatar
dbrazil
Posts: 14
Joined: Mon Apr 26, 2021 5:52 pm

Re: How to get a binary string into a bytearray?

Post by dbrazil » Tue Jul 20, 2021 12:21 am

HI Pip,

I believe your question is in the wrong place, since it doesn't solely relates to the RP2040, but to the MicroPython syntax.

Anyways, the easiest way of dealing with it is converting to integer, and then appending to the bytearray object.

Code: Select all

y = bytearray() #bytearray object to receive your data
for i in num8Bits:
    v = int(i, 2) #Convert string to integer
    y.append(v) #Append integer to byte array
    
print(y)
>>> bytearray(b'~\xc3\xc3\xc3~~\xc3\xc3\xc3\xc3~')


User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: How to get a binary string into a bytearray?

Post by jimmo » Tue Jul 20, 2021 12:23 am

This isn't answering your specific question but if you just want to define a bytes/bytearray from a collection of bits defined in the program, you can use the "0b..." literal prefix.

Code: Select all

b = bytes((0b01111110,
    0b11000011,
    0b11000011,
    0b11000011,            
    0b01111110,
    0b01111110,
    0b11000011,
    0b11000011,
    0b11000011,
    0b11000011,
    0b01111110))
In your code what's going wrong is that when you _print_ a bytearray, it formats it nicely by escaping the non-printables (using the \x syntax) but this isn't how they're internally stored. i.e. bytearray("\\x09") isn't going to do what you want.

If you do want to start with a list of strings-of-bits, then this would work

Code: Select all

num8Bits = ['01111110',
            '11000011',
            '11000011',
            '11000011',
            '01111110',
            '01111110',
            '11000011',
            '11000011',
            '11000011',
            '11000011',
            '01111110']

mybytes = bytes(int(x, 2) for x in num8Bits)
print(mybytes)
(i.e. make a bytes whose bytes are the base-2 conversion of x, where x is each element in num8Bits).

It prints

Code: Select all

b'~\xc3\xc3\xc3~~\xc3\xc3\xc3\xc3~'

User avatar
Pip
Posts: 11
Joined: Mon Jul 12, 2021 6:37 pm
Contact:

Re: How to get a binary string into a bytearray? [Solved]

Post by Pip » Tue Jul 20, 2021 11:49 am

jimmo wrote:
Tue Jul 20, 2021 12:23 am

In your code what's going wrong is that when you _print_ a bytearray, it formats it nicely by escaping the non-printables (using the \x syntax) but this isn't how they're internally stored. i.e. bytearray("\\x09") isn't going to do what you want.


(i.e. make a bytes whose bytes are the base-2 conversion of x, where x is each element in num8Bits).
Ah, I see what is going on now. I was taking things literally as I was seeing them, not understanding what was happening under the hood. My background is in assembly and as such all this abstraction is relatively new to me, and dare I say a little frustrating at times. :evil:

Thanks Jimmo mate, URA*
Last edited by Pip on Tue Jul 20, 2021 12:06 pm, edited 1 time in total.

User avatar
Pip
Posts: 11
Joined: Mon Jul 12, 2021 6:37 pm
Contact:

Re: How to get a binary string into a bytearray?

Post by Pip » Tue Jul 20, 2021 12:04 pm

dbrazil wrote:
Tue Jul 20, 2021 12:21 am
HI Pip,

I believe your question is in the wrong place, since it doesn't solely relates to the RP2040, but to the MicroPython syntax.

Anyways, the easiest way of dealing with it is converting to integer, and then appending to the bytearray object.

Code: Select all

y = bytearray() #bytearray object to receive your data
for i in num8Bits:
    v = int(i, 2) #Convert string to integer
    y.append(v) #Append integer to byte array
    
print(y)
>>> bytearray(b'~\xc3\xc3\xc3~~\xc3\xc3\xc3\xc3~')

Hi dbrazil

Thank you for your response. I am already using the construct:

Code: Select all

num8l = bytearray(b'\x7e\x7e\xc3\xc3\xc3\x7e\xc3\xc3\xc3\xc3\x7e\x7e')
in my code., but wanted to get a visual representation of a character bit map to help during testing, hence my question.

I posted it in the Pico section because it is specific to OLED display manipulation on the SSH1106 board that I am playing with on my Pico. As such I felt it best to lodge my question here.

Once again many thanks for your response

Pip

Post Reply