problem with bytearray conversion

[EasyRider]

Struck a strange output from bytearray conversion.
See bytes 13 and 14.
Appears to be wrong if byte 13 or 14 values exceed 15 (0xF).

I am actually using this in uart.write(bytearray(message)) where the output on uart port is OK as expected , irrispective of byte 13 or 14 value..
But a print(bytearray(message)) produces the strange output for bytes 13 and 14.

Same strange behavior is observed in immediate REPL mode.
Have not looked at other bytes.
Development is on STM32F407 Discovery, fairly recent firmware.

>>> message
[1, 16, 0, 0, 0, 3, 6, 0, 6, 0, 2, 0, 3, 143, 65]
>>> bytearray(message)
bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x06\x00\x02\x00\x03\x8fA')
>>> message[13]=5
>>> bytearray(message)
bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x06\x00\x02\x00\x03\x05A')
>>> message[14]=6
>>> bytearray(message)
bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x06\x00\x02\x00\x03\x05\x06')
>>> message[13]=96
>>> bytearray(message)
bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x06\x00\x02\x00\x03`\x06')
>>> message
[1, 16, 0, 0, 0, 3, 6, 0, 6, 0, 2, 0, 3, 96, 6]
>>> bytearray(message)
bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x06\x00\x02\x00\x03`\x06')
>>>

[pfalcon]

Can you describe what strangeness you see exactly? (Like: I see "<few, not many characters>", instead I expect to see "<few characters>").

[Peter.Kenyon]

What you describe is what I get using python3 on a PC.
Isnt this just the way python stringifies byte arrays ie prints the char or if not printable then the \x value

[EasyRider]

>>> message
[1, 16, 0, 0, 0, 3, 6, 0, 6, 0, 2, 0, 3, 143, 65]
>>> bytearray(message)
bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x06\x00\x02\x00\x03\x8fA')

I would expect byte 13 (dec 143) to be converted to (hex x8F).
Byte 14 (dec 65) to be (hex x41).
Not x8fA' ???

Here is another example where the conversion is OK, as I would expect it.
Conversion of byte 13 and 14 dec values to hex is OK.
dec144 == hex92, dec131 == hex83.

ytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x1f\x00\x02\x00\x03\x92\x83')
[1, 16, 0, 0, 0, 3, 6, 0, 31, 0, 2, 0, 3, 146, 131]

But there is inconsistency in some packet conversions.
Am I missing something?

[EasyRider]

Here is a printout of some other conversions.

I would expect bytes 13 and and 14 to be converted from dec value to appropriate hex value.

bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x02\x00\x02\x00\x03~\x81')
[1, 16, 0, 0, 0, 3, 6, 0, 2, 0, 2, 0, 3, 126, 129]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x03\x00\x02\x00\x03CA')
[1, 16, 0, 0, 0, 3, 6, 0, 3, 0, 2, 0, 3, 67, 65]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x04\x00\x02\x00\x03\xf6\x81')
[1, 16, 0, 0, 0, 3, 6, 0, 4, 0, 2, 0, 3, 246, 129]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x05\x00\x02\x00\x03\xcbA')
[1, 16, 0, 0, 0, 3, 6, 0, 5, 0, 2, 0, 3, 203, 65]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x06\x00\x02\x00\x03\x8fA')
[1, 16, 0, 0, 0, 3, 6, 0, 6, 0, 2, 0, 3, 143, 65]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x07\x00\x02\x00\x03\xb2\x81')
[1, 16, 0, 0, 0, 3, 6, 0, 7, 0, 2, 0, 3, 178, 129]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x08\x00\x02\x00\x03\xe6\x80')
[1, 16, 0, 0, 0, 3, 6, 0, 8, 0, 2, 0, 3, 230, 128]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\t\x00\x02\x00\x03\xdb@')
[1, 16, 0, 0, 0, 3, 6, 0, 9, 0, 2, 0, 3, 219, 64]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\n\x00\x02\x00\x03\x9f@')
[1, 16, 0, 0, 0, 3, 6, 0, 10, 0, 2, 0, 3, 159, 64]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x0b\x00\x02\x00\x03\xa2\x80')
[1, 16, 0, 0, 0, 3, 6, 0, 11, 0, 2, 0, 3, 162, 128]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x0c\x00\x02\x00\x03\x17@')
[1, 16, 0, 0, 0, 3, 6, 0, 12, 0, 2, 0, 3, 23, 64]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\r\x00\x02\x00\x03*\x80')
[1, 16, 0, 0, 0, 3, 6, 0, 13, 0, 2, 0, 3, 42, 128]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x0e\x00\x02\x00\x03n\x80')
[1, 16, 0, 0, 0, 3, 6, 0, 14, 0, 2, 0, 3, 110, 128]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x0f\x00\x02\x00\x03S@')
[1, 16, 0, 0, 0, 3, 6, 0, 15, 0, 2, 0, 3, 83, 64]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x10\x00\x02\x00\x03\xc6\x82')
[1, 16, 0, 0, 0, 3, 6, 0, 16, 0, 2, 0, 3, 198, 130]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x11\x00\x02\x00\x03\xfbB')
[1, 16, 0, 0, 0, 3, 6, 0, 17, 0, 2, 0, 3, 251, 66]


bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x12\x00\x02\x00\x03\xbfB')
[1, 16, 0, 0, 0, 3, 6, 0, 18, 0, 2, 0, 3, 191, 66]

[Peter.Kenyon]

Well I agree it would be more consistent but note that
A is 65 decimal
B is 66 Decimal
@ is 64 decimal

[Damien]

There is no problem, it's just the way that bytes/bytearray objects are printed: Python tries to print printable characters (eg A-Z) and if it can't then it prints in hex format using \x.

If you want a nice print out of the bytes (or bytearray) then you can use binascii.hexlify:

Code: Select all

>>> binascii.hexlify(bytearray(b'\x01\x10\x00\x00\x00\x03\x06\x00\x02\x00\x02\x00\x03~\x81'))
b'011000000003060002000200037e81'

[EasyRider]

Thanks Guys,

I am fairly new to Python and was stumped by the fact that bytearray worked OK in converting an Integer array to bytearray when writing to uart but not actually printing for debugging purposes.

Anyway, have found another option to convert and print correctly Integer array to hex string values.

print('\n'.join([hex(i) for i in message])), message being the Integer array.

Are there any other easier ways in MicroPython?

Regards
John

[pfalcon]

MicroPython follows Python in almost everything and the way to get hexadecimal string from byte strings is (mind the backtick character with hex code 0x60, decimal 96 which apparently confused you):

Code: Select all

>>> import binascii
>>> print(binascii.hexlify(bytearray(b"123`")))
b'31323360'

I recently also added an extension to CPython which allows to specify a separator char:

Code: Select all

>>> import binascii
>>> print(binascii.hexlify(bytearray(b"123`"), b":"))
b'31:32:33:60'

(That's fairly recent addition, ~ week, so probably not in your firmware, and once again, it's extension to standard Python, so use it cautiously). Otherwise, as you show yourself, Python is powerful expressive language, and it's a one-liner to do almost any conversion you like. E.g., extended form of binascii.hexlify() above can be easily replaced with (which also works in standard Python):

Code: Select all

print(":".join(["%02x" % a for a in bytearray(b"123`")]))

The benefit of binascii.hexlify() is memory efficiency (it uses as little memory as required to do conversion, and not a single byte more).

[dhylands]

I also put together a hex/ASCII dumper (ported from some C code I had).
https://github.com/dhylands/bioloid3/bl ... ump_mem.py

It also uses log.py from the same directory so that output can be redirected to say a uart.
https://github.com/dhylands/bioloid3/blob/master/log.py

Sample output:

Code: Select all

>>> from dump_mem import dump_mem
>>> b = bytearray(b'0123\x00\x01This is a test\nto see what happens\n')
>>> dump_mem(b, 'Buf')
Buf: 0000: 30 31 32 33 00 01 54 68 69 73 20 69 73 20 61 20 0123..This is a 
Buf: 0010: 74 65 73 74 0a 74 6f 20 73 65 65 20 77 68 61 74 test.to see what
Buf: 0020: 20 68 61 70 70 65 6e 73 0a                       happens.

I'll have to take a look at updating it to use hexlify