TypeError: 'bytearray' object doesn't support item deletion

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
Post Reply
a-ha
Posts: 21
Joined: Sat Jun 27, 2020 10:42 pm

TypeError: 'bytearray' object doesn't support item deletion

Post by a-ha » Mon Jul 06, 2020 7:25 pm

It appears the del on bytearray is not supported.
I am getting TypeError: 'bytearray' object doesn't support item deletion
Any idea for workaround?
-aha

Code: Select all

    buffer = bytearray()
    ...
    del buffer[0:i + 1]

User avatar
Roberthh
Posts: 2054
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by Roberthh » Mon Jul 06, 2020 7:46 pm

Assign a slice.
buffer = buffer[i:]
not efficient, but works.

a-ha
Posts: 21
Joined: Sat Jun 27, 2020 10:42 pm

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by a-ha » Mon Jul 06, 2020 10:20 pm

That would work, but I am trying to avoid the object copy.
This is a data buffer which can contain huge data/bytes. The ideal is to remove in-line.

User avatar
Roberthh
Posts: 2054
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by Roberthh » Tue Jul 07, 2020 6:04 am

Is the data in some way structured? Then you could handle it packet-wise as a list of packets.

a-ha
Posts: 21
Joined: Sat Jun 27, 2020 10:42 pm

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by a-ha » Tue Jul 07, 2020 6:10 pm

It is being used to read streaming tcp data from the server. At the low level data reading, it is unstructured.

User avatar
jimmo
Posts: 1769
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by jimmo » Wed Jul 08, 2020 5:20 am

a-ha wrote:
Mon Jul 06, 2020 10:20 pm
That would work, but I am trying to avoid the object copy.
Even if "del <bytearray>" was supported, I'm not sure there's a way to avoid a copy. (Unless I'm missing something -- what did you have in mind?)

The best you can do is at least to make the copy "in-place" (i.e. shifting the rest of the bytearray down), by using a slice assignment.

Code: Select all

buffer[0:i + 1] = b''
This will have the effect of shifting the rest of buffer down to the start, but no new buffer will be created in RAM.
Roberthh wrote:
Mon Jul 06, 2020 7:46 pm
Assign a slice.
buffer = buffer[i:]
not efficient, but works.
If your goal is to minimise memory usage, then this might possibly be the more efficient option. As with the previous example, it temporarily results in two copies of the buffer, but the new copy will be sized exactly correctly, and the GC will clean up the old one, so the resulting state is overall that there are `i` total fewer bytes allocated.

chuckbook
Posts: 123
Joined: Fri Oct 30, 2015 11:55 pm

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by chuckbook » Wed Jul 08, 2020 10:13 am

What about:

Code: Select all

lba = 1000         # size of bytearray
n = 10             # number of bytes to delete
ba = bytearray(lba)
mba = memoryview(ba)
mba[0:-n] = mba[n:]
This construct leaves the bytearray intact, which might come in handy for later reuse.

User avatar
pythoncoder
Posts: 4351
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by pythoncoder » Thu Jul 09, 2020 6:00 am

chuckbook wrote:
Wed Jul 08, 2020 10:13 am
...This construct leaves the bytearray intact, which might come in handy for later reuse.
I don't think it does.

Code: Select all

n = 10             # number of bytes to delete
ba = bytearray(range(20))
print(ba)
mba = memoryview(ba)
mba[0:-n] = mba[n:]
print(bytes(mba))
print(ba)
produces

Code: Select all

bytearray(b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13')
b'\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13'
bytearray(b'\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13')
As @jimmo says, the problem is actually fundamental. A bytearray stores the bytes in contiguous memory locations. Deletion of a range must involve potentially large scale data copying, the only question being how fast can you do it? CPython must be doing this under the hood to maintain the properties of a bytearray

As far as I know the only solution is to abandon contiguous storage in favour of structures like linked lists where segments can be mapped out without moving data.
Peter Hinch

chuckbook
Posts: 123
Joined: Fri Oct 30, 2015 11:55 pm

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by chuckbook » Thu Jul 09, 2020 9:45 am

Maybe I wasn't precise enough with my wording.
Intact was meant the way that storage space stays fixed not it's content.
Of course there are better ways to deal with non-contiguous data, but if you have to shift data chunks around it is pretty efficient.

miltmobley
Posts: 25
Joined: Mon Mar 07, 2016 11:44 pm

Re: TypeError: 'bytearray' object doesn't support item deletion

Post by miltmobley » Thu Jul 30, 2020 2:43 am

You are not being completely clear as to what you want to do with this big data.

See https://docs.python.org/3/library/stdty ... #bytearray for info as to what you can do to
a bytearray.

Post Reply