Page 1 of 1
Fastest way to modify a bytearray?
Posted: Thu May 16, 2019 6:23 pm
by ddw
I'm trying to find the fastest code (without using native code generation) to create one bytearray from another by applying a function to each element. So far the best I've found is
def bytearray_mod(b1, b2, mod):
b2[:] = bytearray(map(mod, b1))
Any suggestions will be greatly appreciated!
Re: Fastest way to modify a bytearray?
Posted: Thu May 16, 2019 6:54 pm
by dhylands
bytearrays are modifiable in place, so you can also just modify the elements in place. Not sure if this is faster than what you've suggested, but it uses less memory.
Re: Fastest way to modify a bytearray?
Posted: Thu May 16, 2019 7:06 pm
by ddw
Thanks! What I've posted does already modify in place, right?
Re: Fastest way to modify a bytearray?
Posted: Thu May 16, 2019 10:07 pm
by dhylands
By calling bytearray it winds up allocating a new bytearray.
What I meant is that you can do something like this:
Code: Select all
>>> def bytearray_mod(b1, b2):
... for idx, item in enumerate(b1):
... b2[idx] = item + 100
...
>>> b1 = bytearray([1, 2, 3, 4])
>>> b2 = bytearray(4)
>>> bytearray_mod(b1, b2)
>>> b1
bytearray(b'\x01\x02\x03\x04')
>>> b2
bytearray(b'efgh')
Re: Fastest way to modify a bytearray?
Posted: Thu May 16, 2019 10:46 pm
by ddw
Right, the bytearray() in the map approach causes a temporary new allocation. But it seems to be somewhat faster:
ba_bench.py:
Code: Select all
import time
import random
import array
def timed_function(f, *args, **kwargs):
myname = str(f).split(' ')[1]
def new_func(*args, **kwargs):
t = time.monotonic_ns()
result = f(*args, **kwargs)
delta = time.monotonic_ns() - t
print('function {}: Time = {:6.3f}ms'.format(myname, delta/1000000))
return result
return new_func
def show(a):
for i in range(0,10):
print('%4d' % int(a[i]), end='')
print()
n = 10000
def f(a):
return a+100
ba1 = bytearray(random.randint(0,255) for i in range(0,n))
ba2 = bytearray(random.randint(0,255) for i in range(0,n))
ba3 = bytearray(random.randint(0,255) for i in range(0,n))
@timed_function
def bytearray_for(b1, b2):
for idx, item in enumerate(b1):
b2[idx] = item + 100
@timed_function
def bytearray_map(b1, b2, f):
b2[:] = bytearray(map(f, b1))
bytearray_for(ba1, ba2)
show(ba1)
show(ba2)
print()
bytearray_map(ba1, ba3, f)
show(ba1)
show(ba3)
print()
Running:
Code: Select all
>>> import ba_bench
function bytearray_for: Time = 530.000ms
24 183 65 140 187 173 160 255 100 215
124 27 165 240 31 17 4 99 200 59
function bytearray_map: Time = 384.000ms
24 183 65 140 187 173 160 255 100 215
124 27 165 240 31 17 4 99 200 59
>>>
Re: Fastest way to modify a bytearray?
Posted: Fri May 17, 2019 12:01 am
by dhylands
Yep - I'm not surprised, and I really just wanted to point out that there is a performance/memory usage tradeoff to make here. Pick whichever one is more important for your application.
Re: Fastest way to modify a bytearray?
Posted: Fri May 17, 2019 3:23 am
by jimmo
Also remember that you'll pay a hidden performance cost that the benchmarks don't show for the increased load on the garbage collector.
(I had this exact tradeoff for an application that involved modifying a bytearray to generate animation frames for an LED array... avoiding GC pauses turned out to have a far greater benefit on the overall visual effect than making each frame faster).
(And using things like @micropython.native / @micropython.viper also swung the balance a bit too)