cython - Extra elements in Python list -
using cython, trying convert python list cython array, , vice versa. python list contains numbers range 0 - 255, specify type of array unsigned char
array. here code conversions:
from libc.stdlib cimport malloc cdef to_array(list pylist): cdef unsigned char *array array = <unsigned char *>malloc(len(pylist) * sizeof(unsigned char)) cdef long count = 0 item in pylist: array[count] = item count += 1 return array cdef to_list(array): pylist = [item item in array] return pylist def donothing(pylist): return to_list(to_array(pylist))
the problem lies in fact pieces of garbage data generated in cython array, , when converted python lists, garbage data carries over. example, donothing
should absolutely nothing, , return python list me, unchanged. function testing conversion, when run like:
in[56]: donothing([2,3,4,5]) out[56]: [2, 3, 4, 5, 128, 28, 184, 6, 161, 148, 185, 69, 106, 101]
where data coming in code, , how can garbage cleaned no memory wasted?
p.s. there may better version of taking numbers python list , injecting them unsigned char
array. if so, please direct me better method entirely.
your to_array
has untyped return value. further, assign result untyped value. such, cython forced convert char *
python type.
cython converts bytes
, because char
approximately bytes
. unfortunately, without explicitly-given length cython assumes char *
null-terminated. causes problem:
convert_lists.donothing([1, 2, 3, 0, 4, 5, 6]) #>>> [1, 2, 3]
when there no zeroes, cython read until finds one, going past actually-allocated memory.
you can't for x in my_pointer_arrray
arbitrary cython types. for
loop operates on incorrectly-converted bytes
.
you can fix typing all values hold char
array, passing around length explicitly , looping on ranges (which faster when loop variable typed), or using wrapper of sort. ideas on wrapper arrays use, this question , answer pair has covered.
please note should very careful errors when using manual allocation. malloc
'd data not garbage collected, if error out of code-path you're going leak memory. should check how handle each specific case.
Comments
Post a Comment