Faster way to convert ctypes array to python list?

Question

I was wondering if someone could advise me whether there is a better/faster approach to read data from my C program that outputs two lists of size n. I am using ctypes to call the C program.

The loop I show below works by iterating over a number of scans. For each scan two lists are produced (msX, msY). The c_float data is extracted by using list comprehension loop. Is there a better/faster way to convert the c_float_Array obtained from mzP and mzI to msX and msY?

for scan in xrange(nScans):
    mzP = (c_float * nPoints)() # pointer to list 1, c_float_Array
    mzI = (c_float * nPoints)() # pointer to list 2,  c_float_Array
    mlLib.readData(filePointer, 1, scan, byref(mzP), byref(mzI))
    # The slow part...
    msX = [mzP[i] for i in xrange(nPoints)] # list with mzP data
    msY = [mzI[i] for i in xrange(nPoints)] # list with mzI data

Let me know if my question is not clear. Thanks for your help in advance.

Try with PyPy pypy.org with CPython there is nothing faster that list comprehension. — alec_djinn
– alec_djinn, Commented May 25, 2017 at 12:16
msX = mzP[:] would be faster than a list comprehension, but why do you need a list instead of directly using the ctypes array? If a ctypes array is missing some method that you need, maybe an array.array will suffice? Starting with msX = array.array('f', [0]) * nPoints, you can get a ctypes array that shares it via mzP = (c_float * nPoints).from_buffer(msX). — Eryk Sun
– Eryk Sun, Commented May 25, 2017 at 19:17
Thanks for the comments. I tried your first suggestion and this reduces the runtime to approx. 45% which is great. I will have a look at your other suggestion to see whether it will be suitable. Thanks! — munieq11
– munieq11, Commented May 25, 2017 at 22:06

Jay · Accepted Answer · 2019-06-05 19:44:08Z

4

I may be missing something, but this works for me:

from ctypes import c_float

arr = (c_float * 3)(1,2,3)
arr[:]
#Result: [1.0, 2.0, 3.0]

answered Jun 5, 2019 at 19:44

Jay

2,9485 gold badges36 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Mano-Wii · Accepted Answer · 2017-09-02 20:32:48Z

3

If you prefer, you can convert to an array with np.ndarray:

msX = np.ndarray((nPoints, ), 'f', mzP, order='C')    
msY = np.ndarray((nPoints, ), 'f', mzI, order='C')

answered Sep 2, 2017 at 20:32

Mano-Wii

5926 silver badges21 bronze badges

1 Comment

munieq11 Over a year ago

It does increase the speed by about 20 % on top of what @eryksun suggested. Thanks!

John Zwinck · Accepted Answer · 2017-05-25 12:40:43Z

2

The answer is to use NumPy. You can use NumPy to allocate an array, pass a pointer to its data to your C API which will populate it, and then at the end if you are desperate for a list you can call tolist() on the NumPy array. However, you will likely find that keeping the data stored in a NumPy array instead of a list allows you to accelerate downstream processing.

answered May 25, 2017 at 12:40

John Zwinck

252k44 gold badges346 silver badges459 bronze badges

1 Comment

GRquanti Over a year ago

what if the array to give in input is not empty?

Collectives™ on Stack Overflow

Faster way to convert ctypes array to python list?

3 Answers 3

Comments

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related