Optimize multiple nested for loops in python

Question

Following is the test code, my actual code looks almost similar in which i use the original matrix rather randomly generated. How can I optimize this nested for loops. I know it's possible in python but I am unable to do so.

import time
import numpy as np

a = 1000
b = 500
sum2,sum3,sum4 = 0
t0 = time.time()

x = np.random.random(a*a).reshape([a,a])

for outer1 in xrange(0,a):
    for inner1 in xrange(0,b):
        for outer2 in xrange(0,a):
            for inner2 in xrange(0, a):
                sum2 += x[outer2][inner2]  #this is not the only operation I have   
        for outer3 in xrange(0,a):
            for inner3 in xrange(0, a):
                 sum3 += x[outer3][inner3] #this is not the only operation I have 
        for outer4 in xrange(0,a):
            for inner4 in xrange(0, a):
                sum4 += x[outer4][inner4] #this is not the only operation I have 

print time.time() - t0
print 'sum2: '+str(sum2)+' sum3: '+str(sum3)+' sum4: '+str(sum4)

I am using python 2.7. Thank you.

The additional computations that you do make all the difference in how you could optimize your code. And, also are more likely to be the cause of any bottleneck, rather than these sums. — Dunes
– Dunes, Commented Jan 21, 2012 at 11:08

David Z · Accepted Answer · 2012-01-21 10:32:34Z

2

With Numpy arrays, the way to optimize the computations is to use vectorized operations as much as possible. In your example, since it looks like you're summing the elements of each array, you should keep the array 1-dimensional and just use the sum function directly:

x = np.random.random(a*a)
sum2 = x.sum()

and so on.

Similarly, for your actual code, you will need to translate your loops into vectorized operations. I can't say anything about how to do that without knowing what your actual computation is.

answered Jan 21, 2012 at 10:32

David Z

133k29 gold badges264 silver badges284 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

phimuemue · Accepted Answer · 2012-01-21 10:54:46Z

1

As your code suggests, sum2 depends only on the values outer2 and inner2, and this is done within two loops whose variables are outer1 and inner1. In the code you pasted, you can simply leave out the 2 outer loops (outer1 and inner1), and instead multiply the value of sum2 by a*b. This eliminates two loops and replaces them by a multiplication which should be faster.

I don't know if this is possible with your actual code as well, but in the code you posted, it should be possible.

answered Jan 21, 2012 at 10:54

phimuemue

36.3k11 gold badges95 silver badges124 bronze badges

Collectives™ on Stack Overflow

Optimize multiple nested for loops in python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related