Run multiple python scripts in parallel from master script

Question

I'd like to run multiple python scripts in parallel and start them from a master script. I did find solutions for this in previously asked questions, however, none of these worked if the scripts running in parallel contained loops. Let's for example define two scripts.

Script 1:

array_1 = []

x = 0
while True:
    array_1.append(x)
    x = x + 1

Script 2:

array_2 = []

x = 0
while True:
    array_2.append(x)
    x = x + 1

Now I want to run both processes simultaneously. Previous solutions suggested the following code for a master script:

import script_1, script_2

exec(open(script_1))
exec(open(script_2))

While this is a solution for starting scripts from within another script, however, this will not run the two scripts in parallel. What should such a master script actually look like ?

Thanks for your suggestions!

Edit

I tried the following threading approach:

def function_1():
print('function 1 started...')
    while True:
        print('1')
        sleep(1)

def function_2():
print('function 2 started...')
    while True:
        print('2')
        sleep(1)

thread_1 = Thread(target=function_1())
thread_2 = Thread(target=function_2())
thread_1.start()
thread_2.start()


thread_1.join()
thread_2.join()
print("thread finished")

It doesn't work, only the first function gets started so I get the following output:

function 1 started...
1
1
1
1
1
1

encapsulate child scripts inside functions, run each function in a new thread — Mahrkeenerh
– Mahrkeenerh, Commented Nov 5, 2021 at 11:54
I hope you're not really going to try running that code unless you're deliberately trying to induce a MemoryError — user2668284
– user2668284, Commented Nov 5, 2021 at 12:08
you need to pass the function as an argument, not call it (remove the parenthesis: Thread(target=function1) — Mahrkeenerh
– Mahrkeenerh, Commented Nov 5, 2021 at 12:31

0Interest · Accepted Answer · 2021-11-05 15:58:23Z

When you want to spawn a new thread, you need to pass the address of the function you want the thread to execute, and not to call it. What you are doing here is essentially spawning a new thread that immediately calls function_1() which of course runs forever.

Also, you won't be able to reach this line of code:

print("thread finished")

As the threads are executing a while loop - forever, so it is redundent..

from time import sleep
from threading import Thread


def function_1():
    print('function 1 started...')
    while True:
        print('1')
        sleep(1)


def function_2():
    print('function 2 started...')
    while True:
        print('2')
        sleep(1)


thread_1 = Thread(target=function_1)
thread_2 = Thread(target=function_2)
thread_1.start()
thread_2.start()

thread_1.join()
thread_2.join()
# print("thread finished") - redundant

Collectives™ on Stack Overflow

Run multiple python scripts in parallel from master script

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related