How to implement threading to run two bash shell commands in python?

Question

I have to record a wav file and at the same time I have to analyze it with sox. I am using fifo type file for this operation.

So here I need to start 2 threads at the same time but even if I use the threads I am not able to achieve what I wanna do. Always one executing first and then the other. I want them to be in parallel so that I can do some stuff.

#this should be in one thread
def test_wav(self):
    """ analyze the data """
    bashCommand = "sox {} -n stat".format(self.__rawfile)
    while self.__rec_thread.is_alive():
        process = subprocess.Popen(bashCommand.split(),stdout=subprocess.PIPE,stderr=subprocess.PIPE)
        wav_output = process.communicate()[1] #sox outputs the details in stderr
        #do something and return

#this should be in another thread
def record_wav(self):
    bashCommand = "arecord -d 10 -c 2 -r 48000 -f S32_LE > {}".format(self.__rawfile)
    pid = subprocess.Popen(bashCommand.split())
    pid.wait()
    if pid.returncode != 0:
        raise RecordException("Failed while recording with error {}".format(pid.returncode))

I tried the following code to make them threads but failed(Always one executing first and then the other. I want them to be in parallel so that I can do some stuff). imported from threading import Thread

self.__rec_thread = Thread(target = self.record_wav())
amp_thread = Thread(target = self.test_wav())
self.__rec_thread.start()
amp_thread.start()

EDIT: First its executing the record(it minimum takes 10 sec because of the option -d 10) function completely and then the test wav function. Its like calling them one after another.

@OferSadan Always one executing first and then the other. I want them to be in parallel so that I can do some stuff. — optimus prime
– optimus prime, Commented Jul 4, 2017 at 4:21
I don't know what the sox command does but possibly it's running too fast to even matter if it's threaded or not? Add a print('thread 1 started') to the top of each function and check the output — Ofer Sadan
– Ofer Sadan, Commented Jul 4, 2017 at 4:28
target = self.record_wav() executes record_wav() immediately. You want target = self.record_wav (no parentheses) instead. That's a start. — Tim Peters
– Tim Peters, Commented Jul 4, 2017 at 4:37
@TimPeters Thank You buddy. Its working now :) Post it as an answer. I will accept it — optimus prime
– optimus prime, Commented Jul 4, 2017 at 4:40

Tim Peters · Accepted Answer · 2017-07-04 04:46:32Z

1

... target = self.record_wav() ...

is calling record_wav(): it executes immediately, and the program doesn't proceed until record_wav() completes. You almost always want to pass a function (or method) object to target=, almost never the result of executing the function/method. So just lose the parentheses:

... target = self.record_wav ...

answered Jul 4, 2017 at 4:46

Tim Peters

71.4k14 gold badges133 silver badges140 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

optimus prime Over a year ago

A small doubt here, that I didn't find that in any documentation. Can you point to it?

Tim Peters Over a year ago

f() always calls f - the context is completely irrelevant. If you're not surprised that, e.g., the statement x = f() calls f, then you should be equally unsurprised that target=f() calls f too. The Thread docs clearly do say *target* is the callable object to be invoked by the run() method. self.record_wav is a callable object. self.record_wav() is whatever the heck self.record_wav() returns.

optimus prime Over a year ago

It seems I was drunk :P Now I got it. It's the basics of programming. I don't know how my stupid brain missed understanding it.

Tim Peters Over a year ago

It's a pretty common mistake. Unfortunately, I'm guessing your record_wav() returns None - and target=None is taken to mean Thread.start() won't bother to run anything (see the docs for that). To catch this mistake, it would be more useful if target=None raised an exception instead.

nooper · Accepted Answer · 2017-07-04 06:16:49Z

if you probably use python3, you can use asyncio to run the shell command in goroutines way.

import asyncio
import sys

async def execute(command, cwd=None, shell=True):
    process = await asyncio.create_subprocess_exec(*command,
                                                   stdout=asyncio.subprocess.PIPE,
                                                   stderr=asyncio.subprocess.PIPE,
                                                   cwd=cwd,
                                                   shell=shell)
    std_out, std_err = await process.communicate()

    error = std_err.decode().strip()
    result = std_out.decode().strip()
    print(result)
    print(error)
    return result


if sys.platform == "win32":
    loop = asyncio.ProactorEventLoop()
    asyncio.set_event_loop(loop)
else:
    loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(
        asyncio.gather(execute(["bash", "-c", "echo hello && sleep 2"]), execute(["bash", "-c", "echo ok && sleep 1"])))
except Exception as e:
    raise e
finally:
    loop.close()

Collectives™ on Stack Overflow

How to implement threading to run two bash shell commands in python?

2 Answers 2

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related