I can run this normally on the command line in Linux:
$ tar c my_dir | md5sum
But when I try to call it with Python I get an error:
>>> subprocess.Popen(['tar','-c','my_dir','|','md5sum'],shell=True)
<subprocess.Popen object at 0x26c0550>
>>> tar: You must specify one of the `-Acdtrux' or `--test-label' options
Try `tar --help' or `tar --usage' for more information.
You have to use subprocess.PIPE, also, to split the command, you should use shlex.split() to prevent strange behaviours in some cases:
from subprocess import Popen, PIPE
from shlex import split
p1 = Popen(split("tar -c mydir"), stdout=PIPE)
p2 = Popen(split("md5sum"), stdin=p1.stdout)
But to make an archive and generate its checksum, you should use Python built-in modules tarfile and hashlib instead of calling shell commands.
open() and then hash its content.
tar -c mydir | md5sum. Perhaps you could start a new question, including an interactive terminal session where you run this command, start Python, and run the Python commands, displaying the output.
communicate on the final Popen object, or switch to a modern wrapper like subprocess.run. For many cases, simply pass in a string with shell=True if you want to use shell features like pipes, variables, redirection, job control, etc. Or as the answer suggests, run as little as possible in a subprocess and replace shell commands with native Python where you can (in which case you can often avoid the security implications of shell=True by removing it).Ok, I'm not sure why but this seems to work:
subprocess.call("tar c my_dir | md5sum",shell=True)
Anyone know why the original code doesn't work?
What you actually want is to run a shell subprocess with the shell command as a parameter:
>>> subprocess.Popen(['sh', '-c', 'echo hi | md5sum'], stdout=subprocess.PIPE).communicate()
('764efa883dda1e11db47671c4a3bbd9e -\n', None)
shell=True does something similar.shell=True here.i would try your on python v3.8.10 :
import subprocess
proc1 = subprocess.run(['tar c my_dir'], stdout=subprocess.PIPE, shell=True)
proc2 = subprocess.run(['md5sum'], input=proc1.stdout, stdout=subprocess.PIPE, shell=True)
print(proc2.stdout.decode())
key points (like outline in my solution on related https://stackoverflow.com/a/68323133/12361522):
subprocess.run()['tar c my_dir']or ["tar c my_dir"]stdout=subprocess.PIPE for all processesinput=proc1.stdout chain of output of previous one into input of the next oneshell=True
run over Popen is a good idea when you can, of course (back when the accepted answer was written, run didn't exist).
grep in my command string, which did weird stuff when being supplied to split.
run instead of shlex>>> from subprocess import Popen,PIPE
>>> import hashlib
>>> proc = Popen(['tar','-c','/etc/hosts'], stdout=PIPE)
>>> stdout, stderr = proc.communicate()
>>> hashlib.md5(stdout).hexdigest()
'a13061c76e2c9366282412f455460889'
>>>
