subprocess popen.communicate() vs. stdin.write() and stdout.read()

You can use threads to write and read at the same time, especially if the output only needs to be printed to the user:

from threading
import Thread

def print_remaining(stream):
   for line in stream:
   print(line.decode("utf-8"))

con = a.stdout.readline()
if "FATAL ERROR"
not in con.decode("utf-8"):
   Thread(target = print_remaining, args = [a.stdout]).start()
for cmd in LIST_OF_COMMANDS_TO_SEND:
   a.stdin.write(cmd)

The thing is that when using subprocess.Popen your code continues to be read even before the process terminates. Try appending .wait() to your Popen call (see documentation),

a = subprocess.Popen(execution, bufsize = 0, stdout = PIPE, stdin = PIPE, stderr = STDOUT, shell = False).wait()

Suggestion : 2

Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.,Special value that can be used as the stdin, stdout or stderr argument to Popen and indicates that a pipe to the standard stream should be opened. Most useful with Popen.communicate().,The newlines attribute of the file objects Popen.stdin, Popen.stdout and Popen.stderr are not updated by the Popen.communicate() method.,This will deadlock when using stdout=PIPE or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use Popen.communicate() when using pipes to avoid that.

os.system
os.spawn *

>>> subprocess.run(["ls", "-l"]) # doesn 't capture output
CompletedProcess(args = ['ls', '-l'], returncode = 0)

   >>>
   subprocess.run("exit 1", shell = True, check = True)
Traceback(most recent call last):
   ...
   subprocess.CalledProcessError: Command 'exit 1'
returned non - zero exit status 1

   >>>
   subprocess.run(["ls", "-l", "/dev/null"], capture_output = True)
CompletedProcess(args = ['ls', '-l', '/dev/null'], returncode = 0,
   stdout = b 'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n', stderr = b '')

Popen(["/usr/bin/git", "commit", "-m", "Fixes a bug."])

>>>
import shlex, subprocess
   >>>
   command_line = input() /
   bin / vikings - input eggs.txt - output "spam spam.txt" - cmd "echo '$MONEY'" >>>
   args = shlex.split(command_line) >>>
   print(args)['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"] >>>
   p = subprocess.Popen(args) # Success!

Popen(['/bin/sh', '-c', args[0], args[1], ...])

with Popen(["ifconfig"], stdout = PIPE) as proc:
   log.write(proc.stdout.read())

Suggestion : 3

You can read and write on stdin and stdout even while the subprocess hasn't completed. This could be useful when automating functionality in another program.,The signature for Popen is very similar to the call function; however, Popen will return immediately instead of waiting for the subprocess to complete like call does.,However, if you only need one set of input and output, rather than dynamic interaction, you should use communicate() rather than directly accessing stdin and stdout.,In both the above snippets, the process.poll() is None until the subprocess finishes. This is used to exit the loop once there is no more output to read.

Launching a subprocess

process = subprocess.Popen([r 'C:\path\to\app.exe', 'arg1', '--flag', 'arg'])

Waiting on a subprocess to complete

process = subprocess.Popen([r 'C:\path\to\app.exe', 'arg1', '--flag', 'arg'])
process.wait()

Reading output from a subprocess

process = subprocess.Popen([r 'C:\path\to\app.exe'], stdout = subprocess.PIPE, stderr = subprocess.PIPE)

# This will block until process completes
stdout, stderr = process.communicate()
print stdout
print stderr

In case you want to see the output of a subprocess line by line, you can use the following snippet:

process = subprocess.Popen(<your_command>, stdout=subprocess.PIPE)
   while process.poll() is None:
   output_line = process.stdout.readline()

in the case the subcommand output do not have EOL character, the above snippet does not work. You can then read the output character by character as follows:

process = subprocess.Popen(<your_command>, stdout=subprocess.PIPE)
   while process.poll() is None:
   output_line = process.stdout.read(1)

Suggestion : 4

The communicate() method only reads data from stdout and stderr, until end-of-file is reached. So, after all the messages have been printed, if we call communicate() again we get an error:,Additionally, stderr can be STDOUT, which indicates that the stderr data from the child process should be captured into the same file handle as for stdout.,Writing to a process can be done in a very similar way. If we want to send data to the process's stdin, we need to create the Popen object with stdin=subprocess.PIPE.,To send a message to stdin, we pass the string we want to send as the input argument to communicate():

Popen

Popen(['/bin/sh', '-c', args[0], args[1], ...])

Suggestion : 5

The input argument is passed to Popen.communicate() and thus to the subprocess’s stdin. If used it must be a byte sequence, or a string if encoding or errors is specified or text is true. When used, the internal Popen object is automatically created with stdin=PIPE, and the stdin argument may not be used as well.,Anyone communicating with subrocesses over any text or terminal based protocol should consider using the pexpect library:,I have started a process using Popen . The created active process will be waiting for the input to be passed . I'm trying to send the input using stdin.write('help'). But the input is not reaching to the active process. In linux using proc i'm able to send the inputs but in windows since we dont have i'm stuck.,This seemed like the most obvious solution but it fails miserably. It seems that the first call to communicate also closes the pipe and the second loop raises an exception.

from subprocess
import Popen, PIPE

p = Popen('less', stdin = PIPE)
for x in xrange(100):
   p.communicate('Line number %d.\n' % x)

from subprocess
import Popen, PIPE

p = Popen('less', stdin = PIPE)
for x in xrange(100):
   p.stdin.write('Line number %d.\n' % x)

from subprocess
import Popen, PIPE

out = []
p = Popen('less', stdin = PIPE)
for x in xrange(100):
   out.append('Line number %d.' % x)
p.communicate('\n'.join(out))

from subprocess
import Popen, PIPE

p = Popen('less', stdin = PIPE)
for x in xrange(100):
   p.stdin.write('Line number %d.\n' % x)
p.stdin.close()
p.wait()

import errno

   ...

   try:
   p.stdin.write(input)
except IOError as e:
   if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
   raise

from subprocess
import Popen, PIPE
import errno

p = Popen('less', stdin = PIPE)
for x in xrange(100):
   line = 'Line number %d.\n' % x
try:
p.stdin.write(line)
except IOError as e:
   if e.errno == errno.EPIPE or e.errno == errno.EINVAL:
   # Stop loop on "Invalid pipe"
or "Invalid argument".
# No sense in continuing with broken pipe.
break
else:
   # Raise any other error.
raise

p.stdin.close()
p.wait()

print 'All done!'
# This should always be printed below any output written to less.

Suggestion : 6

In subprocess, Popen() can interact with the three channels and redirect each stream to an external file, or to a special value called PIPE. An additional method, called communicate(), is used to read from the stdout and write on the stdin. The communicate() method can take input from the user and return both the standard output and the standard error, as shown in the following code snippet:,The spawned processes can communicate with the operating system in three channels:,© 2022, O’Reilly Media, Inc. All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.,There's also live online events, interactive content, certification prep materials, and more.

In subprocess, Popen() can interact with the three channels and redirect each stream to an external file, or to a special value called PIPE. An additional method, called communicate(), is used to read from the stdout and write on the stdin. The communicate() method can take input from the user and return both the standard output and the standard error, as shown in the following code snippet:

import subprocessp = subprocess.Popen(["ping", "8.8.8.8", "-c", "3"], stdin = subprocess.PIPE, stdout = subprocess.PIPE) stdout, stderr = p.communicate() print(""
      "==========The ...

Suggestion : 7

For instance, we write

from subprocess
import Popen, PIPE, STDOUT

p = Popen(['myapp'], stdout = PIPE, stdin = PIPE, stderr = PIPE)
stdout_data = p.communicate(input = 'data_to_write')[0]