Jeff Maxim

My experience at Recurse Center

subprocess.Popen: Using Python's Subprocess Module to Pipe Stdin and Stdout Between Processes

| Comments

Let’s say you have two programs that need to communicate with eachother via stdin and stdout. Pipes is one way to accomplish this. A pipe is a connection between two processes such that the stdout of one program becomes the stdin of the other.

With python, this piping process is streamlined with the subprocess module.

Let’s look at an example of how this plays out in python. Take our first program,

import time
import sys

for i in range(0,10):
    print "Hello world " + str(i)

All does is print “Hello World” with an iterator number, and it does this every second. This is our sample output. Now, let’s imagine a second program that is trying to catch the stdout from as stdin. Let’s make that second program called, and it will be based on python’s subprocess module documentation:

import subprocess

proc = subprocess.Popen(['python',''],stdin=subprocess.PIPE)


Running produces our desired result: it catches the stdout from, uses it as stdin, and prints “Hello world” plus an iterator to the console every second.

Now, let’s say you want to catch each line of output from as a variable in, and maybe save that output to a variable in Let’s write, based on this post from stackoverflow, to accomplish that goal:

import subprocess

proc = subprocess.Popen(['python',''],stdout=subprocess.PIPE)

while True:
  line = proc.stdout.readline()
  if line != '':
    print "test: ", line.rstrip()

You’ll notice two key differences between and First, uses a “stdout” arg in it’s call to subprocess.Popen, where uses a “stdin” arg here. Second, replaces’s “proc.communicate()” call with a while-block. This while-block in accomplishes our goal of collecting each line of output from in a variable. With, we just print that variable out. However, you could presumably append that line to a list and save all your line variables in this way.

Any thoughts, questions, comments?