Week 3

Redirection, Pipes, and

Background

Sarah Diesburg
8/3/2010
COP4610 / CGS5765

1
Redirection

Redirection of stdin and stdout happen by

placing a redirection character between
commands
 [command] < [in_file]
 [command] > [out_file]
 [command] < [in_file] > [out_file]

2
Input Redirection

[command] < [in_file]

The command now takes input from in_file
instead of stdin

Examples
 cat < file1.txt
 grep hamburger < menu.txt

3
Output Redirection

[command] > [out_file]

Prints the output from command to out_file
instead of stdout

Example
 echo hello > file1.txt

4
Combination Redirection

[command] < [in_file] > [out_file]

The command takes input from the in_file
instead of stdin and prints output to the
out_file instead of stdout

Example
 cat < file1.txt > duplicate.txt

5
How to Implement Redirection

We need to duplicate standard I/O using the

dup command

int dup(int oldfd);

int dup2(int oldfd, int newfd);

Dup makes a duplicate of the oldfd file

descriptor and returns the new file descriptor

6
Steps for Input Redirection

1. Open the input file

2. Close stdin
3. Issue dup() on the input file
 dup will automatically choose lowest closed file
descriptor stdin
 Input file and stdin are now the same file
descriptor
4. Close input file

7
Input Redirection Visual

fork()

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr

shell shell
Input Redirection Visual
open(input_file, O_RDONLY)

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr

shell shell
Input Redirection Visual
open(input_file, O_RDONLY)

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr
3 input_file

shell shell

Can be any
number > 2
Input Redirection Visual

file descriptors close(0) file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr
3 input_file

shell shell
Input Redirection Visual

dup(3)

file descriptors file descriptors

0 stdin 0
1 stdout 1 stdout
2 stderr 2 stderr
3 input_file

shell shell
Input Redirection Visual

dup(3)

file descriptors file descriptors

0 stdin 0 input_file
1 stdout 1 stdout
2 stderr 2 stderr
3 input_file

shell shell
Input Redirection Visual

close(3)

file descriptors file descriptors

0 stdin 0 input_file
1 stdout 1 stdout
2 stderr 2 stderr
3 input_file

shell shell
Steps for Output Redirection

1. Open the output file

2. Close stdout
3. Issue dup() on the output file
 dup will automatically choose lowest closed file
descriptor stdout
 Input file and stdout are now the same file
descriptor
4. Close output file

15
Output Redirection Visual

fork()

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr

shell shell
Output Redirection Visual
open(output_file,
O_RDWR | O_CREAT | O_TRUNC)

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr

shell shell
Output Redirection Visual
open(output_file,
O_RDWR | O_CREAT | O_TRUNC)

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr
3 output_file

shell shell
Output Redirection Visual

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr
3 output_file

shell close(1) shell

Output Redirection Visual

dup(3)

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1
2 stderr 2 stderr
3 output_file

shell shell
Output Redirection Visual

dup(3)

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 output_file
2 stderr 2 stderr
3 output_file

shell shell
Output Redirection Visual

close(3)

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 output_file
2 stderr 2 stderr
3 output_file

shell shell
Note on Redirection

Only perform redirection in the child process

 Why?

23
Pipes

One program can receive output from

another without an explicit temporary file
command1 | command 2

Same as
command1 > tmpfile
command2 < tmpfile
rm tmpfile

24
Pipes

command1 | command 2

Since two commands will be run, we will

need to create two children

We also need to create an I/O pipe between
the children processes

25
Pipes

Use the system call pipe

int pipe(int filedes[2]);

Creates a pair of file descriptors for pipe I/O

and places them in filedes[]

Returns 0 on success

26
Pipe

int p1_to_p2[2];
pipe(p1_to_p2);

file descriptors
0 stdin
1 stdout
2 stderr

shell
Pipe

int p1_to_p2[2];
pipe(p1_to_p2);

file descriptors
0 stdin
1 stdout
2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1]

shell
Pipe
fork()

int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors

0 stdin 0 stdin
1 stdout 1 stdout
2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] 4 p1_to_p2[1]

shell shell
Pipe
fork()

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 stdin
1 stdout 1 stdout 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] 4 p1_to_p2[1] 4 p1_to_p2[1]

shell shell shell

Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 stdin
1 stdout 1 stdout 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] 4 p1_to_p2[1] 4 p1_to_p2[1]

shell shell shell

Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin dup(4) 0 stdin 0 stdin
1 stdout 1 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] 4 p1_to_p2[1] 4 p1_to_p2[1]

shell shell shell

Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin dup(4) 0 stdin 0 stdin
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] 4 p1_to_p2[1] 4 p1_to_p2[1]

shell shell shell

Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 stdin
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] 4 p1_to_p2[1] 4 p1_to_p2[1]

shell shell shell

Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 stdin
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] 4 p1_to_p2[1]

shell shell shell

Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 stdin
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] execv(bin_path, 4 p1_to_p2[1]
argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 stdin
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] execv(bin_path, 4 p1_to_p2[1]
argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

dup(3)
0 stdin 0 stdin 0
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] execv(bin_path, 4 p1_to_p2[1]
argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

dup(3)
0 stdin 0 stdin 0 p1_to_p2[0]
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] execv(bin_path, 4 p1_to_p2[1]
argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 p1_to_p2[0]
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0] 3 p1_to_p2[0]
4 p1_to_p2[1] execv(bin_path, 4 p1_to_p2[1]
argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 p1_to_p2[0]
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1] execv(bin_path,
argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 p1_to_p2[0]
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1] execv(bin_path, execv(bin_path,
argv) argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 p1_to_p2[0]
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr
3 p1_to_p2[0]
4 p1_to_p2[1] execv(bin_path, execv(bin_path,
argv) argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 p1_to_p2[0]
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr

execv(bin_path, execv(bin_path,
argv) argv)
shell shell shell
Pipe

int p1_to_p2[2]; int p1_to_p2[2]; int p1_to_p2[2];

pipe(p1_to_p2); pipe(p1_to_p2); pipe(p1_to_p2);

file descriptors file descriptors file descriptors

0 stdin 0 stdin 0 p1_to_p2[0]
1 stdout 1 p1_to_p2[1] 1 stdout
2 stderr 2 stderr 2 stderr

waitpid() execv(bin_path, execv(bin_path,

waitpid() argv) argv)
shell shell shell
Background Processes

A process is run in the foreground if the

shell waits for the process to complete before
returning you to the prompt
sarah@trogdor:~$ ./sleep

46
Background Processes

A process is run in the background if the

shell prompt returns right away, even through
the process is still running.
sarah@trogdor:~$ ./sleep &
sarah@trogdor:~$

Look to bash for example

47
Waiting Problem

When we call wait() or waitpid(), the parent

blocks, or waits, until one of the children has
finished
 Fine for foreground processes

How can we make shell continue and return
even if the child has not finished yet?
 Use waitpid() option WNOHANG

48
Background example main loop
int num_processes=0;
int status=0;
/* Main loop */
{
if((pid = fork())==0) {
*/child executes long processes*/
} else {
num_processes++;
if(waitpid(-1, &status, WNOHANG) > 0) {
num_processes--;
}
}
} /* End main loop */

49
What about on exit?

You must wait on all children still running

 Use variable like num_processes in previous
example

50
Foreground Execution

fork()

waitpid(-1, &status, 0)

Shell Shell
Foreground Execution

fork()

waitpid(-1, &status, 0) execv(bin_path, argv)

Shell Shell
Foreground Execution

fork()

waitpid(-1, &status, 0)

Shell Shell
Background Execution

fork()

waitpid(-1, &status,
WNOHANG)

Shell Shell
Background Execution

fork()

waitpid(-1, &status, execv(bin_path, argv)

WNOHANG)

Shell Shell
Background Execution

fork()

waitpid(-1, &status,
WNOHANG)

Shell Shell
Summary

Pieces necessary to complete project 1

 Part 4 Input/Output redirection
 Part 5 Pipes
 Part 6 Background processing

57
Next Recitation

Introduction of project 2
 Kernel modification!

58
Reminders

Project due date

Any questions?

59