UNIX System Programming

Introduction

User UNIX Interface: SHELL

Provides

command line as an interface between the user

and the system
Is simply a program that starts automatically when you
login
Uses a command language
Allows programming (shell scripting) within the shell
environment
Uses

variables, loops, conditionals, etc.

Accepts commands and often makes system calls to carry them

out

Various UNIX shells

sh

(Bourne shell)
ksh (Korn shell)
csh (C shell)
tcsh
bash
Differences mostly in scripting details

The Korn Shell (ksh)

We will

be using ksh as the standard shell

for examples in this class
Language is a superset of the Bourne shell
(sh)

Login scripts
You

dont want to enter aliases, set environment variables, set

up command line editing, etc. each time you log in
All of these things can be done in a script that is run each
time the shell is started
For ksh:
~/.profile - is read for a login shell
~/.kshrc
For

tcsh

~/.login
~/.cshrc

Example .profile (partial)

# set ENV to a file invoked each time sh is started for
interactive use.
ENV=$HOME/.shrc; export ENV
HOSTNAME=`hostname`; export HOSTNAME
PS1="$USER@$HOSTNAME>"
alias
alias
alias
alias
alias

'll'='ls -l'
'la'='ls -la'
'ls'='ls -F'
'rm'='rm -i'
'm'='more'

set -o vi
echo ".profile was read"

stdin, stdout, and stderr

Each

shell (and in fact all programs) automatically

open three files when they start up
Standard input (stdin): Usually from the keyboard
Standard output (stdout): Usually to the terminal
Standard error (stderr): Usually to the terminal

Programs

use these three files when reading (e.g.

cin), writing (e.g. cout), or reporting
errors/diagnostics

Redirecting stdout
Instead

of writing to the terminal, you can tell

a program to print its output to another file
using the > operator
>> operator is used to append to a file
Examples:
man ls > ls_help.txt
Echo $PWD > current_directory
cat file1 >> file2
8

Redirecting stderr
Instead

of writing errors to the terminal, you can

tell a program to write them to another file using
the:
ksh: 2> operator
tcsh: >& operator

Examples

(suppose j is a file that does not exist)

{ajax} ls j
ls: j: No such file or directory
{ajax} ls j >& hello.txt
{ajax} cat hello.txt
ls: j: No such file or directory

Redirecting stdin
Instead

of reading from the terminal, you

can tell a program to read from another file
using the < operator
Examples:
Mail user@domain.com < message
interactive_program <
command_list
10

Pipes and filters

Pipe:

a way to send the output of one command

to the input of another
Filter: a program that takes input and transforms
it in some way
wc - gives a count of words/lines/chars
grep - searches for lines with a given string
more
sort - sorts lines alphabetically or numerically
11

Examples of filtering
ls

-la | more
cat file | wc
man ksh | grep history
ls -l | grep dkl | wc
who | sort > current_users

12

UNIX Tutorial
http://www.ee.surrey.ac.uk/Teaching/Unix/
Google

will give you many links

13

UNIX Filesystem
The

filesystem is your interface to

physical storage (disks) on your machine

storage on other machines
output devices
etc.

Everything

in UNIX is a file (programs, text,

peripheral devices, terminals, )
There are no drive letters in UNIX! The filesystem
provides a logical view of the storage devices
14

Working directory
The

current directory in which you are

working
pwd command: outputs the absolute path
(more on this later) of your working directory
Unless you specify another directory,
commands will assume you want to operate
on the working directory

15

Home directory
A special

place for each user to store personal

files
When you log in, your working directory will
be set to your home directory
Your home directory is represented by the
symbol ~ (tilde)
The home directory of user1 is represented
by ~user1

16

UNIX file hierarchy

/

Directories

may
contain plain files
or other
directories
Leads to a tree
structure for the
filesystem
Root directory: /

bin
dkl
foo.txt
bar.c

users

tmp
kangli

csci1730
abcd
17

Path names
Separate

directories by /
Absolute path
start at root and follow the
tree
e.g. /users/dkl/foo.txt

/
bin

users

dkl
foo.txt

tmp
kangli

csci1730

Relative path

bar.c
abcd
start at working directory
.. refers to level above; . refers to working dir.
If /users/dkl/csci1730 is working dir, all these
refer to the same file
../foo.txt

~/foo.txt

~dkl/foo.txt

18

Types of files
Plain

(- in the first bit)

Most files
Includes binary and text files
Directory

(d)

A directory is actually a file

Points to another set of files
Link

(l): A pointer to another file or directory

Special: e.g. peripheral devices
19

Creating links
ln

s <curr_file> <link_name>
This command creates a symbolic link
The file link_name will be a pointer to the
curr_file which may be in another directory
or even on another physical machine

20

File permissions
Permissions

used to allow/disallow access

to file/directory contents
Read (r) 4, write (w) 2, and execute (x) 1
For owner, group, and world (everyone)
chmod <mode> <file(s)>
chmod 700 file.txt (only owner
can read, write, and execute)
chmod g+rw file.txt
21

Looking at file contents

cat

<filename(s)>

concatenate
output the contents of the file all at once
more

<filename(s)>

Output the contents of a file one screen at a time

Allows forward and backward scroll and search

22

Getting help on UNIX

commands
These notes only give you the tip of the

iceberg for these basic commands

man <command_name> shows you all
the documentation for a command
apropos <keyword> shows you all the
commands with the keyword in their
description
23

The UNIX System

Kernel

Heart of the OS

Process scheduling
I/O control (accesses)
Shell

Interpreter between the user and the

computer
Tools and applications
Accessible from shell
Can be run independently of shell
24

Basic Commands(1)
ls

list files and directories

ls -a
list all files and directories
mkdir make a directory
cd directory change to named directory
cd
change to home-directory
cd ~
change to home-directory
cd ..
change to parent directory
pwd
display current dir path

Basic Commands(2)

cp file1 file2
copy file1 and call it file2
mv file1 file2
move or rename file1 to file2
rm file
remove a file
rmdir directory
remove a directory
cat file
display a file
more file
display a file a page at a time
who
list users currently logged in
lpr -Pprinter psfile print postscript file to named printer
*
?
man

match any number of characters

match one character
command read the online manual page for a command

Basic Commands(3)

command > file

redirect standard output to a file
command >> file append standard output to a file
command < file
redirect standard input from a file
grep 'keyword' file
search a file for keywords
% grep science science.txt

wc file

count number of lines/words/characters in file

% wc -w science.txt

sort sort data (numerically or alphabetically)

Ex:
to sort the list of object, type
% sort < biglist
and the sorted list will be output to the screen.

Unix
Identification and authentication

Users have username

Internally identified with a user ID (UID)

Username to UID info in /etc/passwd
Super UID = 0

can access any file

Every user belong to a group has GID

Passwords to authenticate
in /etc/passwd

Shadow file /etc/shadow

Unix file security

Each file has owner and group
Permissions set by owner

Read, write, execute

Owner, group, other
Represented by vector of four octal values

Only owner, root can change permissions

This privilege cannot be delegated or shared

File system security (access

rights)

-rwxrwxrwx

a file that everyone can read, write and execute

(and delete).

-rw-------

a file that only the owner can read and write - noone else can read or write and no-one has
execution rights (e.g. your mailbox file).

Unix File Permissions

File type, owner, group, others

drwx-----lrwxrwxrwx
-rw-r--r--r-sr-xr-x
-r-sr-sr-x

2
1
1
1
1

jjoshi
jjoshi
jjoshi
root
root

isfac 512
isfac
15
isfac 1754
bin
9176
sys
2196

Aug 20 2003 risk management

Apr 7 09:11 risk_m->risk management
Mar 8 18:11 words05.ps
Apr 6 2002 /usr/bin/rs
Apr 6 2002 /usr/bin/passwd

File type: regular -, directory d, symlink l, device b/c, socket s, fifo f/p
Permission: r, w, x, s or S (set.id), t (sticky)

While accessing files

Process EUID compared against the file UID
GIDs are compared; then Others are tested

Simple Unix
Directory Structure
/
usr

etc

local bin ... class

home
grad

bin
ugrad ...

mmscott jpeckhar...

var ...

Basic Unix commands

cat file
cat file1 file2 ...
cd directory1
cd /usr/bin
cd
clear
cp file1 file2
cp file1 file2 ... dir
ls
ls /usr/bin
more file
mkdir directory
mv file1 file2
mv file1 file2 ... dir
mv dir1 dir2

Concatenate or type out a file

Type out a number of files
Change current directory to directory1
Change current directory to /usr/bin
Change back to your home directory
Clear the current screen
Copy file1 to file2
Copy a number of files to a directory
List the files in the current directory
List the files in the /usr/bin directory
Look at the content of a file with paging, use q to get out
Create a directory
Move file1 to file2, like rename.
Move a number of files into a directory
Move or rename a directory

Starting Vi
Opening an existing file
vi filename

Creating a new file

vi filename
In your workshop directory, create a new file called mysong
vimysong

Vi Modes of Operation
Command Mode
Allows the entry of commands to manipulate text
Default mode when vi starts
Use Escape key to move into command mode

Insert Mode and

Puts anything you type into the current file
To get into insert mode, commands are
a (append) and i (insert)

1. Use the i command to move into insert mode (Press i key).

2. Attempt to type in the title of your favorite song.
3. Use the Esc key to move to command mode.

Exiting the Vi Editor

:q Quit the editor
:q! Quit without saving changes to the file
1. Use the Esc key to make sure you are in command mode.
2. Use the :q command to try to quit vi

3. Use the :q! command to force quit without saving (Enter :q! ).

Saving Changes in vi
:wq Write/save changes and quite
:w Write/Save changes, but dont quit
1.
2.
3.
4.
5.

Type vi mysong to re-edit your song file.

Use the i command to move into insert mode (Press i key).
Retype the title of your favorite song.
Use the Esc key to move to command mode.
Use the :w command to write/save your edits to file.

6. Use the i command to enter insert mode (Enter i ).

7. Type Title: somewhere on the line with the song title.
8. Use the Esc key to move to command mode.

Vi Editor
How

to type commands in command mode

[count] command [where]

count : Its a number
where : Specifies how many lines or how much
of the document the command affects.
It can also be any command that moves
the cursor.

Moving the cursor in vi

1.
2.
3.
4.
5.
6.
7.

h key
move cursor to the left one
position
l key move cursor right one position
j key move cursor down one line
k key
cursor
up file.
one line
Type
vi mysongmove
to re-edit
your song
Use the l command several times to move cursor to the far right
Use the a command to move into append mode (Press a key).
Use the Enter key to start a new line of text.
Type: Artist: and then the name of the artist
Use the Esc key to move to command mode .
Practice moving cursor up, down, left,
and right with h,l,j,k keys.

Simple vi editing commands

r replace one character under the cursor
x
delete 1 character under the cursor.
2x delete 2 characters (3x, etc.)
u undo the last change to the file
1. Use the Esc key to make sure
you are still in command mode.
2. Reposition your cursor and
use the a, l, r and x commands
to repair any typos in your
title and artist, and change
the title to ALL CAPS
3. Use the :w command to save your changes.

Cutting text in Vi
d^
Deletes from current cursor position to the
beginning of the line
d$
Deletes from current cursor position to the
end of the line
Dw
Deletes from current cursor position to the
end of the word
dd
Deletes one line from current cursor position.
Specify count to delete many lines.

Cutting & Yanking Text in Vi

dd
Delete (cut) 1 line from current cursor
position
2dd Delete (cut) 2 lines (3dd to cut 2 lines,
1. Move etc.)
cursor to top line and type dd to cut the title line
2. Use the p command to paste the title line below the
p
paste lines below current line
artist line
3. Use the p command to paste it again.

Cutting & Yanking Text in Vi

yy
yank (copy) a single line
2yy
yank (copy) 2 lines (3yy to copy 3 lines,
etc.)
1. Move cursor to first of the 2 title lines and
P
paste lines before current line
type 2yy to yank/copy 2 lines
2. Move cursor to the first line, then use the capital P
command to paste the two yanked links above the artist

Vi Editor
To go to a specific line in the file
:linenumber
1. Go to the 3rd line by typing :3
2. Go to the 1st line by typing :1
3. Go to the last line by typing G

Vi string/search
/[pattern] search forward for the pattern
?[pattern] search backward for the pattern
n
search for the next instance of a string
1. Search forward for the next line containing the string Title
by typing /Title
2. Search forward for the next instance of Title by typing n
3. Search backward for the most recent instance of Title by
typing ?Title
4. Search backward for the next most recent instance of Title
by typing n

More commands
yl
yank a single character. Specify count to yank more
characters
yw
yank a single word. Specify count to yank more
words
d^
Deletes from current cursor position to the beginning
of the line
d$
Deletes from current cursor position to the
end of the line
Dw
Deletes from current cursor position to the
end of the word

Practice Editing with vi

Take 5 minutes to practice what youve learned by entering as many of the lyrics to the song
as you can.
Use yank and paste to repeat chorus lines.
Use :w to write changes every 30 seconds.
Have one title line at line 1.
Have one artist line at line 2.
Save file and exit vi when finished or time expires.