Unit-1

File handling utilities

security by file permissions
process utilities,
disk utilities
networking commands
backup utilities
text processing utilities
Working with the Bourne shell
What is a shell
shell responsibilities
pipes
input redirection, output redirection,
here documents

Introduction
Unix (officially trademarked as UNIX, sometimes also written as UNIX) is a multitasking, multi-user
computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs,
including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna. The Unix
operating system was first developed in assembly language, but by 1973 had been almost entirely
recoded in C, greatly facilitating its further development and porting to other hardware. Today's Unix
system evolution is split into various branches, developed over time by AT&T as well as various
commercial vendors, universities (such as University of California, Berkeley's BSD), and non-profit
organizations.

Unix features :

Open source
Unix is open source operating system.
Portability:
The system is written in high-level language making it easier to read, understand, change and, therefore
move to other machines. The code can be changed and complied on a new machine. Customers can

Unix programming

Page 1

Unit-1

then choose from a wide variety of hardware vendors without being locked in with a particular vendor.
Machine-independence:
The System hides the machine architecture from the user, making it easier to write applications that can
run on micros, mins and mainframes.

multi-user
more than one user can use the machine at a time supported via terminals (serial or network
connection) .
A multi-user operating system allows more than one user to share the same computer system at the same
time. It does this by time-slicing the computer processor at regular intervals between the various

users.

In the above example, there are five users which share the processor hardware and main
memory on a time basis.
We start off by giving the first user (which we will call Sophia) the processor hardware, and
run Sophia's program for 1/5th of a second. When the time is up, we intervene, save Sophia's
program state (program code and data) and then start running the second users program (for
1/5th of a second).
This process continues till we eventually get back to user Sophia. To continue running Sophia's
program, we restore the programs code and data and then run for 1/5th of a second.
This switching between user programs is done by part of the kernel. To switch from one program
to another requires,
Unix programming

Page 2

Unit-1

a regular timed interrupt event (provided by a clock)

saving the interrupted programs state and data
restoring the next programs state and data
running that program till the next timed interrupt occurs

multi-tasking
more than one program can be run at a time hierarchical directory structure to support the
organisation and maintenance of files
Multi-tasking systems support foreground and background tasks. A foreground task is one that the
user interacts directly with using the keyboard and screen. A background task is one that runs in the
background (it does not have access to the keyboard). Background tasks are usually used for printing or
backups.)

portability
only the kernel ( <10%) written in assembler tools for program development a wide range
of support tools (debuggers, compilers)
Hierarchical File System:
UNIX uses a hierarchile file structure to store information. This structure has the maximum flexibility in
grouping information in a way that reflects its natural state. It allows for easy maintenance and efficient

implementation.

UNIX shell:
UNIX has a simple user interface called the shell that has the power to provide the services that the user
wants. It protects the user from having to know the intricate hardware details.

Unix programming

Page 3

Unit-1

Pipes and Filters:

UNIX has facilities called Pipes and Filters which permit the user to create complex programs from
simple programs.

Utilities:
UNIX has over 200 utility programs for various functions. New utilities can be built effortlessly by
combining existing utilities.
Software Development Tools:
UNIX offers an excellent variety of tools for software development for all phases, from program editing
to maintenance of software,

Unix programming

Page 4

Unit-1

File handling utilities :

CAT Display one or more
files

Cat[-sn][filenames]
If more than one filename is supplied, it is added at the end of the
previous file and displayed.
-n = Display numbers in ascending order in front of each line.
-s = Do not comment about nonexistent files.
filenames = The filename(s) in the order that you want them to be
displayed.

Several UNIX commands are provided for text editing and shell programming. (emp.lst) each line of this file has six fields separated by five delimiters. The details of an employee are
stored in one single line. This text file designed in fixed format and containing a personnel
database. There are 15 lines, where each field is separated by the delimiter |.
$ cat emp.lst
2233 | a.k.shukla | g.m | sales | 12/12/52 | 6000
9876 | jai sharma | director | production | 12/03/50 | 7000
5678 | sumit chakrobarty | d.g.m. | marketing | 19/04/43 | 6000
2365 | barun sengupta | director | personnel | 11/05/47 | 7800

To create, copy, remove and change permissions on files you can use the following
commands

Unix programming

Page 5

Unit-1

Cp-copy the files

Unix programming

Page 6

Unit-1

rm removing the files

Unix programming

Page 7

Unit-1

Cmp----to compare the files

Unix programming

Page 8

Unit-1

diff ---difference between the two files

% diff mon.logins tues.logins

2d1
< bsmith
Unix programming

Page 9

Unit-1

4a4
> jdoe
7c7

< mschmidt
--> proy

touch
The touch command can be used to create a new (empty) file or to update the last
access date/time on an existing file. The command is used primarily when a script
requires the pre-existence of a file (for example, to which to append information)
or when the script is checking for last date or time a function was performed.
Syntax
touch [options] [date_time] file
touch [options] [-t time] file

Common Options
-a change the access time of the file (SVR4 only)
-c dont create the file if it doesnt already exist
-f force the touch, regardless of read/write permissions m change the modification time of the file (SVR4 only)
-t time use the time specified, not the current time (SVR4 only)
CC first two digits of the year
YY second two digits of the year
MM month, 01-12
DD day of month, 01-31
hh hour of day, 0023 mm minute, 00-59
SS second, 00-61

To create a file:
% touch filename
wc

Unix programming

Page 10

Unit-1

wc stands for "word count"; the command can be used to count the number of
lines, characters, or
words in a file.
Syntax
wc [options] file

Common Options
-c count bytes
-m count characters (SVR4)
-l count line
-w count words

If no options are specified it defaults to "-lwc".

ln----creating links
The ln command creates a "link" or an additional way to access (or gives an
additional name to)
another file.
Syntax
ln [options] source [target]

If not specified target defaults to a file of the same name in the present working
directory.
Common Options
-f force a link regardless of target permissions; dont report errors (SVR4 only)
-s make a symbolic link

Examples
A symbolic link is used to create a new path to another file or directory. If a group
of users, for
example, is accustomed to using a command called chkmag, but the command has
been rewritten and
is now called chkit, creating a symbolic link so the users will automatica
automatically execute chkit when they
enter the command chkmag will ease transition to the new command.
A symbolic link would be done in the following way:
% ln -s chkit chkmag
Unix programming

Page 11

Unit-1

The long listing for these two files is now as follows:

16 -rwxr-x--- 1 lindadb acs 15927 Apr 23 04:10 chkit
1 lrwxrwxrwx 1 lindadb acs 5 Apr 23 04:11 chkmag -> chkit
A hard link can only be done to another file on the same file system, but not to a directory
(except by the superuser). A hard link creates a new directory entry pointing to the same inode as
the original file. The file linked to must exist before the hard link can be created. The file will not
be deleted until all the hard links to it are removed. To link the two files above with a hard link to
each other do:
% ln chkit chkmag
Then a long listing shows that the inode number (742) is the same for each:
% ls -il chkit chkmag
742 -rwxr-x--- 2 lindadb acs 15927 Apr 23 04:10 chkit
742 -rwxr-x--- 2 lindadb acs 15927 Apr 23 04:10 chkmag
tee

tee sends standard in to specified files and also to standard out. Its often used in
command pipelines.
Syntax
tee [options] [file[s]]

Common Options
-a append the output to the files
-i ignore interrupts

Examples
In this first example the output of who is displayed on the screen and stored in the
file users.file:
brigadier: condron [55]> who | tee users.file
condron ttyp0 Apr 22 14:10 (lcondronpc.acs.) frank ttyp1 Apr 22 16:19 (nyssa)
condron ttyp9 Apr 22 15:52 (lcondron-mac.acs)
brigadier: condron [56]> cat users.file
condron ttyp0 Apr 22 14:10 (lcondronpc.acs.) frank ttyp1 Apr 22 16:19 (nyssa)
Unix programming

Page 12

Unit-1

condron ttyp9 Apr 22 15:52 (lcondron-mac.acs)

head displaying the beginning of the file
The command displays the top of the file. It displays the first 10 lines of the file, when used
without an option.
head emp.lst
-n to specify a line count
head -n 3 emp.lst
will display the first three lines of the file.

tail displaying the end of a file

This command displays the end of the file. It displays the last 10 lines of the file, when
used without an option.
tail emp.lst
-n to specify a line count
tail -n 3 emp.lst
displays the last three lines of the file. We can also address lines from the beginning of the file
instead of the end. The +count option allows to do that, where count represents the line number
from where the selection should begin.
tail +11 emp.lst
th

Will display 11 line onwards

Different options for tail are:
Monitoring the file growth (-f)
Extracting bytes rather than lines (-c)
Use tail f when we are running a program that continuously writes to a file, and we want to see
how the file is growing. We have to terminate this command with the interrupt key.

Unix programming

Page 13

Unit-1

cut slitting a file vertically

It is used for slitting the file vertically. head -n 5 emp.lst | tee shortlist will select the first
five lines of emp.lst and saves it to shortlist. We can cut by using -c option with a list of column
numbers, delimited by a comma (cutting columns).
cut -c 6-22,24-32 shortlist
cut -c -3,6-22,28-34,55- shortlist
The expression 55- indicates column number 55 to end of line. Similarly, -3 is the same as 1-3.
Most files dont contain fixed length lines, so we have to cut fields rather than columns (cutting
fields).
-d for the field delimiter
-f for the field list
cut -d \ | -f 2,3 shortlist | tee cutlist1
will display the second and third columns of shortlist and saves the output in cutlist1.
here | is escaped to prevent it as pipeline character
To print the remaining fields, we have cut
d \ | -f 1,4- shortlist > cutlist2
paste pasting files
When we cut with cut, it can be pasted back with the paste command, vertically rather than
horizontally. We can view two files side by side by pasting them. In the previous topic, cut was
used to create the two files cutlist1 and cutlist2 containing two cut-out portions of the same file.
paste cutlist1 cutlist2
We can specify one or more delimiters with -d
paste -d | cutlist1 cutlist2
Where each field will be separated by the delimiter |. Even though paste uses at least two files for
concatenating lines, the data for one file can be supplied through the standard input.
Joining lines (-s)

Unix programming

Page 14

Unit-1

Let us consider that the file address book contains the details of three
persons cat addressbook
paste -s addressbook -to print in one single line
paste -s -d | | \n addressbook -are used in a circular manner
sort : ordering a file
Sorting is the ordering of data in ascending or descending sequence. The sort command orders a
file and by default, the entire line is sorted
sort shortlist
This default sorting sequence can be altered by using certain options. We can also
sort one or more keys (fileds) or use a different ordering rule.
sort options
The important sort options are:
-tchar

uses delimiter char to identify fields

-k n

sorts on nth field

-k m,n

starts sort on mth field and ends sort on nth field

-k m.n

starts sort on nth column of mth field

-u

removes repeated lines

-n

sorts numerically

-r

reverses sort order

-f

folds lowercase to equivalent uppercase

-m list

merges sorted files in list

-c

checks if file is sorted

-o flname places output in file flname

Unix programming

Page 15

Unit-1

sort t| k 2 shortlist

sorts the second field (name)

sort t| r k 2 shortlist

or

sort t| k 2r shortlist

sort order can be revered with this r option.

sort t| k 3,3 k 2,2 shortlist

sorting on secondary key is also possible as shown above.

sort t| k 5.7,5.8 shortlist

we can also specify a character position with in a field to be the beginning of sort as
shown above (sorting on columns).

sort n numfile

when sort acts on numericals, strange things can happen. When we sort a file containing
only numbers, we get a curious result. This can be overridden by n (numeric) option.

Unix programming

Page 16

Unit-1

cut d | f3 emp.lst | sort u | tee desigx.lst

Removing repeated lines can be possible using u option as shown above. If we cut out
the designation filed from emp.lst, we can pipe it to sort to find out the unique designations that
occur in the file.

Other sort options are:

sort o sortedlist k 3 shortlist
sort o shortlist shortlist
sort c shortlist
sort t | c k 2 shortlist
sort m foo1 foo2 foo3
uniq command locate repeated and nonrepeated lines
When we concatenate or merge files, we will face the problem of duplicate entries creeping in.
we saw how sort removes them with the u option. UNIX offers a special tool to handle these
lines the uniq command. Consider a sorted dept.lst that includes repeated lines:
cat dept.lst
displays all lines with duplicates. Where as,
uniq dept.lst
simply fetches one copy of each line and writes it to the standard output. Since uniq requires a
sorted file as input, the general procedure is to sort a file and pipe its output to uniq. The
following pipeline also produces the same output, except that the output is saved in a file:
sort dept.lst | uniq uniqlist
Different uniq options are :
Selecting the nonrepeated lines (-u)
cut d | f3 emp.lst | sort | uniq u
Unix programming

Page 17

Unit-1

Selecting the duplicate lines (-d)

cut d | f3 emp.lst | sort | uniq d
Counting frequency of occurrence (-c)
cut d | f3 emp.lst | sort | uniq c
tr command translating characters
The tr filter manipulates the individual characters in a line. It translates characters using
one or two compact expressions.
tr options expn1 expn2 standard input
It takes input only from standard input, it doesnt take a filename as argument. By default, it
translates each character in expression1 to its mapped counterpart in expression2. The first
character in the first expression is replaced with the first character in the second expression, and
similarly for the other characters.
tr |/ ~- < emp.lst | head n 3
exp1=|/ ; exp2=~-
tr $exp1 $exp2 < emp.lst
Changing case of text is possible from lower to upper for first three lines of the file.
head n 3 emp.lst | tr [a-z] [A-Z]
Different tr options are:
Deleting charecters (-d)
tr d |/ < emp.lst | head n 3
Compressing multiple consecutive charecters (-s)
tr s < emp.lst | head n 3
Complementing values of expression (-c)
tr cd |/ < emp.lst
Using ASCII octal values and escape sequences
Unix programming

Page 18

Unit-1

tr | \012 < emp.lst | head n 6

security by file permissions

FILE ATTRIBUTES
. ls l provides attributes like permissions, links, owner, group owner, size, date and the file
name
All attributes of a file except its name and contents are available in a table inode (index
node), accessed by the inode number. The inode contains the following attributes of a file:

File type
File permissions
Number of links
The UID of the owner
The GID of the group owner
File size in bytes
Date and time of last modification
Date and time of last access
Date and time of last change of the inode
An array of pointers that keep track of all disk blocks used by the file

One of the distinguishing features of Unix is that it was designed from its earliest days to be a
multiuser system. In contrast, it is only in recent years that other operating systems have created
true multiuser functionality on a single machine. Because of its multiple-user design, Unix must
use mechanisms that enable users to manage their own files without having access to the files of
other users. These mechanisms are called file ownership and file permissions.
File Ownership
Generally, the files that the user owns are ones that he created, or which were created as a result
of some action on his part. The superuser can change the ownership of any file, whether he
created it or not, with the chown command. For example, if the superuser gives the command
chown jane /home/bill/billsfile
Username versus UID
idea of a username, the name you use when you log in to a Unix machine. The name is assigned
to you by the system administrator . In addition to a username, every user has a numerical ID
number known as a user ID or UID, which is how the user is known to the system. Typically,
these numnumbers are assigned automatically,although they can be specified when an account is
created. The number itself is arbitrary, even though many systems require that ordinary users
have UID numbers above 500.
The superuser always has UID 0.

Unix programming

Page 19

Unit-1

For purposes other than logging in, the username and UID are basically synonymous. For
example, the command
chown jane /home/bill/billsfile could just as easily be rendered as
chown 503 /home/bill/billsfile assuming that Janes UID is 503

Groups
In addition to a UID, you also have at least one group ID, or GID. As with the UID, the operating system
uses GIDs rather than the group names to manage groups. Every user belongs to at least one group, and
may belong to several more. Groups contain users who share certain permissions for certain
activities.You may belong to a group that has the same name as your username. The superuser may add
you to other groups depending on the access you need to certain files or directories. Groups can also own
files, and file ownership can be transferred from one group to another. To do so use the chgroup (change
group) command
chgroup groupname filename

File Permissions
Permissions should be as restrictive as possible. As long as legitimate users can use the file in
intended ways, every other avenue of access should be locked down.
There are three kinds of file permission:
Read (file can be viewed)
Write (file can be edited)

Execute (file can be run as a program)

Likewise, there are three categories of users to whom these permissions can be
applied: User (owner of that particular file)
Group (group to which the file is
assigned) All (all users and groups)

How to Read a Permissions List

just use the ls -l command, and the permission information is printed as part of a directory listing
ls l
-rw-r--r-- 1 root root 1048 Mar 3 2001 aliases rw-r--r-- 1 root root 12288 Sep 8 2003 aliases.db
-rw-r--r-- 1 root root 370 Apr 3 2001 anacrontab
-rw------- 1 root root 1 Apr 4 2001 at.deny

Changing File Permissions

A file or a directory is created with a default set of permissions, which can be determined
by umask. Let us assume that the file permission for the created file is -rw-r--r--. Using chmod
Unix programming

Page 20

Unit-1

command, we can change the file permissions and allow the owner to execute his file. The
command can be used in two ways:

In a relative manner by specifying the changes to the current permissions

In an absolute manner by specifying the final permissions

Relative Permissions

chmod only changes the permissions specified in the command line and leaves the other
permissions unchanged. Its syntax is:

chmod category operation permission filename(s)

chmod takes an expression as its argument which contains:

user category (user, group, others)
operation to be performed (assign or remove a permission)
type of permission (read, write, execute)

Category

operation

permission

u - user

+ assign

r - read

g - group

- remove

w - write

o - others

= absolute

x - execute

a - all (ugo)

Unix programming

Page 21

Unit-1

Let us discuss some examples:

Initially,
-rw-r--r--

kumar metal 1906 sep 23:38 xstart

chmod u+x xstart

-rwxr--r-- 1

kumar metal 1906 sep 23:38 xstart

The command assigns (+) execute (x) permission to the user (u), other permissions remain
unchanged.

chmod ugo+x xstart or

chmod a+x xstart

or

chmod +x xstart

-rwxr-xr-x 1

kumar metal 1906 sep 23:38 xstart

chmod accepts multiple file names in command line

chmod u+x note note1 note3

Let initially,

-rwxr-xr-x

1 kumar metal 1906 sep 23:38

Unix programming

xstart
Page 22

Unit-1

chmod go-r xstart

Then, it becomes

-rwx--x--x 1 kumar metal 1906 sep 23:38 xstart

Absolute Permissions

Here, we need not to know the current file permissions. We can set all nine permissions
explicitly. A string of three octal digits is used as an expression. The permission can be
represented by one octal digit for each category. For each category, we add octal digits. If we
represent the permissions of each category by one octal digit, this is how the permission can be
represented:

Read permission 4 (octal 100)

Write permission 2 (octal 010)
Execute permission 1 (octal 001)

Octal

Permissions

Significance

---

no permissions

--x

execute only

-w-

write only

-wx

write and execute

r--

read only

r-x

read and execute

Unix programming

Page 23

Unit-1

rw-

read and write

rwx

read, write and execute

We have three categories and three permissions for each category, so three octal digits
can describe a files permissions completely. The most significant digit represents user and the
least one represents others. chmod can use this three-digit string as the expression.

Using relative permission, we have,

chmod a+rw xstart

Using absolute permission, we have,

chmod 666 xstart

chmod 644 xstart

chmod 761 xstart

will assign all permissions to the owner, read and write permissions for the group and only
execute permission to the others.

777 signify all permissions for all categories, but still we can prevent a file from being
deleted. 000 signifies absence of all permissions for all categories, but still we can delete a file. It
is the directory permissions that determine whether a file can be deleted or not. Only owner can
Unix programming

Page 24

Unit-1

change the file permissions. User can not change other users files permissions. But the system
administrator can do anything.

The Security Implications

Let the default permission for the file xstart is

-rw-r--r-chmod u-rw, go-r xstart

or

chmod 000 xstart

----------

This is simply useless but still the user can delete this file
On the other hand,

chmod a+rwx xstart

chmod 777 xstart

-rwxrwxrwx

The UNIX system by default, never allows this situation as you can never have a secure system.
Hence, directory permissions also play a very vital role here
Unix programming

Page 25

Unit-1

We can use chmod Recursively.

chmod -R a+x shell_scripts

This makes all the files and subdirectories found in the shell_scripts directory, executable by all
users. When you know the shell meta characters well, you will appreciate that the * doesnt
match filenames beginning with a dot. The dot is generally a safer but note that both commands
change the permissions of directories also.

Directory Permissions

It is possible that a file cannot be accessed even though it has read permission, and can be
removed even when it is write protected. The default permissions of a directory are,

rwxr-xr-x (755)

A directory must never be writable by group and others

Example:

mkdir c_progs

ls ld c_progs

Unix programming

Page 26

Unit-1

drwxr-xr-x 2 kumar metal 512 may 9 09:57 c_progs

If a directory has write permission for group and others also, be assured that every user
can remove every file in the directory. As a rule, you must not make directories universally
writable unless you have definite reasons to do so.

Changing File Ownership

Usually, on BSD and AT&T systems, there are two commands meant to change the
ownership of a file or directory. Let kumar be the owner and metal be the group owner. If sharma
copies a file of kumar, then sharma will become its owner and he can manipulate the attributes

chown changing file owner and chgrp changing group owner

On BSD, only system administrator can use chown

On other systems, only the owner can change both

chown

Changing ownership requires superuser permission, so use su command

ls -l note

-rwxr----x 1 kumar metal 347 may 10 20:30 note

Unix programming

Page 27

Unit-1

chown sharma note; ls -l note

-rwxr----x 1 sharma metal 347 may 10 20:30 note

Once ownership of the file has been given away to sharma, the user file permissions that
previously applied to Kumar now apply to sharma. Thus, Kumar can no longer edit note since
there is no write privilege for group and others. He can not get back the ownership either. But he
can copy the file to his own directory, in which case he becomes the owner of the copy.

chgrp

This command changes the files group owner. No superuser permission is required.

ls l dept.lst

-rw-r--r-- 1 kumar metal 139 jun 8 16:43 dept.lst

chgrp dba dept.lst; ls l dept.lst

-rw-r--r-- 1 kumar dba 139 jun 8 16:43 dept.lst

In this chapter we considered two important file attributes permissions and ownership. After
we complete the first round of discussions related to files, we will take up the other file
attributes.

Unix programming

Page 28

Unit-1

umask: DEFAULT FILE AND DIRECTORY PERMISSIONS

When we create files and directories, the permissions assigned to them depend on the
systems default setting. The UNIX system has the following default permissions for all files and
directories.
rw-rw-rw- (octal 666) for regular files
rwxrwxrwx (octal 777) for directories
The default is transformed by subtracting the user mask from it to remove one or more
permissions. We can evaluate the current value of the mask by using umask without arguments,
$ umask

process utilities

What Processes Are Running?

It is easy to see your own processes by running the ps (process status) command
$ ps
PID TTY TIME CMD 18358
ttyp3 00:00:00 sh 18361 ttyp3
00:01:31 abiword 18789 ttyp3
00:00:00 ps

Process States
Ps-show status of active processes

ps [options]
Common Options
BSD SVR4
-a -e all processes, all users
-e environment/everything
-g process group leaders as well
-l -l long format
-u -u user user oriented report
-x -e even processes not executed from terminals
-f full listing
Unix programming

Page 29

Unit-1

-w report first 132 characters per line

In the Linux ps example shown earlier, the output did not show information about process states.
To have ps output this extra information, using the BSD ps syntax, you can use the u argument: $
ps u
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
reed 18358 0.0 0.7 2460 668 ttyp3 S Sep13 0:00 -sh
reed 18361 0.0 11.6 32936 10756 ttyp3 S Sep13 6:12 abiword
reed 19736 0.0 0.8 2508 784 ttyp3 R 18:12 0:00 ps u
$ ps -l
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
8 S 511 366 360 0 50 20 ? 332 ? pts/2 0:01 bash
8 S 511 360 358 0 40 20 ? 136 ? pts/2 0:00 sh 8
O 511 11820 366 0 50 20 ? 138 pts/2 0:00 ps

System Processes
System processes are programs running behind the scenes handling many essential maintenance aspects
for your system. Normally, system processes do not have a TTY (teletype) in use. Many of these
processes are often called daemons, and they do routine work. The following is an example of system
processes running on a Linux system

$ ps ax
PID TTY STAT TIME COMMAND
1 ? S 0:00 init [3]
2 ? SW 0:00 [migration/0]
3 ? SWN 0:00 [ksoftirqd/0]
4 ? SW< 0:00 [events/0]
5 ? SW< 0:00 [khelper]
6 ? SW< 0:00 [kacpid]
20 ? SW< 0:00 [kblockd/0]
21 ? SW 0:00 [khubd]
31 ? SW 0:00 [pdflush]
32 ? SW 0:00 [pdflush]
33 ? SW 0:13 [kswapd0]
34 ? SW< 0:00 [aio/0]
618 ? SW 0:00 [kseriod]
646 ? SW< 0:00 [ata/0]
647 ? SW 0:00 [khpsbpkt]
670 ? SW 0:00 [kjournald]
788 ? SW 0:00 [kjournald]
793 ? S 0:00 /usr/sbin/syslogd
797 ? S 0:00 /usr/sbin/klogd
816 ? S 0:00 /usr/sbin/sshd
Unix programming

Page 30

Unit-1

829 tty1 S 0:00 /sbin/agetty 38400 tty1

830 tty2 S 0:00 /sbin/agetty 38400 tty2
831 tty3 S 0:00 /sbin/agetty 38400 tty3
It uses the -e switch to output
information on every process:
$ ps -e
PID TTY TIME
CMD 0 ? 0:02 sched
1 ? 0:18 init
2 ? 0:00 pageout
3 ? 8:06 fsflush
314 ? 0:00 sac
224 ? 0:00 utmpd
315 console 0:00 ttymon
47 ? 0:00 sysevent
54 ? 0:00 picld
130 ? 0:00 rpcbind
191 ? 0:00 syslogd
179 ? 0:00 automoun
153 ? 0:00 inetd
205 ? 0:00 nscd
213 ? 0:00 powerd
166 ? 0:00 statd
203 ? 0:00 cron
169 ? 0:00 lockd

Process Attributes
Each process has an environment with various attributes such as command-line arguments, user
environment variables, file descriptors, working directory, file creation mask, controlling terminal
(console),resource limitations, and a lot more. Many of the attributes are shared with the parent process.
To view the various process attributes, you can use the ps -o switch, which is available for both styles of ps

Unix programming

Page 31

Unit-1

Stopping Processes
kill - terminate a process
Ending a process can be done in several different ways. Often, from a console-based command, sending a
CTRL + C keystroke (the default interrupt character) will exit the command. The standard tool for killing a
process is kill. Technically, the kill command does not kill a command,but sends a special signal to the process. Signals
are used for simple communication between processes. To use the kill command, just place the process ID of the

process to signal as the command line argument.For example, to send the default SIGTERM signal to the
process ID 5432, run the command:
kill 5432
To list the possible signal names, run kill with the -l switch:
$ kill -l
HUP INT QUIT ILL TRAP ABRT EMT FPE KILL BUS
SEGV SYS PIPE ALRM TERM USR1 USR2 CLD PWR WINCH URG
POLL STOP TSTP CONT TTIN TTOU VTALRM PROF XCPU

XFSZ WAITING LWP FREEZE THAW CANCEL LOST XRES RTMIN RTMIN+1
RTMIN+2 RTMIN+3 RTMAX-3 RTMAX-2 RTMAX-1 RTMAX

The Process Tree

A process tree displays the lineage of your different processes, placing a child process with its
parent
$ pstree init-+3*[agetty] |events/0-+-aio/0 |
|-ata/0
| |-kacpid
| |-kblockd/0
| |-khelper
| `-2*[pdflush]
|-gconfd-2 |kdeinit-+-artsd
| |-firefox-bin---firefox-bin---3*[firefoxbin] | |-3*[kdeinit]
| `-kdeinit---sh---ssh
Unix programming

Page 32

Unit-1

|-8*[kdeinit] |khpsbpkt |khubd |2*[kjournald]

|-klogd |kseriod |ksoftirqd/0 |kswapd0 |migration/0
|-sshd---sshd---sshd---sh---pstree
|-syslogd
`-xdm-+-X `-xdm--startkde---kwrapper

Zombie Processes
Normally, when a child process is killed, the parent process is told via a SIGCHLD signal. Then the
parent can do some other task or restart a new child as needed. However, sometimes the parent process is
killed. In this case, the parent of all processes, init, becomes the new PPID (parent process ID). You can
often see this indicated by a process ID of 1 as the PPID of some other process.

The /proc File System

The /proc file system is a dynamically generated file system that can be used to retrieve information about
processes running on your system. Depending on the Unix system, the /proc file system can also be used
to retrieve and set other kernel-level configurations
$ ls -l /proc/$$
total 5402
-rw------- 1 jreed build 2736128 Dec 16 12:06 as
-r-------- 1 jreed build 152 Dec 16 12:06 auxv
-r-------- 1 jreed build 32 Dec 16 12:06 cred
--w------- 1 jreed build 0 Dec 16 12:06 ctl
lr-x------ 1 jreed build 0 Dec 16 12:06 cwd ->
dr-x------ 2 jreed build 8208 Dec 16 12:06 fd
-r--r--r-- 1 jreed build 120 Dec 16 12:06 lpsinfo

Shell Job Control

The Unix command-line shell can be used to run programs in the background so you can run multiple
programs at a time. It also can be used to suspend commands and restart suspended commands. To tell the
shell to run a given command in the background, simply append an ampersand to the end of the command
line. For example, to run sleep in the background, issue the command:
$ sleep 60 &
[1] 16556
$

disk utilities
df - summarize disk block and file usage
df is used to report the number of disk blocks and inodes used and free for each file
system. The

Unix programming

Page 33

Unit-1

output format and valid options are very specific to the OS and program version in
use.
Syntax
df [options] [resource]

Common Options
-l local file systems only
(SVR4) -k report in kilobytes
(SVR4) df
Filesystem kbytes used avail capacity Mounted on
/dev/sd0a 20895 19224 0 102% /
/dev/sd0h 319055 131293 155857 46% /usr
/dev/sd1g 637726 348809 225145 61% /usr/local

du - report disk space in use

du reports the amount of disk space in use for the files or directories you specify.
Syntax
du [options] [directory or file]

Common Options
-a display disk usage for each file, not just subdirectories
-s display a summary total only
-k report in kilobytes (SVR4)

du
1 ./.elm
1 ./Mail
1 ./News
20 ./uc
du -a uc
7 uc/unixgrep.txt
5 uc/editors.txt
1 uc/.emacs
1 uc/.exrc
4 uc/telnet.ftp
1 uc/uniq.tee.txt
20 uc
Unix programming

Page 34

Unit-1

networking commands

TELNET and FTP are Application Level Internet protocols. The TELNET and
FTP protocol specifications have been implemented by many different sources,
including The National Center for Supercomputer Applications (NCSA), and many
other public domain and shareware sources rlogin is a remote login service that
was at one time exclusive to Berkeley 4.3 BSD UNIX.
Essentially, it offers the same functionality as telnet, except that it passes to the
remote computer information about the user's login environment. Machines can be
configured to allow connections from trusted hosts without prompting for the
users passwordsA more secure version of this protocol is the Secure SHell, SSH,
software written by Tatu Ylonen and available via ftp://ftp.net.ohiostate.edu/pub/security/ssh.
the r commandsrsh (remotshell), rcp (remote copy), and rlogin (remote login)were prevalent
in the past, but because they
offer little security, theyre generally discouraged in todays environments. rsh and rlogin are
similar in functionality to telnet, and rcp is similar to ftp.
telnet [options] [ remote_host [ port_number ] ]
tn3270 [options] [ remote_host [ port_number ] ]
ftp [options] [ remote_host ]
Common Options
ftp telnet Action
-d set debugging mode on
-d same as above (SVR4 only) -i
turn off interactive prompting
-n dont attempt auto-login on connection
-v verbose mode on
-l user connect with username, user, on the remote host (SVR4
only) -8 8-bit data path (SVR4 only)
telnet solaris
or
telnet 192.168.1
Unix programming

Page 35

Unit-1

telnet oscar.us.ohio-state.edu
tn3270 ohstmvsa.acs.ohio-state.edu
ftp magnus.acs.ohio-state.edu

finger - get information about users

finger displays the .plan file of a specific user, or reports who is logged into a
specific machine. The user must allow general read permission on the .plan file.
Syntax
finger [options] [user[@hostname]]

Common Options
-l force long output format
-m match username only, not first or last names
-s force short output format

Examples
brigadier: condron [77]> finger workshop@nyssa This
is a sample .plan file for the nyssa id, workshop.
This id is being used this week by Frank Fiamingo, Linda
DeBula, and Linda Condron, while we teach a pilot version
of the new Unix workshop we developed for UTS.

Remote login
rlogin [ -l username ] remote_host
rsh [ -l username ] remote_host [ command ]
rcp [ [user1]@host1:]original_filename [ [user2]@host2:]new_filename

where the parts in brackets ([]) are optional. rcp does not prompt for passwords, so
you must have permission to execute remote commands on the specified machines
as the selected user on each machine.
Common Options
-l username connect as the user, username, on the remote host (rlogin & rsh)

Using ssh
ssh (Secure SHell) and telnet are two methods that enable you to log in to a remote
system and run commands interactively;
command hostname
ssh darwin
or
ssh 192.168.1.58

Unix programming

Page 36

Unit-1

backup utilities

Deciding what to back up is a time-consuming process, but well worth the effort, especially if you
have to recover information.

generally want to back up the entire file system. If that isnt possible because of resource
constraints(money and/or time)

Backup Media Types

Magnetic tapeOne of the oldest ways to back up data. It has stood the test of time and proven to be
very durable. The newest versions of magnetic tape are very fast and can store a significant amount of
data .This media is well suited to long-term
storage (20 or more years).
CD/DVD-ROM. The longterm durability of this type of storage has not been confirmed, but is
generally thought to be1020 years, depending on the quality of media chosen. This is the most common
backup hardware for home computers today.

_Zip drivesAn older technology losing market share because of its low capacity and slow speed. Hard
drive/network backupThis type of backup is more and more common with the price of hard

drive space at a low point.

Backup Types
Full: Backs up all data within the scope of the backup definition

Differential: Backs up all data that has changed since the last full backup

Incremental: Backs up all data that has changed since the completion of any other type of Backup

Backup Commands
Using tar
The tar (tape archive) command enables you to create tape archives and, if needed, send them
directly to a tape device. The tar command has many functions, including:
tShows the table of contents for the tarfile
xExtracts or restores the contents of the tarfile
uUpdates the contents of a tarfile
The syntax for creating a tarfile is:
tar -cvf tarfile_name_or_tape_device directory_name

The c argument means to begin writing at the beginning of the tarfile,

the v represents verbose, which provides a lot of output showing what the command is
doing.

The f option notifies the tar command that the user will be providing a tarfile name
instead of using the default identified in /etc/default/tar.
To create a tarfile with the contents of the entire /etc directory to a properly configured
tape device named /dev/rmt0, for example, youd use the following command:
tar -cvf /dev/rmt0 /etc
If you want only a simple listing, you can omit the v option, reducing output; your command
would be
tar -tf /dev/rmt0
or

Unit-1

tar -tf /backups/etc-backup-122004.tar

which would show output similar to the following:
#tar -tf /backups/etc-backup-122004.tar
etc/
etc/profile.d/ etc/profile.d/sshclient.sh
To restore the contents of a tarfile, use the -xvf options to tar. Heres the syntax for the extraction
of a
tarfile:
tar -xvf tarfile_name_or_tape_device directory_name

Compressing with gzip and bzip2

gzip
GNU zip (gzip) is the most popular compression program available for Unix operating systems,
and rightfully so because of its portability among different system architectures and its full
feature set. The command syntax for a simple compression of a file using gzip is: gzip filename
To make a standard text file called mytextfile take up less space by compressing it, you would
run:
gzip mytextfile

To view information on the gzipped file, you can use the -l option, which shows output similar to
the
following:
#gzip -l mytextfile.gz
compressed uncompr. ratio uncompressed_name
614 1660 64.8% mytextfile

bzip2
The bzip2 command offers the same type of functionality as gzip, with some improvements on
the
compression capability.
Heres the syntax for a simple file compression using bzip2:
bzip2 filename

cpio
The cpio (copy in/out) command is more difficult to use than the tar command but provides some special
functionality, such as the capability to save special files

Unix programming

Page 38

Unit-1

text processing utilities:

FILTERS USING REGULAR EXPRESSIONS grep and sed

We often need to search a file for a pattern, either to see the lines containing (or not
containing) it or to have it replaced with something else. This chapter discusses two important
filters that are specially suited for these tasks grep and sed. grep takes care of all search
requirements we may have. sed goes further and can even manipulate the individual characters in
a line. In fact sed can de several things, some of then quite well.
grep searching for a pattern
It scans the file / input for a pattern and displays lines containing the pattern, the line
numbers or filenames where the pattern occurs. Its a command from a special family in UNIX
for handling search requirements.
grep options pattern filename(s)
grep sales emp.lst
will display lines containing sales from the file emp.lst. Patterns with and without quotes is
possible. Its generally safe to quote the pattern. Quote is mandatory when pattern involves more
than one word. It returns the prompt in case the pattern cant be located.
grep president emp.lst

Unix programming

Page 39

Unit-1

When grep is used with multiple filenames, it displays the filenames along with the output.
grep director emp1.lst emp2.lst
Where it shows filename followed by the contents
grep options
grep is one of the most important UNIX commands, and we must know the options that
POSIX requires grep to support. Linux supports all of these options.
-i

ignores case for matching

-v

doesnt display lines matching expression

-n

displays line numbers along with lines

-c

displays count of number of occurrences

-l

displays list of filenames only

-e exp

specifies expression with this option

-x

matches pattern with entire line

-f file

takes pattrens from file, one per line

-E

treats pattren as an extended RE

-F

matches multiple fixed strings

grep -i agarwal emp.lst

grep -v director emp.lst > otherlist
wc -l otherlist will display 11 otherlist
grep n marketing emp.lst
grep c director emp.lst
grep c director emp*.lst
will print filenames prefixed to the line count
grep l manager *.lst
Unix programming

Page 40

Unit-1

will display filenames only

grep e Agarwal e aggarwal e agrawal emp.lst
will print matching multiple patterns
grep f pattern.lst emp.lst
all the above three patterns are stored in a separate file pattern.lst
Basic Regular Expressions (BRE) An Introduction
It is tedious to specify each pattern separately with the -e option. grep uses an expression
of a different type to match a group of similar patterns. If an expression uses meta characters, it
is termed a regular expression. Some of the characters used by regular expression are also
meaningful to the shell.
BRE character subset
The basic regular expression character subset uses an elaborate meta character set,
overshadowing the shells wild-cards, and can perform amazing matches.
*

Zero or more occurrences

g*

nothing or g, gg, ggg, etc.

A single character

.*

nothing or any number of characters

[pqr]

a single character p, q or r

[c1-c2]

a single character within the ASCII range represented by c1 and c2

The character class

grep supports basic regular expressions (BRE) by default and extended regular
expressions (ERE) with the E option. A regular expression allows a group of characters
enclosed within a pair of [ ], in which the match is performed for a single character in the group.
grep [aA]g[ar][ar]wal emp.lst
A single pattern has matched two similar strings. The pattern [a-zA-Z0-9] matches a single
alphanumeric character. When we use range, make sure that the character on the left of the
Unix programming

Page 41

Unit-1

hyphen has a lower ASCII value than the one on the right. Negating a class (^) (caret) can be
used to negate the character class. When the character class begins with this character, all
characters other than the ones grouped in the class are matched.
The *
The asterisk refers to the immediately preceding character. * indicates zero or more occurrences
of the previous character.
g* nothing or g, gg, ggg, etc.
grep [aA]gg*[ar][ar]wal emp.lst
Notice that we dont require to use e option three times to get the same output!!!!!
The dot
A dot matches a single character. The shell uses ? Character to indicate that.
.*

signifies any number of characters or none

grep j.*saxena emp.lst

Specifying Pattern Locations (^ and $)
Most of the regular expression characters are used for matching patterns, but there are
two that can match a pattern at the beginning or end of a line. Anchoring a pattern is often
necessary when it can occur in more than one place in a line, and we are interested in its
occurance only at a particular location.
^

for matching at the beginning of a line

for matching at the end of a line

grep ^2 emp.lst

Selects lines where emp_id starting with 2

grep 7$ emp.lst
Selects lines where emp_salary ranges between 7000 to 7999
grep ^[^2] emp.lst
Selects lines where emp_id doesnt start with 2
Unix programming

Page 42

Unit-1

When meta characters lose their meaning

It is possible that some of these special characters actually exist as part of the text.
Sometimes, we need to escape these characters. For example, when looking for a pattern g*, we
have to use \
To look for [, we use \[
To look for .*, we use \.\*
Extended Regular Expression (ERE) and grep
If current version of grep doesnt support ERE, then use egrep but without the E option.
-E option treats pattern as an ERE.
+

matches one or more occurrences of the previous character

Matches zero or one occurrence of the previous character

b+ matches b, bb, bbb, etc.

b? matches either a single instance of b or nothing
These characters restrict the scope of match as compared to the *
grep E [aA]gg?arwal emp.lst
# ?include +<stdio.h>
The ERE set
ch+

matches one or more occurrences of character ch

ch?

Matches zero or one occurrence of character ch

exp1|exp2

matches exp1 or exp2

(x1|x2)x3

matches x1x3 or x2x3

Matching multiple patterns (|, ( and ))

grep E sengupta|dasgupta emp.lst
We can locate both without using e option twice, or

Unix programming

Page 43

Unit-1

grep E (sen|das)gupta emp.lst

sed The Stream Editor
sed is a multipurpose tool which combines the workof several filters. sed uses for
instructions to act on text. An instruction combines an address selecting lines, with an
action to be taken on them.
sed options address action file(s)
sed supports only the BRE set. Address specifies either one line number to select a single line or
a set of two lines, to select a group of contiguous lines. action specifies print, insert, delete,
substitute the text.
sed processes several instructions in a sequential manner. Each instruction operates on
the output of the previous instruction. In this context, two options are relevant, and probably they
are the only ones we will use with sed the e option that lets us use multiple instructions, and
the f option to take instructions from a file. Both options are used by grep in identical manner.
Line Addressing
sed 3q emp.lst
Just similar to head n 3 emp.lst. Selects first three lines and quits
sed n 1,2p emp.lst
p prints selected lines as well as all lines. To suppress this behavior, we use n whenever we use
command
sed n $p emp.lst
Selects last line of the file
sed n 9,11p emp.lst

find
You can also use the find command to locate files in Unix, although this command may consume
a lot of resources (and make the system respond slowly).
Heres the syntax:
find pathname selection_riteria options

Common Options
For the time search options the notation in days, n is:
Unix programming

Page 44

Unit-1

+n more than n days

n exactly n days
-n less than n days

Some file characteristics that find can search for are:

time that the file was last accessed or changed atime n access time, true if accessed n days ago
-ctime n change time, true if the files status was changed n days ago mtime n modified time, true if the files data was modified n days ago
-newer filename true if newer than filename
-type type type of file, where type can
be: b block special file
c character special file
d directory
l symbolic link
p named pipe (fifo)
f regular file
-fstype type type of file system, where type can be any valid file system type, e.g.: ufs
(Unix File System) and nfs (Network File System)
-user username true if the file belongs to the user username group groupname true if the file belongs to the group groupname
-perm [-]mode permissions on the file, where mode is the octal modes for the chmod
command. When mode is precede by the minus sign only the bits that are set
are compared.
-exec command execute command. The end of command is indicated by and
escaped semicolon (\;). The command argument, {}, replaces the current path name.
-name filename true if the file is named filename. Wildcard pattern matches are allowed if
the meta-character is escaped from the shell with a backslash (\).
-ls always true. It prints a long listing of the current pathname. print print the pathnames found (default for SVR4, not for BSD)
If you want to find the lostfile file, for example, and you think its somewhere in the /usr/share
directory, you can use this command:
find /usr/share -name lostfile -print
There are many options to the find command, and you should refer to the man (manual) pages
for more instructions on its many uses.
% find . -name \*ar\* -ls
326584 7 -rw-r----- 1 frank staff 6682 Feb 5 10:04 ./library
326585 17 -rw-r----- 1 frank staff 17351 Feb 5 10:04 ./standard

wc
Use the wc command to determine the length of a given file. wc uses the
syntax: wc [options] filename
By default, the output shows the length in words. The following table shows the options
available for wc.
Unix programming

Page 45