UNIX

Foundations
THE PROCESS AND THE
KERNEL
 Course Description
 The Goals for this course
 Understand UNIX
 Understand Operating Systems
 Prerequisites:
 COSC 2P13 : Introduction to Operating
Systems
 COSC 2P91 : Procedural Programming
Course Description
 An intensive study of computer operating
system design
 Multiprogramming
 Time-sharing
 Real-time processing
 Job and task control
 Synchronization of concurrent processes and
processors
 Resource scheduling
 Protection
 Management of hierarchical storage.
How to study this course

 Read and remember

 Read the book, remember the concepts and
commands
 Think
 Think operating systems as natural administrative
agents
 Practice
 Coding with UNIX, use and understand of UNIX
commands and get the results
The Textbook Used
 Uresh Vahalia, UNIX Internals: The New Frontiers,
Prentice Hall, 1996
 Why we use this book?
 UNIX is one of the most popular operating systems
of the world.
 If you understand UNIX, you can understand other
operating systems.
 References
 William Stallings, Operating Systems,5th
Ed., Prentice Hall,2005
 A. Tanenbaum, Modern Operating
Systems, 2nd ed. Prentice Hall 2001
 Kay A. Robbins and Steven Robbins,
UNIX Systems Programming, Prentice
Hall,2003
 W. R. Stevens, Advanced Programming
in the UNIX Environment, Addison
Wesley, 1992
 Summary of UNIX
History
XPG
Digital UNIX
SVID POSIX

Solaris ULTRIX
4.4BSD
HP-UX
SVR4
SCO UNIX 4.3BSD
AIX SunOS
SVR3
4.2BSD
XENIX
SVR2
4BSD

3BSD

UNIX
 V1
.
.
Summary of
UNIX History
.
V6
Xenix BSD
V7
PWB
2BSD
.
PWB2 32V .
3BSD
.
2.9BSD
SIII V8
Xenix2 2.10BSD
4BSD

SYSV 2.11BSD
V10 4.2BSD

4.3BSD
SCO V.2 Plan9
Ultrix 4.4BSD
V.3 AIX SUNOS
V.3.2 Mach

SVR4 Solaris OSF1

LINUX
Flexibility

 Traditional Kernel: file, scheduling, executable

file formats
Modern UNIX Kernel
Assessment of UNIX

 Advantages: -
open software process, -
well designed, small and simple kernel,
- text files in
system databases, - simple,
uniform interface to I/O devices,
- portability (written in
C)
Assessment of UNIX

 Disadvantages:
- expansion of the more and more complex I/O
library, - unfriendly
user interface, - building
block approach to tools, but not to the kernel,
- too many
versions and standards, -
monolithic, unmodular and more and more
complex kernel.
UNIX Architecture

 Hardware is surrounded by the operating-

system
 Operating system is called the kernel
 Comes with a number of user services and
interfaces
 shell
 C compiler
 UNIX Architecture

Application Programs

Shell Editors, and

Private User Programs
Compiler Components

Kernel
Compiler

Hardware
 User
Interface

The layers of a UNIX system.

 UNIX Utility Programs

A few of the more common UNIX utility programs required

by POSIX
 UNIX shell programming

 cp src dest
 head –20 file
 ls *.c
 sort <in >out
 sort <in >temp;head –30<temp;rm temp
 sort <in | head –30
 grep ter *.t | sort | head –20 | tail –5 >foo
 sort <x | head &
Process

 In UNIX
 Process is an instance of a running program.
 Lifetime: fork/vfork->exec->exit
 Well-defined hierarchy: parent,child,init,
 init process: the top process
 swapper & pagedeamon
 Orphans: the parent process is terminated.
Process Creation

 Submission of a batch job

 User logs on
 Created to provide a service such as printing
 Spawned by an existing process
Process Termination

 Batch job issues Halt instruction

 User logs off
 Process executes a service request to terminate
 Error and fault conditions
 Reasons for Process Termination
 Normal completion
 Time limit exceeded
 Memory unavailable
 Bounds violation
 Protection error
 example write to read-only file
 Arithmeticerror
 Time overrun
 process
waited longer than a specified
maximum for an event
 Reasons for Process
Termination

 I/Ofailure
 Invalid instruction
 happens when try to execute data
 Privilegedinstruction
 Data misuse
 Operating system intervention
 such as when deadlock occurs
 Parent terminates so child processes
terminate
 Parent request
 Process States
 The Running state
 The process that gets executed (single CPU)
 The Ready state
 any process that is ready to be executed

 The Blocked state

 when a process cannot execute until some event occurs (ex:
the completion of an I/O)
 Process States
 The New state
 OShas performed the necessary actions to
create the process
 has created a process identifier
 has created tables needed to manage the process
 buthas not yet committed to execute the
process (not yet admitted)
 because resources are limited
 The Exit state
 Termination moves the process to this state
 It is no longer eligible for execution
 Tables and other info are temporarily
preserved for auxiliary program
Five-State Process Model
Dispatch Release
Admit
New Ready Running Exit

Time-out

Event Event
Occurs Wait

Blocked
Single Blocked Queue
Ready Queue
Dispatch Release
Admit
Processor

Time-out

Event Wait

Event
Occurs
Blocked Queue
Multiple Blocked Queues
Ready Queue
Dispatch Release
Admit
Processor

Time-out

Event 1 Event 1 Wait

Occurs

Event 1 Queue

Event 2 Event 2 Wait

Occurs

Event 2 Queue
Suspended Processes

 Processor is faster than I/O so all processes

could be waiting for I/O
 Swap these processes to disk to free up more
memory
 Blocked state becomes suspend state when
swapped to disk
 Two new states
 Blocked, suspend
 Ready, suspend
 Process State Transition
Diagram with Two Suspend
ReadyStates

Admit Suspend
Admit

Activate Dispatch
Ready, Ready Running Exit
suspend
Suspend Time out

Event Event
Event
Occurs Wait
Occurs

Activate
Blocked, Blocked
suspend
 Processor State
Information
 Contents of processor registers
 User-visible registers
 Control and status registers
 Stack pointers
 Program status word (PSW)
 contains status information
Process Control
Information
 Additional information needed by the operating
system to control and coordinate the various
active processes
 scheduling and state information
 data structuring
 interprocess communication
 process privileges
 memory management
 resource ownership and utilization
Process Creation

 Assign a unique process identifier

 Allocate space for the process
 Initialize process control block
 Set up appropriate linkages
 Ex: add new process to linked list used for
scheduling queue
 Other
 maintain an accounting file
 When to Switch a
Process
 Interrupts
 Clock
 process has executed for the maximum allowable time
slice
 I/O
 Memory fault
 memory address is in virtual memory so it must
be brought into main memory
 Trap
 erroroccurred
 may cause process to be moved to Exit state
 Supervisor call
 such as file open
 Change of Process
State
 Save context of processor including
program counter and other registers
 Update the process control block with the
new state and any accounting information
 Move process control block to appropriate
queue - ready, blocked
 Select another process for execution
 Update the process control block of the
process selected
 Update memory-management data
structures
 Restore context of the selected process
 UNIX Process State
 Initial (idle)
 Ready to run
 Kernel/User running
 Zombie
 Asleep
 for 4BSD: stopped/suspend
Process states and state
transitions
Process Context
 User address space:
 code, data, stack, shared memory regions,
 Control information:
 u area, proc, kernel stack, ATM
 Credentials:
UID & GID
 Environment variables:
 inherited from the parent
 Hardware context(in PCB of u area):
 PC, SP, PSW, MMR, FPU
User Credentials

 Superuser: UID=0, GID=1

 Real IDs: log in, send signals
 Effective IDs: file creation and
access
 exec:
 suid mode: that of the owner;
 sgid mode: that of the calling process
 setuid / setgid:
 SV & BSD are different with these
saved UID, saved GID in SV
 setgroup in BSD
The u Area
 part of the process space, needed only when running.
 PCB
 proc pointer
 UID s
 Arguments, results, error status from system calls
 Signal handlers and related information
 Info from the program header, text, data, stack size, MM
info
 Open file descriptor
 Vnodes & controlling terminal pointers
 CPU usage statistics, profiling info, disk quotas, & resource
limits
 Per-process kernel stack
 The proc
 Process table
 Id
 Location of the kernel address map for the u area
 Current process state
 Forward and backward pointers in scheduled queue
 Sleep channel (7.2.3)
 Scheduling priority and related (Chapter 5)
 Signal handling info(Chapter 4).
 MM info
 Pointers to link active, free, zombie processes in lists
 Flags
 Pointers to keep structures on a hash queue by PID
 Hierarchy info
Process ID
A typical
pointer to
process hierarchy
Parent Process ID

in 4.3BSD UNIX
parent’s
proc
Pointer to the
younger sibling Pointer to the oldest child
 Typical Functions of an
Operating-System Kernel
 Process Management
 Process creation and termination
 Process scheduling and dispatching
 Process switching
 Process synchronization and support for
inter-process communication
 Management of process control blocks
 Memory Management
 Allocation of address space to processes
 Swapping
 Page and segment management
 Typical Functions of an

Operating-System
I/O Management
Kernel
 Buffer management
 Allocation of I/O channels and devices to processes
 Support Functions
 Interrupt handling
 Accounting
 Monitoring
Operating System
Control Structures
An OS maintains the following
tables for managing processes
and resources:
Memory tables
I/O tables
File tables
Process tables
Memory Tables

 Allocation of main memory to processes

 Allocation of secondary memory to processes
 Protection attributes for access to shared
memory regions
 Information needed to manage virtual memory
I/O Tables

 I/O device is available or assigned

 Status of I/O operation
 Location in main memory being used as the
source or destination of the I/O transfer
File Tables

 Existence of files
 Location on secondary memory
 Current Status
 Attributes
 Sometimes this information is maintained by a
file-management system
Process Table

 Process image consists of program, data, stack,

and attributes
 Attributes
 process control block
Process Control Block
Process Identification
 Unique numeric identifier
 may be an index into the primary process table
 User identifier
 who is responsible for the job
 Execution of the
Operating System
 Nonprocess Kernel
 execute kernel outside of any process
 operating system code is executed as a separate
entity that operates in privileged mode
 Execution Within User Processes
 operating system software within context of a
user process
 process executes in privileged mode when
executing operating system code
 Execution of the
Operating System
 Process-Based Operating System
 major kernel functions are separate processes
 a process is invoked by the operating system
The UNIX kernel
A special program that runs directly on
the hardware.
 Implements the process model and
services.
 Resides on disk, in a file /vmunix or /unix.
 Bootstrapping: loads the kernel.
 Initializes
the system and sets up the
environment, remains in memory before
shut down
UNIX Services
 System call interface
 Hardware exceptions
 Divide by 0, overflowing user stack
 Interrupts
 Devices
 Swapper, pagedaemon
The Kernel interacts with
processes and devices

BACK
 Mode,Space & Context

 By modes: some critical resources can be

protected.
 Kernel Mode: More privileged, kernel functions
 User Mode: Less privileged, user functions
 Virtual Memory
 VM space
 Address Translation Maps
 Memory Management Unit
Kernel data

 Current process & Context switch

 One instance of the kernel
 Global data structure
 Pre-process objects
 System call, Mode Switch
 User area: info. about a process
 Kernel stack:
Context
 UNIX kernel is re-entrant: several processes may be
involved in kernel activities concurrently.
 Execution context
 Process context:
 System context (Interrupt context):
Execution mode and
Context
 Executing in Kernel Mode
 3 types of events:
 Device interrupts
 Exceptions
 Traps or software interrupts
 Dispatch table
 System context: interrupts
 Process context: traps, exceptions & software
interrupts
The System Call Interface

 syscall(): the starting point

 in kernel mode, but in process context.
 Copy arguments , save hardware context on the
kernel stack.
 Use system call number to index dispatch
vector
 Return results in registers, restore hardware
context, to user mode, control back to the
library routine.
 UNIX Interrupt
Handling
 Interrupt handler(interrupt service routine):
 runs in kernel mode and system context,
 not permitted to block.
 the time used to service an interrupt charged to
the interrupted process
 The clock interrupt handler charges the clock
tick to the current process
 ipl(interrupt priority level)- specified for each
interrupt and saved in interrupt register of the
processor status word
 Interrupts are preemptive
Setting the interrupt
priority in 4.3BSD and
SVR4
Interrupt handling
 Synchronization
 UNIXis re-
entrant
 UNIXis non-
preemptive,
keeping the
kernel states
consistent.
 Relinquish CPU
voluntarily.
Blocking Operations

 Blocks the process (make it sleep).

 Lock: a single bit flag
 wanted(flag):
 sleep():
 wake(): wake all the waiting processes.
 upon waking up: check once again to make
sure.
Algorithm for resource
locking
Blocking Interrupts

 Blocking interrupts while accessing

critical sections.
 int x = splbio();/* raise ipl*/
 modify disk buffer cache;
 splx(x); /*restore ipl*/
 Critical region:few & brief.
 Blocked interrupts may access the
critical region.
 Different interrupts may have the same
ipl
 Blocking an interrupt may block others.
UNIX Process Implementation
 New Processes &
fork:Programs
creates a new process.
returns 0 to the child, PID to the
parent
 exec:
 begins to execute a new program
Using fork & exec

 if ((result = fork()==0){
 /* child code*/
 … …
 if (execve(“new program”),…)<0)
 perror(“execve failed!”);
 } else if (result<0) {
 perror(“fork”);/*fork failed*/
}
 /*parent continures here*/
Shell creating a
process

A highly simplified shell

 The ls Command

Steps in executing the command ls typed to the shell

 Process Creation
 Almost an exact clone of the parent.
 Reserve swap space for the child
 Allocate a new PID and proc structure for the child
 Initialize proc structure
 Allocate address translation map (ATM)
 Allocate u area and copy
 Update the u area to refer to the new ATM & Swap space
 Add the child to the set of processes sharing the text region
of the program
 Duplicate the parent’s data and stack regions update ATM
to refer to these new pages.
 Acquire references to shared resources inherited by the
child
 Initialize the hardware context
 Make the child runnable and put it on a scheduler queue
 Arrange to return with 0
 Return the PID to the parent
fork Optimization

 It is wasteful to make an actual copy of the

address space of the parent
 Copy-on-write (SV) : only the pages that are
modified must be copied
 vfork (BSD): The parent loans the address space
and blocks until the child returns to it.
 dangerous!
 csh: exploits it.
Invoking a New Program

 Process address space

 Text: code
 Initialized data:
 Uninitialized data(bss):
 Shared memory(SV):
 Shared libraries:
 Heap: dynamic space
 User stack: space allocated by the kernel
exec System Call
 Parse & access
 Verify the permission
 Read the header and check if valid executable
 If the file has SUID or SGID bits set in its mode, change the
caller’s effective UID or GID to that of the owner
 Copy the arguments to exec and the env. variables into
kernel.
 Allocate swap space for the data and stack region
 Set up the new address space.
 Copy the arguments and env. variables back onto the new
user stack.
 Reset all signal handlers to default actions.
 Initialize the hardware context.
 Process Termination (exit())
 Turns off all signals.
 Closes all open files.
 Releases the text file and other resources such as the current
directory.
 Writes to the accounting log.
 Saves resource usage statistics and exit status in the proc
structure.
 Changes state to SZOMB, and puts the proc on the zombie
process list.
 Makes the init process inherit any live children of the exiting
process.
 Releases the address space, u area, ATM, and swap space.
 Notifies the parent by sending it a SIGCHLD signal.
 Wakes up the parent if it is asleep.
 Calls swtch() to schedule a new process to run.
 Awaiting Process
Termination
 wait(stat_loc);/* SV, BSD & POSIX*/
 wait3(statusp, options, rusagep);
/*BSD*/
 waitpid(pid, stat_loc, options);/*POSIX*/
 waitid(idtype, id, infop,
options);/*SVR4*/
Zombie Processes
 after exit(), process holds only proc
structure and is in zombie state until
cleaned completely.
 wait() (issued by the parent or the init
process)frees the proc structure and
completes process exit,
 If the child dies before the parent which
does not wait for it, the proc is not
released until the system is rebooted.
 Parent can specify that it will not wait
for its children
 LINUX

 Modular structure – collection of modules, some

of them loadable and unloadable on demand,
 Module – object file whose code be linked and
unlinked from the kernel at runtime, executed
in kernel mode on behalf of the current process
 Loadable module characteristic
– Dynamic linking,
-- Stackable
modules – may serve as libraries when they are
referenced by client modules higher up in the
hierarchy, and as clients when they reference
modules further down the hierarchy.
Linux components

 At user level – tasks (specific to Linux –

combine features of processes and
threads);
 At kernel level – interacting collection of
components;
 Hardware components – in this case IA-
64 Intel Itanium architecture
Linux Task Data
Structure
 State
 Scheduling information
 Identifiers
 Interprocess communication
 Links
 Times and timers
 File system
 Address space
 Processor-specific context
Linux States of a
Process/Thread
 Running
 Interruptable
 Uninterruptable
 Stopped
 Zombie