File System and management

File systems: What is a file, user view of files file types and file operations, file types in Unix/Linux and Microsoft file operation commands, file access rights, file storage management, Inode or FAT structure, file control blocks, root file system ,directory file paths, blocks , impact of block size selection Allocation Techniques -: contiguous allocation, chained and indexed allocations, , file related system services, disk access control and scheduling.
Notes Compiled by BJ

Role Of Operating System

The operating system has various roles: Management of the processor: Management of the random access memory Management of input/output:

Management of execution of applications:

Management of authorizations: File management: Information management

Notes Compiled by BJ

File systems: What is a file

A file is a collection of data stored in one unit, identified by a filename. It can be a document, picture, audio or video stream, data library, application, or other collection of data.

A file system is a method of storing and organizing computer files and their data. Essentially, it organizes these files into a database for the storage, organization, manipulation, and retrieval by the computer's operating system.
File systems are used on data storage devices such as a hard disks or CD-ROMs to maintain the physical location of the files. A named collection of related info Consists of asequence of bits, bytes, lines, or records

Notes Compiled by BJ

File Attributes

Name only information kept in human-readable form Identifier unique tag (number) identifies file within file system Type needed for systems that support different types Location pointer to file location on device Size current file size Protection controls who can do reading, writing, executing Time, date, and user identification data for protection, security, and usage monitoring Information about files are kept in the directory structure, which is maintained on the disk
Notes Compiled by BJ

File Operations

File is an abstract data type Create Write Read Reposition within file Delete Truncate Open(Fi) search the directory structure on disk for entry Fi, and move the content of entry to memory Close (Fi) move the content of entry Fi in memory to directory structure on disk
Notes Compiled by BJ

Open Files

Several pieces of data are needed to manage open files: File pointer: pointer to last read/write location, per process that has the file open File-open count: counter of number of times a file is open to allow removal of data from open-file table when last processes closes it Disk location of the file: cache of data access information Access rights: per-process access mode information

Notes Compiled by BJ

Linux/UNIX File Types

Regular file
Text or binary data (e.g. an image).

Directory
Contains other files and directories.

Device special file

An interface to a piece of hardware, such as a printer. You don't have to worry about this.

Notes Compiled by BJ

Microsoft File Types

.doc Microsoft Word File .xls Microsoft Excel File .exe Executable File - one that starts up a software application, for instance .txt A plain text file, can be read by Word, Notepad, Wordpad many others .bmp A full detail (ie large file size) picture.jpg A compressed (ie smaller file size) picture.wav A full detail (ie large file size) sound file .mp3 A compressed (ie smaller file size) sound file

Notes Compiled by BJ

File Types Name, Extension

Notes Compiled by BJ

Access Methods
Sequential Access Direct Access Indexed Access

Notes Compiled by BJ

Sequential Access
The simplest access method is sequential access. Information in the file is processed in order, one record after the other. This mode of access is by far the editors and compilers usually access files in this fashion. Reads and writes make up the bulk of the operations on a file. A read operationread nextreads the next portion of the file and automatically advances a file pointer, which tracks the I/O location. Similarly, the write operationwrite nextappends to the end of the file and advances to the end of the newly written material

Notes Compiled by BJ

Sequential-access File

Notes Compiled by BJ

Simulation of Sequential Access on a Direct-access File

Notes Compiled by BJ

Direct Access
Another method is direct access (or relative access). A file is made up of fixed length logical records that allow programs to read and write records rapidly in no particular order. The direct-access method is based on a disk model of a file, since disks allow random access to any file block. For direct access, the file is viewed as a numbered sequence of blocks or records. Thus, we may read block 14, then read block 53, and then write block 7. Ther

Notes Compiled by BJ

Access Methods

Sequential Access read next write next reset no read after last write (rewrite) Direct Access read n write n position to n read next write next

rewrite n Notes Compiled by BJ n = relative block number

Index and Relative Files

The index, like an index in the back of a book, contains pointers to the various blocks. To find a record in the file, we first search the index and then use the pointer to access the file directly and to find the desired record.

Notes Compiled by BJ

File operation Commands

Cat
Touch Vi

Chmod
Umask Rm rmdir

Notes Compiled by BJ

Root File System

Notes Compiled by BJ

Allocation Methods
An allocation method refers to how disk blocks are allocated for files: Contiguous allocation Linked allocation Indexed allocation

Notes Compiled by BJ

Contiguous Allocation
Contiguous allocation requires that each file occupy a set of contiguous blocks on the disk. Disk addresses define a linear ordering on the disk. With this ordering, assuming that only one job is accessing the disk, accessing block b + 1 after block b normally requires no head movement. When head movement is needed (from the last sector of one cylinder to the first sector of the next cylinder), the head need only move from one track to the next. Thus, the number of disk seeks required for accessing contiguously allocated files is minimal. The IBM VM/CMS operating system uses contiguous allocation because it provides such good performance. Contiguous allocation of a file is defined by the disk address and length (in block units) of the first block. If the file is n blocks long and starts at location b, then it occupies blocks b, b + 1, b + 2, ..., b + n 1. The directory entry for each file indicates the address of the starting block and the length of the area allocated for this file
Notes Compiled by BJ

Contiguous Allocation of Disk Space

Notes Compiled by BJ

DISADVANTAGES Contiguous allocation has some problems, One difficulty is finding space for a new file. Wasteful of space (dynamic storage-Allocation problem) Files cannot grow All these algorithms suffer from the problem of external fragmentation. As files are allocated and deleted, the free disk space is broken into little pieces. External fragmentation exists whenever free space is broken into chunks. It becomes a problem when the largest contiguous chunk is insufficient for a request;
Notes Compiled by BJ

Linked allocation
Linked allocation solves all problems of contiguous allocation. With linked allocation, each file is a linked list of disk blocks; the disk blocks may be scattered anywhere on the disk. The directory contains a pointer to the first and last blocks of the file. To create a new file, we simply create a new entry in the directory. With linked allocation, each directory entry has a pointer to the first disk block of the file. This pointer is initialized to nil (the end-of-list pointer value) to signify an empty file. The size field is also set to 0. A write to the file causes the free-space management system to find a free block, and this new block is written to and is linked to the end of the file. To read a file, we simply read blocks by following the pointers from block to block. So there is no external fragmentation
Notes Compiled by BJ

Linked Allocation

a file of five blocks might start at block 9 and continue at block 16, then block 1, then block 10, and finally block 25. Each block Notes Compiled contains a pointer to the next block. by BJ

Disadvantages
The major problem is that it can be used effectively only for sequential-access files. To find the ith block of a file, we must start at the beginning of that file and follow the pointers until we get to the ith block. Each access to a pointer requires a disk read, and some require a disk seek. Another disadvantage is the space required for the pointers. If a pointer requires 4 bytes out of a 512-byte block, then 0.78 percent of the disk is being used for pointers, rather than for information. another problem of linked allocation is reliability what would happen if a pointer were lost or damaged A bug in the operating-system software or a disk hardware failure might result in picking up the wrong pointer. This error could in turn result in linking into the free-space list or into another file.
Notes Compiled by BJ

Indexed allocation
Linked allocation solves the external-fragmentation and sizedeclaration problems of contiguous allocation but linked allocation cannot support efficient direct access, since the pointers to the blocks are scattered with the blocks themselves all over the disk and must be retrieved in order. Indexed allocation solves this problem by bringing all the pointers together into one location: the index block. Each file has its own index block, which is an array of disk-block addresses. The /"' entry in the index block points to the /"' block of the file. The directory contains the address of the index block To find and read the /th block, we use the pointer in the /"' index-block entry When the file is created, all pointers in the index block are set to nil. Each file has its own index block, which is an array of disk-block addresses.
Notes Compiled by BJ

Indexed Allocation

Brings all pointers together into the

index block.

Logical view.

index table

Notes Compiled by BJ

Indexed Allocation

Notes Compiled by BJ

Indexed allocation
Indexed allocation supports direct access, without suffering from external fragmentation, because any free block on the disk can satisfy a request for more space. Indexed allocation does suffer from wasted space, however. The pointer overhead of the index block is generally greater than the pointer overhead of linked allocation

Notes Compiled by BJ

Disadvantages
Indexed-allocation schemes suffer from some of the same performance problems as does linked allocation. Specifically, the index blocks can be cached in memory, but the data blocks may be spread all over a volume.

Notes Compiled by BJ

Access Rights

Mode of access: read, write, execute Three classes of users RWX a) owner access 7 111 RWX b) group access 6 110 RWX c) public access 1 001 Ask manager to create a group (unique name), say G, and add some users to the group. For a particular file (say game) or subdirectory, define an appropriate access.
owner group public

chmod

761

game

Attach a group to a file chgrp G game

Notes Compiled by BJ

Directory Structure
A collection of nodes containing information about all files.

Directory

Files

F1

F2

F3

F4 Fn

Both the directory structure and the files reside on disk. Backups of these two structures are kept on tapes.
Notes Compiled by BJ

file paths
Absolute or relative path name Creating a new file is done in current directory

Notes Compiled by BJ

Information in a Device Directory

Name Type Address Current length Maximum length Date last accessed (for archival) Date last updated (for dump) Owner ID (who pays) Protection information (discuss later)

Notes Compiled by BJ

Operations Performed on Directory

Search for a file Create a file Delete a file List a directory Rename a file Traverse the file system

Notes Compiled by BJ

Organize the Directory (Logically) to Obtain

Efficiency locating a file quickly. Naming convenient to users.
Two users can have same name for different files. The same file can have several different names.

Grouping logical grouping of files by properties, (e.g., all Java programs, all games, )

Notes Compiled by BJ

Single-Level Directory
A single directory for all users.

Naming problem
Grouping problem

Notes Compiled by BJ

Two-Level Directory
Separate directory for each user.

Path name Can have the same file name for different user Efficient searching No grouping capability
Notes Compiled by BJ

Tree-Structured Directories

Efficient searching Grouping Capability Notes Compiled by BJ

File Storage Management

OS need to maintain several piece of info that can assist management of files Like when file was last updated and whom Unix have audit trail which maintain record of who has accessed when and did what audit trail useful from recovering from crash also detect unauthorized access

Notes Compiled by BJ

A Typical File-system Organization

Notes Compiled by BJ

Storage Structure
Main memory only large storage media that the CPU can access directly Secondary storage extension of main memory that provides large nonvolatile storage capacity Magnetic disks rigid metal or glass platters covered with magnetic recording material Disk surface is logically divided into tracks, which are subdivided into sectors The disk controller determines the logical interaction between the device and the computer

Notes Compiled by BJ

Storage Hierarchy
Storage systems organized in hierarchy
Speed Cost Volatility

Caching copying information into faster storage system; main memory can be viewed as a ast cache for secondary storage

Notes Compiled by BJ

Storage pyramid
Capacity < 1 KB 1 MB 256 MB Registers Cache (SRAM) Main memory (DRAM) Access latency 1 ns 25 ns 50 ns
Better

40 GB
Better

Magnetic disk
Magnetic tape

5 ms
50 sec

> 1 TB

Goal: really large memory with very low latency

Latencies are smaller at the top of the hierarchy Capacities are larger at the bottom of the hierarchy

Solution: move data between levels to create illusion of large memory with low latency
Notes Compiled by BJ

Overview of Storage Structure

Magnetic disks provide bulk of secondary storage of modern computers Drives rotate at 60 to 200 times per second Transfer rate is rate at which data flow between drive and computer Positioning time (random-access time) is time to move disk arm to desired cylinder (seek time) and time for desired sector to rotate under the disk head (rotational latency) Head crash results from disk head making contact with the disk surface

Thats bad
Disks can be removable Drive attached to computer via I/O bus Busses vary, including EIDE, ATA, SATA, USB, Fibre Channel, SCSI Host controller in computer uses bus to talk to disk controller built into drive or storage array

Notes Compiled by BJ

Disk drive structure

Data stored on surfaces
Up to two surfaces per platter One or more platters per disk
head
sector

Data in concentric tracks

Tracks broken into sectors
256B-1KB per sector

platter

track
cylinder

Cylinder: corresponding tracks on all surfaces

Data read and written by heads

Actuator moves heads Heads move in unison

surfaces

spindle
Notes Compiled by BJ

actuator

Moving-head Disk Machanism

Notes Compiled by BJ

 To read a given sector we first move the heads to that sector's cylinder (seek time), then wait for the sector to rotate under the head (latency time), then copy data off of disk into memory (transfer time).

Notes Compiled by BJ

Notes Compiled by BJ

Booting from a Disk

Notes Compiled by BJ

Index Node

Notes Compiled by BJ

Indexed Allocation Mapping (Cont.)

outer-index

index table

file

Notes Compiled by BJ

INODE
The inode pointer structure is a structure adopted by the inode of a file in the Unix File System (UFS) or other related file systems to list the addresses of a file's data blocks. In the past, the structure may have consisted of eleven or thirteen pointers, but most modern file systems use fifteen pointers. These pointers consist of (assuming 12 pointers in the inode) Twelve pointers that directly point to blocks of the file's data (direct pointers) One singly indirect pointer (a pointer that points to a block of pointers that then point to blocks of the file's data) One doubly indirect pointer (a pointer that points to a block of pointers that point to other blocks of pointers that then point to blocks of the file's data) One triply indirect pointer (a pointer that points to a block of pointers that point to other blocks of pointers that point to other blocks of pointers that then point to blocks of the file's data) The structure allows for inodes to describe very large files in a file systems with a fixed logical block size

Notes Compiled by BJ

Notes Compiled by BJ

INODE
The inode pointer structure not only allows for files to easily be allocated to non-contiguous blocks, it also allows the data at a particular location inside a file to be easily located. This is possible because the logical block size is fixed. For example, if each block is 8 kB, file data at 120 to 128 kB would be pointed to by the fourth pointer of the first indirect block The number of pointers in the indirect blocks are dependent on the block size and size of block pointers. Example: with a 512 byte block size, and 4 byte block pointers, each indirect block can consist of 128 (512 / 4) pointers

Notes Compiled by BJ

File Control Block

In Microsoft environment counterpart of inode is File Control Block It stores Filename, Location of secondary Storage, length of file in bytes, date and time of creation, last access etc. Advantage of fcb is that it store file noting which application created it Uses extension name like .txt ,.doc to identify how file was created Uses Simple cluster to store files
Notes Compiled by BJ

File Control Block

Notes Compiled by BJ

File-Allocation Table

Notes Compiled by BJ

Disk Scheduling Policies

First-in, first-out (FIFO)
Process request sequentially Fair to all processes Approaches random scheduling in performance if there are many processes

Notes Compiled by BJ

we assume that the disk head is initially located at track 100. we assume a disk with 200 tracks and that the disk request queue has random requests in it. The requested tracks, in the order received by the disk scheduler, are 55, 58, 39, 18, 90, 160, 150, 38, 184.

Notes Compiled by BJ

Disk Scheduling Policies

Shortest Service Time First
Select the disk I/O request that requires the least movement of the disk arm from its current position Always choose the minimum seek time

Notes Compiled by BJ

we assume that the disk head is initially located at track 100. we assume a disk with 200 tracks and that the disk request queue has random requests in it. The requested tracks, in the order received by the disk scheduler, are 55, 58, 39, 18, 90, 160, 150, 38, 184.

Notes Compiled by BJ

Disk Scheduling Policies

SCAN
Arm moves in one direction only, satisfying all outstanding requests until it reaches the last track in that direction Direction is reversed

Notes Compiled by BJ

we assume that the disk head is initially located at track 100. we assume a disk with 200 tracks and that the disk request queue has random requests in it. The requested tracks, in the order received by the disk scheduler, are 55, 58, 39, 18, 90, 160, 150, 38, 184.

Notes Compiled by BJ

Disk Scheduling Policies

C-SCAN
Restricts scanning to one direction only When the last track has been visited in one direction, the arm is returned to the opposite end of the disk and the scan begins again

Notes Compiled by BJ

we assume that the disk head is initially located at track 100. we assume a disk with 200 tracks and that the disk request queue has random requests in it. The requested tracks, in the order received by the disk scheduler, are 55, 58, 39, 18, 90, 160, 150, 38, 184.

Notes Compiled by BJ