Data Storage Management and Retrieval

Introduction

Why Information Storage?

Digital universe The Information Explosion

21st Century is information era Information is eing created at ever increasing rate Information has ecome critical for success

!e live in an on"command# on"demand $orld

Example% &ocial net$or'ing sites# e"mails# video and photo sharing $e site# online shopping# search engines etc

Information management is a ig challenge

(rgani)ation see' to Store information

Protect

Optimize 2

Leverage the

What is Data?
Collection of raw facts from which conclusions may e drawn!

Data is converted into more convenient form i*e* Digital Data

Video

Photo

Increase in data processing capa ilities +o$er cost of digital storage ,fforda le and faster communication technology Individuals .usinesses

Book

01010101010 10101011010 00010101011 01010101010 10101010101 01010101010

Digital Data

Letter

!ho creates data

Categories of Data

Data can e categori)ed as either structured or unstructured data

Images !orms

"-Mail #ttachments X-Ra s

PD!s

Check

,nstructured (-0*+
Instant Messages Documents

Manuals

(ver /01 of enterprise information is unstructured

Contracts

$e% Pages Rich Media In&oices #udio Video

'tructured ()0*+
Rows and Columns

"

Define Information
!hat do individuals2 usinesses do $ith the data they collect They turn it into information Information is the intelligence and knowledge derived from data

Centrali/ed in.ormation storage and 0rocessing 1etwork 1etwork

$ired

$ireless

$ireless

$ired

,0loading in.ormation

#ccessing in.ormation

.usinesses analy)e ra$ data in order to identify meaningful trends 3or example% .uying ha its and patterns of customers 4ealth history of patients

Creators o. in.ormation

,sers o. In.ormation

Demand .or more In.ormation

$irtuous cycle of information

#

$alue of Information to a &usiness

Identifying ne$ usiness opportunities

.uying2spending patterns

Internet stores# retail stores# supermar'ets Trac'ing shipments# and deliveries

Customer satisfaction2service

Identifying patterns that lead to changes in existing usiness

5educed cost

6ust"in"time inventory# eliminating over"stoc'ing of products# optimi)ing shipment and delivery &ecurity alerts for stolen credit card purchases Communicate to an' customers $ith high account alances a out a special savings plan %

7e$ services

Targeted mar'eting campaigns

Creating a competitive advantage

()am*le of an +rder ,rocessing System

'er&er2 3' Client

L#1

!C '#1

#00lication ,ser Inter.ace

DBM' 'torage #rra

'

Challenges in Managing Data and Information

"40loding digital uni&erse

8ultifold

increase of information gro$th

Increasing de0endenc on in.ormation

The

strategic use of information plays an important role in determining the success of a usiness that is valua le today may ecome less important tomorro$*
-

Changing &alue o. in.ormation

Information

Information /ifecycle Management

Protect

1ew order Value

Process order

Deli&er order

$arrant claim

5ime

!ul.illed order

#ged data

$arrant Voided

Create

#ccess

Migrate

#rchi&e

Dis0ose

A proactive strategy that enables an IT organization to effectively manage the data throughout its lifecycle
.

Information /ifecycle Management ,rocess

Policy-based Alignment of Storage Infrastructure with Data Value

Classi. data 2 a00lications %ased on %usiness rules

Im0lement 0olicies with in.ormation management tools

#,53M#5" D

Integrated managemen t o. storage en&ironmen t

3rgani/e storage resources to align with data classes

!L"XIBL "
01

&enefits of Im*lementing I/M

Improved utili)ation

Tiered storage platforms 9rocesses# tools and automation

&implified management

&implified ac'up and recovery , $ider range of options to alance the need for usiness continuity 8aintaining compliance

:no$ledge of $hat data needs to e protected for $hat length of time .y aligning the infrastructure and management costs $ith information value 00

+o$er Total Cost of ($nership

Storage

Data created y individuals2 usinesses must e stored for further processing Type of storage used is ased on the type of data and the rate at $hich it is created and used Examples%

Individuals% Digital camera# Cell phone# D;D<s# 4ard dis' .usinesses% 4ard dis'# external dis' arrays# tape li rary Centrali)ed% mainframe computers Decentrali)ed% Client server model 02 Centrali)ed% &torage 7et$or'ing

&torage model% ,n evolution

Challenges in Data Storage Management

Availability Data Integrity Security

Manageability

Performance Scalability

Capacity

03

Storage 2echnology and 3rchitecture (volution

+,7

3C &,7

Multi Protocol Router

IP '#1 R#ID #rra '#1 2 1#'

6B3D Internal D#' Time

0"

Data Storage Infrastructure

4ost components
9hysical

and +ogical

Connectivity options
9CI#

IDE2,T,# &C&I optical and dis' drive

&torage options
Tape#

0#

,pplications runs on hosts 4osts can range from simple laptops to complex server clusters 9hysical components of host

4ost

La0to0

'er&er

C9= &torage

L#1

Dis' device and internal memory 4ost to host communications

7rou0 o. 'er&ers

I2( device

7et$or' Interface Card >7IC?

Main.rame

4ost to storage device communications

4ost .us ,dapter >4.,?

0%

4ost5 /ogical Com*onents

8ost #00lications 30erating ' stem DBMS !ile ' stem Volume Manager De&ice Dri&ers 8B# 8B# 8B#

0'

/ogical Com*onents of the 4ost

Application

Interface et$een user and the host Three"tiered architecture ,pplication =I# computing logic and underlying data ases ,pplication data access can e classifies as% Block-le&el access% Data stored and retrieved in loc's# specifying the +., !ile-le&el access% Data stored and retrieved y specifying the name and path of files 5esides et$een the applications and the hard$are 0Controls the environment

(perating system

/ogical Com*onents of the 4ost5 /$M

5esponsi le for creating and controlling host level logical storage

Logical 'torage

9hysical vie$ of storage is converted to a logical vie$ y mapping +ogical data loc's are mapped to physical data loc's
LVM

=sually offered as part of the operating system or as third party host soft$are +;8 Components%

9hysical ;olumes ;olume @roups +ogical ;olumes

Ph sical 'torage

0.

$olume 6rou*s

(ne or more 9hysical ;olumes form a ;olume @roup +;8 manages ;olume @roups as a single entity 9hysical ;olumes can e added and removed from a ;olume @roup as necessary 9hysical ;olumes are typically divided into contiguous eAual" si)ed dis' loc's , host $ill al$ays have at least one dis' group for the Ph (perating &ystem

Logical Volume
Logical Volume Logical Disk Block

sical Volume 1

Ph sical Volume )

Ph sical Volume 9 Ph sical Disk Block

,pplication and (perating &ystem data maintained in separate volume groups

Volume 7rou0

21

/$M ()am*le5 ,artitioning and Concatenation

'er&ers

Logical Volume

Ph sical Volume

Partitioning

Concatenation

20

/ogical Com*onents of the 4ost 7Cont8

Device Drivers

Ena les operating system to recogni)e the device 9rovides ,9I to access and control devices 4ard$are dependent and operating system specific 3ile is a collection of related records or data stored as a unit 3ile system is hierarchical structure of files Examples% 3,T B2# 7T3&# =7IC 3& and ECT22B
22

3ile &ystem

5eacher (,ser+

4ow 9iles are Moved to and from Storage

!ile ' stem Blocks Course !ile(s+ !ile ' stem !iles

1
Configures2 8anages

)
5eside in

9
8apped y a file system to

Disk Ph sical "4tents

LVM Logical "4tents

;
8anaged y dis' storage su system 8apped y +;8 to

:
5esiding in

23

Connectivity

Interconnection et$een hosts or et$een a host and any storage devices 9hysical Components of Connectivity are%
.us#

port and ca le

CP,

B,'

8B#

Ca%le

Disk Port

2"

Connectivity ,rotocol

9rotocol D a defined format for communication et$een sending and receiving devices

Tightly Connected ntitie!

Directly Attached ntitie!

"et#or$ Connected ntitie!

Tightly connected entities such as central processor to 5,8# or storage uffers to controllers >example 9CI? Directly attached entities connected at moderate distances such as host to storage >example IDE2,T,? 7et$or' connected entities such as net$or'ed 2#

,o*ular Connectivity +*tions5 ,CI

9CI is used for local us system $ithin a computer It is an interconnection et$een microprocessor and attached devices 4as 9lug and 9lay functionality 9CI is B22EF it Throughput is 1BB 8.2sec PCI E press

Enhanced version of 9CI us $ith higher throughput and cloc' speed

2%

,o*ular Connectivity +*tions5 ID(:323

Integrated Device Electronics >IDE? 2 ,dvanced Technology ,ttachment >,T,?

8ost popular interface used $ith modern hard dis's @ood performance at lo$ cost Inexpensive storage interconnect =sed for internal connectivity

&erial ,dvanced Technology ,ttachment >&,T,?

&erial version of the IDE 2,T, specification 4ot"plugga le Enhanced version of us provides upto E@ 2s 2' >revision B*0?

,o*ular Connectivity +*tions5 SCSI

9arallel &C&I >&mall computer system interface?

8ost popular hard dis' interface for servers 4igher cost than IDE2,T, &upports multiple simultaneous data access =sed primarily in higher end environments &C&I =ltra provides data transfer speeds of B20 8.2s &upports data transfer rate of B @ 2s >&,& B00?
2-

&erial &C&I

Storage5 Medias and +*tions

8agnetic Tape

+o$ cost solution for long term data storage +imitations

&eAuential data access# &ingle application access at a time# 9hysical $ear and tear and &torage2retrieval overheads

(ptical Dis's

9opularly used as distri ution medium in small# single"user computing environments !rite once and read many >!(58?% CD"5(8# D;D" 5(8 +imited in capacity and speed 8ost popular storage medium $ith large storage capacity 5andom read2$rite access

Dis' Drive

2. application Ideal for performance intensive online