RAINFALL ANALYSIS

IMPLEMENTING ON DATA WAREHOUSE

MAJOR PROJECT REPORT SUBMITTED
IN
PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE DEGREE OF
MASTER OF SCIENCE
IN
COMPUTER SCIENCE

Submitted By:
Jitendra

Guided By:
Dr.Neeraj Bhargava
Head Dept. of Computer
Science.

DEPARTMENT OF COMPUTER SCIENCE

MAHRSHI DAYANAND SARSWATI UNIVERSITY
AJMER (RAJASTHAN)
2009

Acknowledgement
We are unable to find words to express our feelings of gratitude to the
coordinator Department of Computer Application for his keen interest
and exuberant guidance during the completion of project work..
First And foremost we express our deep sense of gratitude of our project
guide Dr. Neeraj Bhargava who initiated us in to this project and
constantly pre shaded us to complete the work he has been a source of
inspiration without whose help, Encouragement Corporation and
endeavor the project would not have got the present shape. In
conceiving and completing our work we have got immense guidance,
practical help and academic support from our teachers, Mr. Vincent
Philips our indebtedness to them is so formal and adequate in
representing the depth of our feelings of gratitude to them. The
completion of this project was a formidable task and we take the
opportunity of thanking our teachers, Classmates, friends and family
who guided helped and worked shoulder to shoulder with us.We once
again thank them because without their co operation we shall not be able
to complete this project.
Again million thanks are due to Head of Department Dr. Neeraj
Bhargava department of computer science provided their valuable
suggestion &guidance.

Department of Computer Application

MAHARSHI DAYANANAD SARASWATI UNIVERSITY

AJMER -305009

Certificate
This is to certify that the Project work entitled Rainfall Analysis
Implementing on DataWarehouse being submitted in partial
fulfillment of the requirement by Jitendra Kapoor for the award of
degree of M.C.A (Lateral Entry), by MAHARSHI DAYANAND
SARASWATI UNIVERSITY, Ajmer is a bonafide record of work
carried out under my supervision.
To the best of my knowledge this project report has been
submitted to any other institution of University for the award of any
other degree or diploma.
Date of Submission:
August

Guided By:
Dr.Neeraj Bhargava
Head
Dept. of Computer Science

ABSTRACT
Data warehousing and electronic commerce are two of the most rapidly expanding
fields in recent information technologies. In this Thesis, we discuss the design of data
warehouses for e-commerce environments. We discuss requirement analysis, logical
design, and aggregation in e-commerce environments. We have collected an
extensive set of interesting OLAP queries for e-commerce environments, and
classified them into categories. Based on these OLAP queries, we illustrate our
design with data warehouse bus architecture, dimension table structures, a base star
schema, and an aggregation star schema. We finally present various physical design
considerations for implementing the dimensional models. We believe that our
collection of OLAP queries and dimensional models would be very useful in
developing any real-world data warehouses in e-commerce environments.

INDEX
List
figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
List of tables

CHAPTER
1
Introduction
. . . . . . . . . . . . . . . . . .1

to

Data

of

Warehousing

What Is a Data Warehouse?. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .1
Retrieves
Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Consolidates Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.....7
Normalized
Data
Store . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 11
Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 13
Business
Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Other
Analytical
Activities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Updated in Batches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 18
Other Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .20
Data Warehousing Today . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 21
5

Business
Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Customer
Relationship
Management . . . . . . . . . . . . . . . . . . . . . . . .22
Data
Mining. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
Master
Data
Management
(MDM). . . . . . . . . . . . . . . . . . . . . . . . . . 25
Customer Data Integration . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .30
Future
Trends
in
Data
Warehousing. . . . . . . . . . . . . . . . . . . . . . . . 24
Unstructured
Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . .33
Service-Oriented Architecture (SOA) . . . . . . . . . . . . . . . . . . . .
. . . .34
Real-Time Data Warehouse . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . .35
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .36
CHAPTER
2
Data
. . . . . . . . . . . . . . . . . . . . 37

Warehouse

Architecture

Data
Flow
Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Single
DDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
NDS + DDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 46
ODS + DDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 51
Federated Data Warehouse . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . .53

System
Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
Case
Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.60
Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 64
CHAPTER
3
Data
Methodology. . . . . . . . .65

Warehouse

Development

Waterfall
Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Iterative Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .71
Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 77
CHAPTER 4 Functional and Nonfunctional Requirements . .
. . . . . .79
Identifying Business Areas. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 80
Understanding Business Operations . . . . . . . . . . . . . . . . . . . .
. . . . 81
Defining
Functional
Requirements . . . . . . . . . . . . . . . . . . . . . . . . . .82
Defining Nonfunctional Requirements . . . . . . . . . . . . . . . . . . .
. . . . 85
Conducting a Data Feasibility Study. . . . . . . . . . . . . . . . . . . . .
. . . . 88
Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .92
CHAPTER
5
Modeling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Data

Designing the Dimensional Data Store . . . . . . . . . . . . . . . . . .

. . . . 94
Dimension
Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101
Date
Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
102
Slowly
Changing
Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .107
Product,
Customer,
and
Store
Dimensions . . . . . . . . . . . . . . . . . . 111
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 118
1.
2.
3.
4.

PHP
MYSQL
WEKA
IV. XAMPP

List of figures :
Figure 1-1. A diagram of a data warehouse system 2
Figure 1-2. A cube with three dimensions 4
Figure 1-3. Simplest form of a data warehouse system 5
Figure 1-4. Star schema dimensional data store 8
Figure 1-5. Normalized data store 10
Figure 1-6. A data warehouse system that uses an enterprise data
warehouse 11
Figure 2-1. A data flow architecture with a stage, ODS, DDS, and MDB
40
8

Figure 2-2. A data flow architecture with control system, metadata ,and
data quality 42
Figure 2-3. Single DDS data warehouse architecture 44
Figure 2-4. Data architecture where the stage ETL and DDS ETL are
combined 45
Figure 2-5. NDS + DDS data flow architecture 47
Figure 2-6. ODS + DDS data flow architecture 51
Figure 2-7. A federated data warehouse from several data warehouses
54
Figure 2-8. Federated data warehouse using enterprise information
integration 55
Figure 2-9. A federated data warehouse from several data marts 56
Figure 2-10. Example of a system architecture for data warehouse 58
Figure 2-11. System architecture for the production environment for
Amadeus Entertainment data warehouse 63
Figure 3-1.Waterfall methodology 66
Figure 3-2.Waterfall methodology with infrastructure setup and project
management 67
Figure 3-4. Iterative cycles 73
Figure 5-1. Product Sales fact table and its dimensions 95
Figure 5-2. Product Sales table (and its dimensions) with timestamp
columns 99.
Figure 5-3.Moving a rapidly changing dimension attribute to the fact
table as a degenerate110

List of Tables:
4-1 Amadeus Entertainment data warehouse system functional
requirements .83
4-2 Amadeus Entertainment data warehouse system nonfunctional
requirements .87
4-3 Amadeus Entertainment data warehouse system risk .89
5-1 product sales fact table .100
5-2 Additional columns on product sales fact table .101
5-3 calendar date attributes. 104
5-4 fiscal calendar attributes columns .105
10

5-5 data dimension .106

5-6 storing historical information as rows. 108
5-7 storing historical information as columns . 108
5-8 A customer dimension .115
5-9 Store dimension 117.

11

12