ETL - Data warehousing project life cycle

A typical ETL datawarehousing project in an MNC consists of following stages:

1. Requirement analysis 2. Warehouse design 3. Build the ETL system 4. Testing 5. Deployment 6. Maintenance

Article will elaborate each of the above stages of the project.

Requirement Analysis
So the first step in creating a data warehouse is to determine what information is needed most. What does management want to know about? What are the most important business questions? To obtain this information, you'll have to perform quite a few interviews

Project Design
This is the single most important phase of the ETL project. A good design would avoid lot of testing and re-working effort.

a. Analyze data sources b. Data modeling c. Capacity planning d. Define Metadata and review plan e. Product selection f. ETL design g. End user application design h. Technical architecture i. Migration plan

This phase starts with analysis of the source data. All source systems are analyzed and the

1/3

ETL - Data warehousing project life cycle

data model is designed to accommodate the data in target system. Plan is made for amount of implementation effort required and also to keep the metadata of the project and project review.

Lot of things goes into selecting the right product to perform the ETL task. Once selected, the ETL implementation framework and design will be accomplished. The steps will be followed with complete architecture design of the ETL process and the migration plans.

Build ETL system

This step is all about implementing the project after the successful completion of the design. Depending on the nature of the project this step can be accomplished either in parallel with design phase or only after the completion of the project design.

a. Establish environment b. Establish processes c. Training d. Implement database e. Develop ETL f. Develop migration scripts g. Review

The whole environment(servers, softwares etc) required for development will be established. Proper project processes will be established and published. Developers will be given brief training of the requirement and more depending on the maturity of the candidate.

Then the data base will be created as per the design and followed by the ETL package creation. The step completes with creation of migration scripts and review.

Testing
Testing typically takes 25% of the development time. The phase starts with preparation of test plans. Then individual system testing will be done, followed by migration scripts testing and performance testing. Finally UAT will be done with the client users.

2/3

ETL - Data warehousing project life cycle

a. Plan b. System testing c. Test migration process d. Performance testing e. User Acceptance testing

Deployment
Deploment phase will take care of the project deployment plans and subsequent project deployment in client environment.

a. Plan b. Deployment

Sustain
This is nothing but the ongoing maintenance activities.

3/3