Vous êtes sur la page 1sur 5

ETL versus ELT Page 1 of 5

..

Log In Create an Account

Blogs Discussions Research Directory

Toolbox for IT Topics Data Warehouse Blogs

ETL versus ELT


Remediator Aug 26, 2008 | Comments (10)

Tweet 1 Recommend 3 Share 0 0

Many of you ask me questions on the primary differences between ELT and ETL - and
to clarify, it's really as simple as Extract-Transform-Load and Extract-Load-Transform.
The primary difference is, where do we do the work, really?

ELT is a domain for a machine that can pull it off. A typical SMP-based RDBMS is Ab Initio Underground
by Remediator
NOT an ETL platform, no matter how much we wish it so.
After spending a number of years with Ab Initio Software
Corporation, this practice leader with a leading
What's that? You're not doing any ELT on your RDBMS? Really? If you're doing even consulting firm now ... more
one on-board stored procedure that processes a batch of data, you're doing ELT.
Now, you may not be doing the "EL" in the ELT, but you're doing the "T", and this is Receive the latest blog posts:

just a formality, because it's where we do the "T" that counts the most, and what's Your email address FOLLOW
involved in that "T", because the "EL" is a nit either way.

Here's an example: In an ETL model, we will perform both row-based and set-based Share Your Perspective
operations in the "T" section, and then load the load-ready AND consumption-ready Share your professional knowledge and experience
data into the target. We don't intend to do any further set-based processing once the with peers. Start a blog on Toolbox for IT today!

data is loaded. BEGIN NOW

In an ELT model, we will do row-level transformation prior to the load, and then load it
in non-integrated form to staging tables. We will then execute the set-based, key-
based integration, you know, the heavy-lifting joins, sorts and rollups.

And many of you recognize this as the model you are trying to escape, if the database
in your environment is an SMP-based RDBMS, and your processing environment is
Ab Initio. You won't need the ELT model, because Ab Initio will do ETL much better
here.

But if we have a platform that can do on-board ELT, that is the large-scale set-based
transforms on board the machine, we have another option, another arrow in our
quiver, so to speak. But don't imagine that your SMP-based RDBMS is the place for it,
because it's not.

Only a handful of platforms out there can do this kind of large-scale integration on-
Categories
board the machine. One is Netezza and one is Teradata. We could possibly include
some others in the mix, but if they are SMP-based, they're not in the running, and Opinion GO
that's just the way it is.

Can you leverage an ELT model? Even if you have Ab Initio, and some of you have
Netezza also, you might find the opportunity to rebalance some of the work, especially
if you find yourself pulling data from Netezza, processing it, and putting it back. It
would be worthwhile to discover how this might be pulled off without the round-trip,
and do all the work in-the-box. Don't get wrapped around the axle on it, though. Ab
Initio has the juice to pull it off, so it's just a matter of optimizing our physical assets

http://it.toolbox.com/blogs/abinitio-underground/etl-versus-elt-26874 29-03-2015
ETL versus ELT Page 2 of 5

(like network bandwidth) more than which product or domain is the right one.

If you don't have Ab Initio and you have a Netezza machine, and you're only using the
machine for simple query acceleration, you have an opportunity before you. To get the
data into the machine in non-integrated form, and then integrate it inside-the-box.

Do a bake-off on the performance if you must, but apart from performance, the data
configuration and overall transport, and the total-points-of-failure have to be
considered as well. If we need to pull data out, process it and put it back, our total
points of failure are much higher than if we do the ETL inside the box.

Read 10 comments

Popular White Paper On This Topic


Gain an Advantage in the Internet of Things
More White Papers

10 Comments

Saugat Dey Sep 3, 2008

Can you please give any case-study ?


Again can you give any tool specific example please?>

Remediator Sep 3, 2008

Tool specific in what sense? The article mentions several technologies that are the
most viable ones for the ETL/ELT context, and calls out the RDBMS as unsuitable for
ELT of any scale.

Please clarify your question

Saugat Dey Sep 7, 2008

I mean doing some comparison of ETL vs ELT using Abinitio, Datastage etc , some
case study kind of examples

Jan 15, 2009

Is there any architecture that describes ELT?

Remediator Jan 17, 2009

An architecture that describes it is easier to find that a hardware platform that will
actually support it. The available hardware directly drives the architecture, and there
are some things in ELT that are not available with standard architectures, and not
advisable at all within the SMP/RDBMS-based systems.

undertow Jan 17, 2009

Remediator,
I been working in abinitio in a env whre we only use serial environemnt . I know we
are not using the tool to the fullest but Lot of ppl are just happy with it.
Anyway i wanted to make sure I truely understand difference between Degree of
parallelism and Types of paralellism and actually get a visual Idea of how the
processes are run internally in parallel Env.

File1 ? Partition by {key RR-Expr} ? compont2


? compont 3

For 20000 rec of 2 distinct key sets i.e 2 sets of data, 1 which go to comp2 and other

http://it.toolbox.com/blogs/abinitio-underground/etl-versus-elt-26874 29-03-2015
ETL versus ELT Page 3 of 5

to comp 3.

If degree of parallelism is 4

4 copies of partition and its following components run and each copy processes
The copy looks like

File1 --> partition by key--> compont2 --> copy 1 processing 10000/4 Records
File1 --> partition by key --> compont2 --copy 2 processing 10000/4 Records
File1 --> partition by key --> compont2 -- copy 3 processing 10000/4 Records
File1 --> partition by key --> compont2 -- copy 4 processing 10000/4 Records

so each component following partition process 10,000/4 recs Vs 20,000 Without


partition and 10,000 without parallelism.
-Right?

USER_1872273 Jan 24, 2010

This discussion about ETL or ELT is the most stupid thing ever... it is like re-labeling
what already exist without creating a new technique...

It is like discussing if the baby should be called John or Paul... despite of the name...
the baby is still the same...

Despite of calling ETL or ELT... we will keep doing the same... dumping data in the
stage area and promote the transformations while loading the data warehouse...

mj62mj62 Apr 9, 2010

I'm looking at balancing more of the load on our database as compared to Ab Initio.

We generate some reports (report tables) that can be done in the DB quite quickly.

We could generate these tables using a big block of SQL and then just use Ab Initio
to pipe the resulting table to another DB where it is required. (Our AI box is already
very busy during this window)

Do you know of any practical standards around using input table for this? Some of
the big reports have lots of columns and the rules for creating them are easier to see
and understand in Ab Initio vs in the SQL. But breaking up the SQL into smaller
chunks seems less efficient as well...

Thank You

aban9 DW Jun 8, 2010

I have seen various datawarehousing where Ab Intio is used as ETL and Teradata is
used as DSS system.
We use both approaches- ETL and ELT to do balance the load in the system as
overall basis.
The basic rule we follow is -
Do ETL in case of row based processing on large dataset and do ETL in cases of set
based operation if the data distribution key is different from aggrgation/integration
key.
And we do ELT when set based operation key is same as data distribution key in
database.In this case we use teradata parallelism without having all-amp joins.so we
are saving network bandwidth and running the process faster and balancing data
processing load in the environment effectively.

USER_1872273 Nov 6, 2010

http://it.toolbox.com/blogs/abinitio-underground/etl-versus-elt-26874 29-03-2015
ETL versus ELT Page 4 of 5

I have not idea how ETL and ELT became technology dependent in your
explanation... it doesn't make any sense to me... those are architecture decisions that
are not limited by technology... despite of the fact that some technologies are more
suitable for ELT than others... but in the end it is about the architecture you pick...
Cheers!

Leave a Comment


Connect to this blog to be notified of new entries.

Name Your email address PREVIEW SUBMIT

You are not logged in.


Sign In to post unmoderated comments.
Join the community to create your free profile today.

Want to read more from Remediator? Check out the blog archive.

Archive Category: Opinion


Keyword Tags: ab initio abinitio ETL ELT Teradata Netezza

Disclaimer: Blog contents express the viewpoints of their independent authors and are not reviewed for
correctness or accuracy by Toolbox for IT. Any opinions, comments, solutions or other commentary
expressed by blog authors are not endorsed or recommended by Toolbox for IT or any vendor. If you feel a
blog entry is inappropriate, click here to notify Toolbox for IT.

Browse all IT Blogs

From Around The Web

Iron.io Blog: How Online Meetings: Lyme Disease: Why The Antioxidant Thats
Edeva Uses MongoDB Evolving to Meet the Antibiotics Fail! 6,000x More Powerful
and IronMQ to Power Needs of the Remote (Envita) Than Vitamin C
Real-time Intelligent Workforce (Live in the Now)
Traffic Systems (Infinite Conferencing)
(Iron.io Blog)

Sleeping Beauty? The Ultimate Web Using Arduino IDE for What did the TV series
Lack of Sleep Causes Personalization Windows with the Intel FRIENDS teach us
Facial Wrinkles Planning Guide IoT Developer Kit about Content
(JuveRest Blog - The (info.evergage.com) (Intel) Marketing?
Sleep Wrinkle Pillow) (Outbrain)

Recommended by

We Recommend From Around The Web


Password-protect Your Outlook Mailbox Voices: In midst of war, life goes on for
and Recover a Forgotten Password Syrian kids (USA Today)

http://it.toolbox.com/blogs/abinitio-underground/etl-versus-elt-26874 29-03-2015
ETL versus ELT Page 5 of 5

Lists of Assumptions & Constraints with Patience is a virtue when it comes to


a Sample investing (Investment - Focus : Smart
Insight from Alliance Trust)
Integrating Quickbooks with asp.NET,
SQL Server, and Microsoft Access What 5G Will Mean for Our Wireless
World (iQ by Intel)
3 Reasons Hackers Want Your CRM Data
The Economist Who Predicted the
4 Essential Things to Know About Web Housing Crisis Sees Risk Ahead for
Conferencing vs. Telepresence China (The Financialist)

Some Tips On How To Disable Startup A Hungry Planet: What would it take to
Programs in Windows 8 feed 9 billion people? (Solazyme)

Recommended by

Toolbox for IT Topics on Toolbox for IT Toolbox.com

My Home Data Center Enterprise Architecture & EAI Networking & Infrastructure About
Topics Data Center Enterprise Architecture & EAI Hardware News
People Networking Privacy
Companies Development Information Management Communications Technology Terms of Use
Jobs C Languages Business Intelligence Work at Toolbox.com
White Paper Library Java Database Operating Systems Advertise
Visual Basic Data Warehouse Linux Contact us
Collaboration Tools Web Design & Development Knowledge Management UNIX Provide Feedback
Discussion Groups Oracle Windows
Blogs Help Topics
Wiki Enterprise Applications Technical Support
CRM IT Management & Strategy Security PCMag Digital Group
Follow Toolbox.com ERP Emerging Technology & Trends Security
AdChoices
Toolbox for IT on PeopleSoft IT Management & Strategy
Twitter SAP Project & Portfolio Management Storage Other Communities
Toolbox.com on Twitter SCM Storage
Toolbox.com on Siebel Cloud Computing Toolbox for HR
Facebook Cloud Computing Toolbox for Finance

Copyright 1998-2015 Ziff Davis, LLC (Toolbox.com). All rights reserved. All product names are trademarks of their respective companies. Toolbox.com is not
affiliated with or endorsed by any company listed at this site.

http://it.toolbox.com/blogs/abinitio-underground/etl-versus-elt-26874 29-03-2015