Académique Documents
Professionnel Documents
Culture Documents
1
SQL Loader Architecture
During processing, SQL*Loader writes messages to
the log file, bad rows to the bad file, and discarded
rows to the discard file.
Marina G. Erechtchoukova 2
Data
Positioned file
Each row starts from the most left position.
Each field takes the number of positions according to
its domain constraint
Files with delimiter
Each record is placed into one row
Each field is delimited by a symbol
3
Control file
Specifies the action:
Insert
Replace
Append
Specifies the data file name and list of fields
Has an extension “.ctl”
4
Control File for Positioned Data File
LOAD DATA
INFILE File_name
REPLACE INTO TABLE Table_name
(field_1 POSITION (SP:EP) data_type_1,
…,
field_K POSITION (SP:EP) data_type_K) )
5
Control File for Data File with
Delimiter
LOAD DATA
INFILE File_name
REPLACE INTO TABLE Table_name
FIELDS TERMINATED BY ‘,’ OPTIONALLY
ENCLOSED BY “’”
(field_1, field_2, …, field_K)
6
Control File Contains Data
Control file is prepared in the same way
INFILE *
BEGINDATA clause is added
Below the clause data are placed according to the
format declared in the control information
7
Using the SQL*LOADER
Create table
Prepare Control file and data file or place data into
control file
Invoke the utility without arguments to get a list
of available parameters .
ex: sqlldr username@server/password
control=loader.ctl sqlldr
username/password@server
control=loader.ct
8
The loader.ctl loads an external data file con
delimiter data.
Can also work with tabulation.
Ex: fields terminated by "\t" fields terminated by
X'09
9
How Does One Load MS-Excel Data
Into Oracle
Open the MS-Excel spreadsheet and save it as a CSV
(Comma Separated Values) file. This file can now be
copied to the Oracle machine and loaded using the
SQL*Loader utility.
There can arise some possible errors since the spread
sheet may contain new line characters ,where as SQL
Loader expects it to be in a single line.
To remove the new line character
(Tools -> Macro -> Visual Basic Editor)
10
Using the SQL*LOADER in ITEC Lab
Login onto sit.yorku.ca
Call SQL*LOADER utility
sqlldr login@studb10g Control_File
System asks for your password
Enter your password
11
If You Get an Error…
Control file has an extension other than “.ctl”
You did not create the table first
You use different column names in the control file
and in the table you created
SQL*LOADER creates “.log” file
SQL*LOADER creates “.bad” file if an error occurs.
12
Example: Control File
LOAD DATA
INFILE 'site.dat'
REPLACE INTO TABLE Site
FIELDS TERMINATED By ',' OPTIONALLY
ENCLOSED BY "'"
(site_id, location)
13
Example: Data File
1, 'Paris'
2, 'Boston'
3, 'London‘
4, ‘Ottawa’
5, ‘Toronto’
14
Example: Data in the Control File
LOAD DATA
INFILE *
REPLACE INTO TABLE building3
FIELDS TERMINATED BY '|'
( B_NAME, HF, DID, B_SIZE)
BEGINDATA
green|y|d3|20
red|y|d1|18
blue|n|d2|16
15
Methods of downoading data to
a flat file
This can be done using SQL*PLUS or Pl/Sql
Ex below using Sql*PLUS & PL/Sql respectively:
set echo off newpage 0 space 0 pagesize 0 feed off head
off trimspool on spool oradata.txt select col1 || ',' || col2 ||
',' || col3 from tab1 where col2 = 'XYZ'; spool off.
declare fp utl_file.file_type; begin fp :=
utl_file.fopen('c:\oradata','tab1.txt','w'); utl_file.putf(fp,
'%s, %sn', 'TextField', 55); utl_file.fclose(fp); end;
16
Data Can be modified as its been loaded.
Column can also be populated with derived and static
values.
Data can also be loaded from multiple files into multiple
table all at once
Ex :LOAD DATA INFILE file1.dat INFILE file2.dat INFILE file3.dat
APPEND INTO TABLE emp ( empno POSITION(1:4) INTEGER
EXTERNAL, ename POSITION(6:15) CHAR, deptno POSITION(17:18)
CHAR, mgr POSITION(20:23) INTEGER EXTERNAL
)
Ex: loading into multiple tables
LOAD DATA INFILE * INTO TABLE tab1 WHEN tab = 'tab1' ( tab
17
FILLER columns are used to skip columns/fields in
the load file, ignoring fields that one does not want.
One can create one logical record from multiple
physical records using one of the following two
clauses:
CONCATENATE - use when SQL*Loader should
combine the same number of physical records
together to form one logical record.
CONTINUEIF - use if a condition indicates that
multiple records should be treated as one. Eg. by
having a '#' character in column 1.
18
Methods of Improving the
performance of a sql loader
A very simple but easily overlooked hint is not to
have any indexes and/or constraints (primary key) on
your load tables during the load process. This will
significantly slow down load times even with ROWS=
set to a high value.
Add the following option in the command line:
DIRECT=TRUE. This will effectively bypass most of
the RDBMS processing.
19
Contd…
However, there are cases when you can't use direct
load. For details, refer to the FAQ about the
differences between the conventional and direct path
loader below.
Turn off database logging by specifying the
UNRECOVERABLE option. This option can only be
used with direct data loads.
Run multiple load jobs concurrently.
20
21