Académique Documents
Professionnel Documents
Culture Documents
-----------------------------------------------------------
-----------------------------------------------------------
Table of Content
_
Introduction
We wanted to populate Siebel Database already existed with a lot of data to create
a scenario in order to test an archive solution integrated with Siebel Application
7.5.3. Thus we need a huge scope of data and a huge numbers of rows to validate
and test performance for this solution.
To populate our Siebel Database with random data, we were obliged to use Siebel
EIM process. So we decided to take advantage of test performance exercise to
execute another complex exercise: Siebel EIM load of user data.
First we will describe EIM basis and we will continue with EIM data export
description.
Then we will explain our first simple example of EIM import and we will describe
all errors and difficulties encountered.
EIM Overview
In Siebel Application User data is populated and managed by you. User data is
stored in one or more tables in the Siebel Database and it implies a lot of
relationships between these tables. It is obviously complex to populate user data
using SQL. First reason is referential integrity (which is maintained
programmatically through ROW_ID) of Siebel data.
Siebel offers you processes to import, delete or export of data into tables. This
process called EIM uses interface tables acting as a staging area between the Siebel
Database and other data sources. Once data entered in EIM tables, EIM task
performs data load into Siebel tables, according to Siebel System rules.
Data Mapping
Data Mapping determines which Siebel table columns will store external source
data and which interface table columns will be used to import from source to
destination. Data Mapping is Business Analyst job and is the more complex and
important part in EIM process. Once you located correct EIM tables and its columns,
you just have to run EIM task to import successfully you user data.
Invoking EIM
All interface tables have 3 columns, which must be populated in addition to the
data that was mapped. A number to identify all records that should be processed in
the batch (IF_ROW_BATCH_NUM) and an unique number to identify the record in
the batch (ROW_ID, which is NOT the same row id exposed previously in this
document), the third one is a status. The EIM batch (or task) is using a
configuration file (.ifb file) to specify the batch, order and type of processing.
Context Description
We decided to work with Siebel Activities and to import a huge set of user data
concerning activities.
As it is written before in the document, we decided to, first export Siebel Data to
better understand EIM mechanism before importing Data into Siebel Tables. To do
that, we chose an activity acting as sample for our export exercise and as template
for our future import exercise.
During almost EIM exercises, which will be described, we worked with this record,
these applets (one list applet and one form applet), this view and this screen.
With Siebel help menu of your browser client, you can obtain such information.
Hereafter you can see all Siebel information about our parameters set we worked
with.
Screen: Activities
Screen
View: Activity List View [Cache Mode : Not Cached]
Business Object: Actio
n
Applets: Applet[0]: Activity List Applet With Navigation; Applet[1]: Activity Form Applet;
Business Components: BusComp[0]: Action; BusComp[1]: Action;
For Siebel and database, we used a power5 server (p570) with AIX 5.3, partitioned
into 3 partitions. One partition for Siebel Web Server, Siebel Server and gateway,
one another for Optim Archive Solution and one last for Database Server (DB2 v8).
EIM Export
To export data, EIM reads the data in the Siebel database tables and places the
information in the appropriate EIM tables. You can then copy data from the EIM
tables into another database. The export process generally populates the
applicable EIM table with a row for every Siebel base table row encountered. As a
consequence, where EIM tables have mappings to multiple Siebel base tables, one
export operation can generate multiple rows within the EIM table governing the
rows encountered within the Siebel base tables.
At the start point, we know which row we want to export: record with description
‘ffr ffr’. But our concern is to know where (which EIM tables) our record will be
exported. So we must find which EIM table will be used before exporting any Siebel
Data.
We know also which view and applet are concerned (see below). So we can guess
which Siebel Table is concerned using Siebel Tools:
Thus we are able to know the base table of our record with Table field in Tools.
We are going to search EIM tables mapping with Siebel table ‘S_EVT_ACT’
We obtain the EIM Table list of EIM tables mapping with ‘S_EVT_ACT’ table
NOTE: the red lines represent old EIM tables with old name convention (name
ending with IF extension) and they are inactive. Now EIM tables beginning with
EIM extension.
If CLEAR INTERFACE TABLE in the configuration file is TRUE, all rows with the
specified batch number are deleted. Otherwise, a warning is issued if rows
already exist with the specified batch number
3 For parent tables, EIM locates child table rows and exports them to their
corresponding EIM tables
Before you initiate an export process, you should verify that rows do not contain
an IF_ROW_BATCH_NUM matching the batch number you plan to use. If such
rows do exist, you should either make sure that they do not contain data you
need to preserve, or change the batch number for the export process. In each
row that you are exporting, you may also want to set the IF_ROW_STAT column
to FOR_EXPORT.
If EIM finds a row whose IF_ROW_BATCH_NUM matches the batch number for
this export process, it does one of the following:
■ Clear the row if the CLEAR INTERFACE TABLES parameter is set to TRUE in
the EIM configuration file
The values for the LAST_UPD and CREATED columns in the EIM tables always
contain the values for the LAST_UPD and CREATED columns from the target
base table. For example, if you use the EIM_CONTACT interface table to export
data from the S_CONTACT and S_ADDR_PER base tables, the values of the
EIM_CONTACT.LAST_UPD and EIM_CONTACT.CREATED columns contain the
data from the S_CONTACT.LAST_UPD and S_CONTACT.CREATED columns,
respectively.
EIM reads a configuration file that specifies the EIM process to perform (import,
update, merge, delete, or export) using the appropriate parameters. The EIM
configuration file (the default file is default.ifb) is an ASCII text file of
extension type .IFB that resides in the Siebel Server/admin directory. Before
you can run an EIM process, you must edit the contents of the EIM configuration
file to define the processes for EIM to perform.
The EIM configuration file begins with a header section used to specify global
parameters that apply to all process sections defined later in the file. Following
the header section, there must be at least one process section with its
associated parameters. Some process section parameters are generic for all EIM
processes. Other process section parameters are specific to a particular EIM
process, such as import.
[Export EIM activities test #1] // the same name as process field value before
TYPE = EXPORT // mandatory
BATCH = 100 // mandatory
TABLE = EIM_ACTIVITY // mandatory
EXPORT ALL ROWS = true // to import row even in child table
All configuration file header must begin with “[Siebel Interface Manager]” and
must contain “PROCESS = ‘xxxx’” field.
Then configuration file section “[xxxx]“ must contain a type (export, import…),
a batch number “BATCH = 100 “ and the name of EIM table which is used
(“TABLE = EIM_ACTIVITY “).
On each pass, EIM processes one EIM table and performs a particular action on
all rows in that table for that batch. Most passes affect only the EIM tables
temporary columns (columns with name starting with ‘T_’); for example,
resolving foreign keys. There are two methods for running an EIM process:
The easiest way to run EIM process is to use GUI, but in our example, we are
using command line and Siebel Server manager.
For your information, hereafter you can find the help guide
The enterprise component is EIM and the task is launched with following
configuration:
You can see all possible parameters for component EIM with the following
command:
When an export process ends, you should carefully check the results to verify
that data was successfully exported. During each export process, EIM writes
comprehensive status and diagnostic information to several destinations.
/sia753_db2/siebsrvr/enterprises/sia753_db2/part1_db2/log
Then you can use your SQL tools (here DB2 Command Center) to check results.
You can verify the batch number (=100) and the batch status (=exported)
EIM Import
Importing data into Siebel base tables is a multi step process that requires
significant effort. You must first load data from an external database into the EIM
tables. Subsequently, you need to run an EIM process to read the data in these EIM
tables and import them into the appropriate Siebel base tables.
To import tables of data, EIM performs a sequence of tasks. Each task involves
multiple passes; at least one pass is required for each EIM table included in the
process. Depending on the type of import process, EIM may repeat several tasks.
The greatest difficulty of EIM import process is obviously data mapping between
external data source (your data) and Siebel tables. Because of huge and complex
data model of Siebel, because of complex Siebel Tables Management processes (a
lot of child-parent relationship for example), it is a so hard exercise to identify
which EIM columns to fill in, which EIM columns are useful and which one are no
useful, which EIM columns point to which Siebel Base Table Columns.
That is for all these reasons described above why we are starting from data
coming from export as a template for import. We are using a copy of row
previously exported (with description ‘ffr ffr’) and we are going to try to import
it with some modifications.
Then, to avoid the difficulty of data mapping we copied this record with some
modifications.
You can see first line for import, which is a copy of the second line above,
which is exported
To finish you complete some of the other fields of record for import,
following values for the same fields of exported record.
You can continue to copy the exported record, as well as you want.
Parameters in the process section apply only to that specific process and
override any corresponding value in the header section for the specific process.
This section describes the parameters used in the process section that are
specific to an import process.
NOTE: The ONLY BASE TABLES IGNORE BASE TABLES, ONLY BASE COLUMNS,
and IGNORE BASE COLUMNS parameters can be used to improve EIM
performance.
We use a different batch number for import (=10 for example) than this one
uses for export (=100). Thus, the new configuration file for import will be the
following:
The last think we should change, is the description field of the record we want
to import (in order to improve checking result after processing). This field
corresponds to EIM Column ACT_NAME as we could see previously.
You are advised to write a personal comment in order to check easier later.
The enterprise component is EIM and the task is launched with following
configuration:
When an export process ends, you should carefully check the results to verify
that data was successfully imported. During each import process, EIM writes
comprehensive status and diagnostic information to several destinations.
First, look at Siebel application to see new record. It is the best way to see
whether import process worked.
Then, query the appropriate EIM tables for rows whose IF_ROW_BATCH_NUM
equals the batch number for the import.
These columns in each EIM table indicate whether a row was imported
successfully, and they identify the pass number on which a row failed. During
various passes of import processing, EIM sets the IF_ROW_STAT value to one of
the values shown in following table:
Most failures are caused by data errors. It is usually faster and easier to correct
the data errors and resubmit the corrected rows as part of a subsequent batch
than to reprocess an entire batch. EIM does not stop when failures occur.
Failures can occur at several steps during the EIM import process. Each type of
failure has a different cause. Read the log files and see the causes listed in the
following table:
Go to SIEBEL_HOME/siebsrvr/enterprises/’name of
enterprise’/’name of siebel server’/log
Edit EIM_*.log
In AIX environment:
Other big difficulty of EIM process is the parallel between the huge numbers of
possible mappings between Siebel attributes and the little documentation about
it.
Thus you need to read log files to correct/improve EIM process and try again to
see whether all stuffs are ok. Some errors causes the import failure, some one
else causes just warning. Check, correct and try again.
Base table:
S_ACT_CAL_RSRC (Calendar Resource for an Activity)
--------------
RESOURCE_ID (Resource ID)
// Explanation of error
This is a foreign key value in the base table and the values in the interface
table did not resolve to existing values. Verify that the IF columns correspond
to existing base table rows.
This failure caused the rows to be eliminated from further processing for this
secondary base table. However, processing of the rows WILL continue for other
destination base tables.
// Rows eliminated
ROW_ID RESOURCE_RES_NUM
-------- ----------------
1 NULL
1-34SSX NULL
-------- ----------------
(all 2 rows which failed)
In the previous example, we just needed to enter a value for the field
‘RESOURCE_RES_NUM’. This error caused the failure of all EIM import process.
---
An other instance of log file is the following. Error with severity 8, which
doesn’t stop the EIM process. If you want, you can fix it or not.
At the file end, you can see the error total. Here 51 errors, which are, in fact,
warnings-like.
Base table:
S_EVT_ACT (Activity)
---------
TODO_APPT_ID (Todo Appointment)
This is a foreign key value in the base table and the values in the interface
table did not resolve to existing values. Verify that the IF columns correspond
to
existing base table rows or new interface table rows.
This failure did not eliminate the rows from further processing. Processing
will continue for other columns in the same and in other destination base
table.
-----------------------------------------------------------------------
What to do after?
As you can guess, it could be very boring to fix all 51 errors. All these errors
refer to foreign key (pointer to another table) problem “This is a foreign key
value in the base table and the values in the interface table did not resolve to
existing values” and we are not obliged to solve it (you can decide to not attach
a contact to an activity for example, without invalidating the activity record at
application level). You have to select foreign key in which you are interested,
depending on your data model or business case, or even your simple choice.
Example
Keeping the former example of errors (51 errors), we are going to show how to
fix a foreign key problem.
We decided (it is our choice) to fix the following error with severity 8:
Interface table:
EIM_ACTIVITY (EIM_ACTIVITY) // for all of the following attributes, values
------------ // are missed to find an existing foreign
key
ACT_AST_BI (Name)
ACT_AST_NUM (Asset number)
ACT_ASTPRDVEN_NAME (Name)
ACT_AST_PRDVEN_BI (Name)
ACT_AST_PRDVEN_LOC (Site)
ACT_AST_PROD_BI (Name)
ACT_AST_PROD_NAME (Name)
ACT_REV_NUM (Rev Num)
Base table:
S_EVT_ACT (Activity)
---------
ASSET_ID (Asset ID) // it is a foreign key pointing to another table
Action
ACT_AST_BI (Name)
ACT_AST_NUM (Asset number)
ACT_ASTPRDVEN_NAME (Name)
ACT_AST_PRDVEN_BI (Name)
ACT_AST_PRDVEN_LOC (Site)
ACT_AST_PROD_BI (Name)
ACT_AST_PROD_NAME (Name)
ACT_REV_NUM (Rev Num)
Indeed, in order to correctly enter right values in EIM columns (values pointing
to an existing row in ‘S_ASSET’ table) to map to a correct foreign key, we need
to understand foreign key mapping:
EIM_ACTIVITY S_ASSET
---------------- ---------
ACT_AST_BI (Name) ?????????
ACT_AST_NUM (Asset number) ?????????
ACT_ASTPRDVEN_NAME (Name) ?????????
ACT_AST_PRDVEN_BI (Name) ?????????
ACT_AST_PRDVEN_LOC (Site) ?????????
ACT_AST_PROD_BI (Name) ?????????
ACT_AST_PROD_NAME (Name) ?????????
ACT_REV_NUM (Rev Num) ?????????
EIM_ACTIVITY S_ASSET
Point
To Foreign key
ASSET_ID
Siebel Tools helps you to manage the foreign key mapping, giving you explicit
columns concerned by the mapping. To see such explicit mapping, let you follow
the following procedure:
These following screenshots show you the mapping between EIM columns
(Interface table data column) and ‘foreign’ table S_ASSET (user key attribute)
EIM_ACTIVITY S_ASSET
Take a glance at user key attribute column. This column indicates you the exact
value to enter in corresponding interface column.
Since a foreign key in a foreign table can point itself to another foreign key and
so on, the search of the right value to enter in foreign key mapping could be so
fastidious. The following section explains you how to resolve foreign key
mappings.
Using the highlighted row example above, we are going to expose the different
step to resolve a foreign key mapping value.
1. Go to S_ASSET Table
PROD_ID/VENR_OU_ID/BU_ID/NAME
The join path must be read from the greatest join sequence number to the
lowest join sequence number.
TIP: you find again this order in the name of the user key attribute
But ACT_AST_PRDVEN_BI column must contain row_id value and not a name
value.
Result:
We must fill in EIM interface column ACT_AST_PRDVEN_BU with the
corresponding value of S_BU.NAME
The EIM_ type interface tables use the xxx_BU/xxx_BI column pairs to map
organizations. For example, the CON_BU/CON_BI column in the EIM_CONTACT
interface table is mapped to the BU_ID column in the S_CONTACT base table.
In order for organizations to be resolved properly, you need to populate the
xxx_BU column with the organization name and leave the xxx_BI column empty.
Do not populate the xxx_BU column with the organization ROW_ID. EIM looks
up the ROW_ID for the organization in xxx_BU and puts it in the
corresponding xxx_BI column!!!