Académique Documents
Professionnel Documents
Culture Documents
Data Service is the primary tool to extract, transform & load data from one or
more source systems into one or more target systems.
Data Services can access data from a wide variety of applications and file sources and can
consume almost every type of datastructured, semi-structured, and unstructuredfrom
those sources.
Service Integration
Message Integration
Data Integration
Data Integration.
Organizational environment
Other factors.
If large no of messages are per second with less or medium amt of data need to be
delivered then SAP PI is the very good choice.
If the data volumes are very large & frequency of data to be transferred is less than
SAP BODS is the choice.
Example:Suppose the requirement is that the data is sent to a business partner twice a
month & the file size is 4 GB. Then in such a case sending the data
to BODS is good
option.
Processing Mode:-
Scheduled / Batch Based:Data services processing is typically scheduled. You can schedule the Batch jobs in
BODS or specifically run it. Pi jobs can also be scheduled but
generally they are
not scheduled.
Event / Trigger Based:BODS has the capability to receive events via web services or other means &
process these events.
PI is typically triggered by incoming data.
Real Time(synchronous/ asynchronous):Data latency for DS is larger( minutes rather than seconds).
PI offers mechanisms for real-time synchronous transactions.
Example:-
Data has to be moved from the database to BW system once in a week or once in a
day then BODS is the goods option.
File has to be picked from the sender system after every 10minutes then SAP PI is the
good option.
Data Processing Level:-
Both data services & PI do well in applying basic transformations. Basic functions
required for transformations are available on both the platforms. eg. String functions,
mathematical functions etc are found in both.
Data service has much fore functionalities related to data cleansing activity. eg.
Validate, Table Comparison, Map_operations etc.
Business Process Management:-
Pi provides full workflow capabilities. System centric & Human centric workflows are
possible in PI.
Data service focus on system centric workflows only. In DS workflows are simple
deals with process sequencing & error handling.
Reliable Messaging:-
Both PI & BODS are capable of handling connectivity between SAP & non-SAP
systems.
A.
File Handling, SAP Applications(RFC, IDOCs, Proxy), Legacy protocols(JMS), Databases.
Database handling is the core functionality of BODS. It is able to extract tables in the
complex manner.
PI is positioned as a SOA middleware. It is the basis of SOA. You can create a web
service from the interfaces used in PI.
In PI you can publish your web services in service registry which is based on UDDI
3.0, where you can search, manage & consume the web services.
BODS acts as service provider or consumer, but doesnt offer further SOA
functionality.
TCO:-
The cost of using SAP PI as an integration tool is based on the overall processed
message volume expressed in GB/month. SAP PI is free to use for integration between SAP
to SAP systems.
In BODS with one application we can cover multiple data management processes so
it lowers TCO.
Example:-
If user need to know the details about the number of materials created in an working
unit then creating the web service for this purpose & consuming it across anywhere is the
right choice & for this SAP Pi is the best option.
Data service is the best option for data integration with SAP Netweaver BW, SAP
HANA & entire Business Objects BI Platform. Data Services is the preferred tool for loading
non-SAP data into SAP HANA. In fact, the data integration capabilities of Data Services are
included with SAP HANA. Additionally, SAP is making major improvements for a seamless
user interface between Data Services and SAP HANA.
Integration Content:SAP delivers prepackaged integration content, which is created based on the SAP
SOA design methodology. This business content includes global data types, service
interfaces, and mapping definitions. IT professionals can use this predelivered content from
SAP to jump-start their SOA implementations in SAP PI.
Third party adapters are also available in SAP PI to meet particular industry
standards.eg. EDI adapters by Seeburger
Development Interface:In BODS all generation & data quality processes are developed & managed in a single
interface i.e. Data Service Designer.
In PI creation of Design & Configuration objects are separated in Integration
Repository & Integration Directory respectively.
Summary:As we have seen the overlap between SAP PI & BODS we came to know that both
are important tools & depending upon the business requirement, cost, available resources,
connectivity options & data volume one would choose SAP PI or SAP BODS for their
integration requirement.
1.
2.
3.
This document provides systematic approach for the data load in three sections:
Configuration Required at SAP Side:
Configuration Required at BODS Side:
Demonstration Sample Load using BODS Job:
Each of above sections provide information as in how to configure the specific parameters
required to get both SAP & BODS ready to communicate with each other. Screen shots
provided for better understanding of the parameters & their configuration.
Configuration Required On SAP (ECC 6.0) Side
Following is the required configuration on SAP side.
1.Display IMG [ /nsale ]
Connect to SAP System using SAP GUI
Create SAP Datastore by providing required SAP Application Server Information &
creadentials
Important : Make sure that in XML file Sending Partner setting i.e. <SNDPRN> should
be same as BODS logical system name. (FRMIDCL100) as shown below.
Objective
Pre Requisites
Introduction to BODS
BODS Architecture
Getting started
o
Repository Creation
Executing jobs
Debugging jobs
Data Profiling
Objective
After reading this article, a normal person with elementary programming knowledge would
be able to work on SAP Business Objects Data Services (which will be referred to as tool,
BODS, Data services or BODI hereafter), and would be in a position to understand its basics
and working. The person reading this book would be able to get acquainted to BODS in less
than a weeks time .This book will also be an entry for professionals who are eager to get on
to the Business Objects Data services world and a confidence booster to attend interviews .
Pre-requisites
Hardware
PC with minimum 8 GB RAM and Processor 3i and above
Software
SAP Business Objects Data Services 4.0 preinstalled. Any RDMS preinstalled (SQL Express
2008 preferred as that will be used for examples and illustration throughout the book).The
RDMS would have to be installed prior to BODS.
Programmer/Consultant/Resource
Basic programming knowledge and basic SQL knowledge. In other words anyone who
understands the terms such as, If then else, Loops, Database, Tables, Query, DML and
DDL statements should be able to follow this tutorial without any trouble. Also someone who
recognizes or is aware of business terms like Data, Transformation, ETL, Data
warehousing, Data Migration, ERP (some these terms are explained briefly in the
following chapters) and familiarity with names of popular databases and softwares and ERPs
like SAP, Oracle, SQL Server.
Exclusions and Inclusions
Not all the functions, transforms, objects, methods or buttons or menus are not
covered in the book.
The latest release Data services 4.0 would be used for illustrations and examples.
Other tools that are installed with the BODS suite including Information steward,
Management console are not covered in the scope of this book.
Introduction
ETL (Extraction Transformation and Loading) Pulling out data from any
system/database/tables ,applying changes to modify the data or applying
programming logic to enhance the extracted data ,and loading data into any other
system/database or tables. E.g.: ETL of data from SQL server database to Oracle.
Data Migration - Moving of data from one place to another. This is a subset of ETL
where data is relocated from one software system or database to another .This also
involves modification and alteration of data.
task without having a local repository. In other words local repository is a mandatory
repository for BODS functioning. At this point we are not bothered about the other two
repositories.
Engine
The BODS Engine executes the jobs created using the Designer. When the BODS application
is started, there are enough Engines launched to effectively accomplish defined tasks.
Job Server
The Job Server is an application that launches the Data Services processing engine and
serves as an interface to the engine and other components in the Data Services suite.
Access Server
The Access Server passes messages between web applications and the Data Services Job
Server and engines.
Datastore
A datastore provides a connection to a data source such as a database. This is a linking
interface between the actual backend database and Data services. Through the datastore
connection, Data Services is able to import descriptions of the data source such as its
metadata.
CMC (Central Management Console)
This is a web based administration tool for BODS which is used for some basic functions such
as repository registration, User Management etc.
These terms should be at the finger tips of a BODS programmer as these would be used very
often while working on BODS.
BODS Architecture:
The
illustration
below
(Figure
1.1)
shows
the
basic
architecture
of
BODS.
Figure 1.1
Above diagram explains the relation among Designer, central and local repository and web
application. We will gradually understand these later. In the next page we will start creating
our first BODS application.Click
Purpose:
The purpose of this document is to up-grade SAP Business Objects Data Services from
11.7/3.X/4.X to SAP Data Services 4.2
Overview:
Environment Details:
Operating system: Windows Server 2008 64 Bit
Database: Microsoft SQL Server 2008 R2
Web Application: Tomcat
SAP Business Objects Tools: SAP Business Objects Information Platform Services 4.1 SP2;
SAP Data Services 4.1 SP2
Migration Tools: Data Services Repository Manager
Repository version: BODI 11.7 or BODS 3.X or BODS 4.X
Installation & Configuration steps to upgrade SAP BODS 3.X/4.X to SAP BODS 4.2:
o
o
SAP Business Objects Information Platform Services 4.1 SP2 Installation &
Configuration Documents:
Prerequisite:
SAP BusinessObjects Business Intelligence (BI) 4.1 SP2 or its higher compatible patches*
OR
SAP BusinessObjects Information Platform Services (IPS) 4.1 SP2 or its higher compatible
patches*
Up gradation & Configuration Documents:
When the prerequisite system check completes successfully, click Next
On the "SAP BusinessObjects Information platform services 4.1 SP2 setup" page, click Next.
Type a password for the CMS Administrator account, and click Next
Installation is in progress
Finish
On the "SAP BusinessObjects Data Services 4.2 SP1 setup" page, click Next.
Click Next to accept the default path to the folder where the program will be installed.
Type a password for the CMS Administrator account, and click Next.
Installation is in progress
Finish
So before login upgrade the repository using SAP Business Objects Repository
manager.Below are the steps for upgrade of local repository details:
Below are the steps for upgrade of local repository & upgrade of secure central repository
Upgrade of Local Repository:
After Installation & Up-gradation of SAP business Objects BOE/IPS Server, SAP Business
Objects Data Services 4.2SP1 below are the status that i upgraded in VM Ware
Then Click on Data Services and update the repository details of SAP Business Objects Data
Services repository.
Click on Logon
Click on ok
New features are added in SAP Business Objects Data Services management console 4.2":
Object Promotion:
Import Configuration
Export Configuration
FTP
Shared Directory
New Features added in Adapter like "Hive Adapter & VCF Adapter"
Changes in "Query and Validation" transform in SAP Business Objects Data Services
Designer 4.2
Changes in "Architecture" of SAP Business Objects Data Services please refer below
upgrade guide for more details.
New Features added like "REST web services": Representational State Transfer (REST or
RESTful) web service is a design pattern for the World Wide Web. Data Services now allows
you to call the REST server and then browse through and use the data the server returns
Relevant Systems
SAP Business Objects Information Platform Services 4.1 SP2/SAP Business Objects
Enterprise Information Management 4.1 SP2
SAP Business Objects Data Services 4.1 SP2
Reference Material:
Upgrade Guide:
https://help.sap.com/businessobject/product_guides/sbods42/en/ds_42_upgrade_en.pdf
https://help.sap.com/businessobject/product_guides/sbods42/en/ds_42_install_win_en.pdf
https://help.sap.com/businessobject/product_guides/sbods42/en/ds_42_install_unix_en.pdf
https://help.sap.com/bods
ABOUT CDC:
About
Articles Related
CDC mechanisms
Type of Integration
Documentation / Reference
OBIA - Creating Custom Indexes in Oracle EBS Source Databases for Incremental Load
Performance
CDC mechanisms
Database Log Miner (Log-based Change Data Capture or Database Log Readers)
Database Triggers. This DBMS feature invokes a prewritten routine each time a specific
set of conditions are met, such as the addition or updating of a record in the database. In
concept, the code that the trigger launches can be written to write a record of the transaction
to a database table and the ETL tool can then poll those tables on a periodic basis. In
practice, this approach is far from foolproof. For example triggers can regularly be
deleted/disabled then re-added/re-enabled in the normal course of a business operation.
Triggers also place a relatively high overhead burden on the source database server
Message Queues. An obvious disadvantage is that they are not a completely reliable
source of change information since they only know about data changes that are sent to them
by the applications and not updates in the database.
Date and time stamps many ERP applications and other data sources maintain data
fields within each record that indicate when it was last changed. The fatal flaw in this
approach is that it relies on the programs that change data to unfailingly update this field. In
addition, this approach can lose track of deletions since the entire record including the time
stamps are gone.
Table comparison with a hash function: OWB - How to implement a type 2 slowly
Type of Integration
data-based,
event-based
or services-based integration.
Documentation / Reference
Kimball University: Six Key Decisions for ETL Architectures
Sources and Targets can be SAP Applications (ERP, CRM etc.), SAP BW, SAP
HANA, Any Relational Database (MS SQL Server, Oracle etc.), Any File (Excel
Workbook, Flat file, XML, HDFS etc.), unstructured text, Web services etc.
ETL technology of SAP BODS can be done in both Batch mode and Real time
mode data integration.
Architecture:
The following figure outlines the architecture of standard
components of SAP BusinessObjects Data Services.
Platform Services (IPS) should be installed for User and Rights security
management from 4.x versions. Data Services relies on CMC (Central
Management Console) for Authentication and Security features. In earlier
versions it was done is Management console of SAP BODS.
BODS Designer:
SAP BusinessObjects Data Services Designer is a developer or designer tool.
It is an easy-to-use graphical user interface wheredevelopers can design
objects that consist of data mappings, transformations, and control logic.
Repository
Repository is the space in a database server which stores the
metadata of the objects used in SAP BusinessObjects Data Services. Each
repositorymust be registered in the Central Management Console (CMC) and
associated with one or more Job Servers which run the jobs you create.
There are three types of repositories used with SAP BODS:
Local repository:
A local repository stores the metadata of all the objects (like projects, jobs,
work flows, and data flows) and source/target metadata defined by
developers in SAP BODS Desinger.
Central repository:
Profiler repository:
Where, CMS repository is used to store the metadata of all the tasks done in
CMC of SAP BO BI platform or IPS.
Information Steward Repository is used to store the metadata of profiling
tasks and objects defined in SAP Information Steward.
Job Server
The SAP BusinessObjects Data Services Job Server retrieves the job
information from its respected repository and starts the data engine to
process the job.
The Job Server can move data in either batch or real-time mode and uses
distributed query optimization, multi-threading, in-memory caching, inmemory data transformations, and parallel processing to deliver high data
throughput and scalability.
Access Server
The SAP BusinessObjects Data Services Access Server is a real-time, requestreply message broker that collects message requests, routes them to a realtime service, and delivers a message reply within a user-specified time
frame.
Management Console
SAP BusinessObjects Data Services Management Console is the Web-based
application with the following properties.
Administration, Impact and Lineage Analysis, Operational Dashboard, Auto
Documentation, Data Validation and Data Quality Reports.
http://scn.sap.com/community/data-services/blog/2013/07/05/scdtype-implementation-in-bods
http://www.dwbi.org/etl/sap-data-services/104-how-to-implementscd-type-2-using-history-preserve-transform-in-data-services
http://wiki.scn.sap.com/wiki/display/EIM/Loading+a+table+with+sur
rogate+key
https://www.youtube.com/watch?v=EhM1r2VLryA
http://wiki.scn.sap.com/wiki/display/EIM/Data+Cleanse+Transform
http://wiki.scn.sap.com/wiki/display/EIM/Data+Quality
Follow below link regarding TDP (Text Data Processing), Global Address
Cleanse Transform, Data Cleanse transform, Address Reference Data, Geocoder
transform, DQM for SAP:
http://wiki.scn.sap.com/wiki/display/EIM/Data+Services+Product+Tu
torials