02 01 PC Architecture&Func

BW Systemadministration Portfolio
Extraction and Data Transfer Data Load Monitoring, Reload and Repair ODS and Datamart Handling Process Chain Architecture & Functionality Process Chain Modelling Aspects Process Chain Monitoring & Error Handling Data Modelling from the technical point of view BW Transport Management System
Periodical jobs and tasks
Monitoring Changerun Load Balancing & Number Range Buffering BW Statistics and Query Tuning with Aggregates(RSRT) Usage of Basis Transactions (SM50, ) for applicational people
SAP AG 2005, BW System Administration / 1
Process Chain Architecture & Functionality
First steps & Basics Specific settings Background Processing & Load Distribution to different Servers Transporting Process Chains Tips & Tricks Implementing new Processes
Introduction: Typical Data Load Cycle

Load into PSA Load into ODS
Activate Data in ODS
Data Load Monitor
Start
Drop Indices Roll up Aggregates
Build DB Statistics
Load into Cube
Data Target Maintenance

Build Indices
Motivation
Automating the BW administration Graphical modelling and monitoring Central monitoring of all BW processes
Openness for all kinds of processes Moving responsibility into the individual processes
3 Different Views to Process Chains

Views related to the working area
Planning view for checking the plan status of the process chain Checking view for check consistency of the process chain Log View for monitoring maintenance activities and executions
Different views to the the activities which are possible: Process chains Process types Data targets InfoSources Logs
working area
There are three main views in the icon bar with different views: Planning view (shows if the process chain is active) - Grey: unplanned processes (e.g. not activated process chain) - Green: planned processes (process chain is active and start process is released) - Yellow: planned but unknown processes - Red: multiple planned processes Checking view (consistency check like double used start variants, missing index deletion and recreation, wrong references in variants, ...) - Green: Error-free processes - Yellow: Process with warnings - Red: Process with errors Log view - Grey: Not yet run - Green: Finished without error - Yellow: running - Red: aborted or failed note that the log information is a usually mixture of - log information of maintenance activites (e.g. new process, change of design, activation) 5
Different Object Trees for Process Chain Administration

For easy administration on process chains different object trees
can be displayed.
Display component
Available Process Types
Process chain
Folders in the process chain are called display components. For mainentance of display components you have to use process chains display components. For reassignment of process chain to different display component For creation of new display component attributes
Different object trees when building process chains

For easy administration on process chains different object trees can be displayed. Log
Display the log tree InfoAreas Search in InfoProvider tree InfoSources Search in InfoSources tree for InfoPackages
In InfoPackage reference to process chain is displayed
Possible processes on data target are displayed

Application Processes
Application processes represent BW activities that are typically performed as part of BW operations. Examples include: Data load Attribute/Hierarchy Change run Aggregate rollup Reporting Agent Settings
Other special types of application processes exist:
Starter process that exists to initiate process chain execution ABAP program Another process chain Customer built process
A starter process is part of every process chain.
Application Process Types

Operating system command
Access to OS commands like dir (list contents of application server file system) Note: There is only an call out to OS, only Standard-Out is given back to BW
Remote process chains

Supports process chains in a data mart environment
Master data reorganization Reporting Agent Settings

Exception reporting Print queries in background Pre-calculating web templates Pre-calculating characteristic variables of type precalculated value sets
Collector Processes
Collectors are used to manage multiple processes that feed into the same subsequent process. The collectors available for BW are:
AND: All of the processes that are direct predecessors must send an event in order for subsequent processes to be executed OR: A least one predecessor process must send an event The first predecessor process that sends an event triggers the subsequent process EXOR: Exclusive OR Similar to regular OR, but there is only ONE execution of the successor processes, even if several predecessor processes raise an event
Collector processes allow the designer of a process chain to trigger a subsequent process based on whether certain conditions are met by multiple predecessor processes. Application processes are the other type of processes these represent BW activities such as aggregate rollup, etc. Although the AND condition is implemented for process chains using the event chain functionality from 2.x, this event chain is internal and cannot be edited.
10
RSPC User Interface: Building a Process Chain

Load Process and Subsequent Processing Data Loading Process Read PSA and update data target Save Hierarchy Further Processing of ODS Object Data Data Export into External Systems Delete Overlapping Requests from InfoCube Data Target Administration Delete Index Generate Index Construct Database Statistics Initial Fill of New Aggregates Roll Up of Filled Aggregates Compression of the InfoCube 1. Drag and Activate ODS Object Data Complete Deletion of Data Target Contents Other BW Processes Attribute Change Run Adjustment of Time-Dependent Aggregates Deletion of Requests from PSA General Services Start Process AND (Last) OR (Each) EXOR (First)
Start TR
Load Data
drop
TR Texts
2. Draw line to connect processes
Attrib. Change TR Characteristics
3. Press Refresh
When you draw a line to connect processes, you are prompted to indicate whether the subsequent process should execute based on the success or the failure of the predecessor process. In other words, it is possible to schedule a process to run only if the predecessor process fails.
11
Start Process
Variant
When and how process chain will be scheduled
Note: For each process chain the start process is mandatory. It is not possible to use the same variant more than once !!!
The process variant could be used just by one process chain. Each process chain could be started manually or via RFC connection with the function module RSPC_API_CHAIN_START.
12
Structure of a process Example on variant

Maintenance of variant is specific for every process type A variant is the configuration of a process of a particular type set at the time of definition (e.g. the variant for a data load process is an InfoPackage)
Type Kind of task

Attribute Change Run ps_attrib2_long PA_ATTRIB2 Execute InfoPackage: ZPAK_4QVBBF65GZONPAP96APH2P3KQ
Variant Configuration
Process = process type + variant

13
Structure of a process Example of process type
Type Attribute Change Run

Execute the hierarchy and attribute change run Process types can be maintained via Settings Process Types
Do not change standard process types (if allowed)
If standard process types can be changed depends on the changeability of name ranges (transaction SE03)
14
Structure of a process Example of variant

Variant Configure the Hierarchy and Attribute Change run
Execute the specific hierarchy and attribute change run for which hierarchy, which InfoObject or which data loading process
15
Process Types Example Execute InfoPackage
16
Maintain Message
Planning view context menu Write a message and fill in recipient and type. Info saved within process variant.
Send with Note
Bwadmin@sap.com BWADMIN@sap.com
Internet Mail Access has to be configured (with SAPCONNECT) for sending Emails from SAP-System to the Internet via SMTP.
17
18
Automatic Insertion of Corresponding Process Types

If a process is inserted into the process chain the corresponding process variants are inserted into the process chain automatically:
Example: Example You drag and drop a data load process to your process chain, the Index drop and the Index create process are automatically inserted. If you want no corresponding processes to be inserted automatically, flag the (user specific) setting in the menu under Settings Default Chains
19
Polling Flag
With setting the polling flag you can control the behavior of the main process when you have distributed processes (Process Chain Attributes Polling). The main job run all the time in parallel (flag is set) or is only triggered by a finished process of the process chain (flag is not set: default).
With setting the polling flag you can control the behavior of the main process when you have distributed processes. Distributed processes, such as the loading process, are characterized as having different work processes involved in specific tasks. With the polling flag you determine whether the main process needs to be kept as long as the actual process has ended. Setting the indicator has the following advantages: A high level of process security is guaranteed External scheduling tools that react only to the SAP internal event 'Batch Process Complete', are also informed about the status of distributed processes. Setting the indicator also has the following disadvantages: Increased demand on resources. Although the CPU is not under pressure during the waiting time, it is during the status checks that run at two minute intervals. You require one more background process.
20
WAIT Time ...

Note: This wait time is just for Debugging purposes, in order to have some time to catch a process for debugging in SM50! Dont use this feature to getting some wait time for a productive process chain. Reason: Sometimes there are alos other processes influenced by this wait time.
21
WAIT Process
Sometimes it is necessary to have some wait time before the subsequent process is started ...
22
Background Server
Note: This setting is just valid, if the process chain is not started directly, but as sub chain!
With this setting you can assign this process chain (especially the Process BI_PROCESS_TRIGGER to a specific background server. Due to the fact that the different processes of this chain are scheduled via event, they are usually started also on this server. Only if there are to less background processes they are distributed to other servers.
For the case that you start the process chain with directly the the setting for background server is given as proposal, when you execute the process chain. But you can change the background server within this step. Please refer to notes: 604448 24092 519059 Collection note on job management Distribution of background jobs on application servers FAQ: Background processing system
23
24
Execution of a direct scheduled process chain

After process chain is started, the different
processes are scheduled and released.
For subchains just process

BI_PROCESS_CHAIN is released. Event triggered
Start process chain
All processes except the BI_PROCESS_TRIGGER are event scheduled All other processes are event triggered
Time scheduled
25
General Rules for Time Scheduled Jobs

Description of Time scheduled Jobs
On each SAP Instance with BTC processes a system program runs periodically (rdisp/btctime=(60)) and picks up as much jobs from the time scheduling queue until all the BTC processes of the Instance are occupied After installation: periodicity of the time scheduler is et to 60s on each SAP Instance. This means there is a fixed realationship for the pickup times between the different SAP Instances.
Result:
one instance is usually the favourite execution server
Recommendation:
Set rdisp/btctime on the different SAP Instances to different values, like 59, 60,61,.... With this setting there is no fixed relationship for the pickup times anymore
Recommendation is just useful for SAP Systems with more than one SAP Instance
26
General Rules for Event Triggered Jobs (1)

Description of Event Triggered Jobs
When an event is triggered, the system checks the SAP parameter rdisp/btcname on the actual SAP Instance and starts the event trigger program on the Instance which is specified with this parameter (default: Central Instance). After this the event scheduler tries to start the Job immediately on this Instance or in case of that all BTC processes occupied, the Job is converted to time scheduled If the Job is converted to time scheduled the mechanism for time scheduled jobs is working (see slide before!)
Result:
If rdisp/btcname is the same on all Instances and there are sufficient BTC processes running on this Instance, this Instance will take almost all event triggered BTC Jobs (this means no load distribution for Process Chain Jobs!)
27

Case 1: Typically for more than 2 SAP Instances Recommendation:
Use one btcname Server for all Instances. This BTC name server (rdisp/btcname) must have less BTC processes (e.g. 2 - 4), so that usually these BTC processes are not sufficient for all jobs which are triggered by event -> conversion to time scheduled takes place Set rdisp/btctime on the different SAP Instances to different values, like 59, 60,61,.... With this setting you get an almost equal distribution of BTC Jobs over all other servers except rdisp/btcname No Backgound Server have to be assigned to the START processes of the different process chains. in order to minimize delay times: rdisp/btctime could be set to 30,31,32, , but not smaller!
Disadvantage:
Delay times are occuring for event triggered jobs , but the more Instances the less delay times.
28

Case 2: 2 SAP Instances Recommendation:
Assign for rdisp/btcname in the Instance profile the own Instance name. Therefore you get different settings for this parameter on different Instances. After this you have to assign dedicated server names for the START processes of the process chains. If there are sufficient BTC processes are configured on the Instances the entire process chain runs on the same Instance. Set rdisp/btctime on the different SAP Instances to different values, like 59, 60, ...
Disadvantage:
Unflexible and getting optimal Job distribution on the different servers is more difficult.
29
Goal: Equal distribution of BTC Jobs over all servers
RZ01:
30
Dedicated server for a direct scheduled process chain
After starting a process chain you are able to assign a

dedicated server for the Job BI_TRIGGER_PROCESS
Due to the fact that the other processes of the chain are
event triggered, they are usually running on the server which is determined with the SAP Parameter /rdisp/btc_name (instance specific!) (see note 24092)
604448 24092 519059 700479 636854
Collection note on job management Distribution of background jobs on application servers FAQ: Background processing system Load balancing of background jobs does not work Jobs remain in the 'Ready' status for a long time
31
Number of required BTC Processes
If no subchains are used there is no special requirement to the

number of BTC work processes.
If complex chains (also in parallel) are executed, you can improve the
overall runtime of the process chains, if there are sufficient BTC processes configured, in order to avoid unnecessary delay times. Sometimes it makes sense to configure more than 2 BTC processes per CPU.
If subchain are used there is a minimum number of BTC

workprocesses necessary in order to avoid dead lock situations
# BTC (minimum) = number of parallel starting subchains at the widest part of the chain + 1 details: see note 621400
rough calculation for the number of BTC processes in order to get best performance and avoid delay times in the batch queue: # BTC (optimal) = number of parallel processes at the widest part of the chain + number of parallel subchains at the widest part + 1
32
Widest part: 2 subchains 3 processes
Minimum mumber of BTC processes: 2+1=3
Optimal number of BTC Processes: 3+2+1=6
33
SM50: Workprocess Overview: 5 BTC configured
SM37: Job Overview: active jobs
Note: BI_Process_Trigger for the second sub chain could not be started immediately, because there are just 5 BTC configured on the SAP Application Server
meaning of this slide there are 5 BTC work processes configured for the BW System (in this case just Central Instance exists) After execution of the process chain all BTC processes are allocated BI_Process_Trigger for the Process Chain itself. This process starts subsequent processes and finishes after the subsequent processes are sending status information back to BI_Process_Trigger PI_Process_Chain processes are starting subsequently the BI_Process_Trigger jobs of their assigned local process chains. Due to the fact that the there are just 5 BTC configured on the SAP Instance, there is one free BTC process missing for BI_Process_Trigger of the second subchain. This job is still waiting in the batch queue for free resources on this or further SAP Instances.
34
Subchains
If you have a complex structure of subchains which are running in parallel, you have to provide a minimum number of BTC processes: # BTC (minimum) = number of parallel subchains at the widest part of the chain + 1
Otherwise you sometimes get a Deadlock situation
SM 37: Job Overview
SM50: Configuration with 3 BTC processes
All BGD processes are blocked by BI_PROCESS_CHAIN
Please also refer to the note: 621400
35
Number of BTC processes

If process chains are used intensively and complex scenarios with lots of subchains are designed, then a sufficient number of BTC processes shall be configured Rule of thumb: #BTC = 2 * #CPUs not valid in all cases anymore In specific cases the runtime for process chains could be optimized by increasing number of BTC processes (e.g. process chains with many parallel running master data loads)
36
Optimize Load distribution

Apply different parameter values for rdisp/btctime on different servers (e.g. 59, 60, 61) (see note 519059)
Each server has its own batch scheduler, which checks periodically after rdisp/btctime the central BTC Queue for waiting BTC jobs Event triggered and immediately started jobs are usually not started view batch scheduler
I f subchains are used, a different background server can be assigned to the subchains with attribute background server in the maintenance view
Usually the processes within one process chain are running on the same server (because they are event-scheduled!)
BTC process definition on the application server

General rule: dont apply too much BTC work processes for an SAP instance (e.g. 1-2 BTC per CPU)
rdisp/btctime: Start Interval for Background Scheduler Specifies the frequency in seconds with which the background processing job scheduler is to be started in a server which can run jobs in the background.
37
Remark: General Rules for immediately scheduled Jobs

Usually immediately scheduled Jobs are not used in BW, besides exceptional cases like restart of process chains, ... Description of Immediately scheduled Jobs
default setting: Jobs that have the 'Immediate start' start condition and no target server are not distributed consistently on the different batch servers. The jobs mostly run on the same server
Result:
The jobs mostly run on the same server
Recommendation:
apply note 700479 in order to get a round robin procedure
38
39
Process Chains: Transport
Process chains are transportable Button for writing to a change request when maintaining a process chain in RSPC Process chains available in the transport connection wizard (administrator workbench) Tlogo Object R3TR RSPC
40
Transport Import Post-Processing Destination
41
Transport - Changeability
Note 337950 Queries (and other objects) cannot be edited
This option allows to deactivate the Change and Transport System for specific BW components
This setting is only intended for systems in production or test systems that are set to not changeable !
42
43
Consideration of automatisms in process chains
The following loading process automatisms are ignored within process chains:
Flag for automatic ODS-activation in ODS-maintenance Flags for automatic deletion/creation of infocube indexes/statistics (Performance-Tabstrip in Infocube Manage) Flags for automatic Infocube Content Deletion or Deletion of overlapping requests in infopackage Flag in Infopackage: PSA + update subsequently in datatargets
There are differences between manual execution and execution in process chain For all these cases there are own process types to get the same functionality and to avoid interdependency / deadlock problems.
44
Consideration of automatisms in process chains

These flags arent processed within a Process Chain !
Instead of this: Process Type Complete Deletion of Data Target Contents
Instead of this: Process Type Read PSA and Update Data Targets
Instead there is: Process Type Delete Overlapping Requests from InfoCube
Only the BW Quality flag is processed, but it is a prerequisite for Process Chains that this flag is set, otherwise unknown kinds of error messages can be displayed
45
Start Process
Variant name and description
Direct scheduling:
Job BI_PROCESS_TRIGGER will be scheduled when the process chain is executed.
Start using Meta Chain of API:

No BI_PROCESS_TRIGGER will be scheduled. Start of the process chain have to be done via FM RSPC_API_CHAIN_START or with another process chain
46
Manual Start of Process Chain

Function Builder: SE37
Sometimes you want to start a process chain immediately. You dont want to change the variant of the start process.
Technical Name of process chain

Each process chain could be started always manually or via RFC connection with the function module RSPC_API_CHAIN_START.
47
Display Component
Typical situation: Folder structure missing! Proposal: Apply your own display components
48
Restartability of Process Types

For each process type (self defined or standard) re-starting is a feature which is set in table: RSPROCESSTYPES
49
Copying Process Chain Step 1 + 2

Transaction Code: =copy
Assign new name:
50
Copying Process Chain Step 3 + 4

Exchange start variant:
Assign new unused start variant:
51
Maintain Process Chains Detail View On
RSPC -> View -> Detail View On:

Technical names Ability to move boxes to re-design the process chain Note: In detail view, a collector process is displayed as multiple collector processes (needed for conditions)
Hidden collector processes are displayed
When collector processes are built into a process chain, there are actually several background jobs scheduled with events in order to construct the conditional nature of collector processes. The simple view displays the processing chain as it exists logically. The detail view displays the processing chain with the extra collector processes.
52
Master Data Reorganization

Available as process type in process chains Minimize table size: improve performance
Attribute/Hierarchy Change Run Query performance
Often required in HR
Two features supported:

Compressing time intervals for time-dependent master data Compressing attributes with same A- and M-version
53
Master Data Reorganization - I

Compressing time intervals for time-dependent MD
Combining records with same attributes and adjacent time intervals
Example: An employees position (time-dependent master data)

01/2001 11/2001
Reorganization
01/2001
11/2001
Development manager Development manager Development manager
2 records
1 record
54
Master Data Reorganization - II

Compressing attributes with same A- and M-version
Deletion of equal A- and M-version of master data records Scenario:
Active data (A-version) already exists Load of new (different) data M-version inserted No activation of loaded master data Load of new data again which equals A-version
active data modified data
blue green
1st upload
blue blue
Reorganization
Example: Master data for article 4711 (attribute: colour)
blue blue
time
no activation 2nd upload
55
56
Appendix: Openness: Implementing a Process

If you...
...Have a simple, stand alone program to run in a chain ...Have a program, which is planned in batch by another program or user
...then you can use the process type ABAP to run your program in a chain. However if you...
...Want to ask predecessor processes of your program ...Write a log , which should be displayed in the process monitor ...Have implemented an own monitor ...Have a customizing screen, which is more difficult than the parameters of a program
...then it is recommended to custom develop own process type
57
Appendix: Implementing a Custom Designed Process

Interfaces: IF_RSPC_*
EXECUTE GET_INFO GET_LOG MAINTAIN GET_VARIANT GET_DEFAULT_CHAIN CHECK GET_STATUS CALL_MONITOR TRANSPORT CONTEXT_MENU Execution of the process Give Information to successors Give back messages Maintenance of variants F4 for variants Give default-chain for a process Check consisteny Give status of instance own monitoring tool give back TLOGO-entry Enhance context-menu
Make entry in view RSPROCESSTYPES
The interfaces are documented in the system
58
Appendix: Implementing a Custom Designed Process

Method called at... Implementing is... mandatory alternatively optional
Interfaces:
EXECUTE GET_INFO GET_LOG MAINTAIN GET_VARIANT
GET_DEFAULT_CHAIN
Maintainance
Runtime
Log View
X X X X X X X X X X X X X X
X X X X X X X X X X X
CHECK GET_STATUS CALL_MONITOR TRANSPORT CONTEXT_MENU

means: e.g.: Interface: IF_RSPC_EXECUTE is executed at runtime of the Process Chain. Usage of this Interface within your self defined process type is mandatory Process Type ABAP just uses IF_RSPC_EXECUTE and IF_RSPC_GET_Variant. No status is given back. e.g.: Interface: IF_RSPC_Get_Status is executed at runtime and when startig log view. It is not mandatory to use this interface within your own process type.
59
Appendix: Generic Services

There are several services in Paket RSPC which shall help you implementing a process type:
Generic variant storage class: CL_RSPC_VARIANT Generic dynpro service for variant maintainance: Function module RSPC_VARIANT_MAINTAIN Generic instance storage class: CL_RSPC_INSTANCE Wrapper class for application log for unambigous assignment of logs to instance CL_RSPC_APPL_LOG Service function group: RSPC_SERVICES
60

02 01 PC Architecture&Func

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

02 01 PC Architecture&Func

Transféré par

Droits d'auteur :

Formats disponibles

BW Systemadministration Portfolio

Periodical jobs and tasks

SAP AG 2005, BW System Administration / 1

Process Chain Architecture & Functionality

SAP AG 2005, BW System Administration / 2

Introduction: Typical Data Load Cycle

Activate Data in ODS

Data Load Monitor

Load into Cube

Data Target Maintenance

SAP AG 2005, BW System Administration / 4

3 Different Views to Process Chains

SAP AG 2005, BW System Administration / 5

Different Object Trees for Process Chain Administration

Available Process Types

SAP AG 2005, BW System Administration / 6

Different object trees when building process chains

In InfoPackage reference to process chain is displayed

Possible processes on data target are displayed

Other special types of application processes exist:

SAP AG 2005, BW System Administration / 8

A starter process is part of every process chain.

Application Process Types

Remote process chains

Master data reorganization Reporting Agent Settings

SAP AG 2005, BW System Administration / 9

SAP AG 2005, BW System Administration / 10

RSPC User Interface: Building a Process Chain

2. Draw line to connect processes

Attrib. Change TR Characteristics

When and how process chain will be scheduled

Structure of a process Example on variant

Type Kind of task

Process = process type + variant

Structure of a process Example of process type

Type Attribute Change Run

SAP AG 2005, BW System Administration / 14

Structure of a process Example of variant

SAP AG 2005, BW System Administration / 15

Process Types Example Execute InfoPackage

SAP AG 2005, BW System Administration / 16

Send with Note

SAP AG 2005, BW System Administration / 17

Process Chain Architecture & Functionality

SAP AG 2005, BW System Administration / 18

Automatic Insertion of Corresponding Process Types

SAP AG 2005, BW System Administration / 19

WAIT Time ...

SAP AG 2005, BW System Administration / 21

SAP AG 2005, BW System Administration / 22

Process Chain Architecture & Functionality

SAP AG 2005, BW System Administration / 24

Execution of a direct scheduled process chain

For subchains just process

Start process chain

General Rules for Time Scheduled Jobs

SAP AG 2005, BW System Administration / 26

General Rules for Event Triggered Jobs (1)

SAP AG 2005, BW System Administration / 27

General Rules for Event Triggered Jobs (2)

General Rules for Event Triggered Jobs (3)

SAP AG 2005, BW System Administration / 29

Goal: Equal distribution of BTC Jobs over all servers