Advanced Performance Optimization with SAP

BW 7.3 and SAP BW 7.4

Dr. Bjarne Berg

COMERIT

Produced by Wellesley Information Services, LLC, publisher of SAPinsider. 2016 Wellesley Information Services. All rights reserved.

In This Session

Get practical tips and techniques for maintaining and cleaning an SAP BW system for
optimal performance, including PSA optimization, compression, maintaining statistical
cubes, and controlling growth, reducing log file sizes, removing DTP temporary storage,
DTP error logs, and temporary database objects

Reduce the size of an SAP BW system by as much as 70% by taking steps such as
removing PSAs, aggregating, and optimizing InfoCubes, and implementing the new LSA+
+ architecture

See how to clean batch tables and reduce the footprints of un-needed data

Learn how to take advantage of new performance features in SAP BW 7.3 and BW 7.4

What Well Cover

SAP BW 7.3 performance basics

Housekeeping tasks in SAP BW 7.3 and 7.4
Demo: Optimal SAP BW 7.4 on HANA performance
12-step pre-HANA migration cleanup tasks
Demo: SAP BW 7.4 performance monitoring
BW optimization after HANA migration
LSA vs. LSA++ optimization and SPOs
Wrap-up

BW 7.3 (Non-HANA) InfoCube Design Line Item Dimensions

Line item dimensions are basically fields that

are transaction-oriented

Once flagged as a line item dimension, the field

is actually
stored in the fact table and has no table joins
Source: SAP AG

This may result in improvements to query speeds for cubes not in BWA or HANA

Explore the use of line item dimensions for fields that are frequently
conditioned in queries. This model change can yield faster queries.
4

BW 7.3 (Non-HANA) InfoCube Design High Cardinality Flags

High-Cardinality flag for large InfoCubes with more than 10 million rows
Real
Exam
p le

At this company there were 11 InfoCubes with a ratio of more than 30% of the records in the dimensions vs.
fact table
SAP recommends for Indexing and performance reasons to flag these as high-cardinality dimensions.
However, it has minor impact to smaller cubes.
In this example, there were four medium and large InfoCubes that are not following the basic design
guidelines, and subsequently had slow performance
Many companies should redesign large InfoCubes with high-cardinality
to take advantage of the standard performance enhancements available

BW 7.3 (Non-HANA) DSO Design and Locks on Large Oracle

Tables
In this example, many of the very
large DSOs are not partitioned, and
several objects have over 250 million
records

Real
Exa

mpl e

Additionally, 101 DSO objects were

flagged as being reportable. This
resulted in System IDs (SIDs) being
created during activation.
Combined, these resulted in frequent
locks on the Oracle database and
failed parallel activation jobs
Partition DSOs. The lock on very large DSOs during parallel loads are well known and SAP has issued
several notes on the topic: 634458 ODS object: Activation fails DEADLOCK and 84348 Oracle deadlocks, ORA-00060.
6

The BW 7.3 DataFlow Generation Wizard

SAP BW 7.3 has a new, step-by-step wizard that allows

you to generate data flows from flat files or existing data
sources

A great benefit is that the wizard works

against any InfoProvider; i.e., you can
use the wizard to create loads from
DSOs to DSOs or InfoCubes
This
Thiswizard
wizardreduces
reducesthe
thenumber
numberor
ormanual
manualsteps
stepsneeded
neededtotoload
loaddata.
data.ItItalso
also
simplifies
simplifiesthe
thedevelopment
developmentprocess
processand
andmakes
makesETL
ETLwork
workmuch
mucheasier.
easier.

Database Performance (Non-HANA Systems)

Real
Exa

Database statistics are used

by the database optimizer to
route queries. Outdated statistics
leads to performance degradation.

Outdated indexes can lead to very poor search performance in all queries where
conditioning is used (i.e., mandatory prompts)
The current sampling rates for this example were too low, and statistics should only be
run after major data loads, and could be scheduled weekly

mpl e

For many systems, database statistics are outdated and may cause database performance to
perform significantly poorer than otherwise would be the case. Sampling should often be
changed and process chains may be re-scheduled.
8

Statistics and Indexes in BW: Top-10 InfoCubes Performance

In this real example, several of the largest InfoCubes have outdated Database statistics, which
may lead to sub-optimal performance. These should be updated as soon as possible.
Real
Exam
p le

Updating DB stats can be done in a weekly job that runs ever weekend.
9

Another example: Global Cache and Query performance

Today over 40% of all queries access the
database instead of the cache. This means
that the cache hit-ratio is only 60%.

Real
Exam
p le

That ratio is very low, and can be improved

upon by changing the size of the cache on
the server
In the last month, the system had 31,955 query execution, of which 12,782 queries accessed
the database instead of the cache. If the hit-ratio is improved by 20%, it would result in
almost 59 minutes of less total wait times per day for the users.
BW comes with the default setting of 100 MB for local cache and
200 MB for global cache.

10

The OLAP Memory Cache Size Utilization

A global cache size of only 200 MB

(default setting) is too small. Most
companies increase this to 400 MB.

The settings can be changed in

RSCUSTV14.

Unfortunately, in this example someone

has decreased the global cache setting
from 200 MB to 136MB in the production
system, instead of increasing it

Real
Exa

mpl e

The decreased cache size may result in

lower hit-ratio, since the cache results gets
invalidated once the max size is filled.
11

Changing the Cache Size

The size of OLAP Cache is also
physically limited by the amount of
memory set in system parameter
rsdb/esm/buffersize_kb.
The settings are available in
RSPFPAR and RZ11.
Since the global cache size is a logical value and the Export/Import SHM gives a physical limit, and
also considering that other applications (such as BCS) might use the Export/Import SHM, SAP
recommend to set the global cache parameter maximally to 90% of the Export/Import SHM buffer.
12

Example: Query Performance of most used InfoCubes

Real
Exa

mpl e

While many SD query executions are very fast (13-14 seconds), other reports in
Finance and Inventory Management is very slow
13

Example: Query Performance 4,300 reports per month

Real
Exam
p le

Of the total queries executed in the system in April-May 2015, 4,075 were from end-users.
The average query run-time took 39.7 seconds, while some reports took 8-9 minutes to run.
Of the slowest interface, the reports in BEx Web Java was executed 2,809 times and took an
average of 45 seconds to execute. In an HANA environment, the same query would have
completed in an estimated time of 7 second.
Installing HANA could result in 426 hours less waiting for
reports by the business users each year

14

Another Example: Query Performance - Details

1,095 query executions took over 69.9 seconds to execute.
The daily inventory report, which is run 7 times each workday takes 2.5 min.

Real
Exam
p le

15

Temporary Tables and Inconsistencies

There are several temp tables that can be obsolete in the older systems (mostly from
previous jobs and internal reports run).
This could be cleaned up using the program SAP_DROP_TMPTABLES when users and
batch jobs are not running on the system (note: this program will remove all temp tables
except temp hierarchy tables and may stop jobs in-progress). It is therefore
recommended that this is done during a weekend outage.
Once this is done, you could consider running SAP_UPDATE_DBDIFF to clean all
obsolete temporary entries. These two efforts can reduce the BW system size and also
provide for a cleaner environment.
Details on how to remove the temporary tables and entries
in can be found in note number 1139396

16

BI Content Consistency Checks (Optional)

If you suspect inconsistency in the

BI content, or are planning to
deploy new BI content, you can run
the BI Content Analyzer. This can
be as a transparent table or loaded to a DSO (tcode RSBICA)

The automated BI Content Analyzer Checks include:

Inactive Transfer Structure checks
List of InfoObjects without an InfoObject Catalog
Inconsistent Roles check
Routines that refer to fixed, programmed structures
Query Elements with Duplicate GUIDs
Several Object Collection Errors
Several Object Status checks
Many checks for Inconsistent Naming Conventions

To help plan any

testing, get a list of
where the objects in
the system are used
(SAP Note: 28022)

17

SAP HANA and SAP BW 7.4

For BW 7.4 on HANA, SAP has continued to move more of the process intensive functions
from the application to the DB server

This takes advantage of the performance improvements of an in-memory DB

It also reduces the need for data transfers between application and DB server
The benefits of this approach are dramatically faster data activation, data
transformations, and query executions

18

Optimized Transformations and Data Activation

Activating data for reporting against
the BW NIRV table was a very slow
process in the older BW systems. With
some number range buffering (BW 7.0)
and package fetch (BW 7.3) this was
somewhat improved, but it was still not
extremely fast
To speed up activation in BW 7.3 on
HANA, and transformations in BW 7.4
on HANA, more of this was moved to
the database layer
Currently, transformations include
conversions, masterdata reads,
formulas, data mapping, and SQL

19

What Well Cover

SAP BW 7.3 performance basics

Housekeeping tasks in SAP BW 7.3 and 7.4
Demo: Optimal SAP BW 7.4 on HANA performance
12-step pre-HANA migration cleanup tasks
Demo: SAP BW 7.4 performance monitoring
BW optimization after HANA migration
LSA vs. LSA++ optimization and SPOs
Wrap-up

20

Pre-Steps Cleaning Up Your BW System

You can save significant amounts of work by doing a cleanup effort

before you start your HANA migration or BW upgrade project

For example, an international company had a BW system with over 108TB, with only
38TB in the production box and the remaining data on their Near-Line Storage (NLS)
solution

This cleaned BW system saved them potentially millions of dollars in hardware and
HANA licensing costs
It is not unusual to reduce a BW system size by
20-30% during a cleanup effort

21

The SAP_BW_HOUSEKEEPING Task List

1.
2.
3.
4.
5.
6.

If you are on 7.0 SP32 or higher, you can generate an SAP BW Housekeeping task list and get automated help in
cleaning the system weeks before upgrading it

Checks BW metadata with DDIC

Deletes RSTT traces
Deletes BW statistical data
Deletes aggregate data via deactivation
Ensures partitioned tables are correctly indexed for PSA
Ensures request consistencies in the PSA

7.
8.
9.
10.
11.
12.

Re-assigns requests written into the incorrect PSA partition

Verifies DataSource segment assignment to PSA
Deletes the entries no longer required in table RSIXW
Clears all OLAP Cache parameters
Repairs InfoCube fact table indices at Data Dictionary level
Reorganizes and deletes bookmark IDs and view IDs

You first have to install the program from SAP Note 1829728 before you can
generate the SAP_BW_HOUSEKEEPING task list using tcode STC01

22

A Tool to Help Migrate and Clean Up

SAP has created a cockpit to:

Clean up the SAP BW system
Reduce system size
Conduct pre-checks (readiness
checks)
Size the system
Find sub-optimal code (i.e.,
transformations)
Look at table distributions and
loads
There are over 235 tests in this tool
These tools are thanks to SAPs Marc Bernard and his team at
SAP Labs Canada
23

More Tips to Make the Database Smaller

Use write-optimized DSOs as first level data stores. These can

easily be off-loaded out of main memory in HANA and save you money.

Keep your Persistent Staging Tables (PSA) clean. BTW: The PSA is often not needed at all in BW
7.4.

If you are on BW 7.3 Service Pack 8 and HANA with at least Service Pack 5, the write-optimized
DSOs and PSAs are flagged as early unload from the HANA memory. This will help you keep
the system smaller and require less memory.

In HANA 1.0 SP-10 we got dynamic tiring that automatically manages objects in memory/disk

You can also flag other InfoCubes, DSOs, tables, and partition them
as not active. If you do so, they will only be loaded into memory when actually required.
The sizing program in SAP Note 1736976 takes these size savings
settings into account when sizing your HANA system

24

What Well Cover

SAP BW 7.3 performance basics

Housekeeping tasks in SAP BW 7.3 and 7.4
Demo: Optimal SAP BW 7.4 on HANA performance
12-step pre-HANA migration cleanup tasks
Demo: SAP BW 7.4 performance monitoring
BW optimization after HANA migration
LSA vs. LSA++ optimization and SPOs
Wrap-up

25

Demo: Optimal SAP BW on HANA Performance

26

What Well Cover

SAP BW 7.3 performance basics

Housekeeping tasks in SAP BW 7.3 and 7.4
Demo: Optimal SAP BW 7.4 on HANA performance
12-step pre-HANA migration cleanup tasks
Demo: SAP BW 7.4 performance monitoring
BW optimization after HANA migration
LSA vs. LSA++ optimization and SPOs
Wrap-up

27

12 Pre-Steps Cleaning Up Your BW System

1.
2.
3.
4.

5.

6.

Clean the Persistent Staging Area (PSA) for data already loaded to DSOs.
Delete the Aggregates (summary tables). They will not be needed again.
Compress the E and F tables in all InfoCubes. This will make InfoCubes much smaller.
Remove data from the statistical cubes (they start with the technical name of 0CTC_xxx).
These contain performance information for the BW system running on the relational
database. You can do this using the transaction RSDDSTAT or the program
RSDDSTAT_DATA_DELETE to help you.
Look at the log files, bookmarks, and unused BEx queries and templates (transaction
RSZDELETE).
Remove as much as possible of the DTP temporary storage, DTP error logs, and
temporary database objects. Help and programs to do this are found in SAP Notes
1139396 and 1106393.
28

12 Pre-Steps Cleaning Up Your BW System (cont.)

7.

For write-optimized DSOs that push data to reportable DSOs (LSA approach),
remove data in the write-optimized DSOs. It is already available in higher-level
objects.

8.

Migrate old data to Near-Line Storage (NLS) on a small server. This will still provide
access to the data for the few users who infrequently need to see this old data. You will
also be able to query it when BW is on HANA, but it does not need to be in-memory.

9.

Remove data in unused DSOs, InfoCubes, and files used for staging in the BW system.
This includes possible reorganization of master data text and attributes using process
type in RSPC.

29

12 Pre-Steps Cleaning Up Your BW System (cont.)

10.

You may also want to clean up background information stored in the table RSBATCHDATA. This
table can get very big if not managed. You should also consider archiving any IDocs and clean the
tRFC queues. All of this will reduce the size of the HANA system and help you fit the system
tables on the master node.

11.

In SAP Note 706478, SAP provides some ideas on how to keep the Basis tables from growing too
fast in the future; if you are on Service Pack 23 on BW 7.0 or higher, you can also delete unwanted
master data directly (see SAP Note 1370848).

12.

Finally, you can use the program RSDDCVER_DIM_UNUSED to delete any unused dimension
entries in your InfoCubes to reduce the overall system size.

30

Deep-Dive: Cleaning Up Your BW System

31

What Well Cover

SAP BW 7.3 performance basics

Housekeeping tasks in SAP BW 7.3 and 7.4
Demo: Optimal SAP BW 7.4 on HANA performance
12-step pre-HANA migration cleanup tasks
Demo: SAP BW 7.4 performance monitoring
BW optimization after HANA migration
LSA vs. LSA++ optimization and SPOs
Wrap-up

32

Demo: SAP BW 7.4 Performance Monitoring

In this demo we will explore the BW 7.4 on HANA DBA Cockpit Features
33

Demo: SAP BW 7.4 Performance Monitoring (cont.)

34

What Well Cover

SAP BW 7.3 performance basics

Housekeeping tasks in SAP BW 7.3 and 7.4
Demo: Optimal SAP BW 7.4 on HANA performance
12-step pre-HANA migration cleanup tasks
Demo: SAP BW 7.4 performance monitoring
BW optimization after HANA migration
LSA vs. LSA++ optimization and SPOs
Wrap-up

35

Converting InfoProviders and/or Data Flows

While not required, InfoCubes can be optimized further

for HANA performance
This basically means flattening the data structures
and removing the dimensions in BW from the physical
layer (they still look as if they exist)

Many refer to this optional step as a functional migration and do this after the HANA migration has been completed,
often as a separate initiative (see SAP Note 1849497)

36

HANA Optimized BW 7.4 DSOs and Performance Improvements

BW optimized DSOs are now created by default in HANA

This
The

means that data activations are done much faster at the HANA database layer

change log is kept in a calculation view resulting in smaller DSOs

HANA optimized DSOs are also available for BW 7.3, but now they are created by
default, so do not convert DSOs to HANA-optimized. Fast activation is available for
all standard DSOs without conversion to HANA-optimized.
37

Converting InfoProviders and/or Data Flows

To help you, the SAP Migration Cockpit also
allows you to migrate your data flows from 3.x to
Data Transfer Processes (DTPs) as used in
versions 7.0 and higher
If you convert the data flows, you get better
automated data package DTP optimization, which
loads data faster into HANA

You can also simulate the data flow before you do the real
conversion. When doing so, data is loaded for both versions (3.x
and 7.x) of the dataflows and the results are stored in cluster
tables. The data is then compared to verify that the dataflow after
migration calculates the same data as it did before migration.
Since the differences are displayed separately, you can analyze the
results and changes in details

38

What Well Cover

SAP BW 7.3 performance basics

Housekeeping tasks in SAP BW 7.3 and 7.4
Demo: Optimal SAP BW 7.4 on HANA performance
12-step pre-HANA migration cleanup tasks
Demo: SAP BW 7.4 performance monitoring
BW optimization after HANA migration
LSA vs. LSA++ optimization and SPOs
Wrap-up

39

EDW Design vs. Evolution

An organization has two fundamental choices:
1. Build a new well architected EDW
2. Evolve the old EDW or reporting system

Both solutions are feasible, but organizations that select an

evolutionary approach should be self-aware and monitor
undesirable add-ons and workarounds

Failure to break with the past can be detrimental to an

EDWs long-term
success

40

Data Design The Use of Layered Scalable

Architecture (LSA)

Since SAP BW 7.3 SP3 we have had a

set of 10 templates to help build a
layered data architecture for largescale data warehousing

The LSA consists logically of:

Acquisition layer
Harmonization/quality layer
Propagation layer
Business transformation layer
Reporting layer
Virtualization layer
41

Example: Current LSA Data Architecture in SAP BW

ERP

BW

DATA
ACQUISITION

CORPORATE
MEMORY

Germany

DATA
PROPAGATION

Germany

BUSINESS
TRANS.

FLEXIBLE
REPORTING

Germany

Germany

DIMENSIONAL
REPORTING

Germany

6 LSA Layers
Europe
(excl.Germany)

Europe
(excl.Germany)

Europe
(excl.Germany)

Europe
(excl. Germany)
Europe
(excl. Germany)

Europe 2

Europe 2

Europe 2

Europe 2

ERP Table

Europe 2
Europe 3

Data Source
41 total
Transferobjects
Rule
Info Source

Europe 3

Europe 3

Europe 3

Data
Acquisition

Europe 3
USA

USA

USA

USA
USA

Americas 1

Americas 1

Americas 1

Americas 1
Americas 1

8 semantic partitions

Americas 2

Americas 2

Americas 2

Americas 2
Americas 2

Asia

Asia

Asia

Asia
Asia

42

Example: Simplified LSA++ Data Architecture

ERP

BW

DATA
ACQUISITION

CORPORATE
MEMORY

DATA
PROPAGATION

BUSINESS
TRANS.

FLEXIBLE
REPORTING

DIMENSIONAL
REPORTING
Remove
3 LSA layers

ERP Table

Europe

Europe

Europe

Americas

Americas

Americas

Asia

Asia

Data Source
Transfer Rule
Info Source

Asia

41 shrinks to 9
total objects

Remove 5 semantic
partitions

43

EDW Complex Layered Architectures

This BPC on BW system was experiencing substantial load

performance issues
Some of this was due to underlying SAP BW configuration, while
some was due to the technical configuration of the data store
architecture and data flow inside SAP BW

2) Long latency with 6 layers of PSA, DSOs,

and InfoCubes before consolidation
processes can be executed.

BPC Staging Cube

(BPC_C01)

Real

Exam

p le

GL Summary Cube
(FIGL_C03)

Production Issues included:

1) Dependent jobs not running sequentially,
i.e., load from Summary cube to Staging
cube is sometimes executed before the
Summary cube data is loaded and
activated, resulting in zero records in the
Staging cube.

Consolidation Cube
(OC_CON)

Consolidation Processes:
1) Clearing
2) Load
3) Foreign Exchange
4) Eliminations
5) Optimizations

Conformed
Reportable
DSO

FIGL_D21

FIGL_D20

FIGL_D17

FIGL_D14

FIGL_D18

Write
Optimized
DSO

FIGL_D15S

FIGL_D13S

FIGL_D10S

FIGL_D08

FIGL_D11S

ECC 6.0
North-America

ECC 4.7
Latin-America

R/3 3.1i
EU

ECC 4.7
ASIA

Persistent Staging Area (PSA)

ECC 6.0
AsiaPacific

44

Fixes to Complex EDW Architecture

The fix to this system included removing the conformed DSO layer, with BEx flags for
DataStores that are never reported on
Also, the BPC Staging cube served
little practical purpose since the data is
Consolidation Processes:
1) Clearing
2) Load
already staged in the GL Summary cube
3) Foreign Exchange
4) Eliminations
and the logic can be maintained in the
5) Optimizations
load from this cube directly to the
Real
consolidation cube
Exam
p le
Consolidation Cube
(OC_CON)

GL Summary Cube
(FIGL_C03)

Long-term benefits included

reduced data latency, faster data
activation, less data replication,
and smaller system backups as
well as simplified system
maintenance.

Write
Optimized
DSO

FIGL_D15S

FIGL_D13S

FIGL_D10S

FIGL_D08

FIGL_D11S

Persistent Staging Area (PSA)

ECC 6.0
AsiaPacific

ECC 6.0
North-America

ECC 4.7
Latin-America

R/3 3.1i
EU

ECC 4.7
ASIA

45

EDW Data Design Classical Use of MultiProvider Hints in BW

Problem: To reduce data volume in each InfoCube,
data is partitioned by Time period.
A query must now search in all InfoProviders to find
the data. This is very slow.
Solution: We can add hints to guide the query execution. In the RRKMULTIPROVHINT table, you can
specify one or several characteristics for each MultiProvider, which are then used to partition
the MultiProvider into BasicCubes.

If a query has restrictions on this characteristic, the OLAP processor is already checked to see which
part of the cubes can return data for the query. The data manager can then completely ignore the
remaining cubes.
An entry in RRKMULTIPROVHINT only makes sense if a few attributes of this characteristic
(that is, only a few data slices) are affected in the majority of, or the most important, queries

(SAP Note 911939. See also 954889 and 1156681).

46

BW 7.3 and Higher (non-HANA) Semantic Partitioned Objects (SPO)

When

DataStores and InfoCubes are allowed to grow over time, the data load and query performance
suffers

Normally

objects should be physically partitioned when the numbers of records exceed 100 200 million

However,

this may be different depending on the size of your hardware and the type of database you use

In

SAP BW 7.3 we get an option to create a Semantic Partitioned Object (SPO) through wizards

You

can partition based on fields such as calendar year, region, country, etc.

47

Data Design Semantic Partitioned Objects

When an SPO is created, a reference structure keeps track of the partitions. The structure is
placed in the MultiProvider for querying.

SPO Wizards create all Data Transfer Processes (DTP), transformations,

filters for each data store, and a process chain
48

What Well Cover

SAP BW 7.3 performance basics

Housekeeping tasks in SAP BW 7.3 and 7.4
Demo: Optimal SAP BW 7.4 on HANA performance
12-step pre-HANA migration cleanup tasks
Demo: SAP BW 7.4 performance monitoring
BW optimization after HANA migration
LSA vs. LSA++ optimization and SPOs
Wrap-up

49

Where to Find More Information

Bjarne Berg and Penny Silvia, Introduction to SAP HANA (3rd Edition) (SAP PRESS, 2014).

http://scn.sap.com/docs/DOC-35096
Michaela Pastor, SAP Business Warehouse 7.4 (SCN, November 2014).

http://help.sap.com/nw_platform
SAP NetWeaver 7.4 on the SAP Help Portal

www.stechno.net/sap-notes.html?view=sapnote&id=153967
SAP BI content release note for BW 7.4

50

7 Key Points to Take Home

SAP BW 7.4 is the first release to take full advantage of SAP HANA
Some of the functions in 7.4 are also available to non-HANA customers
The new CompositeProviders and the Open ODS View make HANA and BW
tightly integrated and capable to support EDWs better
You should break from the past and start designing with the new BW 7.4
features in mind
The new monitoring features in the BW DBA Cockpit and the HANA systems
make it much easier to see what is occurring from a database level for the
non-basis team
Before you size your system, clean it up and save hardware costs
All customers should consider the BW move to HANA in 2016!
51

Your Turn!

How to contact me:

Dr. Berg
bberg@comerit.com
Please remember to complete your session evaluation
52

Disclaimer
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other
countries. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP SE.

53