SAP BW 7.

30 : Performance Improvements in Master-Data related scenarios and DTP Processing

Posted by Girish V Kulkarni

With Data Warehouses around the world growing rapidly every day, the ability of a Data Warehousing
solution to handle mass-data, thus allowing for the ever-shrinking time-windows for data loads is
fundamental to most systems.
BW 7.3 recognizes the need of the hour with several performance related features and in this blog, I
will discuss the performance features related to data loads in SAP BW 7.3, focusing mainly on Master
Data Loads and DTP Processing.
Here is the list of features discussed addressed in this blog -

Master Data
1. Mass Lookups during Master Data Loads
2. The Insert-Only flag for Master Data Loads.
3. The new Master Data Deletion
4. SID Handling
5. Use of Navigational Attributes as source fields in Transformations.
DTP Processing
1. Repackaging small packages into optimal sizes.
MASTER DATA
1. Mass Lookups during Master Data Load
Data loads into a Master Data bearing Characteristic require database look-ups to find out if records
exist on the database with the same key as the ones being loaded. In releases prior to SAP BW 7.3, this
operation was performed record-wise, i.e. for every record in the data-package, a SELECT was executed
on the database table(s). Obviously, this resulted in a lot of communication overhead between the SAP
Application Server and the Database Server, thereby slowing the Master Data loads down. The effect is
pronounced on data loads involving large data volumes.
The issue of overhead between the SAP Application Server and the Database Server has now been
addressed by performing a mass-lookup on the database so that all records in the data-package are
looked-up in one attempt. Depending on the DB platform it can bring up-to 50% gain in load runtimes.
2. The Insert-Only Flag for Master Data Loads
Starting NW 7.30 SP03, this flag will be renamed to New Records Only. The renaming has
been done to align with a similar feature supported by activation of DSO data. (See
blog Performance Improvements for DataStore Objects )
As mentioned above, the Master Data Load performs a look-up on the database for every data-package
to ascertain which key values already exist on the database. Based on this information, the Master Data
load executes UPDATEs (for records with the same key already existing in the table) or INSERTs (for
records that dont exist) on the database.
With the Insert-Only feature for Master Data loads using DTPs, users have the opportunity to
completely skip the look-up step, if it is already known that the data is being loaded for the first time.
Obviously, this feature is most relevant when performing initial Master Data loads. Nevertheless, this
flag can also be useful for some delta loads where it is known that the data being loaded is completely
new.
Lab tests for initial Master Data loads indicate around 20% reduction in runtime with this feature.
The Insert-Only setting for DTPs loading Master Data can be found in the DTP Maintenance screen
under the UPDATE tab as shown below.


Note :
If the Insert-Only flag is set, and data is found to exist on the database, the DTP request will
abort. To recover from this error, the user simply needs to uncheck the flag and re- execute the
DTP.
3. The New Master Data Deletion
Deleting MasterData in BW has always been a performance intensive operation. The reason being that
before any MasterData can be physically deleted, the entire system (Transaction Data, Master Data, and
Hierarchies etc) is scanned for usages. Therefore, if a lot of MasterData is to be deleted, it takes some
time to establish the data that is delete-able (i.e., has no usages) and data that is not (has usages). In
addition, with the classical MasterData Deletion involving large data volumes, users sometimes ran into
memory overflow dumps.
To address these issues, the Master Data Deletion was completely re-engineered. The result is the New
Master Data Deletion. In addition to being much faster than the classical version, the new Master
Data deletion offers interesting new features like Search-modes for the usage check, Simulation-mode
etc. The screen shot below shows the user interface for the new Masterdata Deletion when accessed
via the context menu of InfoObjects in the DataWarehousing Workbench.



Although the new Master Data Deletion has be available for some time now (since BW 7.00 SP 23), it
was never the default version in the system. This implied that the BW System Administrators needed to
switch it ON explicitly. With BW 7.30 however, the New Master Data Deletion is the default version and
no further customizing is necessary to use it.
All further information about this functionality is documented in the SAP note:1370848
underhttps://websmp130.sap-
ag.de/sap(bD1lbiZjPTAwMQ==)/bc/bsp/spn/sapnotes/index2.htm?numm=1370848
It can also be found in the standard SAP BW documentation
underhttp://help.sap.com/saphelp_nw73/helpdata/en/4a/373cc45e291c67e10000000a42189c/frames
et.htm
4. SID Handling
This feature relates to the handling of SIDs in the SAP BW system and while it is certainly relevant for
Master Data loads, it is not restricted to it. The performance improvements in SID handling are relevant
for all areas of SAP BW where SIDs are determined, for example Activation of DSO Requests, InfoCube
Loads, Hierarchy Loads and in some cases, even Query processing.
In BW 7.30, SIDs are determined en-masse meaning that database SELECTs and INSERTs that were done
record-wise previously have been changed to the mass SELECTs (using the ABAP SELECT FOR ALL
ENTRIES construct) and mass INSERTs. The system switches to this mass-data processing mode
automatically when the number of SIDs to be determined is greater than a threshold value. The default
value of this threshold is 500.
The threshold value is customizable of course and that can be done in the SAP IMG for customizing
under the transaction SPRO by following the path: SAP Netweaver -> Business Warehouse ->
Performance Settings -> Optimize SID-Determination for MPP-Databases.
Note: As the threshold value corresponds to the minimum number of SIDs
to be determined in one step, setting the threshold to a very high value
(For example: 100000) causes the system the system to switch back to the
classical behavior.
5. Use of Navigational Attributes as source fields in Transformations
Quite often there are scenarios in SAP BW where data being loaded from a source to a target needs to
be augmented with information that is looked up from Masterdata of Infoobjects. For instance - loading
sales data from a source that contains data on Material level to a DataTarget where queries require the
sales data to be aggregated by Material Group. In such cases, the Master Data Lookup rule-type in
Transformations is used to determine the Material Group for any given Material (given that
MaterialGroup is an attribute of Material).
Although the performance of the Masterdata Lookup rule-type has been optimized in earlier versions of
BW (starting BW 7.0), there is an alternative to this rule-type in BW 7.30. Now, navigational attributes of
Infoobjects are available as source fields in Transformations. The benefits of this feature are two-
pronged.
The fact that the data from the navigational attributes is available as part of the source structure
allows the data to be used in custom logic in Transformations (example : Start Routines).
Secondly, the data from the navigational attributes is read by performing database joins with
the corresponding Masterdata tables during extraction. This helps in improving the performance
of scenarios where a lot of look-ups are needed and/or a lot of data is to be looked-up.
To use this feature in Transformations, the navigational attributes need to be switched ON in the source
InfoProvider in the InfoProvider maintenance screen as below -



Once this is done, the selected navigational attributes are available as part of the source structure of
Transformations as shown below


DATA TRANSFER PROCESS (DTP)
1. Repackaging small packages into optimal sizes
This feature of the DTP is used to combine several data packages in a source object into one data
package for the DataTarget. This feature helps speed up request processing when the source object
contains a large number of very small data packages.
This is usually the case when memory limitations in the source systems (for example: an SAP ERP
system) results in very small data-packages in the PSA tables in BW. This DTP setting can be used to
propagate the data to subsequent layers in BW in larger chunks.
Also, InfoProviders in BW used for operational reporting using Real-time Data Acquisition contain very
small data packages. Typically, this data is propagated within the DataWarehouse into other
InfoProviders for strategic reporting. Such scenarios are also a use-case for this feature where data can
be propagated in larger packets.
As a prerequisite, the processing mode for the DTP needs to be set to Parallel Extraction and Parallel
Processing. Also note that only source packages belonging to the same request are grouped into one
target package.
Below is a screenshot of the feature in the DTP Maintenance.

BW 7.3: Troubleshooting Real-Time Data Acquisition


The main advantage of real-time data acquisition (RDA) is that new data is reflected in your BI reports
just a few minutes after being entered in your operational systems. RDA therefore supports your
business users to make their tactical decisions on a day-by-day basis. The drawback however is that
these business users notice much faster when one of their BI reports is not up to date. They might call
you then and ask why the document posted 5 minutes ago is not visible yet in reporting. And what do
you do now? Ill show you how BW 7.3 helps you to resolve problems with real-time data acquisition
faster than ever before.
First, lets have a look at what else is new to RDA in BW 7.3. The most powerful extension is definitely
the HybridProvider. By using RDA to transfer transactional data into a HybridProvider, you can easily
combine the low data latency of RDA with the fast response times of an InfoCube or a BWA index, even
for large amounts of data. Youll find more information about this combination in a separate blog.
Additionally. BW 7.3 allows for real-time master data acquisition. This means that you can transfer delta
records to InfoObject attributes and texts at a frequency of one per minute. And just like RDA directly
activates data transferred to a DataStore object, master data transferred to an InfoObject becomes
available for BI reporting immediately.
But now, lets start the RDA monitor and look at my examples for RDA troubleshooting. Ive chosen
some data flows from my BW 7.0 test content and added a HybridProvider and an InfoObject. I know
that this flight booking stuff is not really exciting, but the good thing is that I can break it without getting
calls from business users.

Remember that you can double-click on the objects in the first column to view details. You can look up
for example that Ive configured to stop RDA requests after 13 errors.

Everything looks fine. So lets start the RDA daemon. It will execute all the InfoPackages and DTPs
assigned to it at a frequency of one per minute. But wait whats this?

The system asks me whether Id like to start a repair process chain to transfer missing requests to one of
the data targets. Why? Ah, okay Ive added a DTP for the newly created HybridProvider but forgotten
to transfer the requests already loaded from the DataSource. Lets have a closer look at these repair
process chains while they are taking care of the missing requests.

On the left hand side, you can see the repair process chain for my HybridProvider. Besides the DTP, it
also contains a process to activate DataStore object data and a subchain generated by my
HybridProvider to transfer data into the InfoCube part. On the right hand side, you can see the repair
process chain for my airline attributes which contains an attribute change run. Fortunately, you dont
need to bother with these details the system is doing that for you. But now lets really start the RDA
daemon.

Green traffic lights appear in front of the InfoPackages and DTPs. I refresh the RDA monitor. Requests
appear and show a yellow status while they load new data package by package. The machine is running
and I can go and work on something else now.
About a day later, I start the RDA monitor again and get a shock. What has happened?

The traffic lights in front of the InfoPackages and DTPs have turned red. The RDA daemon is showing the
flash symbol which means that is has terminated. Dont panic! Its BW 7.3. The third column helps me to
get a quick overview: 42 errors have occurred under my daemon, 4 DTPs have encountered serious
problems (red LEDs), and 4 InfoPackages have encountered tolerable errors (yellow LEDs). I double-click
on 42 to get more details.

Here you can see in one table which objects ran into which problem at what time. I recognize at a glance
that 4 InfoPackages repeatedly failed to open an RFC connection at around 16:00. The root cause is
probably the same, and the timestamps hopefully indicate that it has already been removed (No more
RFC issues after 16:07). I cannot find a similar pattern for the DTP errors. This indicates different root
causes. Finally, I can see that the two most recent runtime errors were not caught and thus the RDA
daemon has terminated. You can scroll to the right to get more context information regarding the
background job, the request, the data package, and the number of records in the request.

Lets have a short break to draw a comparison. What would you do in BW 7.0? 1) You could double-click
on a failed request to analyze it. This is still the best option to analyze the red DTP requests in our
example. But you could not find the tolerable RFC problems and runtime errors.

2) You could browse through the job overview and the job logs. This would have been the preferable
approach to investigate the runtime errors in our example. The job durations and the timestamps in the
job log also provide a good basis to locate performance issues, for example in transformations.

3) You could browse through the application logs. These contain more details than the job logs. The
drawback however is that the application log is lost if runtime errors occur.

These three options are still available in BW 7.3 they have even been improved. In particular, the job
and application logs have been reduced to the essential messages. Locating a problem is still a
cumbersome task however if you dont know when it occurred. The integrated error overview in the
RDA monitor, BW 7.3 allows you to analyze any problem with the preferred tool. Let me show you some
examples.
Unless you have other priorities from your business users Id suggest starting with runtime errors
because they affect all objects assigned to the daemon. RDA background jobs are scheduled with a
period of 15 minutes to make them robust against runtime errors. In our example, this means the RDA
daemon serves all DataSources from the one with the lowest error counter up to the one which causes
the runtime. The job is then terminated and restarted 15 minutes later. The actual frequency is thus
reduced from 60/h to 4/h, which is not real-time anymore. Lets see what we can do here. Ill double-
click on 10 in the error column for the request where the problem has occurred.

I just double-click on the error message in the overview to analyze the short dump.


Puh This sounds like sabotage! How can I preserve the other process objects assigned to the same
daemon from this runtime error while I search for the root cause? I could just wait another hour of
course. This RDA request will then probably have reached the limit of 13 errors that I configured with
the InfoPackage. Once this threshold is reached, the RDA daemon will exclude this InfoPackage from
execution. The smarter alternative is to temporarily stop the upload and delete the assignment to the
daemon.

The overall situation becomes less serious once the DataSource has been isolated under Unassigned
Nodes. The daemon continues at a frequency of onc per minute although there are still 32 errors left.

Note that most of these errors namely the RFC failures can be tolerated. This means that these
errors (yellow LEDs) do not hinder InfoPackages or DTPs until the configured error limit is reached.
Assume that Ive identified the root cause for the RFC failures as a temporary issue. I should then reset
the error counter for all objects that have not encountered other problems. This function is available in
the menu and context menu. The error counter of an InfoPackage or DTP is reset automatically when a
new request is created. Now lets look at one of the serious problems. Ill therefore double-click on 2
in the error column of the first DTP with red LED.

When I double-click on the error message, I see the exception stack unpacked. Unfortunately that does
not tell me more than I already knew: An exception has occurred in a sub step of the DTP. So I navigate
to the DTP monitor by double-clicking the request ID (217).


Obviously, one of the transformation rules contains a routine that has raised the exception 13 is an
unlucky number. I navigate to the transformation and identify the root cause quickly.

In the same way, I investigate the exception which has occurred in DTP request 219. The DTP monitor
tells me that something is wrong with a transferred fiscal period. A closer look at the transformation
reveals a bug in the rule for the fiscal year variant. Before I can fix the broken rules, I need to remove the
assignment of the DataSource to the daemon. When the corrections are done, I schedule the repair
process chains to repeat the DTP requests with the fixed transformations. Finally I re-assign the
DataSource to the daemon.
The RDA monitor already looks much greener now. Only one DataSource with errors is left. More
precisely, there are two DTPs assigned to this DataSource which encountered intolerable errors, so the
request status is red. Again, I double-click in the error column to view details.

The error message tells me straight away that the update command has caused the problem this time
rather than the transformation. Again, the DTP monitor provides insight into the problem.

Of course GCS is not a valid currency (Should that be Galactic Credit Standard or what?). I go back to
the RDA monitor and double-click on the PSA of the DataSource in the second column. In the request
overview, I mark the source request of the failed DTP request and view the content of the problematic
data package number 000006.

Obviously, the data is already wrong in the DataSource. How could this happen? Ah, okay Its an
InfoPackage for Web Service (Push). Probably the source is not an SAP system, and a data cleansing step
is needed either in the source system or in the transformation. As a short-term solution, I could delete
or modify the inconsistent records and repeat the failed DTP requests with the repair process chain.
Thats all. I hope you enjoyed this little trip to troubleshooting real-time data acquisition, even though
this is probably not part of your daily work yet. Let me summarize what to do if problems occur with
RDA. Dont panic. BW 7.3 helps you to identify and resolve problems faster than ever before. Check the
error column in the RDA monitor to get a quick overview. Double-click wherever you are to get more
details. Use the repair process chains to repeat broken DTP requests.
Disclaimer

The new SAP NetWeaver BW 7.30 hierarchy framework


Posted by Serge Daniel Knapp

Introduction
If you remember older releases of SAP NetWeaver BW hierarchies could only be loaded through the old
3.x data flow. In this case you needed the so called direct update functionality of the corresponding
InfoSource for uploading the hierarchy. This InfoSource 3.x was connected to an 3.x DataSource through
update rules.
Limitations of 3.x data flow for hierarchies
This data flow has to be used in SAP NetWeaver BW 7.x, too, and could not be migrated to the new data
flow. Consequently you always had to deal with two types of data flows in your system. Besides the
heterogeneous aspect the 3.x data flow for hierarchies had a lot of disadvantages:
First, hierarchy DataSources were available only for flatfile and SAP source systems. Besides, end
users could only create own hierarchy DataSources for the flat file system.
Second you could not take full advantage of the new data flow, even some old data flow
features (e.g. the start routine) could not be used. Furthermore, to change the structure of
hierarchies during runtime you had to implement complex scenarios (e.g. with the help of the
analysis process designer APD). The direct update functionality didn't allow you to load the
hierarchy to a DSO or an other arbitrary object and manipulate it according to the end users'
needs.
Third, monitoring was often unclear because the framework was not optimal for segments.
The new BW 7.30 hierarchy framework
With SAP NetWeaver BW 7.30 the hierarchy framework has been improved, you could now use the 7.x
data flow with all its advantages.
First you are able to use any BW object as source for a hierarchy, you are not limited to a
DataSource for hierarchies. This leads to simpler scenarios if you want to transform your
hierarchy according to your needs. You just have to connect your hierarchy through a
transformation and a data transfer process.
Within this transformation you are able to use all features of a transformation, for example
start, end or expert routines. You are not limited as you were in the 3.x data flow.
You can use any DataSource as a source for your hierarchy, you are not restricted to hierarchy
DataSources any more. This makes hierarchy extraction of SAP source systems possible, too.
Last but not least you are now able to take full advantage of all capabilities of the new data flow.
You can distribute the data loaded from one DataSource to several hierarchies and you can use
an arbitrary number of InfoSources in between the DataSource and your hierarchy. A very useful
feature is the automatic filling of the fields CHILDID, NEXTID and LEVEL through the framework if
they are not filled by the source (e.g. if only the PARENTID is provided).
New DataSource structure
If you are familiar with the old hierarchy framework you will notice a new segmentation format for the
hierarchy structure. Let's have a look at the old structure which is shown in the figure below on the left
hand side.

The hierarchy structure contained of the five fields NODEID, IOBJNM, NODENAME, LINK and PARENTID.
The NODEID was the internal number for your node, the field IOBJNM contained the InfoObject name
for your node. The NODENAME contained the value in compounded format (if compounded InfoObjects
were used). This is the case if you use e.g. cost center hierarchies which are compounded to the
controlling area.
The new framework now uses more fields to describe the hierarchy structure. Whereas the first five
fields are the same, the new structure now contains fields for every InfoObject of a compounded
structure. For example, if you use the cost center which is compounded to the controlling area, the new
structure contains both InfoObjects.
New rule type "hierarchy split"
In SAP NetWeaver BW 7.30 both structures are available, the old one and the new one. Please be aware
that most of the DataSources of the Business Content have not been migrated to the new structure.
That could lead to a problem because you have to map the old DataSource structure to the new
structure within the hierarchy. To avoid the problem SAP has introduced a new rule type which
automatically maps the old structure to the new one (see figure).

This new rule type is called "hierarchy split". To use the new rule type you have to map the field
NODENAME to every InfoObject of your structure, in this case the controlling area and the cost center.
The hierarchy split automatically transforms the value of NODENAME to the corresponding values of
controlling area and cost center.
Please remember that this functionality only works if you haven't changed the length of the
corresponding InfoObjects. For example, if you changed the length of the controlling area to 5 digits
instead of 4 you cannot use this feature.
Creating a new DataSource for hierarchies
If you want to create a new DataSource for hierarchies just navigate to transaction RSA1 and open a
DataSource tree of a source system, for example a flat file source system. Right-click on a display
component and create a new DataSource for hierarchies:

After choosing the correct type of DataSource you can edit the properties of your DataSource. The main
properties can be found in the tab "extraction" which is shown in the following figure.

In the top of the screen you can provide information on full/delta loading and the direct access, both
you will know of other DataSources. The interesting parts are highlighted. If you select "hierarchy is
flexible", the DataSource will be created in the new structure, if you leave it blank the DataSource
will consist of the old structure. Second you are now able to create the hierarchy header directly from
the DataSource. In older releases you had to maintain the hierarchy header in the InfoPackage. I think
the other parameters are known, if not just feel free to ask me.
Creating transformation and DTP
After activating your DataSource you can create a transformation between your new DataSource and
the hierarchy of an InfoObject. The following picture shows how this is displayed in the system.

Within the transformation you have to map the corresponding segments to each other. The interesting
one is the mapping of source segment "hierarchy structure" to the correspondent target segment.

You can see that there are a lot of 1:1 rules within the transformation except the mapping of field
NODENAME to 0COSTCENTER and 0CO_AREA. Here, the new rule type "hierarchy split" has been set.
Maybe you have noticed that the key field 0OBJECTID has not been mapped. This field will be used to
load more than one hierarchy to have a unique key. Please be aware that in this case each data package
has to contain exactly one complete hierarchy.
Now let's have a look at the corresponding data transfer process (DTP). There are two major changes
when loading data through a hierarchy DataSource:
1. If you select full extraction mode in the DTP there's a new flag called "only retrieve last request".
This flag is helpful if you don't want to clear your PSA before loading the new hierarchy. Only the
newest request in the PSA will be selected and loaded.
2. In the update tab of your DTP (see figure) you can decide how to update your hierarchy (full
update, update subtree, insert subtree). Furthermore you can activate the hierarchy from within
the DTP, no separate process step in a process chain is necessary to activate the hierarchy!

Now you are able to load a hierarchy from a hierarchy DataSource to an InfoObject.



BW 7.30: Simple supervision of process chains

Posted by Thomas Rinneberg
There has been some moaning about the built-in capabilities to monitor BW process chains. Clicking one
chain after the other to see recent status is just too much effort and transaction RSPCM listing the last
execution is lacking supervision with regards to timely execution. In fact for those of you who do not
want to use the administrator cockpit, there is no time supervision at all (except the insider tip of
transaction ST13 which however is not contained in standard, but an add-on by SAP active global
support).
However with release 7.30 BW development has finally catched up let me show you!
Lets start as always with data warehousing workbench RSA1:

First change catching the eye is a new folder Process Chains in the Modelling View. Seems a small
change, but there comes something important with it search.



At last you are able to search for process chains by name directly in the workbench. Plus, you might
have noticed: The display components are ordered hierarchical like any other folder in RSA1. But not
only the display components, but also the chains itself have a hierarchical structure, displaying the
metachain subchain relationship. So you can directly navigate to the lowest subchain in your
metachain hierarchy if you only remember the name of the top-level master.
But this is not what we were looking for, we spoke about monitoring. So lets go to the Administration
area of RSA1

Ok, same tree, but that does not really make a difference. Lets look at the Monitors.

And choose Process
Chains

If RSPCM was used in this system before upgrade, I will be asked the first question, else the second one.
What does it mean? You might know, that RSPCM requires you to choose the chains, which shall be
monitored. So I understand the second question, this will simplify the setup. But what about the first
question? Well, RSPCM became user-dependent with 7.30. Each user has his own unique selection of
process chains to monitor.
Good thing for most users. But stop what if I have a central monitoring team where all administrators
shall look at the same list? Does each of them need to pick his or her chains himself? No:

Besides the known possibilities to add a single chain resp. all chains of a display component there is the
option to Select Include.

And how do I create an include? I press button Edit Include.


And then I am on the ususal RSPCM list and can add chains. But now, how does the new RSPCM list look
like? Here it is:

The first five columns look pretty familiar. But there is something hidden there as well! Lets click on one
of the red chains:

The system is extracting the error messages from the single processes of the failed process chain run
and directly displays them together with the corresponding message long text. So no more searching for
the red process and navigating and searching the log. One click and you know whats wrong (hopefully ;-
). If not, you still can go to the log view of the process chain by the Log Button on the popup.
And look the button alongside, called Repair, looks interesting. No more error analysis, just press
repair any failure. THAT was what you were looking for all the years, isnt it? Sorry, it is not that simple.
The button will repair or repeat any failed process in the chain run like you could do already before via
context menu. And whether this is a good thing to try still depends on the particular error. But you
should be a little faster now finding it out and doing it.
What about the other two buttons? Lets delay this a little and first go back to the overview table. There
are some new columns:

First, there is a column to switch on and off the Time Monitoring because you might not be interested
in the run time or start delay of any chain in RSPCM, especially considering that you can still schedule
RSPCM to batch to perform an automatic monitoring of your chains and alert you in case of red or
yellow lights. Considering performance, it does not make much of a difference, because the run time
calculations of previous chains are buffered in an extra table. After all, the performance of RSPCM also
greatly improved by following setting:


You can choose to refresh only those chains, which have yellow status or not refresh the status at all but
display the persisted status from the database. For the red and green runs, it is expected, that the status
does not change frequently. If you nevertheless expect a change of such chains, you can also force the
status refresh in the transaction itself by pressing the Refresh All button in the screenshot above.Now,
how come the system judges a run time of 48 seconds Too Long? Did somebody set a maximal run
time for each chain? Let us click on the red icon.


Seems like this chain runs usually not longer than about 35 seconds. Considering this, 48 seconds is too
long. More specifically, the chain takes usually 25.5 plusminus 3.9 seconds. The 48 seconds are about
22.5 seconds more than usual. Especially, these 22.5 seconds are much more than the usual deviation of
3.9 seconds.
So, the system uses some simple statistics to judge on the run time of a chain run. If you like, I can also
give you the mathematical formulas for the statistics, but I guess this is not so interesting for most of the
readers ;-)
To stabilize this statistics, the system even removes outliers before calculating the average run time and
average deviation. If you are interested in the filtered outliers, you can press the Outlier button:

This indeed is an outlier, dont you think? Lets get back in time a little bit and look at older runs of this
chain by pressing the + button twice, which will add twenty additional past values.


Now our outliner is not the biggest anymore. Even more interesting, the chain seemed to stabilize its
runtime over the past weeks. Considering the previous values as well, our current run is no longer
extraordinary long. Is it a false alarm then? No, because if the run time turns back to former instabilities,
you would like to get notified. If that instable behaviour however becomes usual again, the system will
automatically stop to consider this run time extraordinary and thus stop alerting you, because it always
considers the last 30 successfull executions for calculating the statistics for the alerts.
So let us check, what process made this current run longer. We press button Display Processes.

Of course a DTP. You would not have expected something different, would you? I now could click on the
DTP load to open the DTP request log and start analyzing what went wrong, but at this point rather lets
go back to the RSPCM overview list and look at the Delay column.

Today it is the 14.5.2010, so an expected start on 27.4.2010 indeed I would judge heavily delayed. But
how does the system come to the conclusion that the chain should start on 27.4. (plusminus six and a
half days)? The answer comes when clicking on the LED.


Obviously this chain was executed only five times in the past, and it had been started once a week (in
average). Seems like the responsible developer lost interest in this chain meanwhile. If he continues not
to start the chain, the system will stop complaining about the delay and turn the icon grey. Now it is
important to note, that this chain is started either immediately by hand or by event. If it would be
scheduled time periodic in batch, the delay is not calculated in this statistical manner, but the delay of
the TRIGGER job is observed and the alert is raised after 10 minutes of delay.
And now also the question about the two buttons in the status popup is answered: They open up the
shown run time and start time popups.
I hope you have fun using these new functions and appreciate that it is not neccessary to laboriously
customize any thresholds in order to get meaningful alerts.

Performance Improvements for DataStore Objects


Posted by Klaus Kuehnle

A lot of time and effort has been invested in SAP BW 7.30 to improve the performance of operations on
DataStore Objects, such as request activation. The most important improvements are:
1. database partitioning for DataStore Objects
2. mass lookups in activation of requests in DataStore Objects
3. dynamic flag unique data records
4. faster request activation in DataStore Objects on databases with massively parallel processing
architecture
5. lookup into DataStore Objects in transformation
The following sections describe these points in more detail.
(1) Database Partitioning
The E fact tables of InfoCubes can be partitioned, by month or fiscal period, on databases that support
range partitioning (for more details, see here). In SAP BW 7.30, this is now also possible for the active
tables of standard DataStore Objects. More precisely, if a standard DataStore Object has at least one key
field referring to InfoObject 0CALMONTH or InfoObject 0FISCPER, then the active table of this DataStore
Object can be partitioned by one of these key fields. The advantage of this partitioning is faster access to
data due to partition pruning, provided that the partition criterion is part of the selection criterion
(where clause).
To activate partitioning, go to the menu and choose Extras DB Performance DB Partitioning (in
edit mode of a standard DataStore Object).

A popup appears where you can select a key field that refers to the InfoObject 0CALMONTH or
InfoObject 0FISCPER (you cannot select anything if there is no field referring to one of these
InfoObjects).

Any changes made here will become effective after activating the DataStore Object.
Note that you can only change the partitioning settings if the DataStore Object is empty. This means that
you should decide whether you want a DataStore Object to be partitioned before you load any data into
it.
(2) Mass Lookups
Requests are activated in a standard DataStore Object by splitting the data in the activation queue into
packages. These packages are then activated independently of each other (and usually simultaneously).
For each data record of the activation queue, the system performs a lookup into the active table to find
out whether a record already exists in the active table with the same key (i.e. whether the activation will
cause an insert or an update in the active table).
In SAP BW 7.30, the lookup is no longer performed record by record, but for all records in a package in
one go. This decreases the activation runtime by 10-30%, depending on the database.
(3) Unique Data Records Can be Set Dynamically
The last section was about lookups into the active table as part of the activation. This lookup is
performed to find out whether the active table already contains records with the same key as a record
that you want to activate. However, if the active table does not contain any records with the same key
as one of the records that you want to activate (for example, if data for a new year or a new region is
activated), then these lookups can be omitted. This reduces the runtime of the activation, particularly if
the active table contains a lot of records.
You can set the flag Unique Data Records in the settings of a standard DataStore Object to guarantee
that no record in the activation queue will ever have the same key as any record in the active table
during activation. The system will then omit the lookup into the active table. Since this setting is very
restrictive, there will probably not be many occasions where you can risk using this flag.
In SAP BW 7.30, you can set this option for one single activation request only. In other words, you can
specify that for the current activation request, none of the keys in the activation queue occur in the
active table, without making any statement on other activation requests. The system then will omit the
lookup for this activation request only.
To select this option, click Change... next to the text Use DataStore Setting (Unique and Changed Data
Records) in the popup where you choose the requests to be activated.

Another popup appears. Choose New, unique data records only.

Confirm the dialog. The system will now run the activation without lookup.
For activation processes scheduled as part of a process chain, you also have the option to force the
system to omit the lookup for init requests. Choose Change... next to the text Uniqueness of Data:
Use DataStore Settings on the maintenance screen of your variant for an activation.

A popup appears where you can choose Init requests always return unique data records.

Confirm the dialog. The system will now omit the lookup for init requests.
(4) New Activation Method for Databases with Massively Parallel Processing Architecture
The traditional activation of requests in a standard DataStore Object is done by splitting the data of the
activation queue into packages that are activated independently of each other (and usually
simultaneously). The system ensures that records with identical keys are placed into the same package
and stay in their original sequence. It is very important that records with identical keys are activated in
the correct sequence to guarantee correct results. Therefore a package is activated sequentially - record
by record.
Usually there are not many records with identical keys in an activation run. SAP BW 7.30 offers you a
new activation method that can be used on databases with massively parallel processing architecture
(these are currently (January 2011) IBM DB2 for Linux, UNIX, and Windows and Teradata Foundation
for SAP NetWeaver BW).
This new activation method finds all the records in the activation queue that do not have unique keys
and activates them in the traditional way. All the other records have unique keys in the activation queue
and can be activated regardless of the sequence. These records are activated simultaneously using a few
SQL statements. This means that the majority of records are no longer read from the database to the
application server, processed and written back to the database. Instead they are processed directly in
the database. This results in a considerable improvement of the activation runtime, in databases with a
massively-parallel-processing architecture.
This new activation method is used automatically by the system (i.e. no user action is needed), provided
that certain conditions are met (list not complete):
this new activation method is supported for the database used (currently only IBM DB2 for
Linux, UNIX, and Windows and Teradata Foundation for SAP NetWeaver BW, as mentioned
above)
SID generation during activation is not requested
the option unique data records is not set
the aggregation behaviors of all (load) requests that are activated in one go are identical
the option do not condense requests is not set
The performance gains depend on the data as well as on the hardware. In scenarios with very suitable
conditions, performance can improve 2 to 3 times.
(5) Lookup into DataStore Objects in Transformation
In SAP BW 7.30, you can define a transformation rule to fill a field in the transformation target by
reading the value from a DataStore Object (similar to reading from master data). More information on
this can be found in Documentation in section Read from DataStore Object.
This new method of reading from DataStore Objects in a transformation is intended to replace the
routines written by users as transformation rules to read values from DataStore Objects. Besides making
it more convenient to read from DataStore Objects, this replacement will result in performance gains in
many cases (depending on the implementation of the replaced user routine). This is because data is read
from the DataStore Object once per data package instead of record by record.

How to save your Listcube selction screen and the output.

Posted by Nanda Kumar

Hi All,
Just found an quick information in SAP BI from LISTCUBE tcode so thought of sharing the same across
others because I suspect only very very few make use of this .

Have you ever thought of how to save selection screen and output fields in tcode LISTCUBE? If not
below are the steps to be followed to make use of it.

Advantages:
This will help in reusability like avoiding the LISTCUBE tcode again and again for the same
selections.
This will also helps you to reduce certain amount of your manual work.

Here is the steps to be followed.

1. Go to the TCODE LISTCUBE.

2. Select your needed info provider to be displayed.


3. Same time give the program name starting with Z or Y ( This is the program which is going to be
reused) and please make a note of it.



4. Now execute the screen , which displays your selection screen and select your needed field for the
selection, later also select your needed fields for the output using field selection for output tab
.
5. After selecting it, kindly select the Save button which is there on the top of the screen to save it as
variant.



6. You will get the popup screen in which it will ask you to update with Variant name and its Description,
enter the needed info and save it again through the save button or through the file menu Variant ---
>Save.




7. Use can make use of options check box which is available in the screen likeProtect Variant, which
protects others to changes this variant on the same program, means it can be changed only by the
created user, other is not certainly required, still if you need to know its purpose kindly press F1 on
corresponding check box for its use.

8. Now go to TCODE SE38 and give the program name as the one which you gave it in the LISTCUBE
transaction and execute it.



9. Once you execute it you will get the screen as you decided in listcube, click on variant button for save
variants or click on field selection for output tab for your changes like to save another variant or to make
the changes in the current variant.


10. Here if you want to view the save variants from list cube you can use the variant button and select
your variants to display it which has been stored already.



11. Select the need variant and execute the program to get your desired outputs.
Note: you can also store N number of variants from this program itself.

Now instead of going LISTCUBE again and again to make your selection you can make use of this
program when you want to make use of the same info provider to display your output for n number of
times, this will reduce your time in selecting your selection option, and your desired output screen.
Dependencies: If the structure of the Info Provider changes, a program generated once is not adapted
automatically. The program has to be manually deleted and generated again to enable selections of
new fields.

Demystifying aggregates in SAP BI 7.0

Posted by subash sahoo

Aggregates are subsets of InfoCube data, where the data is pre-aggregated and stored in a InfoCube
structure. Aggregates on an Info Cube can be considered similar to that of an Index on a database table.
Creating an Aggregate on an InfoCube is one of the few ways to improve performance of SAP BW
Query. Subset of an InfoCube data is stored in an Aggregate. As a result, the response time that we get
out of reading the data from aggregate will be much faster than reading from an InfoCube.
When a query gets executed it is split into several sub queries. Split of the query is based on the
following rules:
Condition 1: Parts of the query on different aggregation levels are split.
Condition 2: Different Selections on characteristics are combined
Condition 3: Parts on different hierarchy levels or parts using different hierarchies are split.
Aggregates that we build should meet the needs of query navigation and several sub queries within each
query. If we have more than one aggregate built on top of a cube, after the query split, OLAP Processor
searches for an optimal aggregate for each part. This means, a single query using a multiple sub query
can access more than one aggregate to get the desired output. Refer to the flow chart shown below.
At the end, OLAP processor consolidates all the results and gives the desired output.





















If a query performance is poor, we can use the following 2 thumb rules to decide if Aggregates will
actually help improve performance.
Condition 1: Summarization Ratio greater than 10
Summarization Ratio is the ratio of number of records in the cube to the number of records in the
aggregate.
Note: If the summarization Ration is very high, say 300 or more then the aggregate might be very
specific to a query and cannot be used by more than one query.
Condition 2: Percentage of DB time > 30% of the total query run time.
Please note that building an aggregate will only improve the database access time.
There is a tool called Work Load Monitor (ST03N) that is used to track the System Performance.
It basically helps to analyze the queries with highest run time.
DB time can also be determined by executing the Query through RSRT. The EVENT ID 9000 (Data
Manager): highlighted in the screen shot below gives the DB time.








Summarization ratio can be seen by executing the query from RSRT transaction. Select the query you
want to monitor the performance and click on execute and debug button as shown in the screen print
below
Usage: Every time a query hits the aggregate, Usage counter gets incremented by 1. If your Usage
counter is zero, in 99% of the cases, you can safely assume that your aggregates are not being used.














Last Used Date: Along with the Usage Indicator, another important parameter to be kept in mind is the
Last Used Date. For an Aggregate, Usage Indicator might show some big number. But when you
actually take a closer look at the last used date, It might show a date 6 months prior to the current date.
This means that this aggregate has not been used by any query in the last 6 months. The reason might
be that there were many changes made at the query level but the aggregates were not updated or
modified accordingly.
Valuation:
Valuation is the systems best guess of judging how good an aggregate is. Usually, if the summarization
ratio is high, the number of + signs in the valuation will be more. There could be some aggregates where
the number of plus signs in an aggregate is more, but the aggregate as such is never used. Though the
valuation will give a fair idea about the aggregates created, it need not be 100% right.











Once the data is loaded into the cube, following steps are to be carried out- for the data to be available
for reporting at the aggregate level.
Activate and Fill Aggregates should be activated and filled.
Aggregate Rollup: Is a step by which, the newly added data at the cube level gets aggregated
and available for reporting.
Change Run (Master Data Activation): To activate the changes of the master data. All the data
containing the navigational attributes gets realigned.
Adjustment of time dependent aggregates: This is done to recalculate the aggregates with time
dependent navigational attributes.
Any aggregate that is created on a cube has to be periodically monitored for its usage and the
last used date. If there is any aggregate that is not being used, it has to be either modified or
removed
Un-used aggregates add to the load performance. It is always a good practice to drop the
aggregates that are not being used.
Aggregates should be relatively smaller compared to the Info cube.
Too many similar aggregates can be consolidated to a single aggregate.

TOOL - Pre-Analysis of the aggregate filling.
With changes beyond a certain magnitude, modifying the aggregate becomes more time consuming
than reconstructing it. You can change this threshold value.

Steps-In the implementation guide, choose SAP NetWeaver--> Business Intelligence --> Performance
Settings -->Parameters for Aggregates in the section Percentage Change in the Delta Process. In the
Limit with Delta field, enter the required percentage (a number between 0 and 99). 0 means that the
aggregate is always reconstructed. Change these parameters as many times as necessary until the
system response is as quick as possible. We can accord this with the help of TOOL called "Pre-analysis of
aggregate filling".
By default blocksize is set to 100.000.000. SAP recommends changing this setting to a value between
5.000.000 and 10.000.000. In this way, we reduce joining or sorting on disk and also reduce log space
consumption. You should not set BLOCKSIZE to lower values, because this can result in a WHERE
condition that forces the optimizer to use an index other than the clustered index. Suggest to use
5.000.000 for systems with less than 10 GB of real memory. If you have more real memory SAP
recommends setting BLOCKSIZE to a value up to 10.000.000.
Ideally the index on the Time dimensionshould be used when reading data from the fact table, because
the fact table is clustered according to the time dimension. In many cases index on data request
dimension is chosen.
The tool helps you to identify how much block size you can actually generate with your existing
aggregates. Use button "Pre-Analysis of the aggregate filling" in the aggregate maintenance window to
see the SQL statement that is used depending on the value of parameter BLOCKSIZE.
We can explain this statement in ST05 and check, which INDEX is used to access the FACT table and then
act accordingly.

SAP BI Statistics Installation

Posted by nilesh pathak

This blog shares my personal experience during installation of "Standard BI statistics" on my BW system
& it is already provided by SAP as a standard content for analyzing performance of different
cubes,reports in the system.
During intial installation via SPRO transaction, the job RSTCC_ACTIVATEADMINCOCKPIT_NEW was
executed in the background and it was getting cancelled & also gave me dump in ST22 as Assertion
failed.
For resolution of this issue I implemented :"SAP Note-1330447- ASSERT incorrect when RRI jump targets
are activated". Generally it is recommended to install Bi content individually in all the
three environments-devlopment, quality, production otherwise during transporting the objects one may
encounter several issues.
Normally if the BI statistics installation is successfull, the relevant BI statistics cubes and reports are
activated & one should be able to see different statistics in transaction ST03N as well.But when i tried to
open ST03N transaction, I was not able to see any statistical info here as the cube 0TCT_C01 -Front End
and OLAP Statistics-Aggregated was getting zero records everyday even though initialization was
successfull .The data source 0TCT_DS01 during extraction in RSA3 also showed zero records.
When i checked in the view RSDDSTAT_OLAP the data was there & ideally it should have been uploaded
to cube 0TCT_C01(as this cube gets data from this view)but it was not happening in my case.So I tried to
debug in RSA3 in delta mode for data source 0TCT_DS01 & kept the breakpoint on fetch_package
method in function module RSTC_BIRS_OLAP_AGGREGATED_DATA.
The data source 0TCT_DS01 is based on two views- RSDDSTAT_DM & RSDDSTAT_OLAP so while
debugging i found that data was coming in both the views but it was getting filtered out due to settings
maintained in table RSTCTEXTARCT.Generally in RSTCTEXTARCT the step categories - MDX and OLAP are
selected but since my System release was 701 and Support Pack was 4 hence in table RSTCTEXTRACT I
had to set the flag for step category-OTHERS as well to extract the data from data source 0TCT_DS01 in
RSA3.
Hence after maintaining the settings as mentioned above I was able to get data in cube 0TCT_C01
everyday.I hope this blog will provide help regarding the issues that may occur during BI Statistics
Installation.


Delta extraction for 0FI_GL_04 for every 30 minutes

Posted by Ravikanth Indurthi

Note: The below scenario is applicable only for the datasources 0FI_GL_4 (General ledger: Line
Items),0FI_AP_4(Accounts payable: Line Items), 0FI_AR_4 (Accounts receivable: Line Items)

Scenario
In BW, the delta extraction for 0FI_GL_4 from the source system can be loaded as per the processchain
or infopackage schedule but where as, from source system will be able to fetch the delta records as per
the timestamp defined, or depending on system predefined delta pointer. So even if the data is pulled
multiple times Ex:- for every 30 minutes in the same day in the BW system, the deltas will fetch zero
records, if the delta pointer is limited to certain period for Ex:- only once a day in the source system.

Check the below link:
Financial Accounting: Procedure for Line Item Extraction

Constraints
Per day, no more than one delta dataset can be transferred for InforSource 0FI_GL_4. The extracted
data therefore has the status of the previous day. For further data requests on the same day, the
InfoSource does not provide any data.
In delta mode, data requests with InfoSource 0FI_AR_4 and InfoSource 0FI_AP_4 do not provide any
data if no new extraction has taken place with InfoSource 0FI_GL_4 since the last data transfer. This
ensures that the data in BW for Accounts Receivable and Accounts Payable Accounting is exactly as up
to date as the data for General Ledger Accounting.
Because in the source system delta extraction is based on CPU Time, the deltas are fetched once per
day.
So if the source system delta extraction is to be achieved for the timely base, the same data can be
pulled into BW easily by a process chain.
However, you can change this standard delivery. For more information, see note 485958.

Solution
It can be achieved by changing the settings at back end in the source system.
As the requirement is to get the deltas for every 30 minutes (1800 Secs), it needs to be update in the
table BWOM_SETTING. We need to create an entry or change the existing entry for the
field/Param_Name BWFINSAF with value 1800 (secs).
So now the Delta pointer for the datasource will be set to for every 30 mins, So that new deltas are
created in source system.


Coming to BW side, we need to schedule the start variant in the process chain for every one hour ot two
hours, depending on the time it takes for completion of the chain. So now the latest FI GL data can be
reported.



So after loading the data into the cubes, the Bex queries will be able to fetch the latest financial data in
the report output. Even the business object reports, OLAP universes based on the Bex queries will fetch
the latest FI data and show the same data for the BO reports such as WebI or crystal reports based on
the BO Universe.


BW 7.30: Different ways to delete Master data

Posted by Krishna Tangudu in SAP NetWeaver Business Warehouse

In the older version i.e. BI 7.0, we experience Memory
overflows while deleting the master data. Because system checks for all the
dependencies of the data associated with other InfoProviders in the system. This new tool is available in
BI 7.0 (SP 23) as an optional feature. In BW 7.3 it becomes a default option.


I will explain how the new functionality helped
to overcome the above problems with a simple navigation.

Navigation:


<br />



1) Go to DWWB (RSA1).


2) Go to Info Object tree and choose Delete Master
Data from the context menu of the desired Info Object.



In the present
version of Master data deletion functionality, we find that system checks if
the desired info object has any
dependencies i.e. info object being used in Master data like Hierarchies or in
Transaction data like Info Cubes.


So the system takes huge time to delete and also may lead to
memory overflows.


The new functionality looks like as shown below:



We can see four options in this wizard. Let us briefly
discuss each of them.




1) Delete SIDs:


<br /></p>

<p>This option is
used to delete the SID values in the /BIC/S<Info object name>.</p>

<p> We can choose if we want to delete the SIDs
also. If we choose to delete SIDs we have to check option Delete SIDs. This
will ensure that all SIDs are
deleted.</p><p> </p><p>!https://weblogs.sdn.sap.com/weblogs/images/252146487/3d.jpg|height=250
|alt=|width=600|src=https://weblogs.sdn.sap.com/weblogs/images/252146487/3d.jpg!</p><p> </p><
p>System will pop up a warning, before you delete SIDs as
shown
below.</p><p> </p><p>!https://weblogs.sdn.sap.com/weblogs/images/252146487/16d.jpg|height=150
|alt=|width=600|src=https://weblogs.sdn.sap.com/weblogs/images/252146487/16d.jpg!</p><p> </p><
p>SAP recommends not deleting
SIDs as it may lead to consistencies when reloading the same data.* *</p>

<p>In Older version, before deleting we have
to ensure that the chances or reloading the same characteristics is less. Or
else when we reload the data it may lead to inconsistencies if the where-used
list check performed by the system was not complete.</p>

<p>In the newer version, we can check the usage of
the Master data using Search Mode as per our requirement. We have 4 different
search modes which I will be discussing in detail later in the section.</p><p> </p><p>2) Delete Texts:


<br />



This option is used
to ensure that Texts are deleted along with the master data of the info object
selected.



3) Simulation mode:




When we delete
master data, we have to check if it is used in info cubes, hierarchies before
deleting. As we have to lock these dependent objects for deleting data in
them. So we need a down time to execute
deletion.


So it would be better if we know what
objects are getting affected beforehand so that we can delete the dependent
data from these targets, thereby reducing the number of locks and proceeding to
deletion of master data.


Simulation mode helps you in do this by
performing a where-used check based on the search mode you have chosen and
display the logs along with cubes or hierarchies in which the info object is
used.



4) Perform the deletion in background:




We know that every
time we try to delete master data, it takes more than 1 dialog process. To
perform in background we have to use Process chain. We can now do the same from
here as shown below.



Now you can choose the scheduling time from
the below screen to trigger the background job. In this case I choose
"Immediate".



Now you can monitor the background job using SM37. The job will be created by the user name who
created it or choose Application log from the below screen for the same action.



We have 4 search modes, which are used, they are:



1) Only One Usage per value:






0.1. The system performs where-used check and if
it finds a value being used by an InfoProvider then search continues for other values.
0.1. This is the default setting and can be used
if you want to minimize the run time for where-used list check.








On clicking Start option, the simulation starts and
you can find the logs as below:


You can see in the logs, the system creates
a where-used table /BI0/0W00000032 & /BI0/0W00000033 to generate the
where-used list as below and deletes it after the deletion.






It starts a job DEL-D- in background as
we selected background and simulation both. 3 child jobs starting with BIMDCK
are created to compute the where-used list table.







Where-used Table:


<br />



Here you can see the message stating that master data
is deleted in the background as below:

!https://weblogs.sdn.sap.com/weblogs/images/252146487/9d.gif|height=455|alt=|width=395|src=http
s://weblogs.sdn.sap.com/weblogs/images/252146487/9d.gif !

2) One usage per value per Object:




We use this to
know about the usage of master data in all InfoProviders.


The system performs where-used check and if
it finds a value being used by each InfoProvider then search continues for
other values in the InfoProvider.


!https://weblogs.sdn.sap.com/weblogs/images/252146487/12d.jpg|height=250|alt=|width=600|src=h
ttps://weblogs.sdn.sap.com/weblogs/images/252146487/12d.jpg!



This mode helps to find out which
InfoProviders, from which we need to delete data first, before we continue to delete
them all.



3) One usage is sufficient:


<br />



0.1. We use this to
delete the attributes and texts completely from the Info Object.
0.1. If one of the records from master data is
found to be used, the whole process stops saying it is already being used.







4) All usage of All Values:


<br />



All usages are searched in all info
providers along with the count (as in earlier functionality). It will provide
us with complete where-used list and takes the maximum run time.




You can see in the logs, the system creates
a where-used table /BI0/0W00000031 to generate the where-used list as below
and deletes it after the deletion.

In my case I used Only One Usage per value* *and scheduled the deletion in
background after comparing the above results. This helped me to identify the
associated dependencies of the data in the other InfoProviders and reduce extra
locks during deletion.



Selective Deletion:

We can use report RSDMDD_DELETE_BATCH for selective deletion from
any Info Object and give the selections using FILTER button. Simulation options can be given in
P_SMODE.


SAP BW Version 7.3 Overview of promising features!

Added by Shah Nawaz Shabuddin

I am concentrating on some of the new features provided in BW 7.3 and their benefits.
As part of this wiki, I will discuss about the below:
1. Hybrid providers
2. Graphical Dataflow modeler
3. Accelerated data loads
4. Semantic partitioning
5. Integrated planning front end
6. Analytical indexes
There are numerous improvements in release 7.3 new ways of delivering data, new reports and new
BW applications. The changes exist in mainly in the backend, used by the BW Architect and
Development teams. SAP has invested the majority of their frontend development effort in Business
Objects; so with the exception of small minor enhancements, the BEx frontend is unchanged. A number
of the new features rely on a BW Accelerator (BWA), an increasingly important tool for businesses.
1. Hybrid providers
Firstly, lets talk about the Hybrid providers. With hybrid providers, BW Consultants lives are made
easy!!!
To make data held within a DSO available for reporting, in BI7 there are a number of steps well need to
do: Create the DSO, InfoCube, Transformation/DTP, MultiProvider, store in a BWA and connect them all
up, and then schedule and monitor load jobs.
A HybridProvider takes a DSO and does it all for you, removing substantial development and
maintenance effort. Just load your data into a DSO, create a Hybrid Provider and start reporting. You can
even build your HybridProvider on a Realtime Data Acquisition DataSource (RDA), which could
potentially provide near real-time reporting from a BWA.
A typical usage scenario could be that you want to extract your PurchaseOrders from R/3 and make
available for reporting. Using a HybridProvider, as soon as the data is loaded into a DSO they then
become available for reporting with all the benefits of an InfoCube and
BWA.
A hybrid Provider within the Administrator Workbench
2. Graphical Dataflow modeler
A graphical way of building a datamodel has been introduced in BW 7.3. The Graphical Dataflow
Modeler is a new function within the Administrator Workbench (transaction RSA1).
This provides:* Top-down modeling of a new dataflow* Organization for existing dataflows* Creation of
template dataflows using best practice* SAP delivered pre-defined dataflows
Using a drag-and-drop interface, a developer can build a dataflow by selecting either existing objects, or
by directly creating new objects from within the modeler. The entire dataflow can be modeled in this
way, from the DataSource to the MultiProvider, and even as an OpenHub destination.
Any BW system with multiple complex projects would benefit from the retrospective implementation of
dataflows. This tool reduces the effort and complexity associated with the administration and
maintenance of a system. For example, a single dataflow could be delivered for all the data used within
an application, meaning the system will become easier (and therefore cheaper) to support.
3. Accelerated data loads
Accelerated data loads are now possible!!! - SAP has rewritten underlying code for data loads for
optimized performance.
In many places in BW 7.3, the technical coding has been optimized to improve the loading performance.
Load times into DSO objects has been reduced by up to 40% when compared to BI7, plus the loading of
Master Data has been accelerated.
Improvements in load performance translate into benefits for the business. Data can be made available
earlier in the day, reloads are quicker (so cost less) and larger volumes of data can be loaded each and
every day.
4. Semantic partitioning Turned Automatic!!!
With BW 7.3, the BW systems can automatically Semantically Partition your InfoProviders. Using a
wizard interface, you can create multiple partitions that store your data, handle the mappings between
partitions and schedule the loading processes. The need for the manual creation of multiple
transformations, DTPs has been removed J
Semantic Partitioning is where we spread similar data across several InfoCubes or DSOs, split according
to region or year. This allows increased performance and stability, though comes with at a cost of
increased development and maintenance.
A typical scenario would be to Semantically Partition an InfoCube that is used for Global Sales reporting.
By keeping to data for each region or year separate, performance is improved, and maintenance and
daily loading is easier.
Semantic Partitioning development will provide the business with both quicker and more reliable
reporting, at a lower cost. Rather than manually building Semantic Partitioning where critical, you can
use it in smaller applications at little extra cost.

Maintaining a semantically partitioned DSO
5. Integrated planning front end
In my view, the current Integrated Planning (IP) frontend in BI7 has always been unsatisfactory. Whilst it
does the job, I felt that it is:* Slow to use* Requires MS Internet Explorer
Interestingly, SAP has now migrated the IP frontend to a traditional transaction screen within the
SAPGUI. The screens are quicker and easier to use, and have no dependency on either a Portal or Java
stack.
I believe business will benefit through lower development and maintenance costs but I find it so much
nicer to use, this is reason enough for it to be a favorite change!
6. Analytical indexes
An Analytical Index (AI) is a transient dataset that is stored within the BWA. BEx or BusinessObjects can
be used to report on them directly. They are defined and loaded from within the Analysis Process
Designer (APD), and can contain the results of a JOIN between a CSV file and query.
The fact that they do not exist in the database and the ease at which they can be defined makes them a
very interesting concept for inclusion in BW. Up until now, creation of datasets to address specific
requirements has always been a significant exercise and has required transports and loading. So AIs
reduce the development effort needed to produce a fast-performing tool that addresses a specific
current requirement.
Visible Benefits of SAP Netweaver BW 7.3:
So, all in all, SAP BW 7.3 looks quite promising with all the below benefits.
Wizard based system configuration - Supports copying data flows, process chains etc.
Accelerated data loads - SAP has rewritten underlying code for data loads for optimized
performance.
Performance improvement for BW Accelerator* Automated creation of Semantic Partitioned
Objects
Graphical Data flow modeling and best practice data modeling patterns
Admin Cockpit integrated into SAP Solution Manager

Modeling is the next generation of programming languages.

Posted by Kobi Sasson in kobi.sasson

Modeling is the next generation of programming languages, you may say, well you are being a little bit
dramatic here. So let me explain.
Using the conventional way ( e.g. Java, C# etc. ) to create applications is somewhat an art, it require
creativity, a lot of experience and usually a lot of effort. The main reasons for that are clear, but I will
just mention a few- An enterprise application requires you to design and develop your application in a
way that it is maintainable, fulfill all the application standards such as accessibility, security, quality etc.
all of these mean at the bottom-line, time and money.
So I suggest a different way, why not just model the application, no code required, all standards are
already applied on all of your modeled application, and the bottom-line much less money and time, Im
not saying anything new here, but I would like to emphasize on the different aspects of a good
modeling tool :
1. A robust modeling language the modeling language should be robust enough in order to
express most of the requirements from an enterprise application. If it is too simple( such as
configuration tools which allows you to define a very limited set of options and then create an
application out of it) then it will cover only limited set of requirements and therefore will help
only on a very specific cases. On the other hand the modeling language should not be too
complex as it will make the modeling tool tedious and not user friendly, which will not fulfill my
second requirement J.
2. A simple modeling tool modeling tool should be used by business experts and usually not by
developers, so the modeling tool should be simple enough so the business expert will be able to
focus on the business logic and not on technical/complex staff.
3. UI extension points as the modeling language will not cover all customer requirements, a
major point here is to provide the ability to extent the UI offering of the modeling tool. By
providing the UI extension points the customers can cover all of his specific requirements.
4. Connectivity to various backend the ability to consume services is a must, but in the enterprise
world today companies have various sources of data ( Web services, Databases , R3 ). If you can
consume different subsets of data you are able to create applications which displays and
consolidate data from various data sources.
5. LCM- one last and very important in the enterprise world is to enable customers to deliver the
applications from development to production or to enable to work on the applications with
multiple users.


Visual composer do just that. We still have some points that can be improved but in general Visual
composer fulfill the above requirements, and therefore is the right solution for creating a fully fledged
enterprise applications.
I will appreciate your comments and feedback, and of course, if you think on a concrete requirement
that is missing in Visual Composer, please feel free to suggest an idea in Visual Composer Idea place.

BW 7.30: Dataflow Copy Tool: How to use Dataflow Copy tool to speed up the development activity

Posted by Krishna Tangudu in SAP NetWeaver Business Warehouse

As you know creating a data flow takes considerable development time. This affects TCO aspects of the
project. To reduce the total cost of development activity, BW 7.3 release supports the copying of
existing BW 7.x data flows, i.e. data sources, data targets, transformations and also process chains.
In this blog, I would like to explain how to use Dataflow Copy tool (wizard based tool). You will also
understand how to copy an existing BW 7.x data flow and create data flows. This will help you to
decrease the overall time of your development.
I present you with sample navigation, using this wizard based tool.

Navigation:
1) Go to DWWB (RSA1)
2) Select Copy Data flow from the context menu of the "Info provider" or "Process Chain".
In my case, I have selected an info provider (Ex: Sales Overview).



3) This action leads to the below screen, where I have to select whether to copy the upward, downward
or both the data flows of the cube.

In this case, I choose "Downward".

4) After continuing the above step, we will have an option to collect the dependent objects like IPs and
DTPs as below.



I choose YES to continue. This will help in creating required IP's and DTP's automatically.

5) This will lead to Dataflow copy wizard. Where I will configure below steps:

A) No. Of Copies
B) Source Systems
C) Data Sources
D) Info Provider
E) Transformations
F) Directly dependent Process
G) Indirectly dependent process

Now lets discuss more about features present in the wizard.
We can see that wizard is divided into 2 parts. Left side tells you the step which is currently in progress
as well as the errors RED traffic light (if any) in the steps. If you find all these steps with GREEN
traffic light, then you can precede with copying the data flow else the wizard will not allow you to do the
same.

We will now discuss in detail about what each step in this wizard is responsible for:

A) No. Of Copies:
We can determine the number of copies to be created in this step. We can create a maximum of
99999999999 (We may not use these many copies in the real environment).We need to mention the
replacement for placeholders for technical & description names as you can see below.

In my case, I will create only 1 copy. If it is "one" copy, we need not worry about the replacement
holders as shown above.

Now I press continue to proceed to the next step in the wizard i.e. Source Systems.

B) Source Systems:
We cannot create a new source system or change the existing source system for the copy flow. But we
can create a data source from the existing source system to the new source system i.e. we can create an
identical data source for a flat file system based on SAP source system (or we can use an existing flat file
system).
We have an additional feature in the wizard. We can checkDisplay Technical Names to display
technical names of the objects as shown below.

If we assign a wrong object or no object, you can find RED status as shown below.

Now I will assign a flat file source system (ex: YFF_SALES) to our new flow as shown below.


Now the status will turn to GREEN and we can precede to the next step i.e. Data Sources.

C) Data Sources:
In this Step, I will use the option Create with Template. This means we create a new data source with
help of new template.

Now I proceed to the next step i.e. Info Provider.

D) Info providers:
We can use the existing info providers or copy to new info providers but we cannot use overwrite here.
If in case this option is available, then if we have changed anything to the original info provider to
accommodate the changes this would not reflect in OVERWRITE and hence may lead to the data loss.
Hence MERGE option is used.


E) Transformations:
Before copying transformations, we have to check if source and target fields are present in the Target
object. If we dont have those fields present in the Target object that particular rule cannot be copied
(as the field in Target is missing).

Now I proceed to the next Step i.e. Directly Dependent Process.

F) Directly Dependent Process:
In this step, the processes are displayed that depend on an object of the data flow. For example, using
the change run if we include any Master data in the flow.
In this case we have IPs and DTPs to be created (As I selected this option in the previous steps).

Now I proceed to next step i.e. Indirectly Dependent Process.

G) Indirectly Dependent Process:
In this step, the processes are displayed that depend on a direct data-flow-dependent process. For
example, I have to use Error DTP (as I am using a DTP here).

With these I have completed all the important steps and I can proceed with copying the data flow.

Now system will ask you if I want to continue with data flow copy in the dialog or background.

I choose "Dialog" here, and the logs are displayed as below.

If we want to check the logs once again, we can use transaction RSCOPY.

We can also see our newly created objects.

6) Hence the copying is completed. Hope you understood the benefits of this new tool.
Major Lessons Learnt from BW 7.30 Upgrade

Dear SAP BW-ers,

This is my 1st time blog posting experience, I would like to share with you my journey when upgrading
to BW 7.30 SP05 from BW 7.01 SP07.

Whatever I've written below is purely my opinion and not my employer.

It has been a painful project yet rewarding since this will enable the BW to be on HANA and have much
better integration with other Business Objects tools.

Data Modelling Issue
There is a new data validation check procedure in BW 7.30, some of our daily batch jobs fails
because of this new behavior. We have to remove conversion routine in 0TCTTIMSTMP and also write
an ABAP routine to do the conversion during data loading. The same thing happened for 0WBS_ELEMT,
we wrote a routine to fix this thing as well.

Cannot activate an InfoSource after the upgrade, run program RS_TRANSTRU_ACTIVATE_ALL in
debugging mode and set the value of the parameter "v_without" to "x".

3.x DBConnect Data source stopped working after upgrade, needs to be regenerated because it
was corrupted after the upgrade Go-Live

BWA Issue
If you are running on BWA 7.20 lower than revision 22, please set the query properties back into
using Individual access instead of Standard/Cluster access. You can also do mass maintenance of every
queries in an InfoProvider in transaction RSRT. If you don't do this you will be having major performance
problem, for example a query that used to take 9 secs before upgrade will come back in 800 seconds if
the cluster access is enabled.

Reporting Issue
Error #2032 because of dashboard crosstab data binding, we experienced this with most query
that has 0CALDAY. You have to remove the 0CALDAY and then put it back again in the query so that the
BICS connection will be refreshed from the query to the dashboard.

UOM conversion issue after the upgrade for some query, implements this SAP Note 1666937 to
solve the issue.

Cell defined value was not impacted by scaling factor in BW 7.01, but in BW 7.30 it does. We have
to make lots of adjustment in few queries because of this.

A condition on a hidden key figures no longer supported in BW 7.30, again some queries has to be
adjusted because of this.

Dashboard Design 4.0 cannot be connected to BW 7.30 system until we upgrade SAP GUI version
to 7.20 SP10.

This is the list of major issues that we encountered so far a week after Go-Live; hope that this will help
your journey. I personally say to run the upgrade better we need to have a copy of production right
before the upgrade and do the heavy testing few times and involve the business when doing so. You will
then expect only few minor issues during the go-live.