Cognos ReportNet

Performance Tuning

Cognos Confidential

Agenda

Overview
Infrastructure tuning
Web servers J2EE application servers

Operating systems
Databases

Application Design Services Tuning Feature Specific Considerations Basic Monitoring

Cognos Confidential

Agenda

Overview
Infrastructure tuning
Web servers J2EE application servers

Operating systems
Databases

Application Design Services Tuning Feature Specific Considerations Basic Monitoring

Cognos Confidential

System

CPU Speed
Improve overall performance of each request

Disk I/O
Improve overall performance of data access, including temporary space and database files
O/S Network

Network I/O
Fast network speeds reduce contention between tiers
CPU Speed Disk I/O

O/S
Tune parameters that limit the ability of processes to utilize resources
Cognos Confidential

E.g.) An Infrastructure Built to Perform

Tier 1
Load Balancing

Web Server

Web Server

Tier 2
Load Balancing Fast Storage Server CRN Server

Fast Storage Server

CRN BIBUS

Tier 3
Database Server

Directory Server

CRN Servers

Database Server

CRN Repository

Authentication Source

User Database

Fast Storage Server

Cognos Confidential

Web Servers

Use a scalable gateway implementations

Apache Mod, ISAPI, servlets, etc More scalable than CGI implementations (default)

200 concurrent users supported on 2 x 2.8 GHz (< 20% cpu utilization)
Approximately 30 web requests per second

General Considerations (all brands):

Tune web servers according to expected user loads Consider capacity, load balancing and failover

Cognos Confidential

J2EE Application Servers

Used as a servlet container to manage BI requests

Cognos ReportNet dispatcher component java.exe process

Support Java versions are 1.3.x

1.4 being considered for future releases

2 major considerations:
# of threads Amount of memory (heap size)

Use monitoring tools to assess thread requirements

As a general rule, configure 2.5 threads * # of concurrent users

Configure 1024 MB or higher for best performance

Cognos Confidential

IBM WebSphere

Assume WebSphere has been installed & configured

CRN p2pd.war file has been deployed

Cognos Confidential

IBM WebSphere Java Memory Allocation

Navigate
Servers -> Application Servers -> server1 -> Process Definition -> Java Virtual Machine

Change Maximum Heap Size to 1024 Mb or higher (if available) Add Generic JVM Arguments such as -Xms256M -Xmx768M
Used to specify garbage collection memory and how objects age

Cognos Confidential

IBM WebSphere Thread Allocation

Navigate
Servers -> Application Servers -> server1 -> Web Container -> Thread Pool

Change Maximum Size to 2.5 threads * # of concurrent users

Check Is Growable checkbox
Additional threads will be allocated if required

Cognos Confidential

BEA WebLogic

Assume WebLogic has been installed & configured

CRN p2pd.war file has been deployed

Cognos Confidential

http://e-docs.bea.com/wls/docs81/perform/topten.html

BEA WebLogic Native IO

Navigate
Servers -> Server running CRN -> Configuration > Tuning tab

Ensure Enable Native IO is checked

Cognos Confidential

BEA WebLogic Accept Backlog

Navigate
Servers -> Server running CRN -> Connections > Tuning tab

Set Accept Backlog to 50 or higher

Cognos Confidential

BEA WebLogic Configure Execute Queue

Navigate
Servers -> Server running CRN -> Monitoring > General tab -> Monitor All Active Queues ->

Configure Execute Queue -> default

Set values based on expected concurrency 1 thread per concurrent user

Cognos Confidential

BEA WebLogic Monitoring Threads

Navigate
Servers -> Server running CRN -> Monitoring > Performance tab

Ensure Idle Threads are not being exhausted under load

Cognos Confidential

BEA WebLogic JVM Tuning

Edit the StartManagedWebLogic.cmd

Specifies settings to:
Start CRN server Identify CRN configuration details (p2pd.war file, etc) Customize JAVA_OPTIONS and MEM_ARGS

WebLogic bugs have been encountered in testing (bug #) Create .hotspot_compiler file
exclude java/lang/Long toString exclude weblogic/socket/NTSocketMuxer processSockets

Drop where Java is invoked

Confirm workaround by following startup message:
CompilerOracle: exclude java/lang/Long toString CompilerOracle: exclude weblogic/socket/NTSocketMuxer processSockets

Cognos Confidential

Apache Tomcat Java Memory Allocation

Set using Cognos Configuration or startup.bat in CRN\tomcat directory

Change Dispatcher Maximum memory in MB to 1024 Mb or higher (if available)

Cognos Confidential

Apache Tomcat Thread Allocation

Navigate to CRN Tomcat directory -> server.xml -> coyote section

Set maxProcessors to 2.5 threads * # of concurrent users Set acceptCount to 500 connections
Before: <!-- Define a non-SSL Coyote HTTP/1.1 Connector on port 8080 --> <Connector className="org.apache.coyote.tomcat4.CoyoteConnector" port="9300" minProcessors="5" maxProcessors="75" enableLookups="true" redirectPort="9443" acceptCount="100" debug="0" connectionTimeout="60000" useURIValidationHack="false" disableUploadTimeout="true"/> After: <!-- Define a non-SSL Coyote HTTP/1.1 Connector on port 8080 --> <Connector className="org.apache.coyote.tomcat4.CoyoteConnector" port="9300" minProcessors="5" maxProcessors="1000" enableLookups="true" redirectPort="9443" acceptCount="500" debug="0" connectionTimeout="60000" useURIValidationHack="false" disableUploadTimeout="true"/>
Cognos Confidential

Apache Tomcat Thread Allocation

maxProcessors
Maximum number of threads that can be simultaneously processed

acceptCount
Length of the request queue before additional requests are refused

enableLookups
Specifies whether DNS lookups are enabled Some customer situations have benefited from switching to False

Cognos Confidential

Garbage Collection (GC)

Develops as a result of re-allocated memory over time

Objects become orphaned and are no longer accessible via a program pointer

Different algorithms exist to discard unused objects

Generational GC uses logic to avoid having to parse every object

Concept of generations young vs. old

Memory pools of different ages are used

GC more relevant on large systems (see chart below)

Cognos Confidential

Survivor Ratios

Two generations Young and Old

Young composed of:
Eden used for newly allocated objects 2 survivor spaces
1 for aging objects 1 for copying to old generation (tenured)

Cognos Confidential

Databases

Database performance tuning is a subject of its own

Database technologies Database design Database parameter tuning

Need to work closely with database expertise

Ensure queries issued by Cognos are being processed as efficiently as possible

Provide insight as to more efficient query design

Cognos Confidential

Operating System

Windows 2000 Registry entries to support high load applications

SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\TCPTimedWaitDelay=30s SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPort=10000 or higher SYSTEM\Current\SessionManager\Subsystem\Windows Shared Section to 3072

AIX Environment variable to enhance multi-threaded applications

Greater priority is given to the processing of threads AIXTHREADSCOPE=S and AIXTHREAD_SCOPE=S

HP-UX Kernel parameters for high use of threads

Max_thread_proc max threads allowed per process (ceiling of nkthread) Nkthread total number of kernel threads that can run (ceiling of 30,000)

Solaris
No specific settings
Cognos Confidential

Agenda

Overview
Infrastructure tuning
Web servers J2EE application servers

Operating systems
Databases

Application Design Services Tuning Feature Specific Considerations Basic Monitoring

Cognos Confidential

Application Tuning

ReportNet

Application Server

Java VM

O/S

Machine

Cognos Confidential

Application Design

Database Design
Indexes, data types Primary and foreign keys

Framework Manager Design

Builds on database design
Provides the basis for query generation

Report Design

Model Design

Report Design
Query Report Layout

Database Design

Cognos Confidential

Report Design

Report Design is critical!

Font size has no impact on response time Font choice choose appropriate font for a performing application Fixed Width columns spend time on authoring a good report

Be frugal with information avoid redundant or unnecessary information

Each column added reduces throughput by approximately 10%
Pages Per Second Per CPU

Cost Per Additional Column

10% reduction in Pages Per Second Per CPU

Adding columns (data or ink) is expensive

4 # of Colum ns

Cognos Confidential

Report Design Font Choice

Andale WT font is embedded by default

Ensures that CRN can display all supported languages Drawback is that Andale font is approximately 20 Mb in size
Separate cost for each report processed Tradeoff Performance

Resolution is to select lightest possible font to support language requirements of an application

E.g. Arial font is only 100 Kb in size

Cognos Confidential

Report Design Fixed Width Columns

CRN uses a flow model to build reports

Report layout specifications are determined during report execution
Leads to rapidly application deployment Allows adjustment to variable sized data values

Drawback is that CRN must do multiple passes of data to determine appropriate size
Tradeoff Performance (particularly PDF)

Resolution is to fix the size of objects (titles, columns, etc)

Disables built-in CRN intelligence Tradeoff Authoring time

Cognos Confidential

Fixing Column Widths

Ensure that data containers have been customized

Cognos Confidential

Introducing warpproperties.xml

Advanced properties used to configure report processing

Located in CRN_ROOT\configuration directory Disabled by default (.sample extension is used) Enabled by:
Changing warpproperties.xml.sample to warpproperites.xml Restarting CRN

Configuration setting applicable to specific features and functionality

Note Changes to an enabled warpproperties.xml file must be accompanied by restarting CRN

Cognos Confidential

Temp Files

Temp files may be created if report processing activity exceeds memory thresholds
Large reports or load situations whereby memory is exhausted Monitor CRN_Root\temp directory for cclvpage*.tmp files

Drawback is that CRN must write to disk (usually a slow device) to build the entire report
Resolution is to allocate more memory (if available) to avoid writing to disk

Accomplished by changing the VirtualMemoryDiagnostics property to a value of 2 (unlimited use of memory)

<property>VirtualMemoryDiagnostics</property> <value type="long">2</value>
Cognos Confidential

Temp Files

Use of memory over disk has proven beneficial in a real customer example
Report processing time was reduced from over 2 hours to 11 minutes

Cognos Confidential

Workshops #1 to #3 1 hr

Cognos Confidential

Agenda

Overview
Infrastructure tuning
Web servers J2EE application servers

Operating systems
Databases

Application Design Services Tuning Feature Specific Considerations Basic Monitoring

Cognos Confidential

Services Tuning

Services tuning can be conducted at various scope levels

Environment scope all servers and all services Server scope server by server Service scope service by service

Tuning by scope is especially important when different server capacities are being used

Cognos Confidential

Services Tuning

Tools -> Server Administration -> Tuning Settings

Cognos Confidential

Maximum number of report processes

Represents the maximum BIBuses that can be spawned on a server

BI Buses are used for querying and report generation Separate setting for Batch and Report services Considerations:
# of CPUs Clock rate of CPUs (E.g. 2.0 GHz) Memory (estimate 200 Mb per BIBus) Contention between batch and interactive report processing

Default value = 2 Recommended value = 2 * # of CPUs per server Use monitoring to determine ideal setting
Cognos Confidential

Example 1: Setting maximum processes

No overlap between batch and interactive usage

Time
Batch processing period Interactive processing period

Batch and Interactive usage periods are distinct Total processes = 2 * 4 CPUs = 8 Interactive processes = 8 4 CPUs Batch processes = 8

Cognos Confidential

Example 2: Setting maximum processes

Some overlap between batch and interactive usage

Time
Batch processing period Interactive processing period

Batch and Interactive usage periods are in part contention

Throttle down number of processes (determine ratio by level of importance) Administer daily not recommended

Total processes = 2 * 4 CPUs = 8 Interactive processes = 8 4 CPUs

Cognos Confidential

Batch processes = 8

Example 3: Setting maximum processes

Complete overlap between batch and interactive usage

Time
Batch processing period Interactive processing period

Two options:
1. 2. Consider dedicated servers for each of interactive and batch processing Throttle down number of processes (determine ratio by level of importance)

Total processes = 2 * 4 CPUs = 8 Batch processes = 8

Total processes = 2 * 4 CPUs = 8 Interactive processes = 8

4 CPUs
Batch

4 CPUs
Interactive

Cognos Confidential

Affinity Connections

Represents the number of threads allocated per BIBus

Affinity connections are distinguished by transaction type Low affinity connections (E.g. run report)
Request that can be processed efficiently by any report server

High affinity connections (E.g. next page)

Request that performs most efficiently when directed to the report server

Considerations:
Clock rate of CPUs (E.g. 2.0 GHz) Type of transactions

Cognos Confidential

Affinity Connections

Default values (each setting is per BIBus)

Low affinity = 4 High affinity = 1

Recommended values
Low affinity = 2
High affinity = 1

Cognos Confidential

Workshop #4 10 mins

Cognos Confidential

Queue Time Limit

Maximum amount of time that a request will remain in the dispatcher queue for an available report service connection
User receives error if queue time limit is exceeded
A process was not available within the configured time limit

Consider the memory requirements to manage more queued requests if limit is set high
Set a low limit to help troubleshoot slow performance and request latency

Cognos Confidential

Process Capacity

Assigns a proportion of total requests to be handled by a dispatcher

Considerations:
# of servers # of CPUs per server Clock rate of CPUs (E.g. 2.0 GHz) Relative performance measures (provided by hardware vendors)

Default value = 1.0

If all servers are configured using the default, each server will process an equal number of requests
Cognos Confidential

Example: Setting Process Capacity

Requests

Scenario:
Note - All CPUs are equivalent (architecture, speed, etc) 4 CPUs 2 CPUs 2 CPUs

Server 1

Server 2

Server 3

Server 1 can handle 2x more requests than Server 2 and Server 3

Therefore, process capacity would be:

Server 1: 2.0 Server 2: 1.0 Server 3: 1.0
Cognos Confidential

Agenda

Overview
Infrastructure tuning
Web servers J2EE application servers

Operating systems
Databases

Application Design Services Tuning Feature Specific Considerations Basic Monitoring

Cognos Confidential

Batch Reporting

Batch reporting is advantageous for many reasons:

Maximize the use of hardware during off-peak hours Minimize the complexity of BI transactions during peak hours Report generation can be aligned with data refresh cycles

Individual schedules can be created on a report-by-report basis. Best used when:

Only a few reports need to be scheduled Reports need to be generated on different time schedules

Jobs can be created to schedule many reports. Best used when:

Many reports need to be generated on same time schedule

Cognos Confidential

Batch Reporting Jobs

Strategy is based on:

Time constraints Number of reports to be run Number of low affinity connections available

If # of reports <= low affinity connections available

Use 1 job and execute all reports at once (for a given schedule)

If # of reports > low affinity connections available

Use N jobs where N is the # of low affinity connections available Divide the # of reports to run by N and assign each group to a job Set each job to run on the same schedule Execute all jobs in sequence
Cognos Confidential

Reports <= # of low affinity connections

All at once - Reports will run in parallel

Cognos Confidential

Reports > # of low affinity connections

In sequence - Reports will run in one at a time

Cognos Confidential

Example Scenario

Assumptions
Need to run 112 reports nightly 2 report servers are available and configured to use 8 low affinity connections each (16 in total)

# of reports > low affinity threads available (I.e. 112 > 16)
Use 16 individual jobs with 7 reports each
100 = 16x where X is the # of reports per job (X = 7) Therefore, 112 reports will be executed using:
16 jobs executing in parallel Each job will process 7 reports in sequence

Cognos Confidential

Example Scenario

16 low affinity connections available

Job #1
16 jobs are setup and execute simultaneously

Job #1
Report #1 Report #2 Report #3 Report #4 Report #5 Report #6 Report #7

16 Jobs configured to run 6 reports in sequence = 112 reports

Cognos Confidential

Bursting

Concurrent PDF Bursting offered in 1.1

1 database query to retrieve all information N threads to assemble personalized reports
N should be configured based on server capacity (# of cpus) In general, 1 thread per CPU Configured in the CRN_ROOT/configuration/warpproperties.xml
<structure> <!--Number of threads in the bursting thread pool. Default: 3--> <!--<property>BurstThreadPoolSize</property> <value type="long">3</value>--> </structure>

Concurrent bursting for other report output types being considered for future releases
Cognos Confidential

Demo #1

Show sample report that can be burst

Cognos Confidential

Workshop #5

Cognos Confidential

Demo #2

Run a burst job using a multi-processor server

Filter on Order number < 1200 Use the 1 assembler thread configuration (warpproperties.xml enabled)

Monitor the CPU utilization on the CRN Report server

Cognos Confidential

Demo #3

Run a burst job using a multi-processor server

Filter on Order number < 1200 Change the burst thread property (1 thread per CPU) Restart CRN

Monitor the CPU utilization on the CRN Report server

Cognos Confidential

Prompting

Prompting is advantageous for various reasons:

Provides dynamic reporting to end users Filtered result sets means less report processing

Possible use scenarios:

All prompts on a single page Prompts separated on multiple pages Cascading prompts Optional prompts

Cognos Confidential

Prompt Caching in 1.1

Prompt caching can be enabled to improve performance within a report session

Report session Activities performed from report instantiation to completion

Open Report Select prompts Finish Report

Designed to:
minimize number of database queries
<structure> <!-- MaxCachedQueries is the maximum number of queries the Prompt Manager will attempt to cache. Valid values are: 0=OFF (default) -1=NoLimit n=Cache up to 'n' queries --> <!-- <property>MaxCachedQueries</property> <value type="long">-1</value> --> </structure>

Expansion of prompt caching features being considered for future releases

Cognos Confidential

Query Reuse

Available in 1.1 MR2

Enabled by default

Used when same results can be used or new results a subset of cached
Results are cached for the entire duration of a RS, QS and RV session

Introduces a new affinity type session affinity

Designed to:
Minimize number of database queries

Expansion of query reuse features being considered for future releases

Cognos Confidential

Query Reuse

Disabled using CQEconfig.xml

queryReuse property

Set value = 0 to force database access Set value = -1 to use cache whenever possible Set value = N to force database access after N cache hits

Cognos Confidential

Agenda

Overview
Infrastructure tuning
Web servers J2EE application servers

Operating systems
Databases

Application Design Services Tuning Feature Specific Considerations Basic Monitoring

Cognos Confidential

Monitoring

Windows
Task Manager Perfmon

UNIX
top
ps ef pstat Mpstat

Other monitoring tools: nmon, topas, 3rd party products, etc

Cognos Products
Server Administration tools Run a live user
Cognos Confidential

Server Administration tool

Cognos Confidential

Server Administration tool

Total number of requests processed per minute Total time (sec) to process a request Services selected for monitoring

Server and service type

Queue Time before a request can be processed Dispatcher queue timeout limit (sec)

Cognos Confidential

Number of BIBuses started Maximum number of BIBuses

Process Monitoring

Ensure expected number of BIBuses are created

Windows Task Manager to verify UNIX tops or ps

If appropriate number of BIBuses not created:

CPU utilization on report servers may suffer

Possible reasons:
Number of affinity connections per BIBus is set too high Not enough requests to exhaust maximum BIBuses

Cognos Confidential

Request Distribution Monitoring

Ensure requests per minute are distributed as per server capacity

Use Server Administration to monitor

Change distribution model through process capacity settings

Cognos Confidential

Things to Avoid

Cognos ReportNet monitoring is expensive

Use during investigative phases do NOT leave these interfaces open while not in use

Major changes to process and affinity setting model

Bigger is NOT always better

Cognos Confidential

Questions?

Cognos Confidential