Vous êtes sur la page 1sur 78

Introduction

Introduction

Contents
Introduction 1
1 NetAct Architecture and Components 3
1.1 NetAct Platform Components 4
1.2 Virtualization 6
1.3 Software Environment 8
1.4 NetAct 15 System Functionalities Overview 10
1.5 NetAct Applications environment: WebSphere Applications Server 12
1.6 Load Balancer 14
1.7 NetAct Mediations Environment: Apache Service Mix 16
1.8 NetAct Reliability 19
2 NetAct Administration: Tasks and Responsibilities 31
2.1 Purpose of NetAct Administration 32
2.2 Responsibilities of a NetAct system Administrator 34
3 NetAct Troubleshooting: General Strategies and Tasks 37
3.1 Overview 38
4 NetAct Health Check Tools 41
4.1 Preventive Health Check 42
4.2 Service Manager 45
4.3 WebSphere Application Server tools: Unify 56
4.4 WebSphere Application Server tools: PM Side 58
4.5 NetAct Logs 61
4.6 Log Monitor Tool 67
5 Exercises 69
Exercise 1 70
Exercise 2 72
Exercise 3 74
Exercise 4 77

OS90531EN15GLA0
1
© Nokia Solutions and Networks 2015
Introduction

OS90531EN15GLA0
2
© Nokia Solutions and Networks 2015.
Introduction

1 NetAct Architecture and Components

OS90531EN15GLA0
3
© Nokia Solutions and Networks 2015
Introduction

1.1 NetAct Platform Components


The hardware platform of NetAct consists of three main parts, namely servers,
storage devices and network equipment, such as switches and routers. NetAct
utilizes x86-64-based servers with multi-core processors and supports blade
solutions. NetAct operating environment is fully virtualized.
NetAct’s software architecture is three-tiered:
 Presentation tier
 Business logic tier and
 Data tier.
Business logic and Data tiers are separated from Presentation tier by firewalls and
reside inside the secure domain.
Presentation tier hosts client applications. The client applications access Business
logic tier and reside typically on a user workstation, for example, a laptop or a
desktop computer. The recommended workstation is an office PC with either
Windows or Linux operating system.
Business logic tier hosts the business logic and data access specific for NetAct. Data
tier hosts NetAct data storages, that is, the database and directory servers. Both
Business logic tier and Data tier utilize virtual machines running on one or more host
servers dedicated to NetAct.

OS90531EN15GLA0
4
© Nokia Solutions and Networks 2015.
Introduction

NetAct 15 Architecture
Components

7 OS90531EN15GLA0
OS90531EN81GLA1 © Nokia
@Nokia Solutions
Solutions and
and Networks
Networks 2014
2015

Fig. 1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
OS90531EN15GLA0
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
© Nokia Solutions and Networks 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.2 Virtualization
NetAct 15 delivers network functionalities deployed into a virtualized environment. In
a virtualized infrastructure (VI), software is set up in such a way that it can operate
independently from the underlying hardware. The physical resources of multiple
machines are shared across the whole infrastructure, and they act as a resource
pool. In a VI, the physical hardware resources are divided into smaller units, virtual
machines (VMs) that have their own allocated virtual CPU and memory.
The virtualization layer (HW virtualization capability) is provided by the VMware
hypervisor. Servers simply add I/O, CPU and RAM computing resources to the HW
pool and virtualization provides a logical view of data, computing power, storage
capacity and other server resources.
There are no longer dedicated hardware roles. Instead, NetAct applications,
mediations, and services running on a number of VMs are powered by a hardware
resource pool. Each VM is dimensioned and optimized for the service or services
running in it. In this way, the same NetAct functions can be provided but with better
high availability and scalability options.
NetAct is connected to the network elements through southbound interfaces, and to
the user workstations and to users’ external systems through northbound interfaces.
The data communication network (DCN) backbone provides the infrastructure that
connects network elements to NetAct and different NetAct sites.

OS90531EN15GLA0
6
© Nokia Solutions and Networks 2015.
Introduction

NetAct 15 Architecture and Components


Virtualized Infrastructure

NetAct services and applications Virtual machines (VMs) running NetAct


services and applications

Bare-metal ESXi hypervisor enabling full


virtualization of each x86 server, abstracting
the physical hardware from the NetAct
Virtual Infrastructure
services and applications

HP x86 G8 blade servers with 10G interfaces


and Virtual Connect Flex 10 Ethernet
Modules in blade chassis.

Cisco, Juniper or HP - DCN

Fiber channel SAN connecting the blade


SAN SAN chassis to the storage.

EMC/HP physical storages containing the


ESXi hosts root disks, VM root disks, DB
disks and global shares.
NEs

8 OS90531EN81GLA1 ©@Nokia
Nokia Solutions
Solutions and
andNetworks
Networks2014
2015
OS90531EN15GLA0

Fig. 2

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.3 Software Environment


NetAct 15 contains several software layers that are installed in the correct order.
After hardware and OEM software installation, NetAct SW platform is installed to the
servers. On top of NetAct 15, there are Mediations, adaptations and applications.
The following list presents the current version information about the key 3rd party SW
components used in the NetAct 15 release. Please note that this information is non-
binding and the list is subject to changes.
If any third-party software is needed, the required licenses are provided by Nokia.

Function Software Used


Operating System Unify Side: RedHat Enterprise Linux (RHEL) version 6.6 on NetAct VMs, SuSe Linux for
VCA and VDP and Windows Server 2008 for Node Managers.
PM Side: RedHat Enterprise Linux (RHEL) version 5.8
Database Oracle Enterprise Edition Version 11R2

Application Server WebSphere Application Server (WAS) version 8.0

Load Balancer Unify Side: LINUX Virtual Server (LVS) provided by RHEL.
PM Side: IBM Edge LBD
Business Logic J2EE Version 1.4

JBI Server Apache ServiceMix 3.4

Java JRE Version 7 for apps and JRE 1.5 for WAS

Virtualization VMware vSphere Enterprise Plus 5.5 and VMware vCenter Server Standard for vSphere
5.5
Authentication/ LDAP Version 3 / RedHat Directory Server provided by RHEL
Directory Server
File System NFS Version 3 provided by RHEL

OSI Stack Marben

IP Connectivity FTP, NFS, SSH, HTTP, SNMP, NTP, BIND and DNS protocols, provided by RHEL

OS90531EN15GLA0
8
© Nokia Solutions and Networks 2015.
Introduction

NetAct 15 Architecture and Components


Software Environment

Application Server: IBM WebSphere version 8.0 (Unify Side) & version 6.1 (PM Side)

Business Logic: JBI Server: Cache: Perl:


J2EE version 4 Apache Service Mix 3.4 JBoss Cache Version 5.8.8

Java: JRE 7 for applications & JRE 1.5 for WebSphere

Database: Oracle Enterprise Edition 11G R 11.2.0.3.0

Operating System Unify Side: RHEL 6.5 Operating System


Node PM Side: RHEL 5.8
Load Balancer: LINUX Virtual Server (LVS) File Share: NFS v 3 Managers:
Windows File Share: NFS v 3
IP Connectivity: FTP, NFS, SSH, HTTP, SNMP, NTP, BIND and DNS
Server
2008R2 Load Balancer:
Directory Server: RHDS 9.0 IBM EDGE LB

Virtualization components
vSphere: VMware vSphere Enterprise Plus 5.5

vCenter: VMware vCenter Server for vSphere Plus 5.5 VDP: vSphere data Protection

Hypervisor: EXSi Linux: SuSe for VMware Server Appliances

9 OS90531EN81GLA1 OS90531EN15GLA0 © NokiaSolutions


@Nokia Solutions and
andNetworks
Networks 2015
2014

Fig. 3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.4 NetAct 15 System Functionalities Overview


NetAct system functionalities can be distributed in different groups regarding the
services provided and the implementation used for such services. Each group
provides functionalities like applications deployment, southbound mediations,
northbound mediations, load balancing, fm-pipe processing, pm-pipe processing,
Topology and Configuration Management as depicted in the next figure.

1.4.1 PM in NetAct 15: PM Side and Unify Side


The PM components (backend and UI) in NetAct 15 are based on OSS 5.x
processes. For this implementation, the PM processing components and OSS 5.x
platform and dependent OSS services have been introduced into NetAct 15. Hence
some of the VMs in NetAct 15 are dedicated to PM and are called as PM nodes or
PM side. The other components (depicted in next figure in blue scheme) correspond
to the latest NetAct 15 platform and are called the Unify side.

OS90531EN15GLA0
10
© Nokia Solutions and Networks 2015.
Introduction

NetAct 15 Architecture and Components


System Functionalities Overview

Southbound – Northbound Mediations


JBI Instances XoH nbi-fmascii

3GPP CORBA NBI: FM, PM,CM Common -mediati NWI3 Q3/MML NX2S nbi-snmp
PM Side

Unify Side WAS: JEE Cluster


Applications, SNMP SB Mediations, CM
Operations, NASDA (topology services)
PM - pipe
Aux
rep
serv LDAP as as as as

linas linas fm-pipes


DB NFS JBI dmgr

LDAP instances
Node Node Node Node
Load Agent Agent Agent Agent
LB LB DNS
Balancers

NFS

Virtual Infrastructure

10 OS90531EN81GLA1 OS90531EN15GLA0 © Nokia


@NokiaSolutions and
Solutions Networks
and 2014
Networks 2015

Fig. 4

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.5 NetAct Applications environment: WebSphere


Applications Server
In NetAct 15, IBM WebSphere Application Server (WAS) is at the heart of the Java
EE runtime environment. It is the WAS that hosts the various enterprise applications
for NetAct 15, both in the Unify and PM Side. WAS is installed in two or more VM
nodes depending on the NetAct deployment type (small, mainstream or large). In
addition to WAS, there is also IBM HTTP Server installed on each WAS node.
There is a load balancer function running in a separate VM that provides a single
point of connectivity to NetAct 15. This load balancing functionality is provided by
Linux Virtual Server (LVS) in the Unify Side and IBM Edge Load balancer in PM Side.
The load balancer servers are running on designated VMs.
WebSphere Application Server also communicates with other backend systems like
Oracle DB and LDAP.

OS90531EN15GLA0
12
© Nokia Solutions and Networks 2015.
Introduction

NetAct Architecture and System Components


Applications Environment : WAS – Unify side

VM11** VM20**
WebSphere JDBC
Load AS
HTTPS
Balancer IBM HTTP node
HTTPS server1
Server agent
Database
jms- web-
HTTPS
server1 server1

HTTPS
HTTPS

VM21**
WebSphere
Firewall HTTPS AS
IBM HTTP node
server1
Server agent LDAP
jms- web-
server1 server1

Client VM22**

VM23**
Backend

1 **Note: Example
© Nokia Solutionsfor NetAct
and Networks 2014 Deployment Type Mainstream: VM22 and VM23

have the same configuration and connectivity as VM20 and VM21

Fig. 5

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.6 Load Balancer


Traffic and requests to NetAct services are distributed among multiple instances of
those services that are running in multiple virtual machines. This ensures high
availability of the service and also optimal distribution of load across the participating
NetAct virtual machines.
The load balancer service is configured to work as a highly available service. For the
Unify Side, there are two instances of the LB service running on two different virtual
machines configured in active-backup setup:
 lb-unify-primary
 lb-unify-backup
The active load balancer serves two roles:
 To balance the load across the real servers.
 To check the integrity of the services on each real server. .

In NetAct 15, the lb-unify-primary is running in the VM11 node, while the lb-unify-
backup is running in VM06 node. These services balance the load on the WAS
servers are running in the corresponding WAS VM nodes (for example VM20, VM21,
VM22, VM23 in a mainstream deployment).
The active load balancer also dynamically monitors the overall health of the specific
services on the WAS servers. If a service on a WAS server malfunctions, the active
load balancer stops sending jobs to that server until it returns to normal operation.

For the PM Side, there are also two instances of the LB service running on two
different virtual machines configured in active-backup setup:
 lb-n85vm26
 lb-n85vm27
In PM side, the load balancer function is provided by the IBM Edge product.

OS90531EN15GLA0
14
© Nokia Solutions and Networks 2015.
Introduction

NetAct Architecture and Components


Load Balancer: Unify Side

Unify Load balancer nodes WAS nodes WebSphere


JDBC
IBM HTTP
AS
Load Balancer HTTP S
HTTPS
server1
node
agent
Server
HTTPS jms- web-
Database
server1 server1

NE Fm-pipe nodes
SA SA SA
BC

NMR
BC BC
SA
Firewall Service Mix
Instances

Web
Services Common Mediations nodes
SA SA SA
BC

NMR
BC BC

Client
SA
Service Mix
Instances

LDAP
RedHat Directory Server
1 © Nokia Solutions and Networks 2015

Fig. 6

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.7 NetAct Mediations Environment: Apache Service


Mix
Most of the NetAct mediations, the Fault Management functionality and other
functions have been built using Apache Servicemix JBI (Java Business Integrations).
JBI is a Java-based standard that defines runtime architecture for plug-ins to
interoperate via a mediated message exchange model. This Java based event
handling architecture can work in single node or multiple node environments.
JBI is designed to ease the work required to make systems to communicate together.
In NetAct this is implemented for the network element integrations and their protocols
to be transformed into O2ML standard language in the mediation layer

In the diagram in the following page, each box represents a JBI instance i.e. Apache
ServiceMix process.
NetAct Mediation Framework includes separate mediations for Common South
bound (NE3S/WS, 3gpp corba sbi), NWI3, Q3, NBISNMP and NBI (NE3S).

GEP
Generic Event Processing handles the FM function.

Common container
.NE3S/WS SBM, 3GPP Corba SBM and NetAct internal mediations directly deploy
into this instance.

NWI3, Q3, XoH, NX2S, SNMP NBI, FMASCII NBI, 3GPP CORBA NBI
All other containers are installed by the respective mediations using their
corresponding JBI instances.

OS90531EN15GLA0
16
© Nokia Solutions and Networks 2015.
Introduction

NetAct Architecture and Components


Apache ServiceMix: JBI Environment

JMX-based mgmt
application
Installation

Deployment

Control

Monitoring

1 © Nokia Solutions and Networks 2015

Fig. 7

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

Apache Service Mix JBI


Instances in NetAct

All deployment configuration, run time environment and logging is located under
the /opt/oss directory in the VM running the respective service

Common
Mediations
nbi3gc
NE3S/WS SBM – 3gpp NWI3 q3user nbisnmp 3GPP CORBA
CORBA SBI
NAC mediations
NBI

common_mediations/ nwi3/ q3/ nbisnmp nbi3gc/

XoH rac3gp nbi3gcpm


GEP
fmascii 3GPP Corba 3GPP Corba
fm-pipe
NX2S CMBULK NBI PM NBI

fmascii/ gep/ XoH/ rac3gp/ nbi3gcpm/


NX2S/

1 © Nokia Solutions and Networks 2015

Fig. 8

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
OS90531EN15GLA0
18 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
© Nokia Solutions and Networks 2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.8 NetAct Reliability


System reliability refers to all features included in NetAct platform that facilitate high
availability and disaster avoidance of the NetAct system. The following sections
explain the concepts and solutions in more detail.
High availability aims to eliminate single points of failure (power supply, disk space,
CPU, software or connectivity) by introducing redundancy of in software and
hardware components.
Connections between any two servers and inside the storage system are built to
incorporate redundancy, that is, there always are two independent paths between any
two components in a virtualized cluster. The same principle applies also to routers
and firewalls. High availability guarantees that critical resources are available for
NetAct operations even in failover situations.
Load balancing concerns other server configurations than single server configuration.
In multi-server configuration, load balancing ensures that server load is distributed
among the servers in an application server cluster. Automatic load balancing is not
used on the data tier, but balances traffic on the application.

OS90531EN15GLA0
19
© Nokia Solutions and Networks 2015
Introduction

NetAct Architecture and Components


Reliability

Online /
High Load System Self- Health
offline Backup
Availability Balancing
and Recovery monitoring Check

Hardware Distribution
Configuration of Resources VMWare Data Internal Fault
in the Virtual Protection Management
Infrastructure Tools to
verify
Virtual
Infrastructure
NetAct
Distribution System
of resources Internal Status
NetAct
in the Performance
Software Backup Tool
Application Management
Components
Server Cluster

1 © Nokia Solutions and Networks 2015

Fig. 9

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
20
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. .Solutions
. .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.8.1 High availability in the virtual infrastructure


With virtualization in the NetAct architecture, the physical hardware resource pool
and the virtualization layer (enabled by the hypervisor installed in each server) are
configured to optimize the use of computing resources in a given NetAct deployment
configuration.
NetAct provides high availability (HA) in three dimensions: it provides hardware
redundancy, virtual machine availability, and service monitoring. Hardware
redundancy is built into the virtual infrastructure (VI) at various levels. Connections
between physical servers and the storage are designed in a redundant way: at least
two independent paths exist between physical servers and the storage. Routers and
firewalls are also configured in a redundant way. Redundancy guarantees that
resources are available for the essential NetAct operations (for example, fault
management event flow, performance management transfer, and MMI access to the
network elements) and for any failover scenario.
The virtual infrastructure takes care of virtual machine (VM) availability. It monitors
the hardware and each virtual machine’s operating system for failures. For example,
if a server fails, the virtual infrastructure management software restarts the affected
VMs in an active host with sufficient computing resources.
In the VI, a virtual manager (vManager) monitors the NetAct VMs for any failed
service. Service Monitoring, on the other hand, checks the services inside the VM so
that if a service fails, it tries to restart the affected service for a configured number of
times. If the service still does not start, then Service Monitoring places the service in
the failed state. If the failed service is marked critical, then the virtual manager
requests the virtual infrastructure management software to reboot the whole VM.
These mechanisms ensure that virtual machines and the services running in them
are always available to provide the operating environment to manage network
elements.
The following lists the virtualization HA mechanisms in use in NetAct:

 Virtual Machine (VM/guest) availability monitored by Virtual Infrastructure Manager


 Service availability monitored on guest System by Watchdog
 J2EE Application availability guaranteed with WebSphere cluster
 Automatic restart of failed application or virtual machine in case of application or
guest failure
 Priorities and VM startup order
 Redundant HW components: Running NetAct node (VM) can be relocated to
another physical server online
There is no service interruption for end users

OS90531EN15GLA0
21
© Nokia Solutions and Networks 2015
Introduction

NetAct Architecture and Components


Reliability: High Availability in Virtualized Infrastructure

• VMware HA monitors VMs to


detect OS and HW failures
• Restarts VMs on other
physical server in the HW
resource pool automatically
when server failure is
detected
• It also protects from
VM OS failures

1 © Nokia Solutions and Networks 2015

Fig. 10

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
22
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. Solutions
. . .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.8.2 High Availability: Failure in ESXi host case


1. One physical server (ESXi host) fails
2. VMware high availability notices that one ESXi host has gone down
3. VMware restarts the virtual machines on top of another ESXi host based on
available resource information in the virtual infrastructure

1.8.3 Service Monitoring


Service monitoring is divided into two levels:

 Each VM has a Service Monitor that manages service(s) inside the VM


 vManager manages the Service Monitor in VMs

Monitoring Workflow:
Services are started/stopped/monitored by services’ own /etc/init.d/<service> -scripts
When there is a problem with the service, it will be restarted by local Service Monitor
Monitoring definitions are in /var/opt/cpf/ha/smanagerConf.xml
If the local Service Monitor cannot restart the service and the service is critical, vManager
restarts the VM in the same host or relocates the VM to another host
JBI Server and J2EE Server high availability for the applications remains the same

OS90531EN15GLA0
23
© Nokia Solutions and Networks 2015
Introduction

NetAct Architecture and Components


Reliability: High Availability in Virtualized Infrastructure – services Monitoring
status

status
status

status status

restart VM status restart status


LDAP Proxy
restart

status
DNS LB LB NFS NFS
vCenter

1 restart VM
© Nokia Solutions and Networks 2015

Fig. 11

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
24
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. Solutions
. . .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.8.4 Load balancing functions in NetAct


In NetAct, both the traffic and computing capacity are load-balanced. Load balancing
is provided through the following:

 Distributed Resource Scheduler (DRS)


 Linux Virtual Server (LVS), IBM Edge Load Balancer
 WebSphere Application Server (WAS) workload manager
The NetAct system contains load balancers dedicated to different kinds of traffic.
Load balancers distribute load between certain VMs or clusters assigned to them.
LVS, Edge Load Balancer and WAS are among those used to improve system
response time, optimize resource utilization, and promote higher throughput.
The Distributed Resource Scheduler (DRS) is used in NetAct for balancing the
computing capacity load of the hosts and virtual machines. DRS functions within one
cluster: it monitors the use of CPU and memory resources, and when required, it
migrates virtual machines from one host to another. When a virtual machine is
enabled, DRS places the virtual machine on the most suitable host having sufficient
computing power (CPU and memory resources).

OS90531EN15GLA0
25
© Nokia Solutions and Networks 2015
Introduction

NetAct Architecture and Components


Reliability: Load Balancing functions in NetAct

DRS places virtual machines so that


the load across the cluster is balanced,
and cluster-wide resource allocation
policies are enforced. JEE Cluster
Load distribution among all Application Servers

Applicati Applicati Applicati Applicati


Resource Pool
on on on on
VMware VMware VMware Server Server Server Server
ESX ESX ESXi
Deploy-
ment
Manager

Physical Servers Node Node Node Node


Agent Agent Agent Agent
Load
Load balancers distribute load between Balancer
certain VMs or clusters assigned to them.
LVS and WAS are among those used to
improve system response time, optimize
resource utilization, and promote higher
Virtual Infrastructure
throughput
1 © Nokia Solutions and Networks 2015

Fig. 12

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
26
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. .Solutions
. .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.8.5 Backup
1.8.5.1 Disk-based backups
The backup solution implements backup and restore functionality for the NetAct
system. It consists of two components: vSphere Data Protector (VDP) by VMWare
and backuptool, which is the disk based backup tool by Nokia Solutions and
Networks. VDP is used to create image backups of the virtual machines, and
backuptool is used to backup database and global file systems. Both components
store backups on disks.
The backup solution takes online database backups of Oracle database, backups of
LDAP data in the Directory Server, file level backups of the global (shared) disk, and
image backups of the virtual machines in the NetAct virtual cluster. It also takes
manual and offline backups. An online backup of the file system is taken when the
system is up and running while offline backup is taken when NetAct services are shut
down. Online backups of Oracle database can be full or incremental. Taking backups
of the database, LDAP data, and global disk can be done from the command line or
by using scheduled automatic backups. Taking backups of the virtual machines can
be done from the VDP Web client or by using scheduled automatic backups.
Backups are then taken to a dedicated virtual backup storage, which can be located
inside the normal NetAct online storage or it can be located in a separate disk
storage system.
In addition, the backup solution also restores backups from the backup storage.
Restoring backups can be done from the command line, from the VDP Web client, or
from both. Restoring global file systems can be done manually file by file.
The backup and restore solution also stores all messages, errors, and backup status
into a log for viewing. It has an open backup interface description that enables the
use of a backup system from a third-party vendor to take backups of the NetAct
system.

1.8.5.2 Open backup solution


With the open backup interface, operators can build their own backup solution and it
allows them to use a backup system from a third-party vendor without using the
backup solution delivered by Nokia Solutions and Networks.
Open backup data volume
The volume of data to be backed up consists of two parts:
• data on the file system
• data in the Oracle database
The data in the file system includes the data on the root disks of all virtual disks of all
NetAct virtual nodes.
The data in the Oracle database includes network configuration and measurement
data. The amount of data depends on your parameters, for example, the size of the
managed network and the configuration of the system.

OS90531EN15GLA0
27
© Nokia Solutions and Networks 2015
Introduction

NetAct Architecture and Components


Reliability: Backup and Recovery Management

VMWare vSphere Data


Protector (VDP)
Disk Based Used to create image
Solution backups of the virtual
machines

Backup and NetAct Backup Tool


Recovery
Solutions To execute backups of
oracle database and
global file systems

Open Backup
Solution

1 © Nokia Solutions and Networks 2015

Fig. 13

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
28
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. .Solutions
. .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

1.8.6 System Self-Monitoring


NetAct monitors its own performance and processes, both in the hardware and
software. System self-monitoring processes and forwards performance management
(PM) and fault management (FM) data collected from the NetAct system to NetAct
itself. The results of the self-monitoring process can be seen in performance
management reports for PM data and as alarms for FM data.
The solution is based on the O&M Agent software, which is installed on all NetAct
virtual machines and which starts to function immediately after installation. O&M
Agent provides the central functionality for the system self-monitoring: it raises
alarms and cancels them when the fault situation has been resolved. O&M Agent
contains the Integration Framework and Core components, and it also has an internal
database for active self-monitoring of FM and PM data.
The self-monitoring data flow proceeds as follows:
1. NetAct monitors its own software and hardware and collects FM and PM data.
2. HP SIM collects the alarms associated with the NetAct hardware, such as
servers, data communication network (DCN) equipment, and storage hardware
and sends these data to Integration Framework.
3. O&M Agent further processes FM and PM data and forwards the data back to
NetAct itself as events and measurements using the NE3S/WS southbound
interface.
4. FM and PM pipes transfer alarms and measurements to the database, from
which the FM and PM tools (Monitoring Desktop and Performance Manager,
respectively) can retrieve the needed data for processing or for reporting. The FM
and PM pipes also transfer the data to NetAct’s northbound interface, where self-
monitoring information can be forwarded to other network management systems
for further processing.
Monitoring of NetAct hardware
The monitored NetAct hardware is integrated to HP SIM during NetAct installation.
HP SIM monitors server hardware, DCN routers and switches, firewalls, and storage
devices. During the automated NetAct installation, HP SIM automatically discovers
and identifies the HP Proliant servers that are part of the NetAct cluster. Other
hardware used, such as the EMC VNX storage system or the DCN switches, can be
manually integrated with HP SIM.
HP SIM collects fault management (FM) data from the hardware in the managed
system and forwards these to O&M Agent for further processing. Critical and
important hardware faults that require attention are interpreted by the HP SIM
service, which categorizes the alarms into three groups: server hardware faults, DCN
hardware faults, and storage hardware faults. In addition to displaying the alarms in
the HP SIM GUI, the HP SIM service sends notifications to O&M Agent, which in turn,
forwards them to NetAct Monitor. If there is a need to find out more about a particular
hardware alarm, it is possible to check the information from HP SIM.

OS90531EN15GLA0
29
© Nokia Solutions and Networks 2015
Introduction

NetAct Architecture and Components


Reliability: System Self Monitoring

HW HP SIM NB Interface

SW Performance
Database Manager
FM data PM data Monitoring
Collection Collection Desktop

O&M Adaptation
Agent

O&M Agent Core PM pipe FM pipe


Data Processing

NE3S/WS SB Interface Mediation

1 © Nokia Solutions and Networks 2015

Fig. 14

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
30
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. .Solutions
. .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

2 NetAct Administration: Tasks and


Responsibilities

OS90531EN15GLA0
31
© Nokia Solutions and Networks 2015
Introduction

2.1 Purpose of NetAct Administration


As a network management system designed to operate and maintain a complex
telecommunications network, NetAct needs to be up and running always. NetAct
administration is essentially managing system reliability that is provided by the
hardware, operating systems, third-party software, services, tools, interfaces, and
applications comprising NetAct.
In NetAct, system reliability is ensured through the following basic features:
 High availability in the hardware configuration, virtual infrastructure, and
selected software components to eliminate single points of failure in power,
disk, memory, CPU, network connectivity, virtual machine, or service
availability
 Load balancing, which takes care of the distribution of server load within a
WebSphere Application Server cluster and the distribution of resources within
the virtual infrastructure
 Online and offline backup and recovery solutions using VMware Data
Protection and the NetAct Backup Tool
 System Self-monitoring for internal fault and performance management
 Preventive Health Check, which is a tool that verifies the NetAct System status

NetAct Administration also involves troubleshooting the NetAct system.


Troubleshooting involves determining first whether the problem occurs in the NetAct
system itself, in some network elements, or in all network elements. After determining
the location of the problem, you can then proceed by checking the Domain Name
Service (DNS), Lightweight Directory Access Protocol (LDAP), the different high
availability services, and the NetAct processes as they also affect the functionality of
the entire system. The order for the rest of the system checks can vary. For example,
the administrator can use the vSphere Web Client to manage and troubleshoot the
NetAct virtual infrastructure. For more information, see Troubleshooting NetAct
documents in the online documentation.
For different levels of troubleshooting, log files can provide helpful information in
maintaining and fixing issues in the system. Event log file is used to trace system
events in NetAct, both successful and failed events. This log file is automatically
created and maintained by the system. It can be used to monitor problems in the
system that needs to be corrected by the system administrator. Security and trace
logs are important elements for securing the NetAct system. They provide information
on security events of the entire system and trace those events that affect all
components of NetAct. For more information about logs, see the NetAct Logging
Reference in the online documentation.

OS90531EN15GLA0
32
© Nokia Solutions and Networks 2015.
Introduction

NetAct OSS Administration


Purpose

NetAct is a critical System


that impacts all the NetAct Administration
Telecommunications network

• NetAct must be up and • Manage System


running always Reliability

Hardware Operating 3rd Party


Systems Software
Services
Interfaces Applications
and tools

6 © Nokia
@NokiaSolutions
Solutionsand
andNetworks
Networks2014
2015

Fig. 15

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
OS90531EN15GLA0
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
© Nokia Solutions and Networks 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

2.2 Responsibilities of a NetAct system Administrator


System administrators are the most privileged users in the NetAct system as it is their
responsibility to set up and maintain the operating environment for other users. They
carry out routine maintenance tasks to ensure that resource utilization is optimized,
they check network performance to better respond to resource contention, and they
do basic troubleshooting in case of service or network problems.
NetAct system administrators also manage the NetAct hardware, which consists of
three main items: the servers, the storage devices, and the network equipment such
as switches and routers. Along with this role, they administer the virtual infrastructure
and its corresponding resources. They also maintain the services that provide the
runtime environment to manage network elements. The components that need to be
managed include basic services (such as the directory service, database service, and
domain name service to name a few) as well as southbound and northbound
interfaces.
NetAct administration consists of a set of proprietary as well as third-party tools and
instructions that enable administrators to perform their responsibilities. System
administrative tasks include:
Monitoring the NetAct health and status:
 Monitor system activity, disk usage and log files.
 Monitor and maintain the NetAct database.
 Monitor system security.
 Check the provided reports and alarms.
Managing the network operations
 Create groups and user accounts needed for managing the network based on
the plans.
 Plan how network management is setup.
Maintaining the system
 Back up files and databases.
 Forecast the need for NetAct upgrades and capacity enhancements.
 Install new software and hardware.
 Shut down and restart the system, when necessary.
 Supporting end users when they encounter problems with NetAct or the
integrated network
In short, system administrators ensure that NetAct is performing well and that
network operations are running as they should.

OS90531EN15GLA0
34
© Nokia Solutions and Networks 2015.
Introduction

NetAct OSS Administration


Administrator Tasks

• Monitor system activity, disk usage and log files.


Monitoring the NetAct • Monitor and maintain the NetAct database.
health and status: • Monitor system security.
• Check the provided reports and alarms.

• Create groups and user accounts needed for managing the


Managing the network network based on the plans.
operations • Plan how network management is setup.

• Back up files and databases.


• Forecast the need for NetAct upgrades and capacity
enhancements.
Maintaining the system
• Shut down and restart the system, when necessary.
• Supporting end users when they encounter problems with
8
NetAct or the integrated network
© Nokia Solutions and Netw orks 2014

Fig. 16

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
OS90531EN15GLA0
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
© Nokia Solutions and Networks 2015
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

OS90531EN15GLA0
36
© Nokia Solutions and Networks 2015.
Introduction

3 NetAct Troubleshooting: General Strategies


and Tasks

OS90531EN15GLA0
37
© Nokia Solutions and Networks 2015
Introduction

3.1 Overview
This section describes the basic principles of troubleshooting, that is, how you should
proceed when you detect a fault in your system and begin the troubleshooting
procedure. You must follow the instructions mentioned in this section before using
any of the recovery procedures.
You should first determine whether the problem occurs only in some network
elements or all network elements. After determining the location of the problem,
proceed by checking DNS, LDAP, high-availability services and the NetAct services,
as they affect the functionality of the entire system. The order for the rest of the
system checks can vary.
1. To check if all services are in started or adopted state, enter:
[root] #smanager.pl status
2. To check if all the Unify applications are in STARTED state in the virtual machine
where WAS is running, enter:
[root] #/opt/cpf/bin/cpfwas_listClusterApplications.sh
Note:
You can also execute /opt/oss/NSN-nasmercury/bin/mpf_status.sh to
check WebSphere application status.
3. To check the memory on a virtual machine, enter:
[root] #free -m
Check the memory utilization on physical RAM and also check swap memory.
WARNING
Ensure that swap memory is not utilized for longer duration.

4. To check the CPU usage on a virtual machine, enter:


[root] #sar
WARNING
Also, check if the usage is more than 60% over a period, for example 1hr.
5. To check if database connectivity is working on a virtual machine, enter:
[omc] $sqlplus omc/<omc password>
SQL prompt must appear on the screen.
6. To check if LDAP connectivity is working on a virtual machine, enter:
[omc] $ldapsearch -D cn=Manager -w <password> -x
where <password> is the password for cn=Manager.

If the above mentioned checks are fine, refer to specific sections based on the
reported problem (for example, for issues related to NetAct Monitor, refer to Fault
management). If the issue is related to network element integration, refer to the
corresponding integration document.

OS90531EN15GLA0
38
© Nokia Solutions and Networks 2015.
Introduction

NetAct Troubleshooting
General Strategies

Locate Problem

NetAct related Check HA Services (monitoring


(all NEs affected or NEs
Check DNS and virtual services)
groups affected)

Check LDAP Check NetAct Services

Implement
single Network Element troubleshooting
related accordingly to previous
steps results

1 © Nokia Solutions and Networks 2015

Fig. 17

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

NetAct Troubleshooting
General Strategies
Check services
• #smanager.pl status

Check load balancer / unify / PM side


• Unify side: #ps-ef | grep pulse
• PM side: #ps –ef| grep ibmlb2

Check WAS applications


•Unify side:
#/opt/cpf/bin/cpfwas_listClusterApplications.sh
•PM side: #linas_applications_status.sh

Check DB
• $sqlplus omc/<omc password>

Check LDAP
1
• $ldapsearch -D cn=Manager -w <password> -x
© Nokia Solutions and Networks 2015

Fig. 18

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
40
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. Solutions
. . .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

4 NetAct Health Check Tools

OS90531EN15GLA0
41
© Nokia Solutions and Networks 2015
Introduction

4.1 Preventive Health Check


Preventive Health Check (PHC) is a command-line tool that is used to check the
current state of the NetAct system through the summary report it generates and the
logs that it collects. The tool can give the system administrator helpful insights about
the system, especially when troubleshooting so that it is best to execute it in the
following scenarios:
 after every NetAct deployment
 when there are failures within NetAct
 when there is a need to monitor the NetAct system at regular intervals
The tool is modular and covers test cases for both RHEL and NetAct. The RHEL
module test cases perform the operating system and hardware-related tests. NetAct
test cases are grouped into the following sub-modules:
• Configuration management
• Fault management
• High availability
• NetAct system information
• Performance management
• Websphere Application Server
You can run the tool from one VM using the mhcf user, which is automatically
created with the installation, and as a result, get the status of all the VMs. Every tool
execution creates a new folder that contains the summary of all the findings. The
reports generated are available in .html, .log, and .xml file formats. All logs
collected are saved in a .tar file.
For more information, see Preventive Health Check User Manual under the
Troubleshooting category.

OS90531EN15GLA0
42
© Nokia Solutions and Networks 2015.
Introduction

NetAct HealthCheck Tools


Preventive HealthCheck
PHC is a command-line tool that is used to check the current state of the NetAct
system through the summary report it generates and the logs that it collects.

PHC gives the administrator PHC covers test cases for both
insights about the system, RHEL Operating System and
especially when troubleshooting. NetAct. NetAct test cases are
It is best to execute it in the grouped into the following sub-
following scenarios: modules:

• after every NetAct deployment • Configuration management


• when there are failures within • Fault management
NetAct • High availability
• when there is a need to monitor • NetAct system information
the NetAct system at regular • Performance management
intervals • Websphere Application Server

1 © Nokia Solutions and Networks 2015

Fig. 19

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

NetAct HealthCheck Tools


Preventive HealthCheck
Preventive Health Check is installed on all cluster nodes. Therefore, you can run the
health check from any node.

To run Preventive Health Check:


1. Log in as mhcf user to the node where you want to run the health
check.
2. Change the directory to /opt/oss/NSN-mhcf:
3. Start Preventive Health Check by executing:
./mhcf.pl –c <configuration file> -loglevel <value>
where:
• <configuration file> - configuration file for the complete test suite
• <value> - value of the log level (default value 4)

• To list all the available test cases, execute:


./mhcf.pl -c data/cfg_netact8.xml –list
1 © Nokia Solutions and Networks 2014

Fig. 20

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
OS90531EN15GLA0
44 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
© Nokia Solutions and Networks 2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

4.2 Service Manager


NetAct services are managed using a Service Manager tool. The script
/opt/cpf/bin/smanager.pl is an enhanced replacement of the OSS 5.x script
hamgrmx.pl.
Service manager covers all nodes and their services in NetAct 15. It also provides
options to start/stop services as well start/stop the nodes and the whole of NetAct.
The output of the command ‘/opt/cpf/bin/smanager.pl status’ provides the
status of the services in each of the nodes. There is no concept of service relocation
to another node in NetAct 15. There is no virtual IP associated with each service in
NetAct 15; hence it is marked as standAlone.
View the status of a specific service using the command:
smanager.pl status service <name_of_service>
View the status of the services on a specific VM using the command:
smanager.pl status node <name_of_node_or_VM>

OS90531EN15GLA0
45
© Nokia Solutions and Networks 2015
Introduction

NetAct HealthCheck Tools


Service Manager
Status of NetAct
DomainNode Service Status VirtualIP
Site: Local
All
n83-vm03
#smanager.pl status
netfs-n83-vm03 started standAlone
cpfnx2s-n83-vm03 started standAlone
OMAgentAF-n83-vm03 started standAlone
ssh-n83-vm03 started (u) standAlone
ntp-n83-vm03 started (u) standAlone
DNS-Master started (u) standAlone
dirsrv started (u) standAlone
tomcat_vcentplg started standAlone
n83-vm04
netfs-n83-vm04 started standAlone Status (u): unstoppable:
cpfnx2s-n83-vm04 started standAlone It is not possible to stop
OMAgentAF-n83-vm04 started standAlone
ssh-n83-vm04 started (u) standAlone this service using smanager.pl
ntp-n83-vm04 started (u) standAlone
db started standAlone
n83-vm05
cpfnx2s-n83-vm05 started standAlone
OMAgentAF-n83-vm05 started standAlone
ssh-n83-vm05 started (u) standAlone
ntp-n83-vm05 started (u) standAlone
nfs started (u) standAlone
dirsrv-secondary started (u) standAlone
n83-vm06
netfs-n83-vm06 started standAlone
cpfnx2s-n83-vm06 started standAlone
OMAgentAF-n83-vm06 started standAlone
ssh-n83-vm06 started (u) standAlone
ntp-n83-vm06 started (u) standAlone
pulse-jbi started standAlone
n83-vm07
netfs-n83-vm07 started standAlone
cpfnx2s-n83-vm07 started standAlone
OMAgentAF-n83-vm07 started standAlone
ssh-n83-vm07 started (u) standAlone
ntp-n83-vm07 started (u) standAlone
1 © Nokia Solutions OMAgent
and Networks 2014started standAlone
vcenterselfmon started standAlone
hpsim started standAlone

Fig. 21

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
46
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. .Solutions
. .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

NetAct HealthCheck Tools


Service Manager

Status Description

Starting The service is currently starting

Started The service is running and being monitored by the Service Monitoring application

Stopping The service is stopping

Stopped The service was stopped by an smanager command or was manually stopped
The service has failed to recover, and the Service Monitoring application is trying
Recovering
to start the service
Failed The service has stopped, and recovery efforts has failed

Maintenance The service is not monitored by the Service Monitoring application

No connection The smanager application cannot establish a connection with the service

Frozen The service is under maintenance mode

Config Error The specified service control script does not exist.

1
Locked The service is disabled
© Nokia Solutions and Networks 2014

Fig. 22

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
47
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

4.2.1.1 smanager.pl services description


4.2.1.2 smanager.pl - main stream deployment
Service Description

netfs-n85vm03 nfs client service


cpfnx2s-n85vm03 internal subsystems communication
OMAgentAF-n85vm03 self monitoring client
ssh-n85vm03 ssh service
n85vm03 nslcd-n85vm03 ldap client service
ntp-n85vm03 NTP service
DNS-Master DNS master Server
dirsrv LDAP master Server
tomcat_vcentplg integration with virtualized environment

netfs-n85vm04 nfs client service


cpfnx2s-n85vm04 internal subsystems communication
OMAgentAF-n85vm04 self monitoring client
n85vm04 ssh-n85vm04 ssh service
nslcd-n85vm04 ldap client service
ntp-n85vm04 NTP service
db ORACLE Database Server

cpfnx2s-n85vm05 internal subsystems communication


OMAgentAF-n85vm05 self monitoring client
ssh-n85vm05 ssh service
n85vm05 nslcd-n85vm05 ldap client service
ntp-n85vm05 NTP service
nfs NFS service
dirsrv-secondary LDAP Server - Secondary

netfs-n85vm06 nfs client service


cpfnx2s-n85vm06 internal subsystems communication
OMAgentAF-n85vm06 self monitoring client
n85vm06 ssh-n85vm06 ssh service
nslcd-n85vm06 ldap client service
ntp-n85vm06 NTP service
lb-unify-backup LOAD Balancer - Backup

OS90531EN15GLA0
48
© Nokia Solutions and Networks 2015.
Introduction

netfs-n85vm07 nfs client service


cpfnx2s-n85vm07 internal subsystems communication
OMAgentAF-n85vm07 self monitoring client
ssh-n85vm07 ssh service
nslcd-n85vm07 ldap client service
n85vm07
ntp-n85vm07 NTP service
OMAgent NetAct self- monitor core service
vcenterselfmon NetAct virtual infrastructure monitor service
hpsim Integration with HP SIM

ftp-n85vm08 FTP service


netfs-n85vm08 nfs client service
cpfnx2s-n85vm08 internal subsystems communication
OMAgentAF-n85vm08 self monitoring client
ssh-n85vm08 ssh service
nslcd-n85vm08 ldap client service
n85vm08 ntp-n85vm08 NTP service
pingServer-n85vm08 auxiliary service for mediations
PasswordChangeWS-
n85vm08 auxiliary service for mediations
tnameserver-n85vm08 auxiliary service for mediations
nbi3gc 3GPP Corba general services and FM
nbiim 3GPP Corba general services and FM
rac3gp 3GPP Corba CM

ftp-n85vm09 FTP service


netfs-n85vm09 nfs client service
cpfnx2s-n85vm09 internal subsystems communication
OMAgentAF-n85vm09 self monitoring client
ssh-n85vm09 ssh service
nslcd-n85vm09 ldap client service
ntp-n85vm09 NTP service
n85vm09 pingServer-n85vm09 auxiliary service for mediations
PasswordChangeWS-
n85vm09 auxiliary service for mediations
tnameserver-n85vm09 auxiliary service for mediations
nwi3 NWI3 Mediation+E53
nwi3-http NWI3 - Http Service for NWI3 Mediation
jacorb NWI3 - java for Corba Service for NWI3 Mediation
saucnt Cloud Manager Service

OS90531EN15GLA0
49
© Nokia Solutions and Networks 2015
Introduction

ftp-n85vm10 FTP service


netfs-n85vm10 nfs client service
cpfnx2s-n85vm10 internal subsystems communication
OMAgentAF-n85vm10 self monitoring client
ssh-n85vm10 ssh service
nslcd-n85vm10 ldap client service
n85vm10 ntp-n85vm10 NTP service
pingServer-n85vm10 auxiliary service for mediations
PasswordChangeWS-
n85vm10 auxiliary service for mediations
tnameserver-n85vm10 auxiliary service for mediations
osi OSI stack service for Q3 mediation
q3user Q3 Mediation

netfs-n85vm11 nfs client service


cpfnx2s-n85vm11 internal subsystems communication
OMAgentAF-n85vm11 self monitoring client
ssh-n85vm11 ssh service
n85vm11
nslcd-n85vm11 ldap client service
ntp-n85vm11 NTP service
lb-unify-primary LOAD Balancer - Primary
socks SOCKS protocol - NetAct connectivity service

ftp-n85vm12 FTP service


netfs-n85vm12 nfs client service
cpfnx2s-n85vm12 internal subsystems communication
OMAgentAF-n85vm12 self monitoring client
ssh-n85vm12 ssh service
nslcd-n85vm12 ldap client service
ntp-n85vm12 NTP service
n85vm12 pingServer-n85vm12 auxiliary service for mediations
PasswordChangeWS-
n85vm12 auxiliary service for mediations
tnameserver-n85vm12 auxiliary service for mediations
nx2s NX2S Mediation
xoh XoH Mediation
ne3sws_dynamicadaptation Dynamic Adaptations Services
generic_mediations Generic Mediations

OS90531EN15GLA0
50
© Nokia Solutions and Networks 2015.
Introduction

netfs-n85vm13 nfs client service


cpfnx2s-n85vm13 internal subsystems communication
OMAgentAF-n85vm13 self monitoring client
ssh-n85vm13 ssh service
nslcd-n85vm13 ldap client service
ntp-n85vm13 NTP service
n85vm13 pingServer-n85vm13 auxiliary service for mediations
PasswordChangeWS-
n85vm13 auxiliary service for mediations
tnameserver-n85vm13 auxiliary service for mediations
fmascii FMASCII Northbound interface Mediation
nbisnmp SNMP Northbound Interface Mediation
nbine3s NE3S/WS Northbound Interface Mediation

ftp-n85vm14 FTP service


netfs-n85vm14 nfs client service
cpfnx2s-n85vm14 internal subsystems communication
OMAgentAF-n85vm14 self monitoring client
ssh-n85vm14 ssh service
n85vm14 nslcd-n85vm14 ldap client service
ntp-n85vm14 NTP service
nbi3gcpm-n85vm14 3GPP Corba PM Northbound Interface Mediation
pingServer-n85vm14 auxiliary service for mediations
PasswordChangeWS-
n85vm14 auxiliary service for mediations
tnameserver-n85vm14 auxiliary service for mediations

ftp-n85vm15 FTP service


netfs-n85vm15 nfs client service
cpfnx2s-n85vm15 internal subsystems communication
OMAgentAF-n85vm15 self monitoring client
ssh-n85vm15 ssh service
n85vm15 nslcd-n85vm15 ldap client service
ntp-n85vm15 NTP service
nbi3gcpm-n85vm15 3GPP Corba PM Northbound Interface Mediation
pingServer-n85vm15 auxiliary service for mediations
PasswordChangeWS-
n85vm15 auxiliary service for mediations
tnameserver-n85vm15 auxiliary service for mediations

n85vm16 netfs-n85vm16 nfs client service

OS90531EN15GLA0
51
© Nokia Solutions and Networks 2015
Introduction

cpfnx2s-n85vm16 internal subsystems communication


OMAgentAF-n85vm16 self monitoring client
ssh-n85vm16 ssh service
nslcd-n85vm16 ldap client service
ntp-n85vm16 NTP service
common_mediations-
n85vm16 NE3S/WS, Southbound 3GPP Corba mediations
httpd-n85vm16 common mediations auxiliary http service
pingServer-n85vm16 auxiliary service for mediations
PasswordChangeWS-
n85vm16 auxiliary service for mediations
tnameserver-n85vm16 auxiliary service for mediations
DNS-Slave DNS Server slave

netfs-n85vm17 nfs client service


cpfnx2s-n85vm17 internal subsystems communication
OMAgentAF-n85vm17 self monitoring client
ssh-n85vm17 ssh service
nslcd-n85vm17 ldap client service
ntp-n85vm17 NTP service
n85vm17 common_mediations-
n85vm17 NE3S/WS, Southbound 3GPP Corba mediations
httpd-n85vm17 common mediations auxiliary http service
pingServer-n85vm17 auxiliary service for mediations
PasswordChangeWS-
n85vm17 auxiliary service for mediations
tnameserver-n85vm17 auxiliary service for mediations

netfs-n85vm18 nfs client service


cpfnx2s-n85vm18 internal subsystems communication
OMAgentAF-n85vm18 self monitoring client
n85vm18 ssh-n85vm18 ssh service
nslcd-n85vm18 ldap client service
ntp-n85vm18 NTP service
fm_pipe-n85vm18 FM FUNCTIONALITY 1

netfs-n85vm19 nfs client service


cpfnx2s-n85vm19 internal subsystems communication
OMAgentAF-n85vm19 self monitoring client
n85vm19 ssh-n85vm19 ssh service
nslcd-n85vm19 ldap client service
ntp-n85vm19 NTP service
fm_pipe-n85vm19 FM FUNCTIONALITY 2

OS90531EN15GLA0
52
© Nokia Solutions and Networks 2015.
Introduction

ftp-n85vm20 FTP service


netfs-n85vm20 nfs client service
dmgr-n85vm20 WAS - Deployment Manager of WAS cluster
was-n85vm20 WAS - Application Server
ihs-n85vm20 WAS - IBM Http Server
n85vm20 jetty-n85vm20 WAS - Java Http Server
cpfnx2s-n85vm20 internal subsystems communication
OMAgentAF-n85vm20 self monitoring client
ssh-n85vm20 ssh service
nslcd-n85vm20 ldap client service
ntp-n85vm20 NTP service

ftp-n85vm21 FTP service


netfs-n85vm21 nfs client service
was-n85vm21 WAS - Application Server
ihs-n85vm21 WAS - IBM Http Server
jetty-n85vm21 WAS - Java Http Server
n85vm21
cpfnx2s-n85vm21 internal subsystems communication
OMAgentAF-n85vm21 self monitoring client
ssh-n85vm21 ssh service
nslcd-n85vm21 ldap client service
ntp-n85vm21 NTP service

ftp-n85vm22 FTP service


netfs-n85vm22 nfs client service
was-n85vm22 WAS - Application Server
ihs-n85vm22 WAS - IBM Http Server
jetty-n85vm22 WAS - Java Http Server
n85vm22
cpfnx2s-n85vm22 internal subsystems communication
OMAgentAF-n85vm22 self monitoring client
ssh-n85vm22 ssh service
nslcd-n85vm22 ldap client service
ntp-n85vm22 NTP service

ftp-n85vm23 FTP service


netfs-n85vm23 nfs client service
was-n85vm23 WAS - Application Server
ihs-n85vm23 WAS - IBM Http Server
n85vm23
jetty-n85vm23 WAS - Java Http Server
cpfnx2s-n85vm23 internal subsystems communication
OMAgentAF-n85vm23 self monitoring client
ssh-n85vm23 ssh service

OS90531EN15GLA0
53
© Nokia Solutions and Networks 2015
Introduction

Service Description

lb-n85vm26 load balancer - PM PIPE


nfs-n85vm26 NFS server - PM PIPE
n85vm26 netfs-n85vm26 nfs client service
db_crons CRON functionalities - PM PIPE
system supporting service - PM PIPE

lb-n85vm27 load balancer - PM PIPE


n85vm27 netfs-n85vm27 nfs client service
message supporting service - PM PIPE

netfs-n85vm28 nfs client service


httpd-n85vm28 supporting service - PM PIPE
base-n85vm28 supporting service - PM PIPE
cmessage supporting service - PM PIPE
n85vm28 dns-namesurfer supporting service - PM PIPE
osscore supporting service - PM PIPE
osscore2 supporting service - PM PIPE
nww supporting service - PM PIPE
mvi supporting service - PM PIPE

netfs-n85vm29 nfs client service


n85vm29 rep-n85vm29 REP FUNCTIONALITY
base-n85vm29 supporting service - PM PIPE

dmgr_linas-n85vm30 Deployment Manager - PM WAS


was_linas-n85vm30 Application Server - PM WAS
nodeagent_linas-n85vm30 Node Agent - PM WAS
n85vm30 ihs_linas-n85vm30 IBM Http Server - PM WAS
netfs-n85vm30 nfs client service
sqm_eem-n85vm30 SQM Service
sqm_jboss-n85vm30 SQM Service

was_linas-n85vm31 Application Server - PM WAS


nodeagent_linas-n85vm31 Node Agent - PM WAS
ihs_linas-n85vm31 IBM Http Server - PM WAS
n85vm31 netfs-n85vm31 nfs client service
sqm_eem-n85vm31 SQM Service
sqm_jboss-n85vm31 SQM Service
sqm_jboss-n85vm31 SQM Service

OS90531EN15GLA0
54
© Nokia Solutions and Networks 2015.
Introduction

NetAct HealthCheck Tools


Service Manager: services description
Service Description

netfs-n85vm03 nfs client service


cpfnx2s-n85vm03 internal subsystems communication
OMAgentAF-n85vm03 self monitoring client
ssh-n85vm03 ssh service
n85vm03 nslcd-n85vm03 ldap client service
ntp-n85vm03 NTP service
DNS-Master DNS master Server
dirsrv LDAP master Server
tomcat_vcentplg integration with virtualized environment

netfs-n85vm04 nfs client service


cpfnx2s-n85vm04 internal subsystems communication
OMAgentAF-n85vm04 self monitoring client
n85vm04 ssh-n85vm04 ssh service
nslcd-n85vm04 ldap client service
ntp-n85vm04 NTP service
db ORACLE Database Server
1 © Nokia Solutions and Networks 2015

Fig. 23

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

4.3 WebSphere Application Server tools: Unify


Working as root user in a WAS node you can:
4.3.1.1 Check the status of WAS related services
# /opt/cpf/bin/cpfwas_status_was_services.sh –all
With the above script, you can also check specifically the status of application
servers and node agent (-as), deployment manager (-dmgr) and IBM HTTP server (-
ihs).
4.3.1.2 Stop WAS related services
# /opt/cpf/bin/cpfwas_stop_was_services.sh –all
With the above script, you can also specifically stop the application servers and node
agent (-as), deployment manager (-dmgr) and IBM HTTP server (-ihs).
4.3.1.3 Start WAS related services
# /opt/cpf/bin/cpfwas_start_was_services.sh –all
With the above script, you can also specifically start the application servers and node
agent (-as), deployment manager (-dmgr) and IBM HTTP server (-ihs).
4.3.1.4 Restart node agents
# /opt/cpf/bin/cpfwas_restartNodeAgents.sh
4.3.1.5 Display WAS Topology components
# /opt/cpf/bin/cpfwas_listTopology.sh
4.3.1.6 Restart an application server cluster
Run the following script to restart a specific cluster:
# /opt/cpf/bin/cpfwas_restartCluster.sh <clustername>
4.3.1.7 Integrated Solutions Console ISC
In NetAct, ISC serves as the administrative console that is used to manage
WebSphere Application Server. It supports a full range of product administrative
activities for the whole WebSphere cell.
The ISC is accessed through the VM node which is running Deployment Manager via
the following URLs:
https://<dmgr node FQDN>:9043/ibm/console or
https://<dmgr node FQDN>:9060/ibm/console

OS90531EN15GLA0
56
© Nokia Solutions and Networks 2015.
Introduction

NetAct HealthCheck Tools


WAS Tools
Task Command Description
Provides the status of application
Check the status of #/opt/cpf/bin/cpfwas_status_was_services.sh –all servers and node agents (as),
WAS related services deployment manager (dmgr) and IBM
HTTP server (ihs).
Stops the application servers and node
Stop WAS related #/opt/cpf/bin/cpfwas_stop_was_services.sh –all agent (as), deployment manager (dmgr)
services and IBM HTTP server (ihs).

Starts the application servers and node


Start WAS related #/opt/cpf/bin/cpfwas_start_was_services.sh –all
agent (as), deployment manager (dmgr)
services and IBM HTTP server (ihs).

Restart node agents #/opt/cpf/bin/cpfwas_restartNodeAgents.sh


List the NetAct
#/opt/cpf/bin/cpfwas_listClusterApplications.sh
Applications in WAS
Find out the names of
#/opt/cpf/bin/cpfwas_listTopology.sh
the application server
clusters

Restarting an APPS #/opt/cpf/bin/cpfwas_restartCluster.sh


cluster <clustername>

Starting all Stopped


#/opt/cpf/bin/cpfwas_startStoppedApplications.sh
Applications
1 © Nokia Solutions and Networks 2014

Fig. 24

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

4.4 WebSphere Application Server tools: PM Side


Working as root user in a PM WAS node you can:

4.4.1.1 Check the status of WAS related services and communication to


Oracle and LDAP
# linas_status.sh

4.4.1.2 Check the status of PM applications


# linas_apps_status.sh

4.4.1.3 Display additional information about the LinAS cluster configuration


#mercury_status.sh

4.4.1.4 Start / stop / restart PM Applications


#deploy_ear.sh <start / stop/ restart> <application name>

4.4.1.5 Restart WAS PM Cluster


#mercuryCluster.sh <restart / ripplestart>

OS90531EN15GLA0
58
© Nokia Solutions and Networks 2015.
Introduction

NetAct HealthCheck Tools


WAS Tools: PM Side
Task Command
Check the status of WAS related
# linas_status.sh
services

List the NetAct Applications in WAS # linas_apps_status.sh

Restarting the cluster # mercuryCluster.sh restart/ripplestart

Starting /Stopping Applications #deploy_ear.sh start/stop <application_name>

Applications in PM WAS nodes:


STARTED process is running in every LinAS server
PARTIALLY STARTED process is running in one or several LinAS servers but not
in every PM WAS node
STOPPED process is stopped in every server

If1 application is PARTIALLY STARTED it should be restarted.


© Nokia Solutions and Networks 2015

Fig. 25

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .
OS90531EN15GLA0 . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
. Solutions
© Nokia . . and. Networks
. . .2015. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

NetAct HealthCheck Tools


WAS Tools: Integrated Solutions Console

https://<DMGR node FQDN>:9043/ibm/console or https://< DMGR node >:9043/admin


http://< DMGR node >:9060/ibm/console or http://< DMGR node >:9060/admin

user: wassrvid ,
default password: wassrvid

1 © Nokia Solutions and Networks 2014

Fig. 26

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
60
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. Solutions
. . .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

4.5 NetAct Logs


4.5.1 Virtualization Logs (located on ESXi Hosts)
Log Location Description
/var/log/vmware/vpx/* vCenter logs
vmware.log VM specific log on ESXi host
/var/log/vmksummary.log Summary of ESXi host startup and shutdown, and an hourly
heartbeat with uptime, number of virtual machines running, and
service resource consumption
/var/log/vmkernel.log Core VMkernel logs, including device discovery, storage and
networking device and driver events, and virtual machine
startup
/var/log/syslog.log Management service initialization, watchdogs, scheduled tasks
and DCUI use
/var/log/hostd.log Host management service logs, including virtual machine and
host Task and Events, communication with the vSphere Client
and vCenter Server vpxa agent, and SDK connections

4.5.2 Computing Platform Logs - Linux


Log Description
/var/log/boot Logs messages from the boot and initialization
scripts
/var/log/btmp Logs bad console login attempts
/var/log/cron Logs commands and scripts that have been run
by LINUX cron
var/log/messages Logs messages produced by the LINUX system,
NTP, load balancer
/var/log/secure Logs SSH logins
/var/log/maillog Logs mail daemon messages
/var/log/rpmpkgs Logs all the rpms installed. Equivalent to ‘rpm –
qa’
/var/log/spooler Logs print messages and events in case the
system is connected to a printer
/var/log/wtmp Logs information of users who have successfully
logged-in and logged-out of the system
/var/log/yum.log Logs YUM events
/var/log/ha Service manager log files

OS90531EN15GLA0
61
© Nokia Solutions and Networks 2015
Introduction

4.5.3 Computing Platform Logs - Databases and Corba


Log Description
/var/log/cpf/oracle/ Oracle database installation log file
/opt/oracle/admin/oss/adump/*.aud Database security auditing files
/opt/oracle/product/11gR2/rdbms/audit/ The default location for the security
* audit files, if the AUDIT_FILE_DEST
parameter is not specified.
/opt/oracle/diag/rdbms/oss/oss/trace/* Database log files
/opt/oracle/product/11gR2/network/log/ Oracle client logs
*
/opt/oracle/product/11gR2/network/trac Network listener trace file directory
e/*
/opt/oracle/product/11gR2/sysman/log/* Management agent log file
/opt/JacORB/etc/jacorb/* CORBA related log files
/var/log/dirsrv/slapd- Logs connections, authentications,
<instance_name>/access searches, addition, modification,
deletion and replication
/var/log/dirsrv/slapd- Logs LDAP related operational errors
<instance_name>/errors

4.5.4 Load Balancer and WebSphere Application Server Logs


Log Description
/opt/IBMHttpServer/logs/* HTTP server log files
/opt/WebSphere/AppServer/profiles IBM service log. To be viewed using
/<WAS node>/logs/activity.log showLog tool
/opt/WebSphere/AppServer/profiles WebSphere Application Server log
/<WAS node>/logs/server1/* files
/var/log/messages Load Balancer messages from Linux
Virtual Server

OS90531EN15GLA0
62
© Nokia Solutions and Networks 2015.
Introduction

4.5.5 Applications Logs


Application Node Location
Adaptation Manager WAS nodes /var/opt/oss/log/am/*.log

PM Platform Logs REP node /var/opt/oss/log/pm/*.log

CM Analyzer WAS nodes /var/opt/oss/log/rac/*.log

CM Editor WAS nodes /var/opt/oss/log/rac/*.log

CM Operations Manager WAS nodes /var/opt/oss/log/rac/*.log

CM Reference WAS nodes /var/opt/oss/log/rac/*.log

Software Manager WAS nodes /var/opt/oss/log/swm/*.log

Addon Manager WAS nodes /var/opt/oss/log/cap/*.log

User Management WAS nodes /var/opt/oss/log/um/*.log

Network Element Access Control WAS nodes /var/opt/oss/log/neac/neac.log

Permission Management WAS nodes /var/opt/oss/log/um/*.log

Performance Manager LinAS nodes /mnt/sharedrw/content3/tmp/jflx.log

Admin Toolkit LinAS nodes /mnt/sharedrw/content3/tmp/jflx.log

Administration of Measurements WAS nodes /var/opt/oss/log/aom/*.log

Operating Documentation WAS nodes /var/opt/oss/log/onlinehelp/*.log

MOF Desktop WAS nodes /var/opt/oss/log/mofd/*.log

Common Applications WAS nodes /var/opt/oss/log/cap/*.log

NE3S/WS South Mediation Common- /var/opt/oss/log/NSN-


mediations common_mediations/*.log
3GPP CORBA South Mediation Common- /var/opt/oss/log/common_mediations/*.log
mediations
3GPP CORBA North Mediation nbi3gc node /var/opt/oss/log/nbi3gc/*.log

SNMP North Mediation nbisnmp node var/opt/oss/log/nbisnmp/*.log

FM platform (GEP) FM Pipe /var/opt/oss/log/gep/*.log


nodes

Alarm Pipe Supervision FM Pipe /var/opt/oss/log/gep/AlarmPipeSupervision.l


nodes og

OS90531EN15GLA0
63
© Nokia Solutions and Networks 2015
Introduction

4.5.6 South Mediations Logs


Mediation Logs Location and files

/var/opt/oss/log/common_mediations
oss_activityX_X.log
Common Mediations: NE3S/WS,
oss_errorX_X.log
southbound CORBA
oss_trace0_0.log
Service-Mix log directory

/var/opt/oss/log/snmp-mus/
oss_activityX_X.log
SNMP (in WAS nodes)
oss_errorX_X.log
oss_traceX_X.log
XoM /var/opt/oss/log/xoh
oss_activityX_X.log
oss_errorX_X.log
oss_trace0_0.log
Service-Mix log directory
NX2S /var/opt/oss/log/nx2s
oss_activityX_X.log
oss_errorX_X.log
oss_trace0_0.log
Service-Mix log directory
NWI3 /var/opt/oss/log/nwi3
FileMapping.log
PMAcknowledgerActivity.log
PMEventActivity.log
PMEventFiles.log
PMOesWriter.log
PublishActivity.log
RegistrationActivity.log
TopologyActivity.log
oss_activity0_0.log
oss_error0_0.log
oss_trace0_0.log
persistence.log
rac_cm_events.log
Service-mix Log directory

OS90531EN15GLA0
64
© Nokia Solutions and Networks 2015.
Introduction

Mediation Logs Location and files


Q3/MML /var/opt/oss/log/q3user
oss_activityX_X.log
oss_errorX_X.log
oss_trace0_0.log
Service-Mix log directory
/var/opt/oss/log/q3mml
/var/opt/oss/log/q3cm
/var/opt/oss/log/q3em

OS90531EN15GLA0
65
© Nokia Solutions and Networks 2015
Introduction

NetAct HealthCheck Tools


NetAct Logs: Applications, fm-pipes, Mediations
Application Node Location
Adaptation Manager WAS nodes /var/opt/oss/log/am/*.log Log reference in
PM Platform Logs REP node /var/opt/oss/log/pm/*.log the document
CM Analyzer WAS nodes /var/opt/oss/log/rac/*.log
CM Editor WAS nodes /var/opt/oss/log/rac/*.log
CM Operations Manager WAS nodes /var/opt/oss/log/rac/*.log
CM Reference WAS nodes /var/opt/oss/log/rac/*.log
Software Manager WAS nodes /var/opt/oss/log/swm/*.log
Addon Manager WAS nodes /var/opt/oss/log/cap/*.log
User Management WAS nodes /var/opt/oss/log/um/*.log
NEAC WAS nodes /var/opt/oss/log/neac/neac.log
Permission Management WAS nodes /var/opt/oss/log/um/*.log
Performance Manager LinAS nodes /mnt/sharedrw/content3/tmp/jflx.log
Admin Toolkit LinAS nodes /mnt/sharedrw/content3/tmp/jflx.log
AoM WAS nodes /var/opt/oss/log/aom/*.log
Operating Documentation WAS nodes /var/opt/oss/log/onlinehelp/*.log
MOF Desktop WAS nodes /var/opt/oss/log/mofd/*.log
Common Applications WAS nodes /var/opt/oss/log/cap/*.log
NE3S/WS SBI Common-mediations /var/opt/oss/log/NSN-common_mediations/*.log
3GPP CORBA SBI Common-mediations /var/opt/oss/log/NSN-common_mediations/*.log
3GPP CORBA NMI nbi3gc node /var/opt/oss/log/NSN-nbi3gc/*.log
SNMP North Mediation nbisnmp node var/opt/oss/log/nbisnmp/*.log
FM platform (GEP) FM Pipe nodes /var/opt/oss/log/gep/*.log
Alarm Pipe Supervision FM Pipe nodes /var/opt/oss/log/gep/AlarmPipeSupervision.log

1 © Nokia Solutions and Networks 2015

Fig. 27

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
66
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. Solutions
. . .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

4.6 Log Monitor Tool


As there are many virtual machines representing a single NetAct cluster, it is difficult
to check logs on each of these virtual machines and troubleshoot the problems in
NetAct system.
Log Monitor tool provides a real-time view of logs of all virtual machines. It is installed
on the virtual machine where Self Monitoring is running.
Log Monitor tool is not enabled by default after installation. You can enable or disable
Log Monitor tool at runtime, and also verify the status of Log Monitor tool.
When log monitor is enabled, logs are mounted from all virtual machines, and log
monitor creates the following directories:

Logs Location on Server Mount Location on Self Monitoring


Virtual Machine
Application /var/opt/oss/log /var/opt/oss/logc/<nodeName>/osslogdir
Logs
WebSphere /opt/WebSphere/AppServer/profiles/ /var/opt/oss/logc/<nodeName>/dmgr
Logs dmgr/logs
/opt/WebSphere/AppServer/profiles/ /var/opt/oss/logc/<nodeName>/applicationserver
<nodename>/logs
/opt/IBMHttpServer/logs /var/opt/oss/logc/<nodeName>/httpserver
Deployment /oss/global/var/mediation/south/met /var/opt/oss/logc/<nodeName>deployment
Logs a/import
NetAct PM /var/opt/nokia/oss/local/common/log /var/opt/oss/logc/<nodename>/onepm_commonlog
logs
/var/opt/nokia/oss/local/common/var /var/opt/oss/logc/<nodeName>onepm_trace
/tmp/trace

OS90531EN15GLA0
67
© Nokia Solutions and Networks 2015
Introduction

NetAct HealthCheck Tools


Log Monitor Tool

A log directory specific to Log Monitor tool is created during installation. However,
log folders specific to virtual machine are created when the tool is invoked. When
enabled, the script collects logs at run time.
Log directories
To enable collecting the logs from the virtual machines: can be viewed
1. Log in to vcenterselfmon virtual machine as omc user. only as omc user
2. To view log directories of:
a) NetAct nodes excluding all Performance Manager virtual machines, execute:
./logmonitorconsole.sh -enable
Or
b) NetAct and NetAct Performance Manager VMs except VMs running was_linas
services, execute:
./logmonitorconsole.sh -enableWithPM
This invokes the tool and collects the logs from the virtual machines. The logs collected are
stored at /var/opt/oss/logc directory.

1 © Nokia Solutions and Networks 2015

Fig. 28

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OS90531EN15GLA0
. . . . .
68
. . . . . . . . . . . . . . . . . . . . . . . . . . . .© Nokia
. Solutions
. . .and .Networks
. .2015.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction

5 Exercises

OS90531EN15GLA0
69
© Nokia Solutions and Networks 2015
Introduction

Exercise 1
Title: NetAct Administration
Objectives: Explain the main goals of NetAct Administration.
List the components of the Reliability implementation in NetAct
Explain the purpose of the main NetAct Administration Tasks.
Pre-requisite: none
Task
Please answer the following questions.

Query
1. What is the purpose of NetAct Administration? Why is important to guarantee the
Reliability of NetAct System?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

2. Explain the purpose of the High Availability implementation, Load Balancing


Implementation and Backup and Restore.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

3. What is monitored in the Self-Monitoring part of the system Reliability?


________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

OS90531EN15GLA0
70
© Nokia Solutions and Networks 2015.
Introduction

4. List the main administrator tasks and describe the purpose of each of them.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

5. How is Security related to the System Reliability?


________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

OS90531EN15GLA0
71
© Nokia Solutions and Networks 2015
Introduction

Exercise 2
Title: NetAct Reliability
Objectives: Describe the functionality of the high availability in the
virtualized platform.
Explain how the virtualized infrastructure responds to different
the different failure scenarios.
Describe the purpose of the backup solutions implemented in
NetAct.
Describe the information obtained by the Self Monitoring
Functionality.
Pre-requisite: none
Task
Please answer the following questions.

Query
1. Explain how high availability is implemented in the virtualized environment?
Which components of the virtualized environment provide the high availability
functions?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

2. In the virtual environment, what happens when a server is broken? How the
NetAct system will react to this kind of failure with regards to the availability of the
applications and functions? Shutdown a server and check what happens to the
functions in NetAct.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

OS90531EN15GLA0
72
© Nokia Solutions and Networks 2015.
Introduction

3. The JEE Cluster or WebSphere application cluster provides the applications


functionalities in NetAct. How this type of cluster guarantees high availability of
the applications deployed? Shutdown one of the WAS nodes (different from
DMGR) and check the impact on the applications in NetAct.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

4. Test the functionality of load balancer. Display the interfaces information of the
two load balancer virtual machines. What can you conclude based on the current
information? Shutdown the active load balancer. What happens with the data
going through such interfaces?
________________________________________________________________
________________________________________________________________
________________________________________________________________

5. Describe the backup implementations and solutions available in NetAct. Verify if


backups have been executed for file system, oracle database and virtualized
environment. Execute a backup of the virtual environment.
________________________________________________________________
________________________________________________________________
________________________________________________________________

6. What types of information is possible to obtain with the Self Monitoring


functionality? Where can you see in NetAct this information?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

OS90531EN15GLA0
73
© Nokia Solutions and Networks 2015
Introduction

Exercise 3
Title: NetAct Architecture and System Components
Objectives: Describe the functionality of the high availability in the
virtualized platform.
Explain how the virtualized infrastructure responds to different
the different failure scenarios.
Describe the purpose of the backup solutions implemented in
NetAct.
Describe the information obtained by the Self Monitoring
Functionality.
Pre-requisite: none
Task
Please answer the following questions.

Query
1. Describe in general terms the virtualization environment for the NetAct 15
platform.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

2. List the three configuration types of NetAct 15. How is the system size scaled
taking into account the virtualization? Please check Operating Documentation if
answer is not known.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

OS90531EN15GLA0
74
© Nokia Solutions and Networks 2015.
Introduction

3. List the southbound interfaces and the network domains that are connected
through them.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

4. Execute the command to find the NetAct applications in WAS, Unify Side and PM
Side. Write the command and list at least three of the applications. Do the same
but this time using the Integrated Services Console. Verify that the results are the
same.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

5. Stop the WAS related services. Specifically, what components are stopped? Start
the services again and verify that the services are in a normal status. Write the
commands to execute these tasks
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

6. Explain the main functions of service engines and binding components. What
other component types can be found in the JBI architecture?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________

OS90531EN15GLA0
75
© Nokia Solutions and Networks 2015
Introduction

7. List of 3 different protocols handled by Apache components


________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

8. What are the High Availability solutions implemented for ServiceMix components
in NetAct? Stop one of the PM pipes and verify the impact in the functionality via
the Monitor Application.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

OS90531EN15GLA0
76
© Nokia Solutions and Networks 2015.
Introduction

Exercise 4
Title: NetAct Health Check Tools
Objectives: Revise the Status of NetAct Functionality using smanager.pl
tool.
Setup Preventive Health Checks
Analyze the Log Outputs from Log Monitor Tools
Pre-requisite: none

Task
Please connect to the NetAct 15 environment with the credentials provided by the
trainer and execute the following tasks.

Query

1. Execute the smanager.pl command. Which of your virtual machines are in a


status that can indicate problems in the system? What are the functions of these
virtual machines?
________________________________________________________________
________________________________________________________________
________________________________________________________________

2. Use the diagnostic tool to evaluate the current status of the test system. What
are your findings about the system? List the test cases you are using and what is
applicability of them.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

OS90531EN15GLA0
77
© Nokia Solutions and Networks 2015
Introduction

3. Start the Collection of Logs using the Log Monitor Tool. Where can you actually
see those logs? List some of the log files available.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
_____________________________

4. Create a task to Schedule the Preventive Health Check in your test system.
Schedule the first execution to be in the next 10 minutes. How can you verify the
health check was executed?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
____________________________________________________________

OS90531EN15GLA0
78
© Nokia Solutions and Networks 2015.

Vous aimerez peut-être aussi