High Level Solution Approach

POC Control Switch Next Generation

Architecture
Dual DL380 with Integrated NMHOST option

MotorolaConfidentialRestrictedPage1

Date8/12/2009

Revision History

Revision

Date

Author

Description

R00.00.01

12/04/09

YuMei
Bennett

Initial Release

MotorolaConfidentialRestrictedPage2

Date8/12/2009

Content

Background
Project Assumptions
Scope
Solution Description
Solution Assumptions
System IP Network Design
Middleware Functions
Application Considerations

MotorolaConfidentialRestrictedPage3

Date8/12/2009

Background
POC current platform is reaching end of life
P4 NMHOST EOL 2008, project to replace it with Xeon server is in
progress.
SanNet II RAID EOL Sep. 2009, selection of next generation SAS
RAID is on going.
CPCI chassis EOL June 2009, the main objective of this project.

POC current SW
OS Linux kernel (2.2) is out-dated
OS package is home grow, no clear middleware package
separation
Employ application level redundancy as well as board level
redundancy

MotorolaConfidentialRestrictedPage4

Date8/12/2009

Background
Evaluation of HW platform have resulted 3 options
HP DL 380 2U server with 2 Nehalem quad core cpu
Emerson c2000 3U ATCA chassis, capable of 2 ATCA payload
blades. Each blade have 2 Nehalem quad core cpu
Emerson 1440 13U 14 slot ATCA chassis, 2 slot for IP switches and
12 slot for ATCA payload blade. Each blade have 2 Nehalem quad
core cpu.

SW Strategies
Clear middleware package
Linux kernel upgrade to 2.6
Go to commercially supported OS distributions.

MotorolaConfidentialRestrictedPage5

Date8/12/2009

Background
Three High Level Solution Approach
HW

OS

Middleware

Solution A

HP DL380

RedHat

CCP

Solution B

Emerson
c2000

WindRiver

Open SAF

Solution C

Emerson
1440

WindRiver

Avantellis

MotorolaConfidentialRestrictedPage6

Date8/12/2009

Scope
This HSLA is about solution A!
As the first step, define one system option that has
Simple configuration
Least effort
Enable the ability to produce effort estimate quickly to different
configurations, including SE, dev, CCP, NPI and test

Not necessarily the final commercial roll out solution!

Solution A have other alternatives and add on options
This is the third HLSA for solution A
For the purpose of effort estimation for two DL380 with
integrated NMHOST, no Xeon box configuration!

MotorolaConfidentialRestrictedPage7

Date8/12/2009

Scope
Existing Control Switch

Next Gen Control Switch

HOW?
Ret
ain
Int e

Slots 5-21 are for

the CPU Cards

e
grat

CPCI Chassis

Replace
DL 380 G6
MotorolaConfidentialRestrictedPage8

Date8/12/2009

Project Assumptions
This is only for CS next gen
M3 target is 1H 2011
SAS RAID is required and only required for Next Gen.
The effort to migrate CS SCSI RAID to SAS RAID is NOT
part of this HLSA!
Final HW will be same DL 380 model but with newer
dual six core westmere cpu which is only available in
mid 2010

MotorolaConfidentialRestrictedPage9

Date8/12/2009

Solution Description
Double DL 380 G6 server replace current 21 slot CPCI
chassis and NMHOST
DL 380 G6 is a 2U rack mount server with dual quad core cpu,
24GB RAM
8 Ethernet ports with one 4 ports NIC extension card
2 72GB HDD

Capacity of CS should be comparable to best

configuration cPCI fully loaded chassis.
System cost is only a fragment
cPCI(92K)+2Xeon(12K)=104K;
2xDL380=10K

MotorolaConfidentialRestrictedPage10

Date8/12/2009

Solution Description
CurrentcPCI

RAID

NMHOST01

NMHOST02

Custer
service

Customer Edge
Switch/Router
RAINLINK

Customer
IP Network
Customer Edge
Switch/Router

ZNYX2

ZNYX1

VRRP

MotorolaConfidentialRestrictedPage11

....

cPCI Chassis

Date8/12/2009

Solution Description

RAID

Customer Edge
Switch/Router

Customer
IP Network

HP DL 380

HP DL 380

Customer Edge
Switch/Router

MotorolaConfidentialRestrictedPage12

Date8/12/2009

Solution Assumptions
RedHat Enterprise 5 running on DL 380
CCP team provides middleware for DL 380
No IP switch
DL380 will boot from local drive, unlike CPCI cpu cards net
boot from NMHOST/RAID
SNMP functionality will be the responsibility of application
layer for both client and agent.
Associated customer documentation need to be updated per
next gen design

MotorolaConfidentialRestrictedPage13

Date8/12/2009

System IP Network Design

Assumptions

POC system do not rely on customer IP equipment for

any internal system traffic switching, routing and
monitoring
Customer IP network is viewed as IP cloud, POC system
provides redundant connections for each type of traffic,
it does not care redundancy scheme employed at
customer IP interface.

MotorolaConfidentialRestrictedPage14

Date8/12/2009

System IP Network Design

cPCI
All traffics in/out of system through Znyx IP switch

Znyx is a layer 2 and 3 switch

Runs VRRP for redundancy
Traffic separation by VLAN configuration
IPMH has two externally visible floating IP, one for mobile signaling traffic and one for CS/AD traffic.
NMHOST has one externally visible floating IP for O&M traffics
Each MRS has a fixed externally visible IP for media traffic.

VLAN
subnet name

Zhp#
(VLAN)

Switches

Ports

IP Subnet

VRRP

Private

Zhp0

Znyx

1-20

192.168.chid.0/24

192.168.0.252

Management

Zhp1

Znyx/customer
switch/Nmhost
/IPMH

21

Customer provides
(typical 10.x)

m.m.m.252

Mobile

Zhp2

Znyx/MRS/IPMH
customer switch

22

s.s.s.0/24 (typical 10.y)

s.s.s.252

RAINLINK

Zhp99

23

10.254.254.0/24

10.254.254.252

MotorolaConfidentialRestrictedPage15

Date8/12/2009

System IP Network Design

cPCI

SAS
1

Edge Switch

Customer
IP Network
Edge Switch
3

1
9
2
0
2
1
2
2

1
9
2
0
2
1
2
2

ZNYX02

ZNYX01

NMHOST01
3

cPCI
Chassis

RAID
3

NMHOST02
1

SAS
19NMHOST01;private192.168.0.0/24
20NMHOST02;private192.168.0.0/24
21ManagementandinterCS/ADtraffic;m.m.m.0/24
22Mobilesignalingandmedia;s.s.s.0/24

MotorolaConfidentialRestrictedPage16

Date8/12/2009

System IP Network Design

Next Gen
Redundant direct connections between all elements
No IP switches
No VLAN concept
Traffic separation by physical LAN direct connections
Subnet Name

System
Element

Ports

IP Subnet

InterDL380
PrivateLAN

DL380

DL380
5&6

192.168.20.0/24

InterCS<>AD&
Management
PublicLAN

DL380
CustomerEquipment

DL380
2&3

Customer provides
m.m.m.0/24

Customer provides
Ex. m.m.m.252

Mobile
Public LAN

DL380
Customer Equipment

DL380
1&4

Customer provides
s.s.s.0/24

Customer provides
Ex. s.s.s.252

MotorolaConfidentialRestrictedPage17

Floating public IP

Date8/12/2009

System IP Network Design

Next Gen
Inter CS <-> AD & Management m.m.m.0/24

SAS

Mobile - s.s.s.0/24
23

1
2

Edge Switch
1
3

Customer
IP Network

DL380 G6
CS-1

4
3

Mobile - s.s.s.0/24

Inter CS <-> AD & Management m.m.m.0/24

Private

Inter CS <-> AD & Management - m.m.m.0/24

Mobile - s.s.s.0/24
3
4

192.168.10.0/24

Edge Switch
2
2

RAID
2

DL380 G6
CS-2

1
1

Mobile - s.s.s.0/24
Inter CS <-> AD & Management - m.m.m.0/24

MotorolaConfidentialRestrictedPage18

SAS

Date8/12/2009

System IP Network Design

Next Gen
4 external IP connections

DL380 CS01 Ethernet ports 1 and 4 are bonded together Signaling and Media traffic.
DL380 CS01 Ethernet ports 2 and 3 are bonded together CS/AD and management traffic.
DL380 CS02 Ethernet ports 1 and 4 are bonded together Signaling and Media traffic.
DL380 CS02 Ethernet ports 2 and 3 are bonded together CS/AD and management traffic.

1 internal IP connections

DL380 Port 5 and 6 are bonded together for inter-DL380 communications

9 IP addresses need to be provided by customer, 4 for bonded connections and 3 for

logical floating IP for mobile, inter CS/AD and system O&M traffics. 2 for MRS NAT IP
that lives on each DL380.
Port bonding and bridging are used to bond multiple ports towards the same
destination with single physical IP to provide link level redundancy
Logical floating IP will be used to provide redundancy for all customer visible
traffics.
CCP middleware will perform physical layer monitoring on all DL380 Ethernet ports.
6 Ethernet ports required on DL 380
Adds inter-DL380 connection, sits on a separate private VLAN.

MotorolaConfidentialRestrictedPage19

Date8/12/2009

Middleware Functions
Assume all hardware monitoring of DL380 will be done by middleware layer (CCP).

Middleware layer will use openHPI for hardware monitoring. For devices that cannot be monitored using OpenHPI like
Ethernet ports, file system utilization, CCP will use other methods to monitor those devices.
Application need to register with middleware layer to receive hardware state change notifications
CPU and memory utilization monitoring will be done by APP team.
Two disk drives running as RAID1 internal to DL380 transparent to the application

CCP logs will be stored on local disk along with OS logs

Separate disk partition for OS and CCP

Application probing and app state updates

Application task can register for task probing

An API is provided to receive updates about all tasks that are probed by CCP.

Node state updates

Application can register for node state updates.

Hardware states, CCP tasks and application will be considered in the calculation of node state. The node will transition to
multiple states (maintenance -->cold-standby) before getting to "active" state.
Application can post events using CCP API to restart or shutdown application node.
NMHOST will continue to monitor the state of the DL380 using the existing app level interface. CCP will monitor DL380 locally
and take recovery action if required.

Critical hardware failures without redundancy will cause node reboot

Non-critical application failures will be reported by CCP layer and recovery handled by app layer
Assume CCP layer does application node restart for critical app failures without node reboot.

Individual process level redundancy management at APP layer

This is for entire app sequence up and down
Application will be started/stopped in layers through the framework provided by CCP
Startup/shutdown scripts will be provided by application team

MotorolaConfidentialRestrictedPage20

Date8/12/2009

Middleware Functions
Watchdog timer will be configured by CCP. Watchdog timeout will cause system
reboot and dump of kernel memory will be collected before reboot.
ccpsnap (collection of debug info)
Utility used to collect/gather debug info. Configuration of this utility can be
updated to collect app debug data. Can also be integrated with app if app has
a similar utility.

CCP middleware OS requirement

Disk partition requirement
OS build should support NETLINK socket
OS build should support Evlog, support to forward kernel and syslog events to
evlog subsystem.

Middleware/HA services startup

CCP task will be started from /etc/inittab.
CCP will update /etc/inittab as part of post installation to add the CCP task.
This task will start all other CCP tasks.

Delivery of CCP content to app

CCP will deliver CCP content as a rpm to app team.
CCP will use OS distro from the vendor for testing.

MotorolaConfidentialRestrictedPage21

Date8/12/2009

CS APP Team Functions

OS for DL380

Receiving RedHat package, managing patches including security patches

Deliver OS SW package to build team and SW factory

Installation and Upgrades for DL380

DL380 will NOT perform net boot like cPCI payload cards. All SW packages including OS,
middleware and OAMP all stored locally
Provide MOP for DL380 scratch installation and field upgrade
Firmware upgrades method of procedures

Two disk drives running RAID1 internal to DL380 transparent to the application

RAID configuration and Disk partitioning

App will use these drives for scratch install
CCP logs will be stored on local disk along with OS logs
Separate disk partition for OS and CCP and OAMP

Ethernet port configuration

Responsible for Ethernet port and IP config/bond/redundancy.

Customer IP Information collection
DHCP or manual delivery to CCP middleware. Potentially at RAID/NMHOST first as it is
today.

Either by file or via CCP API

MotorolaConfidentialRestrictedPage22

Date8/12/2009

CS APP Team Functions

APP processes running on DL380 Same as it does today

Crisscross active/backup CCSW process configuration across two DL380

AD cache and IPMH will run FT across two DL380
No enhanced MRS. Normal MRS runs multiple active instances on both DL380. Each with a fixed public IP
AD caches preserved like it does today across cpu board
CCSW, IPMH and AD cache failure does not lose any established call sessions
MRS is an exception, call hosted on that MRS process will drop
Number of each functional process (CCSW, IPMH, AD cache and MRS) will be optimized for max. capacity. This
will require proto type modeling

Redundancy Scheme

Two DL380 will function in load sharing mode in normal condition, failover to either single box when necessary
The backup for the CCSW, ADHLR and IPMH on one DL380 will be on the other
When one DL380 fails, the other one will only have actives running without any backups until the failed
DL380 back in service
MRS apps have no backups, there will be equal number of MRS apps on each of the DL380s.
There will not be any movement of MRS from one DL380 to another on hardware failure.
The number of MRS on each DL380 should be such that they can take the entire call load when one
DL380 fails.
IPMH handles external visible floating IP for SIP signaling and CS/AD traffic, these floating IP failover will
be managed by CEM as it does today.
NMHOST services such as billing and logging will run on one of the DL380 in normal operation, when
failure happens to one of the NMHOST service, restart failed service on the same DL380. When DL380
that runs NMHOST service fails, all service should be migrated to other live DL380
No cluster service running on DL380. CCP is responsible for node restart.

MotorolaConfidentialRestrictedPage23

Date8/12/2009

CS APP Team Functions

NMHOST Services Related Changes
Application redundancy frame work related changes
PoC Middleware (CEM/CDE) remain the same for fault tolerance and application process
management
PoC Middleware (CEM) will work with CCP API on DL380. CEM will be the critical PoC
application monitored by CCP.
NMHOST services migration need not to rely on cluster service.

O&M aspect changes

PoC Middleware (CEM) will be link to the CCP high availability library to receive
events/notifications from CCP.
App start up/shutdown scripts for CCP to use through CEM with CCP. CCP will only start
PoC Middleware agent on DL380 CEM.
Start/stop CCP functions running on DL380
GUI Chassis, Card and Process configuration page will be yanked

SNMP events/alarms/state change events

PoC middleware (CEM on DL380) will consume events/notifications from CCP and
convert them to internal format. Then will be relayed with any modifications needed
using the existing North Bound interfaces on the nmhost application.

MotorolaConfidentialRestrictedPage24

Date8/12/2009