GGSN Monitoring & First Layer Handling

December 9th, 2013 by Doddy W., PSNOS Dept., CNO Div.

Preface

GGSN is the core network node of GPRS system

GGSN functions:
Assign IP address to subscriber
Perform AAA, maintain connection to SGSN, generate charging, etc

Page 2
GGSN Hardware

Ericsson SSR 8020

Huawei UGW9811
PGP-X16
Ericsson M320

Ericsson M120

?
Page 3
GGSN APN Mapping

Hostname Hardware Location

GGSNTBS1 Ericsson M120 TB Simatupang, Jakarta

GGSNTBS3 Ericsson M320 TB Simatupang, Jakarta

GGSNTBS4 Ericsson M320 TB Simatupang, Jakarta

GGSNTBS5 Ericsson SSR8020 TB Simatupang, Jakarta

GGSNBRN1 Ericsson M120 Buaran, Jakarta

GGSNBRN2 Ericsson M320 Buaran, Jakarta

GGSNBRN3 Ericsson M320 Buaran, Jakarta

GGSNBRN4 Ericsson SSR8020 Buaran, Jakarta

GGSNPKB1 Ericsson M320 Arifin Ahmad, Pekanbaru

GGSNSBY1 Huawei PGP-X16 Gayungan, Surabaya

Page 4
GGSN Coverage

Hostname APN Handling Coverage

GGSNTBS1 - Internet / telkomsel, mms West & Central Java, Kalimantan, East Java,
- internet.r / telkomsel.r , blackberry.net.r, Balinusra, Sulawesi
mms.r (roaming)
- corporate APN
GGSNTBS3 internet/telkomsel, blackberry.net, mms West & Central Java, Kalimantan, East Java,
Balinusra, Sulawesi
GGSNTBS4 internet/telkomsel, blackberry.net West & Central Java, Kalimantan, East Java,
Balinusra, Sulawesi
GGSNTBS5 blackberry.net, internet/telkomsel (trial) West & Central Java, Kalimantan

GGSNBRN1 - Internet / telkomsel, mms West & Central Java, Kalimantan, East Java,
- internet.r / telkomsel.r , blackberry.net.r, Balinusra, Sulawesi
mms.r (roaming)
- corporate APN
GGSNBRN2 internet/telkomsel, blackberry.net, mms West & Central Java, Kalimantan, East Java,
Balinusra, Sulawesi
GGSNBRN3 internet/telkomsel, blackberry.net West & Central Java, Kalimantan, East Java,
Balinusra, Sulawesi
GGSNBRN4 blackberry.net, internet/telkomsel (trial) West & Central Java, Kalimantan

GGSNPKB1 internet/telkomsel, blackberry.net Sumatera

Page 5 GGSNSBY1 blackberry.net East Java, Balinusra, Sulawesi

GGSN Interface (1 of 2)

Gn
Gi
Gx
Ga Ga Gmb

Gx
Gy
GGSN
Gp
Gmb Gn Gi
LIG
Gp LIG
Gy

Page 6
GGSN Interface (2 of 2)

Gn network Gi network

Internet

SGSN

Gn Blackberry
Network Gn Server

GGSN

Corporate
Network
SGSN

Page 7
Physical Topology

Te-7/1
Gi Core Te-7/2
Gate
GGTBS1
xe-0/0/0
Te-8/2 Internet

Te-4/3
Te-4/2

Te-4/4
Te-8/4 Gate
xe-1/0/0
Te-7/3 TBS Physical Connection
Te-7/4

GGTBS3
xe-0/0/0

2A
xe-1/0/0

5A
6A

3A
GGTBS4 5B
M320 xe-0/0/0 xe-3/0/0 6B IPS
xe-0/0/5 xe-3/0/1 2B M8000-
xe-0/0/10 3B
xe-1/0/2 TBS1
xe-0/0/15
xe-3/0/2 Te-4/1
xe-2/0/0 xe-3/0/3 Te-4/2
xe-2/0/1 Gi_Switch_TBS1 xe-0/0/20 Te-4/3
xe-2/0/2 xe-0/0/21
Gi, PBR, NAT,Radius xe-2/0/3 xe-0/0/25 PS1
and Load Balancer xe-2/0/4 xe-0/0/26
xe-2/0/5 xe-0/0/30
xe-2/0/6 xe-0/0/31
VIPRION1TBS xe-1/0/0
xe-2/0/7
xe-1/0/1
1/2.1 xe-1/0/25 xe-3/0/4
F5 PB100 Google Cache
1/2.2 xe-1/0/26 xe-3/0/5
xe-0/0/35 Google
Cache
xe-3/0/6
VIPRION2TBS xe-3/0/7
ge-1/0/30

xe-1/0/20
xe-1/0/15
1/2.1 xe-1/0/6

xe-1/010
ge-1/0/31

xe-1/0/5
F5 PB100
1/2.2 xe-1/0/11 ge-1/0/32
Google

SW1-Port 49
SW1-Port 50
SW1-Port 49

SW1-Port 49
SW1-Port 50

SW1-Port 50
xe-1/0/16 ge-1/0/33 Cache

Cluster # 1
SAPC2 SAPC1

Cluster # 2
xe-1/0/21

Cluster # 3
ge-1/0/34
F5_PBR_ 1/2.1
Intersite

SASN
TBS1

SASN
SASN
1/2.2
1/2.3 Module

SW-2Port 50
SW2-Port 50

SW2-Port 50
SW2-Port 49

SW2-Port 49

SW2-Port 49
1/2.4
Xe-1/0/10

xe-1/0/20
xe-1/0/15

RD1 ge-1/0/31
xe-1/0/5

xe-1/0/25 ge-1/0/32
RD2 2/2.1 xe-1/0/26 ge-1/0/33
2/2.2 xe-0/0/0 ge-1/0/34 Google
Cache
2/2.3 xe-0/0/5 xe-3/0/4
2/2.4 xe-0/0/10 xe-3/0/5
xe-0/0/15
xe-0/0/35
F5_PBR- 1/2.1 xe-2/0/0 xe-3/0/6 Gx and Gy
xe-3/0/7 Google
TBS2 1/2.2 xe-2/0/1 Cache
ge-1/0/30
1/2.3 xe-2/0/2 xe-1/0/2
1/2.4 xe-2/0/3 xe-1/0/1
RD1
xe-1/0/0
RD2 2/2.1 xe-2/0/4 xe-0/0/31
Gi_Switch_TBS2
2/2.2 xe-2/0/5 xe-0/0/30
2/2.3 xe-2/0/6 xe-0/0/26
xe-0/0/25
2/2.4 xe-2/0/7
xe-0/0/21 PS3
xe-1/0/6 xe-0/0/20
xe-1/0/11
Gi VAS xe-1/0/16
xe-3/0/2 Te-9/3
xe-3/0/3 Te-9/4
Anti xe-1/0/21
Anti
Spam xe-3/0/0
Spam xe-3/0/1
xe-9/0/0 SRX
DNS DNS
xe-10/0/0
ns1-3g-tbs ns2-3g-tbs
WAP
xe-9/1/0

LAN1
LAN1
LAN2

LAN2
GW S R X FW
WAP Force-10 xe-10/1/0
GW

Page 8 WAP
GW
Bluecoat DMZ
Service Topology APN internet/telkomsel

Spam
DNS Gi
Filter
SGSN

gate-router

F5 F5
Gn Load Load
Internet
Balancer Balancer
Network

GGSN SASN
gate-router

SGSN Proxy

Page 9
Service Topology APN blackberry.net

SGSN
Blackberry
network
grx-router

Gn VPN Links
via TELIN/AICENT
Network

GGSN Blackberry
grx-router server
SGSN

Page 10
Service Topology APN mms

SGSN

F5
Gn Load
Balancer
Network
MMS
GGSN Proxy MMSC

SGSN

Page 11
Service Topology Roaming APN

Roaming Partner GPRS Roaming Exchange Telkomsel

network network network

SGSN /
t.r
grx-router rne el.r
te s
in om
Gi elk
t
via AICENT/TELIN

GRX Gn
Network Network Gi blackberry.net.r

via AICENT/TELIN
Gi
GGSN m
m
s.r
grx-router
SGSN

Page 12
GGSN Monitoring (1 of 3)

Tools:
Eniq
Cacti http://10.251.15.85/cacti/

Parameters to be monitored:
APN level
Active PDP number
Available IP pool
Uplink & downlink bytes/packets
PDP activation attempt
Node level
Gn, Gi throughput
Routing Engine CPU utilization & temperature
GTP-U & GTP-C CPU utilization & memory usage
Page 13
GGSN Monitoring (2 of 3)

APN level

Page 14
GGSN Monitoring (3 of 3)

Node level

Page 15
First Layer Handling Daily Checklist (1 of 4)

@GGTBS1-re1> show system uptime

Current time: 2013-12-08 21:51:58 WIT
System booted: 2013-12-05 23:45:34 WIT (2d 22:06 ago) Pengecekan uptime
Protocols started: 2013-12-06 00:01:42 WIT (2d 21:50 ago)
Last configured: 2013-12-06 20:57:01 WIT (2d 00:54 ago) by bennypn
9:51PM up 2 days, 22:06, 1 user, load averages: 0.50, 0.42, 0.42

@GGTBS1-re1> show system commit

0 2013-12-06 20:57:01 WIT by bennypn via cli commit synchronize Pengecekan perubahan
1 2013-12-06 20:43:16 WIT by bennypn via cli commit synchronize
2 2013-12-06 00:32:52 WIT by ericsson via cli commit synchronize
konfigurasi terakhir

@GGTBS1-re1> show chassis environment

Class Item Status Measurement
Temp PEM 0 OK
PEM 1 OK Pengecekan
Routing Engine 0 OK 43 degrees C / 109 degrees F
Routing Engine 0 CPU OK 44 degrees C / 111 degrees F temperature
Routing Engine 1 OK 42 degrees C / 107 degrees F
Routing Engine 1 CPU OK 45 degrees C / 113 degrees F

Page 16
First Layer Handling Daily Checklist (2 of 4)

@GGTBS1-re1> show chassis routing-engine

Routing Engine status:
Slot 1:
Current state Master
Election priority Backup
Temperature 42 degrees C / 107 degrees F
CPU temperature 45 degrees C / 113 degrees F
DRAM 3584 MB
Memory utilization 20 percent
CPU utilization:
User 11 percent Pengecekan
Background 0 percent routing engine
Kernel 9 percent
Interrupt 2 percent
Idle 78 percent
Model RE-A-2000
Serial ID 9009050302
Start time 2013-12-05 23:45:34 WIT
Uptime 2 days, 22 hours, 8 minutes, 20 seconds
Last reboot reason Router rebooted after a normal shutdown.
Load averages: 1 minute 5 minute 15 minute
0.23 0.36 0.40

Page 17
First Layer Handling Daily Checklist (3 of 4)

@GGTBS1-re1> show ospf neighbor instance Gn_vrf

Address Interface State ID Pri Dead
Pengecekan
221.132.243.113 xe-0/0/0.32 Full 221.132.254.65 1 37 routing OSPF
221.132.243.121 xe-1/0/0.33 Full 221.132.254.73 1 30
Gn network

@GGTBS1-re1> show ospf neighbor instance Gi_?

Gi_blackberry.r
Gi_inetcorporate
Gi_inetspec
Gi_mms Pengecekan
Gi_telkomsel routing OSPF
Gi_telkomsel.r
Gi_test Gi network
Gi_tsel

@GGTBS1-re1> show ospf neighbor instance Gi_telkomsel

Address Interface State ID Pri Dead
192.168.0.10 xe-0/0/0.101 Full 192.168.200.101 0 37
192.168.0.11 xe-0/0/0.101 Full 192.168.200.102 0 33
192.168.0.26 xe-1/0/0.102 Full 192.168.200.101 0 31
192.168.0.27 xe-1/0/0.102 Full 192.168.200.102 0 34

Page 18
First Layer Handling Daily Checklist (4 of 4)

@GGTBS1-re1> show log messages | last 5

Dec 8 22:07:29 2013 GGTBS1-re1 (FPC Slot 5, PIC Slot 0) gc50
gtpcd[257]:gc-5/0/0: <PDP>: <04>: pdp_RadiusRecvAcctRespRadius
failure: status=RADIUS_SERVERDOWN, sending gtp failure cause=204
imsi=510100342629156, msisdn=6285203629156, nsapi=5, apn=telkomsel,
idx=0, s2a_wlan=N, type=21, ip=182.12.227.114, ip_dual=-,
Pengecekan log
sgsn_ip_c=221.132.235.21, sgsn_ip_u=221.132.235.21, teid_data=153047,
pdp_id=aaaaaaaaa2641030 STATE:[SSM = 'CREATE', S =
'PDP_ST_RADIUS_START'], first occurrence at 22:07
Dec 8 22:07:29 2013 GGTBS1-re1 (FPC Slot 4, PIC Slot 0) gc40
gtpcd[253]:gc-4/0/0: <PDP>: <00>: pdp_DeletePdpRsp: delete rsp from
221.132.233.99 for PDP ID aaaaaaaa80542270 was REJECTED (v1Cause=192),
first occurrence at 22:07

@GGTBS1-re1> show log chassisd | last 5

Dec 6 00:32:49 ch_fru_power_sequencer FEB 3 step 0
Dec 6 00:32:49 ch_fru_power_sequencer FEB 4 step 0
Dec 6 00:32:49 ch_fru_power_sequencer FEB 5 step 0
Dec 6 00:32:49 ... power sequencer finished ...
Dec 6 00:32:49 CHASSISD_PARSE_COMPLETE: Using new configuration

Page 19
First Layer Handling Service Checklist

@GGTBS1-re1> show services epg pgw statistics apn telkomsel

APN: telkomsel
Active PDP contexts: 445128
With traffic class conversational: 0 Pengecekan statistik
With traffic class streaming: 0
With traffic class interactive: 444573 per APN
With traffic class background: 555
Active EPS bearers: 0
Active dedicated EPS bearers: 0
Active EPS bearers for WLAN: 0
Active IPv6 PDP contexts: 0
Active IPv4v6 PDP contexts: 0
Active IPv6 EPS bearers: 0
Active IPv4v6 EPS bearers: 0
Attempted PDP activations: 65203433, Completed: 64740019

--lines skipped--

Radius statistics:
Failed RADIUS Authentication procedures: 0
Failed RADIUS Accounting procedures: 689757
RADIUS accounting servers:
Server IP address: 192.168.0.14
Accounting requests sent: 129384101
Accounting responses received: 128694343
Accounting requests timed out: 689757
Accounting requests retransmitted: 0
Page 20
Invalid authenticators received: 1
First Layer Handling Service Checklist

@GGTBS1-re1> show services epg pgw statistics imsi 510101772143945

International mobile subscriber identity: 510101772143945
Mobile station ISDN number: 6282217143945
Pengecekan statistik
NSAPI/Bearer ID: 5
APN: telkomsel user
PDP address: 182.12.174.62
Framed Route: 0.0.0.0/ 0
Charging ID: 3779425136
Gi/SGi DiffServ class: AF22, Gn/S5 DiffServ class: AF22
Maximum Bit Rate Downlink (kbps): 8640
Maximum Bit Rate Uplink (kbps): 5760
GTP-C version: 1
Session activation timestamp: Sun Dec 8 22:13:36 2013
Total session duration: 0h 0m 23s
Idle time: 0h 0m 1s
Packets uplink: 166
Packets downlink: 149
Bytes uplink: 37948
Bytes downlink: 61210

Serving Node:
Serving Node Type: SGSN
Control plane IP address: 221.132.227.35
Data plane IP address: 221.132.227.35

Page 21
Escalation Process

1st level - Packet Switch Network Operation Support (PSNOS) team

Moch SN Tsalasanto (Soleh), moch_sn_tsalasanto@telkomsel.co.id,
08111987745
Doddy Widanto, doddy_widanto@telkomsel.co.id, 08119935594
Alex Atmana, alex_atmana@telkomsel.co.id, 08119953112
Benny P Nugraha, benny_p_nugraha@telkomsel.co.id, 08119783464
Dwi SG Utomo, dwi_sg_utomo@telkomsel.co.id, 0811276002

2nd level - Senior Engineer Packet Switch Operation Expert Job

Coordinator
Eko Prasetyo, eko_prasetyo@telkomsel.co.id, 0811939961

Page 22
Question

Page 23
Page 24