Cambium High Availability For Wireless Manager Server On A Linux Platform Setup Guide

Cambium
High Availability for

Wireless Manager
Server on a Linux
Platform
Setup Guide
System Release
4.0
Issue 6 September 2013

© 2013 Cambium Networks. All Rights Reserved.
Accuracy
While reasonable efforts have been made to assure the accuracy of this document, Cambium Networks assumes
no liability resulting from any inaccuracies or omissions in this document, or from use of the information obtained
herein. Cambium reserves the right to make changes to any products described herein to improve reliability,
function, or design, and reserves the right to revise this document and to make changes from time to time in
content hereof with no obligation to notify any person of revisions or changes. Cambium does not assume any
liability arising out of the application or use of any product, software, or circuit described herein; neither does it
convey license under its patent rights or the rights of others. It is possible that this publication may contain
references to, or information about Cambium products (machines and programs), programming, or services that
are not announced in your country. Such references or information must not be construed to mean that Cambium
intends to announce such Cambium products, programming, or services in your country.
Copyrights
This document, Cambium products, and 3rd Party Software products described in this document may include or
describe copyrighted Cambium and other 3rd Party supplied computer programs stored in semiconductor
memories or other media. Laws in the United States and other countries preserve for Cambium, its licensors, and
other 3rd Party supplied software certain exclusive rights for copyrighted material, including the exclusive right to
copy, reproduce in any form, distribute and make derivative works of the copyrighted material. Accordingly, any
copyrighted material of Cambium, its licensors, or the 3rd Party software supplied material contained in the
Cambium products described in this document may not be copied, reproduced, reverse engineered, distributed,
merged or modified in any manner without the express written permission of Cambium. Furthermore, the
purchase of Cambium products shall not be deemed to grant either directly or by implication, estoppel, or
otherwise, any license under the copyrights, patents or patent applications of Cambium or other 3rd Party supplied
software, except for the normal non‐exclusive, royalty free license to use that arises by operation of law in the sale
of a product.
Restrictions
Software and documentation are copyrighted materials. Making unauthorized copies is prohibited by law. No part
of the software or documentation may be reproduced, transmitted, transcribed, stored in a retrieval system, or
translated into any language or computer language, in any form or by any means, without prior written permission
of Cambium.
License Agreements
The software described in this document is the property of Cambium and its licensors. It is furnished by express
license agreement only and may be used only in accordance with the terms of such an agreement. See the end
user licenses published (redundantly) in the user guide or the server administration guide for Release 4.0.

© 2013 Cambium Networks, Inc. All Rights Reserved.
Table of Contents
Accuracy...................................................................................................................................... 2
Copyrights ................................................................................................................................... 2
Restrictions ................................................................................................................................. 2
License Agreements.................................................................................................................... 2
1 About High Availability.................................................................................... 7
1.1 Prerequisites for High Availability .................................................................................. 7
1.1.1 Prerequisites Common to All HA Deployments ...................................................................... 7
1.1.2 Prerequisites Unique to Same Subnet Deployments .............................................................. 8
1.1.3 Prerequisites Unique to Differing Subnet Deployments......................................................... 8
1.2 How the Switchover on Differing Subnets Works .......................................................... 9
1.3 Caveats for High Availability Operations ...................................................................... 10
2 Setting Up High Availability ............................................................................13
2.1 Setting Up the WM and Network File Servers.............................................................. 13
2.2 Configuring the Related Services.................................................................................. 18
3 Upgrading to WM Release 4.0 SP 3 ................................................................19
4 Starting and Stopping the WM Service .............................................................21
5 Changing the IP Address of a High Availability Server.........................................23
6 Testing High Availability .................................................................................27
7 Troubleshooting High Availability Setups ..........................................................31
7.1 Responding to trouble cases......................................................................................... 31
7.2 Resyncing the HA Databases ........................................................................................ 31
7.3 Preventing HA Boot Errors............................................................................................ 32
8 Disabling and Uninstalling High Availability.......................................................35
Resources for Support ..........................................................................................37
Network Updater Help.............................................................................................................. 37
Community Forum .................................................................................................................... 37
Technical Support ..................................................................................................................... 38
List of Figures
Figure 1: Example switchover configuration for differing subnets............................................................. 10
Figure 2: License Manager remote, database replication .......................................................................... 13
Figure 3: License Manager on each server, database replication............................................................... 14
Figure 4: License Manager remote, database remote (no replication) ...................................................... 14
Figure 5: License Manager on each server, database remote (no replication) .......................................... 15

List of Procedures
Procedure 1: To disable SELinux ................................................................................................................... 8
Procedure 2: To check and change the interface names of the Linux servers ............................................. 8
Procedure 3: To enable RIP Version 2........................................................................................................... 8
Procedure 4: To set up WM and Pacemaker servers.................................................................................. 13
Procedure 5: To upgrade to Service Pack 3 and get its HA features and fixes ........................................... 19
Procedure 6: To upgrade to Service Pack 3 and forfeit its HA features and fixes ...................................... 19
Procedure 7: To start the WM service........................................................................................................ 21
Procedure 8: To check the status of High Availability ................................................................................ 21
Procedure 9: To stop the WM service ........................................................................................................ 21
Procedure 10: To change only the physical IP address of both HA servers................................................ 23
Procedure 11: To change only the virtual IP address of the HA servers..................................................... 24
Procedure 12: To change both the virtual and physical IP addresses of the HA servers ........................... 24
Procedure 13: To simulate the failure of the primary server ..................................................................... 27
Procedure 14: To simulate the failure of the secondary server ................................................................. 28
Procedure 15: To resync the HA databases after only one server has been down seven days.................. 31
Procedure 16: To resync the HA databases after both servers have been down seven days .................... 32
Procedure 17: To prevent HA boot errors .................................................................................................. 32
Procedure 18: To disable High Availability ................................................................................................. 35
Procedure 19: To uninstall High Availability ............................................................................................... 35
High Availability for WM on Linux Issue 6
September 2013
1 About High Availability

High Availability (HA) is an optional Wireless Manager (WM) software feature that supports a
designated backup WM server fully ready for service whenever it is needed. This provides
 management server redundancy
 optional replication of the database
 replication of other important files, regardless of whether you configure database replication
 seamless management and trap handling through primary server outages.
Technical support for your migration is available through contact on the sources listed in the user
guide section titled "Contacting Support Representatives."
1.1 Prerequisites for High Availability
1.1.1 Prerequisites Common to All HA Deployments

The High Availability feature requires all of the following, regardless of whether the server IP
addresses exist within the same subnet:
 dedicated installation packages from Cambium Networks
 two Linux devices for WM servers with
− one specified as primary, the other as secondary (backup)
− a dedicated IP address assigned to each server (see Prerequisites Unique to Same
Subnet Deployments on Page 8 or Prerequisites Unique to Differing Subnet
Deployments on Page 8)
− each having its firewall disabled
− Security‐Enhanced Linux (SELinux) disabled on each (see Procedure 1 on Page 8)
− identical interface names (see Procedure 2 on Page 8)
− one operator‐configurable virtual IP address that links to both of these servers
− connectivity to License Manager
− multicasting kept enabled in each server and in the network (for example, in switches
and routers)
 one further additional device for a standalone MySQL Server where remote database is
preferred over database replication
 Pacemaker with CoroSync software installed on the separate network file server to
− broker communications and perform load balancing between the WM servers
− monitor and manage their services
 operator mapping of related services on each server device to reflect the high availability
deployment (mapping is not automated by the scripts from the distribution)
 operator mapping from each server to the remote network file server
 uninstallation, when desired, through only the documented, supported sequence.
7
Issue 6 High Availability for WM on Linux
September 2013
Procedure 1: To disable SELinux

1. At the Linux OS prompt, enter setenforce 0.
2. Open the file /etc/sysconfig/selinux for editing.
3. Reset the SELINUX value to SELINUX=disabled.
4. Write and close the file.
5. Reboot the server device.
Procedure 2: To check and change the interface names of the Linux servers
1. At the Linux OS prompt, enter ifconfig.
RESULT: The system response is composed as follows:
ifconfig interface_name IP_address netmask mask broadcast broadcast_address
2. Repeat Step 1 on the other server.
3. If either name needs to be reset to be identical to the other, enter the following command:
ifconfig old_name name new_name
1.1.2 Prerequisites Unique to Same Subnet Deployments

Where the dedicated IP address assigned to each server exists in the same subnet, multicasting
must be kept enabled in each server and in the network (in switches and routers, for example).
1.1.3 Prerequisites Unique to Differing Subnet Deployments

Where the dedicated IP address assigned to each server exists in a different subnet, the following
additional requirements apply:
 The Linux version must be 6.x.
 The following router settings are in force:
 Routing Information Protocol (RIP) Version 2 is enabled as the routing protocol
(see Procedure 3 on Page 8)
 The router configuration includes the subnet of the
◦ first server
◦ second server
◦ virtual IP address (VIP), if it differs from the first two
Procedure 3: To enable RIP Version 2

1. Open the command‐line interface of your Cisco router.
RESULT: The system provides the Router> command prompt.
2. Enter enable.
RESULT: The system provides access to the privileged EXEC mode of commands and
changes the command prompt to Router#.
8
September 2013
3. Enter conf t.
RESULT: The system provides access to the global configuration mode and changes the
command prompt to Router(config)#.
4. Enter router rip.
RESULT: The router is enabled to send and receive RIP Version 2 packets.
5. Enter version 2.
RESULT: The router will globally use Version 2.
6. Enter network first_subnet.
7. Enter network second_subnet.
8. If the virtual IP address exists on a subnet different from those of the first two (servers),
enter network VIP_subnet.
9. Enter network router_IP_subnet.
10. Enter no validate-update-source.
RESULT: The router will not perform validation checks on the source IP address of incoming
RIP updates.
11. Enter end.
RESULT: The command prompt returns to Router#, and the command mode returns to
privileged EXEC.
1.2 How the Switchover on Differing Subnets Works

For the case where the primary and secondary Linux servers exist on different subnets, the high
availability distribution (from the file HA_LinuxOrCentOSVersionNumber.x.zip) installs
Quagga Routing Software by Savannah on each server. This software instructs the router to listen for
Routing Information Protocol (RIP) messages that it will send about the servers.
When the virtual IP (VIP) connection becomes active between the primary and secondary server, this
software sends an RIP update from the primary server to the router about the connection, which
causes the router to add the VIP prefix to its routing table. This entry establishes the next hop
(192.168.1.1 in the example configuration shown in Figure 1 on Page 10).
9
September 2013
Figure 1: Example switchover configuration for differing subnets
When a failover occurs, the VIP connection between the servers drops and, as a result, the routing
software stops sending the RIP messages to the router, which then experiences a timeout and
removes the routing table record of the VIP prefix.
When the secondary server (HA Server 2 in the example) raises the VIP connection to become
primary, the routing software restarts in that server and notifies the router in an RIP update message.
Upon receipt of this update, the router adds a record of the VIP prefix to its routing table, with
192.168.2.1 (in the example) as the next hop.
The high availability framework from the HA software distribution automatically starts and stops the
routing software as these events occur. No operator intervention is required to start or stop Quagga.

1.3 Caveats for High Availability Operations

The following caveats apply to any High Availability deployment of WM:
 No other network software is enabled on either WM server device.
 While both WM servers remain in communication, switchover (swap of roles for primary and
backup server) is smooth. However, if the servers lose communication with each other, then
each behaves like the primary server role.
 Database replication is according to the MySQL Server replication feature. Where replication
is configured, the database in the designated primary server is always considered the master
copy, and in the secondary server is always considered the slave copy.
 If the secondary server is ever out of service for longer than seven days, then the replicated
database that it maintains becomes corrupted.
10
September 2013
 Restarts of the primary and secondary server must occur within seven days of each other to
prevent corruption of the database that the secondary server maintains. However, a script
that the distribution provides ensures the proper sequence and synchronization of server
restarts as long as both are in service. If an out‐of‐service condition prevents the script from
ensuring this, then the operator must deliberately restart the primary server and then start
the secondary server (after verifying that the primary is again successfully operating).
 A switchover event drops client sessions that were up in the primary server. As in single‐
server deployments, a dropped client session spawns a message in the client, advising the
user that the connection has been interrupted, that the interface should be closed, and that
a new client session should be launched.
 WM and MySQL services must be controlled by only the high availability feature and its
scripts (without operator intervention).

11
September 2013
2 Setting Up High Availability

important ........... Before performing the procedures that this chapter provides, carefully review both
Prerequisites for High Availability on Page 7 and Caveats for High Availability
Operations on Page 10.
2.1 Setting Up the WM and Network File Servers

To set up the servers, perform the following steps.
Procedure 4: To set up WM and Pacemaker servers

1. Decide whether you want License Manager installed on each High Availability server device
or on a separate (third) server within network reach of them.
note ............. Any of the Figure 2 through Figure 5 configurations is supported for High Availability.
Figure 2: License Manager remote, database replication
Client/Device
Related Servers High Availability WM

License Manager Virtual IP
R0KD
Primary Backup Server
DHCP Server
WM WM
DNS
BAM MySQL MySQL
RADIUS
13
September 2013
Figure 3: License Manager on each server, database replication
Client/Device

Virtual IP
R0KD
DHCP Primary Backup Server

Server
WM WM
DNS
BAM MySQL MySQL
RADIUS License License

Manager Manager
2. Decide whether you want database replication (Figure 2 or Figure 3 above) or a single
database on a third server (Figure 4 or Figure 5 below).
Figure 4: License Manager remote, database remote (no replication)
Client/Device MySQL

License Manager Virtual IP
R0KD
Primary Backup Server
DHCP Server
WM WM
DNS
BAM
RADIUS
14
September 2013
Figure 5: License Manager on each server, database remote (no replication)
Client/Device MySQL

Virtual IP
R0KD
DHCP Primary Backup Server

Server
WM WM
DNS
BAM License License

Manager Manager
RADIUS
3. Install WM Release 4.0 and License Manager as follows:
◦ On each server that is not currently running WM Release 4.0, perform the installation
according to the instructions that the WM Release 4.0 quick start guide provides.
◦ On each server that is currently running one of those releases, perform the upgrade
according to the instructions that the WM Release 4.0 server administration guide
provides.
◦ While performing these WM software installation processes, install
− License Manager in the configuration that you decided upon in Step 1 of this
procedure.
− MySQL Server in the configuration that you decided upon in Step 2 of this
procedure.
4. Install your WM license and device licenses, referring to either the quick start guide or the
server administration guide as your reference.
5. On one of the WM servers, perform the following steps:
a. Use the WM Administrator Tool to configure the server. (See the server administration
guide.)
b. Use the Discovery Configurator as many times as needed until all of the elements in
your network have been discovered. (See the Release 4.0 user guide.)
c. Ensure that this server can push configuration values, generate reports, receive traps,
and in all other ways perform as the WM management server described in the user
guide, without throwing unexpected errors to the logs.
d. Stop the server.
6. On the other WM server, perform Step 5 of this procedure.
7. In the /etc/hosts files on each server device, add an entry that points to the other server
device by hostname.
8. Access the web site
http://www.cambiumnetworks.com/support/management‐tools/wireless‐manager/.
15
September 2013
9. Download the High Availability package appropriate to your operating system version, either
◦ HA_5.x.zip for Linux 5.x or CentOS 5.x
◦ HA_6.x.zip for Linux 6.x or CentOS 6.x
10. Enter the following command:
unzip Path/ZipFileName –d /usr/local/cambium/wm/server/ha/packages/
11. Enter the following command:
scp /usr/local/cambium/wm/server/ha/packages/HighAvailabilityResources/*
/usr/local/cambium/wm/server/ha/packages/
12. On one of the WM servers, perform the following steps:
a. If the DVD drive is not /media/RHEL_release i386 DVD/, perform the following
steps:
1) Change directory to /usr/local/cambium/wm/server/ha/packages.
2) Open the file yum-rh-dvd.conf for editing.
3) Beneath the line name= Server Base, edit the value between
baseurl=file:// and Server so that it is consistent with the location of your
DVD drive, except representing each space with %20.
note ............. View the default on this line as an example of the required syntax.
4) Save and close the file.
b. Insert your Red Hat media into the DVD drive.
c. Change directory to /usr/local/cambium/wm/server/ha.
d. Open the file pacemaker.conf for editing.
e. For VIRTUAL_IP, set the address to the Virtual IP address that will always correlate to
whatever server is primary at the time.
f. For SAME_SUBNET, set the value as
 yes for the same subnet.
 no for a different subnet.
g. For BINDING_IP, set the address to the subnet IP address of the local WM server.
For example
 if the local interface is 192.168.5.92 with netmask 255.255.255.0, set
bindnetaddr to 192.168.5.0.
 if the local interface is 192.168.5.92 with netmask 255.255.255.192, set
bindnetaddr to 192.168.5.64.
h. For INTERFACE_NAME, set the value returned for interface from the ip addr
command for the static IP address that you assigned to the machine. (The default
interface name is eth0.)
i. For CLUSTER_NODE1 and CLUSTER_NODE2, set the names to the hostnames of the
local WM servers.
j. For REPLICATE_MYSQL
 set the value to yes if your configuration is like Figure 2 or Figure 3 on Page 14.
 set the value to no if your configuration is like Figure 4 or Figure 5 on Page 15.
k. Save and close the file.
l. Enter ./pacemaker_install.sh.
16
September 2013
m. Only if in Step 10h you set REPLICATE_MYSQL to yes, perform the following two
substeps:
1) Enter ./create_db_rep_user.sh.
2) At the prompt, type in the password associated with the user root.
n. Remove the Red Hat media from the DVD drive.
13. On the other WM server, perform Step 10 of this procedure.
14. On only one of the WM servers, perform the following steps:
a. Insert the Red Hat media into the DVD drive.
b. Enter the following commands:
 corosync-keygen
This command creates the authentication keys.
 scp /etc/corosync/authkey
root@hostname:/etc/corosync/authkey (where hostname identifies
the other WM server)
This command transfers the authentication keys to the other server.
c. Remove the Red Hat media from the DVD drive.
15. On one of the servers, enter ./ha_enable.sh.
16. On the other server, enter ./ha_enable.sh.
17. Only as described under Starting and Stopping the WM Service on Page 21, start both of the
WM servers.
18. As described under Starting and Stopping the WM Service on Page 21, check the status of
High Availability.
19. If the status tool does not report that both servers are online as shown below, verify that
you performed all the steps in order and have correctly set the parameters of the
pacemaker.conf file. If you discover any discrepancies, repeat the installation.
============
Last updated: Wed May 15 01:23:22 2013
Stack: openais
Current DC: HA_hostname1 - partition with quorum
Version: 1.0.8-9881a7350d6182bae9e8e557cf20a3cc5dac3ee7
2 Nodes configured, 2 expected votes
0 Resources configured.
============
Online: [ HA_hostname1 HA_hostname2]

If the status tool does report that both servers are online as shown above, proceed.
20. After the status tool has reported that both servers are running, enter on only one server
./wm_ha_initialize.sh.
note ............. The initialize tool has configured Pacemaker to control and monitor WM in the High
Availability configuration.
21. Ensure that the status tool output now indicates that one server is primary and the other is
backup.
17
September 2013
2.2 Configuring the Related Services

The related servers and their services (some of which are shown on Pages 13 and 15) require
attention and some or all may require adjustment as described in this section.
If you use DHCP server on both WM servers, identically configure DHCP on each of them, initially and
whenever you make a change of this configuration in one them. Either
 map the virtual IP address to the MAC address of each server.
 set each DHCP server to be responsible for only a unique subset of the range of assignable IP
addresses.
If you use DNS server on both WM servers, identically configure DNS on each of them, initially and
whenever you make a change of this configuration in one them. Either
 point to the Virtual IP address (which always correlates to current primary server) as the
lone DNS server
 the actual IP addresses of both WM servers as multiple DNS servers.
If you deploy a BAM server, use the WM Administrator Tool to identically configure BAM in each,
initially and whenever you make a change of this configuration in one them. Point to the Virtual IP
address of WM.
If you deploy a RADIUS server, identically configure RADIUS server on each of them, initially and
whenever you make a change of this configuration in one them. Point to the Virtual IP address of
WM.
18
September 2013
3 Upgrading to WM Release 4.0 SP 3

Service Pack 3 for WM Release 4.0 includes features and fixes for High Availability, but how this
service pack is installed will determine whether those features and fixes are available on the HA
server. Procedure 5 will make them available; Procedure 6 will not.
Procedure 5: To upgrade to Service Pack 3 and get its HA features and fixes
1. Either
a) Uninstall HA on both servers.
b) Install Service Pack 3 on both servers.
or
a) Install Service Pack 3 on both servers.
b) Uninstall HA on both servers.
2. Install HA on both servers.
Procedure 6: To upgrade to Service Pack 3 and forfeit its HA features and fixes
1. Use Procedure 9 on Page 21 to stop HA on both servers.
2. Install Service Pack 3 on both servers.
3. Use Procedure 7 on Page 21 to start HA on both servers.
19
September 2013
4 Starting and Stopping the WM Service

important ........... Do not use the server start, stop, or restart tools described in other WM documents.
Those are for only single-server deployments.
The High Availability distribution included dedicated scripts for starting the WM service, checking the
status of High Availability, and stopping the WM service.
Procedure 7: To start the WM service

1. On the server that you want to initially be the primary server, perform the following steps:
a. Change the working directory to /usr/local/cambium/wm/server/ha (if not
already in this directory).
b. Enter ./wm_ha_start.sh.
2. Enter crm_mon to open an instance of the Pacemaker console to the dynamic output of its
monitoring tool.
3. Wait until the output of the tool has identified the running server as primary.
4. Press Ctrl+C to close the monitoring console.
5. On the other server, perform Step 1 of this procedure.
6. Use Procedure 8 to check the server roles and the status of the HA feature.
Procedure 8: To check the status of High Availability

7. On either server (the second is suggested, since this procedure typically follows
Procedure 7), either
◦ enter ./wm_ha_status.sh to launch the High Availability status tool, which
generates HA status output once and then automatically exits.
◦ enter crm_mon to open an instance of the Pacemaker console to the dynamic output of
its monitoring tool.
8. Review the output.
9. If you executed crm_mon in Step 2, press Ctrl+C to close the monitoring console.
note ............. In future status checks, you can add the -1 option to the crm_mon command to cause
Pacemaker to perform a single check, report the static output of that check, and then
automatically exit its console.
Procedure 9: To stop the WM service

1. On the secondary server, perform the following steps:
a. Change the working directory to /usr/local/cambium/wm/server/ha (if not
already in this directory).
b. Enter ./wm_ha_stop.sh.
2. On the primary server, perform Step 1 of this procedure.
21
September 2013
5 Changing the IP Address of a High Availability

Server
The contents of this chapter are based on the following assumptions:
 All commands are run from the directory /usr/local/cambium/wm/server/ha.
 Only IP addresses are being changed, not also hostnames.
This chapter provides steps
 to change only the physical IP address of both HA servers (Procedure 10 on Page 23)
 to change only the virtual IP address of the HA servers (Procedure 11 on Page 24)
 to change both the virtual and physical IP addresses of the HA servers (Procedure 12
on Page 24)
Procedure 10: To change only the physical IP address of both HA servers

2. On each HA server in either order, enter ./ha_disable.sh.
3. As described in the WM server administration guide, stop License Manager. (If LM is
installed on both servers, stop it on both.)
4. According to operating system‐specific procedures, change the physical IP address of each of
the servers.
5. In the /etc/hosts file of each server, edit its hostname line to reflect its new physical IP
address.
important ........... Leave the hostname itself unchanged. Changing it would require a complete HA
deployment installation.
6. On each server, enter service network restart.
note ............. The above step is required following any change in the hosts file.
7. Open the file pacemaker.conf for editing.
8. Change the value of BINDING_IP.
9. Save and close the file.
10. Open the file /etc/corosync/corosync.conf for editing.
11. Find the string bindnetaddr.
12. Change its value to the BINDING_IP that you set in Step 8.
14. Update the Hostname / IP field of the Licensing panel in the WM Administrator Tool to
reflect the new IP address in each server. (If LM is installed on both servers, update it on
both.)
15. Click the Save Configuration button.
16. Click the Restart LM button.
17. In the Licensing panel of the WM Administrator Tool on each server, click the License Status
button.
23
September 2013
18. If LM fails to show licenses for both servers, check and correct the IP address configured in
the Licensing panel for the server that has no license.
19. On each HA server in either order, enter ./ha_enable.sh.

Procedure 11: To change only the virtual IP address of the HA servers

1. On the secondary server, enter ./ha_disable.sh.
2. On the primary server, enter ./ha_disable.sh.
3. If the servers are still on SP 2 or are operating on SP 3 via Procedure 6: To upgrade to Service
Pack 3 and forfeit its HA features and fixes on Page 19, enter the following command on
each:
rm –f /var/lib/mysql/*.info
5. For VIRTUAL_IP, set the address to the new Virtual IP address that will always correlate to
8. Use Procedure 7 on Page 21 to start the HA service on both servers.
9. On the primary server, enter crm_mon.
10. Reading the system output, verify that both servers are operating and online.
11. Press Ctrl+C to restore the command prompt.
12. Wait for three minutes.
13. On either server, but not both, enter ./wm_ha_initialize.sh.
RESULT: After approximately ten minutes, the primary server begins operating as the
newly assigned virtual IP address.

Procedure 12: To change both the virtual and physical IP addresses of the HA servers
2. On each HA server in either order, enter ./ha_disable.sh.
3. As described in the WM server administration guide, stop License Manager. (If LM is
installed on both servers, stop it on both.)
4. According to operating system‐specific procedures, change the physical IP address of each of
the servers.
5. In the /etc/hosts file of each server, edit its hostname line to reflect its new physical IP
address.
important ........... Leave the hostname itself unchanged. Changing it would require a complete HA
deployment installation.
6. On each server, enter service network restart.
note ............. The above step is required following any change in the hosts file.
24
September 2013
7. Update the Hostname / IP field of the Licensing panel in the WM Administrator Tool to
reflect the new IP address in each server. (If LM is installed on both servers, update it on
both.)
8. Click the Save Configuration button.
9. Click the Restart LM button.
10. In the Licensing panel of the WM Administrator Tool on each server, click the License Status
button.
11. If LM fails to show licenses for both servers, check and correct the IP address configured in
the Licensing panel for the server that has no license.
12. If the servers are still on SP 2 or are operating on SP 3 via Procedure 6: To upgrade to Service
Pack 3 and forfeit its HA features and fixes on Page 19, enter the following command on
each:
rm –f /var/lib/mysql/*.info
14. For VIRTUAL_IP, set the address to the new Virtual IP address that will always correlate to
15. For BINDING_IP, set the address to the subnet IP address of the local WM server.
17. Open the file /etc/corosync/corosync.conf for editing.
18. Find the string bindnetaddr.
19. Change its value to the BINDING_IP that you set in Step 15.
23. On the primary server, enter crm_mon.
24. Reading the system output, verify that both servers are operating and online.
25. Press Ctrl+C to restore the command prompt.
26. Wait for three minutes.
27. On either server, but not both, enter ./wm_ha_initialize.sh.
RESULT: After approximately ten minutes, the primary server begins operating as the
newly assigned virtual IP address.
25
September 2013
6 Testing High Availability

This chapter provides procedures to test High Availability by simulating the failure of one server, then
the other. At their start, these procedures assume that
 all commands are executed from /usr/local/cambium/wm/server/ha.
 both servers are operating.
 the hostnames are as follows:
− hostname1 is the primary server.
− hostname2 is the secondary server.
Procedure 13: To simulate the failure of the primary server

1. On either server, enter ./wm_ha_status.sh.
2. In the system output, make note of the
◦ process names for the primary and secondary servers (wm-primary-svr and
wm-backup-svr, respectively in the example below)
◦ hostnames quoted for the primary (beneath Resource Group: wm-active) and
the secondary (beneath Resource Group: wm-passive) server.
EXAMPLE:
Resource Group: wm-active
failover-ip (ocf::heartbeat:IPaddr): Started hostname1
wm-primary-svr (lsb:wm-primary-svr): Started hostname1
Resource Group: wm-passive
wm-backup-svr (lsb:wm-backup-svr): Started hostname2
3. In your web browser, use the virtual IP address of the primary server to access the initial
web page for a WM client session launch.
4. By any one of the following means, disrupt the primary server:
◦ Kill the wmserver process as follows:
− At the Linux OS prompt, enter service wmserver status.
RESULT: The system responds wmserver (pid wm_pid) is running...
− Enter kill -9 wm_pid.
note.............. Killing wmserver does not free the port 9090. Free this port to start the wmserver again.
◦ If you noted the process name of the primary server in Step 2, enter service
wm-primary-svr stop.
◦ Unplug its power cable.
◦ Click the Stop WM Server button in the WM Server panel of the its WM Administrator
Tool interface.
5. After five minutes, enter ./wm_ha_status.sh on the original secondary server.
27
September 2013
6. In the system output, verify that hostname1 of the original primary server is shown as
OFFLINE, and hostname2 of what was the secondary server is shown as Online:
EXAMPLE:
Online: [hostname2]
OFFLINE: [hostname1]

7. Verify that hostname2 of the original secondary server is now shown as Online, and
associated with failover-ip and the process name of the primary server
(wm-primary-svr).
note ............. During the switchover, the client session is disconnected, but now can be reconnected.
8. On the original primary server, enter ./wm_ha_stop.sh.
9. On the same server, enter ./wm_ha_start.sh.
10. Verify that the system output resembles the following:
Online: [ hostname1 hostname2 ]

This indicates that the HA feature is properly operating with the original secondary server now in the
role of the primary server, and the original primary in the role of secondary.
Procedure 14: To simulate the failure of the secondary server

1. On either server, enter ./wm_ha_status.sh.
2. In the system output, make note of the
◦ process names for the primary and secondary servers (wm-primary-svr and
wm-backup-svr, respectively in the example below)
◦ hostnames quoted for the primary (beneath Resource Group: wm-active) and
the secondary (beneath Resource Group: wm-passive) server.
EXAMPLE:
3. In your web browser, use the virtual IP address of the primary server to access the initial
web page for a WM client session launch.
28
September 2013
4. By any one of the following means, disrupt the primary server:
◦ Kill the wmserver process as follows:
− At the Linux OS prompt, enter service wmserver status.
RESULT: The system responds wmserver (pid wm_pid) is running...
− Enter kill -9 wm_pid.
note.............. Killing wmserver does not free the port 9090. Free this port to start the wmserver again.
◦ If you noted the process name of the primary server in Step 2, enter service
wm-primary-svr stop.
◦ Unplug its power cable.
◦ Click the Stop WM Server button in the WM Server panel of the its WM Administrator
Tool interface.
5. After five minutes, enter ./wm_ha_status.sh on the primary server.
RESULT: The system responds as follows:

Node hostname2: standby (on-fail)
Online: [hostname1]

wm-backup-svr (lsb:wm-backup-svr): Started hostname2 FAILED
Failed actions:
wm-backup-svr_monitor_60000 (node= hostname2, status=complete): not running
6. Observe that
◦ since the secondary server (hostname2) has failed, there is no backup/passive server.
If the primary server (hostname1) fails at this state, then no Wireless Manager service
will be available.
◦ the client session is not disconnected when the secondary server fails, so a client
disconnection is not an indication.
7. On the original secondary server, enter ./wm_ha_stop.sh.
8. On the same server, enter ./wm_ha_start.sh.
9. Verify that the system output resembles the following:
Online: [ hostname1 hostname2 ]

This indicates that the HA feature is properly operating with the original primary server in the role of
primary, and the original secondary in the role of secondary.
29
September 2013
7 Troubleshooting High Availability Setups

7.1 Responding to trouble cases
note ............. All commands advised in this chapter are executed from the directory
/usr/local/cambium/wm/server/ha.

important ........... The steps in this section are effective only if they are taken within the first seven days
of server failure. If this much time has elapsed, follow the guidance provided in
Resyncing the HA Databases on Page 31.
Attempt the following remedies for observed trouble:
 If the network cable is accidentally unplugged, either
− restart the affected system.
note ............. Restart may take more than 30 minutes, depending on the server’s current state and HA
services running on it. Using the script wm_ha_cleanup.sh can reduce this time. This
script will kill all the services started by HA, free all the allocated resources, and restart the
system.
− execute wm_ha_cleanup.sh.
 If communication between both the server is blocked (by either firewall or network fault),
ensure that each server can ping the other and the firewall is not running on either of them.
note ............. In this case, when communication is inhibited between the servers, they begin acting as
primary (stand-alone) WM servers. The loss of polling data (statistics) may occur, but will
not corrupt the database.
 If the output of the command wm_ha_status.sh shows the status of one of the HA
servers as failed, standby, or offline
1. stop the affected HA server by executing the command wm_ha_stop.sh.
2. then start the affected HA server again using the command wm_ha_start.sh.
 If the resource wm-active fails sometime after wm_ha_start.sh is executed on the
server
1. restart the affected system or execute wm_ha_cleanup.sh.
2. then start the HA services by executing wm_ha_start.sh.
7.2 Resyncing the HA Databases

After seven days down, perform the procedure that applies from this section.
Procedure 15: To resync the HA databases after only one server has been down seven days
1. On each server, enter cd /usr/local/cambium/wm/server/ha.
2. On each server, enter./wm_ha_stop.sh.
3. On each server, enter./wm_ha_start.sh.
4. On the primary server, repetitively enter the following command until the server returns its
status as primary: ./wm_ha_status.sh
31
September 2013
5. On the secondary server, enter ./wm_ha_start.sh.
6. On the secondary server, repetitively enter the following command until the server returns
its status as secondary: ./wm_ha_status.sh.
note ............. For the HA feature to work, both servers must be online, with one recognized as primary
and other as secondary.
Procedure 16: To resync the HA databases after both servers have been down seven days
1. On each server, execute
/etc/init.d/mysqld status.
2. If either or both server fails to respond with a password prompt and then values of counts
such as uptime, threads, and average queries per second, perform the following steps on the
server(s):
a. Enter service mysql start.
b. Enter mysql -u user -p password status.
3. On each server, perform the following steps:
a. Run the following query:
SELECT max(UPDATE_TIME) AS last_update FROM
INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='WMSDB'
GROUP BY TABLE_SCHEMA;
b. Compare the date in the output on both the servers, and consider the server with the
later date to be the primary server for the purpose of Step 4 of this procedure.
c. Enter \q.
4. Perform Procedure 15: To resync the HA databases after only one server has been down
seven days on Page 31.
7.3 Preventing HA Boot Errors

The following procedure is required only if HA is operating on Service Pack 2 and Procedure 5: To
upgrade to Service Pack 3 and get its HA features and fixes on Page 19 has not been performed. For
example, if Procedure 6: To upgrade to Service Pack 3 and forfeit its HA features and fixes on Page 19
was performed instead of Procedure 5, then the following procedure must be performed to avoid
boot errors.
Procedure 17: To prevent HA boot errors

1. On each server (that is operating on SP 3 via Procedure 6 or is still on SP 2,
enter uname –n, noting the system response as its hostname.
2. Enter service heartbeat restart.
3. If the system does not return a heartbeat OK message, continue.
4. Enter rpm –q heartbeat –d.
RESULT: The system returns the current path to the three files that are critical to
implementation of a heartbeat on this two‐node cluster.
5. On each server, enter the following commands to copy those files:
cp path/authkeys /etc/ha.d/
cp path/ha.cf /etc/ha.d/
cp path/haresources /etc/ha.d/
32
September 2013
6. Enter cd /etc/ha.d/.
7. Open the file authkeys for editing.
8. Append the following two lines to the bottom of the file:
auth 2
2 sha1 HA
note ............. The string HA here is a placeholder for any string, but the string must be identical on both
servers.
10. Enter chmod 600 /etc/ha.d/authkeys.
RESULT: No one except you can read or write the file.
11. Open the file ha.cf for editing.
12. In the order shown here, append the following lines to the end of the file:
bcast eth0
udpport 694
auto_failback on
node hostname1
node hostname2
where hostname1 and hostname2 are is the strings derived from performing Step 1
on each server.
13. Search the file contents for a duplicate instance of each of the added lines, commenting out
or removing any that you find.
15. Retry Step 2.
33
September 2013
8 Disabling and Uninstalling High Availability

If you ever want to disable the High Availability feature, perform the following steps.
Procedure 18: To disable High Availability

1. On the secondary server, perform the following steps:
a. Change the working directory to /usr/local/cambium/wm/server/ha
(if not already in this directory).
b. Enter ./wm_ha_stop.sh.
2. On the primary server, perform Step 1 of this procedure.
3. While still in the working directory /usr/local/cambium/wm/server/ha
of the secondary server, enter ./ha_disable.sh.
4. While still in the working directory /usr/local/cambium/wm/server/ha
of the primary server, enter ./ha_disable.sh.
5. On the server that will be the single‐server WM instance, use the WM Administrator Tool
to start WM, as described in the server administration guide. (Do not use the
wm_ha_start.sh script.)

To remove the High Availability feature and all of the scripts and configuration files that have
operated it, perform the following steps.
Procedure 19: To uninstall High Availability

1. Change the working directory to /usr/local/cambium/wm/server/ha.
2. Insert your Red Hat media into the DVD drive.
3. Enter ./pacemaker_uninstall.sh.
35
September 2013
Resources for Support

Network Updater Help
These release notes call attention to Release 4.0 changes in the Network Updater tool. The effects of
these changes are fully described in the document Network Updater On‐Line Help for Release 4.0.
The user should search that document for information before consulting other sources of
information. From the Network Updater application main menu, select HelpContents to open that
resource.
Community Forum
The technical support Community Forum is part of the support web site and can be used for asking
questions directly to the support team. Questions and answers are accessible to all so that any
customer can benefit from the same dialogue. To access this forum, visit
http://forum.cambiumnetworks.com/. The following is an example of the contents of the forum
page:
37
September 2013
This forum requires authentication for posting.
Technical Support
You can obtain support for Wireless Manager from any or all of the following sources:
 Wireless Manager setup guide, administration guide, and release notes.
 Cambium Networks support web page: http://www.cambiumnetworks.com/support.
This page provides links to information on all products and tools, as well as access to
customer support materials and interactive support forums. Some of these resources are
restricted to registered users and channel partners.
 the Community Forum. Visit http://forum.cambiumnetworks.com/. See Community Forum
on Page 37.
 direct contact with Cambium Networks Technical Support. This contact is available 7 days
a week, 24 hours a day. To find the appropriate phone number based on your country or
region, visit http://www.cambiumnetworks.com/support/contact‐support/.
 a technical support case, which you can open at
http://www.cambiumnetworks.com/forms/contact‐technical‐support/. The case captures
basic information about answers you are seeking or the problem that your network is
experiencing and provides this to the support team, who are available 7 days a week,
24 hours a day, and will respond. They will also provide a case number by which you and
they can continue to track progress on issues that require deeper investigation.
38

Cambium High Availability For Wireless Manager Server On A Linux Platform Setup Guide

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Cambium High Availability For Wireless Manager Server On A Linux Platform Setup Guide

Transféré par

Droits d'auteur :

Formats disponibles

Cambium

High Availability for

Issue 6 September 2013

1 About High Availability

1.1 Prerequisites for High Availability

1.1.1 Prerequisites Common to All HA Deployments

Procedure 1: To disable SELinux

1.1.2 Prerequisites Unique to Same Subnet Deployments

1.1.3 Prerequisites Unique to Differing Subnet Deployments

Procedure 3: To enable RIP Version 2

1.2 How the Switchover on Differing Subnets Works

Figure 1: Example switchover configuration for differing subnets

1.3 Caveats for High Availability Operations

2 Setting Up High Availability

2.1 Setting Up the WM and Network File Servers

Procedure 4: To set up WM and Pacemaker servers

Figure 2: License Manager remote, database replication

Related Servers High Availability WM

BAM MySQL MySQL

Figure 3: License Manager on each server, database replication

Related Servers High Availability WM

DHCP Primary Backup Server

BAM MySQL MySQL

RADIUS License License

Figure 4: License Manager remote, database remote (no replication)

Related Servers High Availability WM

Figure 5: License Manager on each server, database remote (no replication)

Related Servers High Availability WM

DHCP Primary Backup Server

BAM License License

Online: [ HA_hostname1 HA_hostname2]

2.2 Configuring the Related Services

3 Upgrading to WM Release 4.0 SP 3

4 Starting and Stopping the WM Service

Procedure 7: To start the WM service

Procedure 8: To check the status of High Availability

Procedure 9: To stop the WM service

5 Changing the IP Address of a High Availability

Procedure 10: To change only the physical IP address of both HA servers

Procedure 11: To change only the virtual IP address of the HA servers

6 Testing High Availability

Procedure 13: To simulate the failure of the primary server

Resource Group: wm-active

Online: [ hostname1 hostname2 ]

Resource Group: wm-active

Procedure 14: To simulate the failure of the secondary server

Resource Group: wm-active

Online: [ hostname1 hostname2 ]

Resource Group: wm-active

7 Troubleshooting High Availability Setups

7.2 Resyncing the HA Databases

7.3 Preventing HA Boot Errors

Procedure 17: To prevent HA boot errors

8 Disabling and Uninstalling High Availability

Procedure 18: To disable High Availability

Procedure 19: To uninstall High Availability

Resources for Support

Vous aimerez peut-être aussi