Académique Documents
Professionnel Documents
Culture Documents
Jeri Korkki
IBM Distinguished Engineer
Version: 20150501
IBM Internal Use Only – Contains GTS Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
This Work Product is intended for IBM Global Technology Services employees
developing network solutions that include SoftLayer an IBM company
This Work Product contains short overview of SoftLayer and its compute
capabilities, servers, storage, Operating Systems, virtualization, services,
networking and service management; basics that you need to know to be
able to design network solutions. The work product introduces SoftLayer
network components and the basics of SoftLayer internal and external
connectivity.
This work product explains the high-level network architecture of
SoftLayer’s data centers and private network, and shows the different how
IBM Strategic Outsourcing Extended Premises and Customer Premises can
be connected to customer's SoftLayer environments.
Note:
This work product contains IBM Global Technology Services developed intellectual
capital
The IBM intellectual capital, the know-how, techniques, methods and
information learned through services engagements is of great value to IBM and
is an asset that needs to be protected in the same way as any physical asset.
The IBM Global Technology Services professionals can use content from this
work product for services proposals and paid services engagements. Do not
distribute this document and its content externally or give a copy to an IBM
Business Partner.
Send any feedback, corrections, comments and suggestions for change to:
Jeri Korkki/Finland/IBM@IBMFI
2 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Table of contents
• SoftLayer – Introduction
• SoftLayer Network Overview
• Rack Architecture
• SoftLayer Computing - Basics
• SoftLayer Networking - Basics
• Interconnecting SoftLayer with IBM SO Customer
3 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
4 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
5 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
6 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Servers - Overview
Bare metal servers
Physical Server Physical Server Physical Server
• Single-tenant physical server Single-tenant Multi-tenant Single-tenant
• Dedicated to a single customer Optional (unmanaged) Managed Managed
Hypervisor Hypervisor Hypervisor
Dedicated rack
• Customer has an option to buy a dedicated rack Bare Metal Server Virtual Server Virtual Server
Public Node Private Node
• All servers and switches in one physical rack
SoftLayer
Private, self managed virtualization environment Managed
• Customer buys a required number of bare metal servers
• To build a private, self managed virtualization
Configurations and Options
environment, contract unmanaged hypervisor (VMware, • Models with Hourly billing
Bare Metal Servers Bare Metal Server
Citrix Xen, Microsoft Hyper-V, Virtuozzo) from SoftLayer • Latest Multi-core CPUs
as a Mass Storage
• GPUs (for HPC and VDI)
with monthly billing • Mass storage servers Any Hypervisor
• Add bare metal servers as mass storage as needed • Redundant power supplies Management VM or
• Etc. Virtual Machines Bare Metal Server
Virtual Server Instances on a Public node Private Cloud
• The resources of a physical server are shared with
multiple customers (multitenant environment)
Virtual Server Instances on a Private node
• The resources of a physical server are dedicated
• Customer can consume all resources of the server
• One customer can have one or more virtual machines in
the same server, not sharing with other customers
OpenPOWER-based “bare metal” servers
- based on IBM POWER8
- for Linux
7 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Optional:
Storage - Overview Networked Storage
- Legacy iSCSI Shared SAN
- iSCSI Consistent Performance Storage
Local Storage on bare metal servers: - Dedicated SAN
• SSD, SATA, SA-SCSI - with RAID capability - QuantaStor Storage Appliance
- Shared Legacy NAS
Remote Block Storage Options - Consistent Perfomance File Storage
• Legacy iSCSI Shared SAN [managed by SL] - Object Storage
• iSCSI Consistent Performance Storage [managed by SL]
• Dedicated SAN [managed by SL]
• QuantaStor Storage OS over a bare metal server [managed by Customer]
Shared Legacy NAS [managed by SL]
• Accessed via CIFS, cost effective, Reliable, 2 TB or less
SoftLayer Private
• Many to one – many servers can use one NAS account
Network
• Intended for “off-server” backup Archives Layer 3 IP Network
Note:
• Network Storage connects over server’s Ethernet interface Supermicro 1, 2 and 4U
• No Fiber Channel, No FCoE (Fiber Channel over Ethernet) x86 rack mounted servers
8 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
DEVELOPMENT ADMINISTRATOR
API1 Message queue Monitoring Flex images RescueLayer
Managed hosting
Platform management
1Application programming Interface (API): 2Intrusion detection system and intrusion protection system; 3Secure socket layer (SSL); 4Domain name server (DNS);
5Content delivery network (CDN); 6Storage area network (SAN); 7Network-attached storage (NAS)
9 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
10 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
11 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Public Network
Every SoftLayer data center and network PoP has multiple 10 Gbps multi-homed connections with bandwidth
from independent top-tier transit and peering network service providers.
Network traffic from anywhere will connect to the closest network PoP and travel directly across SoftLayer’s
network to its data center, minimizing the number of network hops and handoffs between providers.
• Unmetered inbound Internet bandwidth
• Metered and unmetered outbound bandwidth
• Multiple internet backbone connections
Transit Network: TeliaSonera
• Peering and Transit connections
• Automated IP routing and management Transit Network: NTT
12 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
https://www.euro-ix.net/tools/peering_matrix
13 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Transit Networks
14 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Public IP Addresses
SoftLayer Bare Metal servers and Virtual Server Instances (VSI) come IPv4 Addresses Monthly
Primary Address Included
with one Primary Public IPv4 Internet Static Public Address $1.00
Portable Public Address $2.00
• Primary IP addresses are bound to each individual server
Global Address $20.00
Additional, Secondary Public IPv4 Internet addresses can be purchased in
quantities of 1, 2, 4, 8, 16, or 32
IPv6 addresses Monthly
• With limitations and requiring ARIN (American Registry for Internet Numbers) justification
Primary Address Included
• When you order an additional IP address, SoftLayer typically assigns addresses /64 Block Static Public Addresses $4.00
from an existing subnet – if exchausted a new subnet is created /64 Block Portable Public $10.00
Addresses
• Types of Secondary Public IP addresses: /64 Block Global Addresses $20.00
• Static Public IP Addresses – a block of IPs that are routed directly to a specific
server on the SoftLayer network
• Portable Public IP Addresses – can be used on multiple servers within a single
Public VLAN at the same time – typicall assigned to virtual machines
• Global IP address – is a static IP address that can be transferred between
servers associated with the account that owns the subnet. It can help shift
workloads across data centers too. This is most similar to Amazon Elastic IPs Microsoft Excel
without the limitation of being restricted to a region 97-2003 Worksheet
15 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Private Network
All SoftLayer data centers and PoPs are connected by SoftLayer’s private network backbone. The private
network enables customers to seamlessly connect their services in SoftLayer data centers around the world.
• High-speed redundant private network VRFs that SoftLayer maintains – contains all routes for SoftLayer’s internal use,
cross-connects with Telco partners and for the customers backend connectivity
• Network Points of Presence (NPoP) are maintained by SoftLayer – SoftLayer data centers are connected to NPoPs in
Equinix/Telx/InterXion, etc. facilities with high-capacity redundant fiber connections
• Move data between servers at no cost, and take advantage of our update and patch servers, software repositories,
backend services, and more without interfering with public network traffic.
• Multiple 10 Gbps fiber backbone with automatic fail-over
• Unmetered bandwidth on Private Network
• Secure, customer-configurable private VLANs
• Private VLANs may be spanned between data centers
• Servers available with port speeds up to 10Gbps
• Free server-to-server cross connects
• Private Local DNS Resolvers
• Centralized NAS and Block storage resources
• Private OS update, reload and change servers
• SoftLayer software repository
• McAfee security update server
16 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Private Network…
17 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Network Overview
18 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Rack Topology
19 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
• Admin access through the SoftLayer Portal / API SoftLayer VPN Public Network
Concentrators 1G
• Administrative access only
• not for application/end user traffic
Customer’s
SSL
SoftLayer The Internet
Administrator
20 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
The Internet
21 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
22 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
vNIC SoftLayer
• Mobile Apps to keep administrators connected SoftLayer IMS Managed
• tickets Infrastructure VSI
management system
• servers vNIC
• Bandwidth
http://knowledgelayer.softlayer.com/articles/control Customer
Portal Primary Subnet
Public VLAN
Shared Optional:
SoftLayer VPN • Firewall
Concentrators • Load Balancer
Your Management
• Application Accelerator
and Automation SoftLayer
Application Public Network
Frontend Customer
1G Routers (FCR)
Your SoftLayer IPsec
Administrator The Internet
SSL
23 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
24 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
• each server will only be allocated one Primary IP - all other IP addresses vNIC SoftLayer
associated with that server must be from a secondary subnet, which may be VSI Managed
purchased at any time through the Customer Portal vNIC
• default gateway is set to Backend Customer Router (BCR) Primary Public address Primary Public address
• Future servers purchased by the customer will be placed to the same VLAN
• unless another VLAN is specified when ordering the new device Primary Subnet (Public IPs x.x.x.x/29)
Public VLAN
• Prior to first server order being placed, the customer can request for a larger
Optional:
contiguous (larger than 10.x.x.0/26) Private IP address block • Firewall
• Load Balancer
• Bare Metal and VSI servers also attach to the Public VLAN • Application Accelerator
SoftLayer
• Public subnet’s default gateway is set to Frontend Customer Router (FCR) Public Network
Frontend Customer
• Note: Public Internet connection can be disabled if not needed Routers (FCR)
• Customer can request more than one VLAN – see next page
The Internet
25 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
• Servers can optionally be ordered with Primary Public IPv6 address Primary Private address Primary Private address
• Additional IPv6 Internet addresses are available in /64 blocks
• Global IPv6 IP addresses are available as /64 blocks with a limit of 5 vNIC SoftLayer
Managed
per customer VSI
vNIC
Notes:
Primary Public address Primary Public address
• Primary Private and Primary Public IP addresses are bound to each
individual server and can not be moved unless the server is cancelled Primary Subnet (Public IPs)
Public VLAN
• Each server may only be allocated one Primary IP address – all other IP
addresses associated with the server (i.e. alias IP addresses or Virtual Optional:
• Firewall
Machines on bare metal servers), must be from a Secondary address • Load Balancer
block that may be purchased at any time through the Customer Portal. SoftLayer
• Application Accelerator
Public Network
• Customer can order bare metal server without public interface or the
Frontend Customer
public Internet connection can be disabled if not required Routers (FCR)
The Internet
26 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Private
Private
IP
IP
• Portable Public IP Addresses – is a block of Public IP addresses that
Server Server
can be used on multiple servers within a single Public VLAN
Portable
Portable
Public
Public
• Portable IP addresses are switchable within a VLAN from server to server –
Static
IP
IP
IP
IP
IP
Typically assigned to virtual machines
• Customer is responsible for managing tracking the address assignment
Portable
• Global IP – is a static Public IP address that can be transferred between IP
servers associated with the SoftLayer account that owns the subnet Other
Global SoftLayer
• Global IP is not restricted to a VLAN and can help shift workloads across data IP Data
Center
centers
• This is most similar to Amazon Elastic IPs without the limitation of being Static Portable Subnet
restricted to a region
Primary Subnet
Public VLAN
IPv4 Addresses Monthly
Primary Address Included Public Network
Portable Private Address Free Frontend Customer
Routers (FCR)
Static Public Address $1.00
Portable Public Address $2.00
The Internet
Global Address $20.00
27 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Portable
Portable
Portable
Portable
Private
Private
VLAN
IP
IP
IP
IP
IP
IP
• Portable IP addresses are switchable within a VLAN from server to server –
Typically assigned to virtual machines
VM VM VM VM
• For each portable subnet requested, three IPs are devoted to Network,
Hypervisor
Broadcast and Gateway traffic – This means if a block of eight (/29) IPs is
Server Server
issued, three (3) are reserved for the aforementioned traffic and five (5) IP
Public
Public
IP
IP
addresses are available for use on VMs
Primary Subnet
Public VLAN
Public Network
Frontend Customer
Routers (FCR)
The Internet
28 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
• Secondary IP addresses (Alias address or Virtual Machine address) associated Primary Subnet (Public IPs)
with a server must be from a secondary subnet, which may be purchased at any Public VLAN
time through the Customer portal. Optional:
• Only devices assigned to same VLAN can talk to each other - Network Gateway • Firewall
• Load Balancer
[Vyatta] can be used to route between VLANs - see later pages • Application Accelerator
SoftLayer
• The Backend and Frontend Customer Routers will drop all packets of which Public Network
The Internet
29 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
eth3 - Optional
NOTE:
eth1
• SoftLayer servers don’t have multiple NICs – such as typically found in a traditional
SO environment where one NIC is for Production, one for SAN, one for Admin, Public VLAN
Backups, Management, vMotion, etc. Primary Subnet
Optional:
• Everything runs through one (or pair of) NICs on the same SoftLayer Private • Firewall
• Load Balancer
Network • Application Accelerator
SoftLayer
Public Network
Frontend Customer
Routers (FCR)
The Internet
30 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Internet Firewalls 1 of 6
Host-based firewall on SoftLayer server (aka Software firewall)
• Helps customer to secure a server through blocking specific traffic
distinguished by the source IP or target IP address and port number
• Software Firewall can protect both private and public interfaces
• Install and provision through Customer Portal:
Private VLAN
• Host-based firewall is self managed through SoftLayer VPN Gateway Primary Subnet
• Windows Firewall
• Installed by Default Application
• Configured with the following ports
• RDP 3389; FTP 20,21; HTTP 80; HTTPS443 Software
Firewall
• DNS 53; SMTP 25; POP 110; IMAP 143
• IDENT 113; ICMP echo reply
• If Plesk is installed: Ports open per Plesk requirements SoftLayer assigned Public
• Linux Firewall Internet address
• IPTables is installed
Primary Subnet
• APF – Advanced Policy Firewall
Public VLAN
• Others: IPFW, SmoothWall, IPCop, Ebox
Customer Optional:
Note: Portal • Load Balancer
• Application
Software Firewall does not meet IBM service security requirements Accelerator
SoftLayer
Firewall Public Network
Administrator Frontend Customer
Routers (FCR)
SSL
User The Internet
31 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Internet Firewalls 2 of 6
Standard Firewall Service Monthly
Standard Hardware Firewall
10Mbps Hardware Firewall $49.00
• Shared FortiGate 3950B Firewall 100Mbps Hardware Firewall $99.00
• Multi-tenant use of FortiGate hardware firewall 1Gbps Hardware Firewall $199.00
2Gbps Hardware Firewall (in select $399.00
• Virtual Firewall is applied to an individual server Data Centers)
• Sold based on Port speed – Can be ordered with the purchase of server 10Gbps Hardware Firewall (in $899.00
select Data Centers)
• Protects Public Internet interface only
• Provisioned from the Customer Portal without service interruption
SoftLayer Private Network
• Managed through Customer Portal (or API) - simplified user interface with
Private VLAN
limited configuration options
Primary Subnet (10.x.x.0/26)
• Firewall rules apply for all IPs addresses assigned to a single server
• Upto 50 firewall rules
• Portable IP addresses can be protected, but this can not be configured through the
vNIC vNIC SoftLayer
Portal and requires raising manual service ticket Managed
App App
VSI VSI
Notes: vNIC vNIC
• Standard Hardware Firewall is strictly for filtering traffic by IP address and SoftLayer assigned Public Internet addresses
port. It does not support VPN termination, NAT, DMZ, Intrusion prevention
and Anti-virus – if you need those, use Fortigate Security Appliance. Primary Subnet (Public IPs)
Public VLAN
• Standard Firewall only filters inbound traffic - outbound traffic is not blocked
• You can not have Standard Hardware Firewall (shared) and Dedicated
Customer
Hardware Firewall on same Public VLAN Firewall Portal
Administrator SoftLayer Public
Network
SSL
User The Internet
32 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Internet Firewalls 3 of 6
Dedicated Hardware Firewall Dedicated Firewall Service Monthly
1Gbps Dedicated Hardware $999.00
• Dedicated FortiGate Firewall Firewall
1Gbps Dedicated Hardware $1,998.00
• Protects one, multiple or even all servers on the same Public VLAN Firewall with High Availability
• Protects Public Internet interface only
• High Availability option
• available as new order only; cannot be upgraded or downgraded
• Provisioned from the Customer Portal without service interruption
SoftLayer Private Network
• Managed through Customer Portal (or API) - simplified user interface with
Private VLAN
limited configuration options
Primary Subnet (10.x.x.0/26)
• no direct login to FortiGate firewall
Notes:
• Dedicated Hardware Firewall is strictly for filtering traffic by IP address and vNIC vNIC SoftLayer
port. It does not support VPN termination, NAT, DMZ, Intrusion prevention App App
VSI VSI Managed
App
vNIC vNIC
and Anti-virus – if you need those, use Fortigate Security Appliance
• Dedicated Firewall only filters inbound traffic, outbound traffic is not blocked SoftLayer assigned Public Internet addresses
• Portable IP addresses can be protected, but this can not be configured Primary Subnet (Public IPs)
through the Portal and requires raising manual service ticket Public VLAN
• You can not have Standard Hardware Firewall (shared) and Dedicated
Hardware Firewall on same Public VLAN Customer
Firewall Portal
• Upto 50 firewall rules Administrator SoftLayer Public
Network
SSL
User The Internet
33 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Internet Firewalls 4 of 6
FortiGate 300 Service Monthly
FortiGate 300 series Security Appliance
Fortigate® Security Appliance $999.00
• Dedicated Fortigate hardware Fortigate® Security Appliance $1,998.00
• FortiGate 310B or 300C with High Availability
SSL
User The Internet
34 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Primary Subnet
Customer Data Center Public VLAN
Application SoftLayer
Customer’s Firewall with Your Firewall Public Network
IPSec VPN termination Administrator
CE SSL
Private Addresses The Internet
IPsec VPN tunnel
35 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Primary Subnet
Customer Data Center Public VLAN (Transit VLAN)
Application
Customer’s Firewall with Your Firewall SoftLayer Public Network
IPSec VPN termination Administrator
CE SSL
Private Addresses The Internet
IPsec VPN tunnel
36 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
• Can be activated and configured in real-time through Customer Portal Local Load Balancing with SSL - 500 Connections $199.99
Local Load Balancing with SSL - 1000 Connections $399.99
• Limited (basic) configuration options
• Load balancing method may be updated at any time through the
Customer Portal:
• Round robin, lowest latency, least connections, shortest response
• Public Network load balancing only
• Optional SSL offloading to streamline performance Customer 2 Customer ’s environment
Local Load
Customer 1
• Reduces the number of SSL certificates required Balancer
Local Load
Balancer
• by processing and securely decrypting incoming traffic at the load Server 1 Server N
balancer instead of at each individual server
SoftLayer
• Performance: 250-2500 cps (SSL: 250-1000 cps) Assigned VIP
Customer
• Local load balancing is Layer 4 only Portal Primary Subnet
Primary Subnet
• No console access Public VLAN Public VLAN
• Manage through Customer Portal, CLI or API SoftLayer Public SoftLayer Public
Network Network
Your SoftLayer
Administrator
SSL
User The Internet
37 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
• Manage through Customer Portal, CLI or API SoftLayer Public SoftLayer Public
Network Network
Your SoftLayer
Administrator
SSL
User The Internet
38 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Primary Subnet
Public VLAN
SSL
User The Internet
39 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
40 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
41 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
• Do you need redundant Internet connection and DDoS protection on customer side? Access to resources
SoftLayer has multi homed, redundant Internet connections. On Private VLAN
vNIC SoftLayer
SoftLayer Application Managed
VSI
IMS
vNIC
42 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Secure the Internet connection with IPsec VPN using FortiGate Appliance
Services Network
1. Purchase SoftLayer account and required virtual (VSI) and/or bare metal server(s)
Optional: Optional:
• For bare metal servers, select: single/redundant NICs and interface speed Network Storage Services
2. Obtain public Internet address(es) from SoftLayer • iSCSI Shared SAN • DNS Resolver
• Shared NAS / FTP • Evault
• Select: Network port speed and Internet bandwidth • Dedicated SAN • LockBox
3. Obtain FortiGate 300 series Security Appliance for IPsec VPN connection • Object Storage • Provisioning
• OS Updates
• Standard and Dedicated Hardware Firewalls do not support IPsec VPN • Software Repository
4. Design and implement the Solution and the Network inside SoftLayer • IPMI over Ethernet
• Monitoring
• Servers, Application, VLANs, Subnets
• Security, etc.
• Servers and Applications use SoftLayer provided public Internet addresses
5. Configure IPsec VPN tunnel in FortiGate 300 series Security Appliance using SSL VPN MBR
to access Fortigate BCR
Private VLAN
6. Request AT&T (or other Communications Service Provider) to provide an Internet Primary Subnet (10.x.x.0/26)
access and to configure the Internet facing Firewall (set the firewall rules)
Access to resources
• Do you need redundant Internet connection and DDoS protection on Customer side? On Private VLAN
43 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
3. Connecting SoftLayer Private Network over the Internet with link encryption – single VLAN
Secure the Internet connection with IPsec VPN using Vyatta CE or VyOS installed to a
Virtual Server Instance (VSI) Services Network
1. Purchase SoftLayer account and at least one virtual (VSI) for Vyatta – and additional Optional: Optional:
Network Storage Services
VSIs and bare metal servers as required • iSCSI Shared SAN • DNS Resolver
2. VSI’s public Internet address will be used by Vyatta • Shared NAS / FTP • Evault
• Dedicated SAN • LockBox
• Select required Internet bandwidth • Object Storage • Provisioning
3. Obtain Vyatta CE or VyOS software and install it to VSIs • OS Updates
• Software Repository
• Do you need redundant Vyatta design? • IPMI over Ethernet
4. Design and implement the Solution and the Network inside SoftLayer • Monitoring
• Servers, Applications, VLANs and Subnets • Security, etc.
• Servers and Applications use SoftLayer assigned Private 10.x.x.x addresses MBR
• Modify routing on the SoftLayer Hosts to point to Vyatta as a gateway BCR
• Define NATing on Vyatta to avoid possible 10.x.x.x address overlap
Private VLAN
• Configure the IPSec VPN tunnel – could also use Generic Routing Encapsulation Primary Subnet (10.x.x.0/26)
(GRE), or OpenSSL VPN
SoftLayer allocated 10.x.x.x addresses
Access to resources
On Private VLAN
5. Request AT&T (or other Communications Service Provider) to provide an Internet SoftLayer
vNIC
access and to configure the Internet facing Firewall (set the firewall rules) Application Managed
SoftLayer VSI
• Do you need redundant Internet connection and DDoS protection on EP side? IMS
vNIC
SoftLayer has multi homed, redundant Internet connections.
Customer SoftLayer allocated Internet addresses
IBM Administrator Portal
outside IBM network Primary Subnet (Public Addresses)
Public VLAN
Management through
SSL VPN over the Internet
DDCN Architecture
Frontend Customer
EP or CP 1G Routers (FCR)
Application Customer’s
Firewall
Customer’s Private address
IPsec VPN tunnel
CE SoftLayer assigned 10. or NATed address
Private Addresses The Internet
44 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Secure the Internet connection with IPsec VPN using Gateway Appliance (Vyatta)
Services Network
1. Obtain a SoftLayer account and Gateway Appliance (Vyatta)
Optional: Optional:
• Vyatta will become the default gateway – all traffic will route through it Network Storage Services
• iSCSI Shared SAN • DNS Resolver
2. Obtain additional VSIs and bare metal servers as needed • Evault
• Shared NAS / FTP
3. Gateway Appliance’s public Internet address will be used by Vyatta • Dedicated SAN • LockBox
• Object Storage • ......
• Select Network port speed and Internet bandwidth
• Gateway Appliance allows you to connect and route multiple VLANs MBR
• Do you need redundant Gateway / Vyatta design? BCR
4. Design and implement the Solution and the Network inside SoftLayer Private VLAN (Transit VLAN)
• Servers, Application, VLANs, Subnets and Vyatta Firewalling / Routing between Primary Subnet (10.x.x.0/26)
VLANs
Private VLAN
• Applications have SoftLayer assigned Private 10.x.x.x addresses
• Define NATing on Vyatta to avoid possible 10.x.x.x address overlap Primary Subnet (10.x.x.0/26)
802.1q
• Configure the IPSec VPN tunnel trunk Portable 10.x.x.x
SoftLayer allocated 10.x.x.x addresses
5. Request AT&T (or other Communications Service Provider) to provide an Internet vNIC
Application
access and to configure the Internet facing Firewall (set the firewall rules) VM VM
SoftLayer
• Do you need redundant Internet connection and DDoS protection on EP side? vNIC
IMS
SoftLayer has multi homed, redundant Internet connections.
Customer Portable Subnet
IBM Administrator Portal
outside IBM network 802.1q Primary Subnet
trunk Public VLAN
Management through
SSL VPN over the Internet Primary Subnet
DDCN Architecture
Public VLAN (Transit VLAN)
EP or CP 1G
Application Customer’s Frontend Customer
Firewall Routers (FCR)
Customer’s Private address
IPsec VPN tunnel
CE SoftLayer assigned 10. (or NATed) address
Private Addresses The Internet
45 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Direct Link
Extends customer premises network into SoftLayer Private Network SoftLayer Network
Location PoP Provider
• Available in 20 SoftLayer NPoPs (Network Points of Presence) North America
Atlanta, GA ATL01 Telx
• Customer orders a dedicated connection from CSP (Communication Service Provider) to Chicago, IL CHI01 Equinix
SoftLayer Network PoP and a cross-connection to SoftLayer XCR (X-connect) router Denver, CO DEN01 Coresite
Dallas, TX DAL03 Equinix
• private 1 or 10 Gbps connection from customer’s data center to customer’s SoftLayer VLAN
Los Angeles, CA LAX01 Coresite
• unfettered access to customer’s servers on SoftLayer platform Miami, FL MIA01 Terremark
• data is isolated in a secure private virtual network, but not encrypted – encryption may need to New York City, NY NYC01 Telx
be added to protect sensitive, business-critical data if data is not encrypted on application San Jose, CA SJC02 Equinix
Seattle, WA SEA02 The Westin
• all customers share SoftLayer private network bandwidth Toronto, Ont, CA TOR02 Cologix
• X-connect router provides non-redundant connection – two Direct Link connections to two Washington, D.C. WDC02 Equinix
SoftLayer data center is needed for full redundancy APAC
Hong Kong HKG01 Pacnet
• Flat monthly fee for each Direct Link based on the port speed Customer select’s Melbourne MEL02 NextDC
Singapore SNG01 Equinix
• Traffic across Direct Link and on SoftLayer backbone between data centers is free and
Sydney SYD02 Equinix
unmetered Tokyo TOK01 Equinix
EMEA
• Access link and CE router cost up to the SoftLayer X-connect router is Customer
Amsterdam AMS02 Equinix
responsibility Frankfurt FRA01 InterXion
London LON01 TeleCity
Customer data center Paris PAR02 Equinix
SoftLayer data center
46 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
SoftLayer Interconnect
Extends customer premises network into SoftLayer CMS Data
Center link SoftLayer PoP code Provider
• SLI is available at 12 x IBM Cloud Managed Services (CMS) data centers North America
Boulder, US A Denver, CO DEN01 Coresite
• 10 Gbps connections from Frontbones to two SoftLayer data centers B Denver, CO DEN01 Coresite
RTP, US A Atlanta, GA ATL01 Telx
• Enables private 1 or 10 Gbps redundant connections from Customer’s data B Washington, D.C. WDC02 Equinix
center to servers in CMS and SoftLayer Toronto, CA A Toronto, CA TOR02 Cologix
B Toronto, CA TOR02 Cologix
• Logical Connection for existing CMS Frontbone connected customers Europe
Portsmouth, UK A London LON01 TeleCity
• Physical Connection new 1/10 Gbps access circuits to CM Frontbone B London LON01 TeleCity
Lisbon, PT A Amsterdam AMS02 Equinix
• access link cost to Frontbone is customer’s responsibility B Frankfurt FRA01 InterXion
• data is isolated in a secure private virtual network, but not encrypted – Barcelona, ES A Amsterdam AMS02 Equinix
B Frankfurt FRA01 InterXion
encryption may need to be added to protect sensitive, business-critical data
Montpellier, FR A Amsterdam AMS02 Equinix
• all customers share the Frontbone to CMS and SoftLayer connections B Frankfurt FRA01 InterXion
Ehningen, DE A Amsterdam AMS02 Equinix
• One time connection charge and flat monthly fee for each connection, B Frankfurt FRA01 InterXion
based on the port speed selected – data traffic is free and unmetered Winterthur, CH A Amsterdam AMS02 Equinix
B Frankfurt FRA01 InterXion
AP and LA
Tokyo, JP A Tokyo TOK01 Equinix
B Tokyo TOK01 Equinix
Hortolandia, BR A Hortolandia, BR
IBM CMS data center B Hortolandia, BR
Sydney, AU A Sydney SYD02 Equinix
SoftLayer data center B Sydney SYD02 Equinix
Customer data center
47 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
SoftLayer Interconnect…
Customers connect either with Physical or a Logical connection LAX
BLD SEA
• Physical Connection – New 1 or 10 Gbps access link to Frontbone routers
SJ
• each physical connection comes with a Logical connection
DEN
• access link costs to Frontbone is customer’s responsibility CHI
• all customers share the Frontbone to CMS and SoftLayer connections DAL
• SoftLayer Interconnect can provide redundant connectivity HOU
• Logical Connection – for Customers that are already have access link to Frontbone WDC
RTP
ATL
MIA
NYC
IBM CMS data center
TOR TOR
SoftLayer data center
HOR MEX
Customer data center SoftLayer HOR
Network NHB
PoP LON
CMS Private Network
Customer VLAN LIS PAR
Customer BBR SoftLayer Private Network
CE
Access Customer VLAN BAR AMS
Link(s)
Customer Network FRA
MOP
Frontbone SoftLayer data center
SIN
WIN
TOK
SoftLayer Data Center EHN HGK
SoftLayer Network PoP TOK SYD
SoftLayer Backbone Network (N x 10 Gbps) BBR SoftLayer Private Network MEL
SYD
SoftLayer Interconnect (N x 10 Gbps) Customer VLAN
Customer access link ( 1 or 10 GBPS)
48 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
4. Request AT&T (or other Communications Service Provider) to provide dedicated BBR
vNIC SoftLayer
connection and CPE routers between customer and SoftLayer X-connect router Application VSI Managed
• SoftLayer Direct Connect is non-redundant router, do you want to have vNIC
redundant connection from your data center to X-connect router? X-connect
Router
Primary Subnet
Public VLAN
DDCN Architecture
SoftLayer assigned 172.x.x.x addresses Frontend Customer
EP or CP Routers (FCR)
Application Customer’s
Firewall SoftLayer assigned 10.x.x.x address CE
CE The Internet
SL defined 172.x.x.x Dedicated Connection (MPLS, Wave,...)
49 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
vNIC
4. Request AT&T (or other Communications Service Provider) to provide dedicated X-connect
Router
connection and CPE routers between customer and SoftLayer X-connect router
• SoftLayer Direct Connect is non-redundant router, do you want to have Primary Subnet
redundant connection from your data center to X-connect router? Public VLAN
172.x.x.x
DDCN Architecture
NAT
Private addresses (i.e. 192.x.x.x) 172.x.x.x -> Frontend Customer
EP or CP <- 192.x.x.x Routers (FCR)
Application Customer’s
Firewall SoftLayer assigned 10.x.x.x address CE
CE The Internet
Private (i.e. 192.x.x.x) Dedicated Connection (MPLS, Wave,...)
50 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
7. Connecting SoftLayer via Direct Link – GRE tunneling with no overlapping addresses
GRE tunnel between Customer Premises and SoftLayer without NAT translation
Services Network (10.0.0.0/14)
1. Obtain a SoftLayer account and Gateway Appliance (Vyatta)
Optional: Optional:
2. Obtain additional VSIs and bare metal servers as needed Network Storage Services
• iSCSI Shared SAN • DNS Resolver
3. Gateway Appliance allows you to connect and route multiple VLANs • Evault
• Shared NAS / FTP
• Do you need redundant Gateway / Vyatta design? • Dedicated SAN • LockBox
• Object Storage • ......
4. Design and implement the Solution and the Network inside SoftLayer
• Servers, Application, VLANs, Subnets, Vyatta MBR
• Servers and Applications use Private 10.x.x.x SoftLayer defined addresseses BCR
5. Request SoftLayer to provide cross connection to external customer Private VLAN (”Vyatta Transit VLAN”)
• SoftLayer assigns 8 IP addresses in 172.16.0.0/12 for a Direct Link Primary Subnet (10.x.x.0/26)
connection as a default Private VLAN
• Setup GRE tunnel between Gateway Appliance [Vyatta] and Customer
Primary Subnet (10.x.x.0/26)
Firewall (or router)
• Traffic between Customer Private (i.e. 192.x.x.x) and 10.x.x.x subnets will flow Portable 10.x.x.x
without NAT translation SoftLayer allocated 10.x.x.x addresses
51 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
8. Connecting SoftLayer via Direct Link – GRE tunneling with IP Alias Addresses
• Servers can communicate with SoftLayer Services using Primary IP addresses MBR
• Management access is through servers Primary addresses BCR
• Addresses that can not be used with IP Aliasing in SoftLayer:
Private VLAN (”Vyatta Transit VLAN”)
• Primary subnets directly connected to Vyatta
• SoftLayer services network (10.0.0.0/14) Primary Subnet (10.x.x.0/26)
• The /31 assigned to the Direct Link port on the XCR (10.x.x.x/31)
Private VLAN
• SoftLayer routeable 10.x.x.x IP’s assigned to the Vyatta transit (10.x.x.0/26)
• Customer’s VLAN Primary subnet (10.x.x.0/26) Primary Subnet (10.x.x.0/26)
• Refer to “What IP ranges do I allow through the firewall?” for exact address ranges: Portable 10.x.1.0/26
http://knowledgelayer.softlayer.com/faq/what-ip-ranges-do-i-allow-through-firewall
Alias Primary
• Gateway Appliance [Vyatta] is needed to perform routing and to set up the
GRE tunnel BBR
Application
• Design routing and name resolution with 10.x.1.x
Alias addr
X-connect
Router
Primary Subnet
Public VLAN
10.x.x.x/
31 Primary Subnet
DDCN Architecture
Public VLAN (Transit VLAN)
EP or CP Private 10.x.x.x Subnet addresses
Application Customer’s Frontend Customer
Firewall SoftLayer assigned Alias 10.x.x.x address CE Routers (FCR)
52 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
9. Connecting SoftLayer via Direct Link – GRE tunneling with partially overlapping addresses
Find 10.x.x.x subnet from SoftLayer that does not overlap with Customer
Services Network (10.0.0.0/14)
premises network
Optional: Optional:
1. Even when 10.0.0.0/8 is used in Customer premises network, not all 10.0.0.0/8 Network Storage Services
subnets necessarily overlap with SoftLayer network • iSCSI Shared SAN • DNS Resolver
• Shared NAS / FTP • Evault
2. Identify the network address spaces used on Customer premises network
• Dedicated SAN • LockBox
3. Find if there is a possibility to obtain non-overlapping subnets – ask Cloud • Object Storage • ......
technical sales to know the current private network IP subnet assignment
MBR
• Servers and Applications use Private 10.x.x.x SoftLayer defined addresseses
BCR
• Subnets that don’t overlap Customer premises network may not always be
Private VLAN (”Vyatta Transit VLAN”)
available
Primary Subnet (10.x.x.0/26)
4. If non-overlapping subnets are available, the subnets must be ordered to secure
them Private VLAN
5. Request SoftLayer to provide cross connection to external customer Primary Subnet (10.x.x.0/26)
6. Setup GRE tunnel between Gateway Appliance [Vyatta] and Customer Firewall (or Portable 10.x.x.x
router)
• Traffic between Customer Private 10.x.x.x and SoftLayer 10.x.x.x subnets will BBR
vNIC vNIC
flow without NAT translation Application
VM VM
• The connection can be encrypted
X-connect
Router
Primary Subnet
Public VLAN
53 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
10. Connecting SoftLayer via Direct Link – GRE tunneling and Virtual LAN Overlay Network
• Traffic between Customer Private 10.x.x.x and virtual LAN overlay 10.x.x.x Private VLAN - Transit
Primary Subnet
subnets on SoftLayer will be routed without NAT translation Portable Subnet
BBR
• The connection can be encrypted
For more information about designing virtual LAN overlay technology, see GTS
App
Software Defined Networking (SDN) and Network Function Virtualization (NFV) NSX Edge
X-connect VM VM
Reference Architecture: https://w3- Router
03.ibm.com/tools/cm/iram/assetDetail/generalDetails.faces?v=*&guid=A2E519EC-7FDB-18DB-B550- VXLAN Transport Zone
D25AFDA416FF vDS vDS
vSphere Host vSphere Host
VMware NSX deployment
172x.x..x
DDCN Architecture
54 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
55 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
56 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
DDCN Architecture
Frontend Customer
EP or CP GSNI to Private
Routers (FCR)
Addresses NAT
Customer’s
Firewall Customer’s Private address
IPsec VPN tunnel
CE SoftLayer assigned 10. or NATed address
Private Addresses The Internet
57 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
Manage objects on SoftLayer through IBM Internal Interconnect Service and Internet VPN
GSNI MBR
IBM Blue BCR
Management Private VLAN
Tool Access IBM
Firewall Primary Subnet (10.x.x.0/26)
Administrator
CR SoftLayer allocated 10.x.x.x addresses
Centralized Resources
GSNI address
IPsec VPN tunnel
GSNI addresses
The Internet
IIIS Service
58 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
59 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation
IBM GTS Architecture and Solution Design
60 IBM Internal Use Only – Contains GTS Developed Intellectual Capital © 2015 IBM Corporation