IZO Private

Microsoft Azure ExpressRoute

Service assurance and delivery
guidelines
 Author: John Robinson Selvaraj

Date: 12th Dec 2014

Reviewed by Raju Raghavan

Version Date Changed By Description

1.0 12th Dec 2014 John Initial Draft

2.0 6th Mar 2015 John Provisioning steps

2.1 11th Mar 2015 John Prefix limit

2.2 7th April 2015 John Support process

2.3 15th April John Change is public services IP pool

2.4 23rd April John Config Template

2.5 1st July 2015 John Support portal login credentials added

John O365 services Assurance/Deliver guidelines added

2.6 13th Oct 2015 Azure Premium Add-on

2.7 25th Nov 2015 John Adding bullets to the O365 design

2.8 9th Dec 2015 John ER modify process

2.9 2nd Feb 2016 John QOS for O365

3.0 4th April 2016 John New site addition (Ashburn, SJ)

Distribution
GIMEC, LOB, SE, SA, SD, IT, Service Planning and Realization
Table of Contents

1. Introduction .................................................................................................................................... 5

2. IZO Private Service Architecture: .................................................................................................... 5

A. Existing Cloud Delivery Model ........................................................................................................ 5

B. IZO Private Service .......................................................................................................................... 5

C. IZO Private Layer-3 Network Architecture...................................................................................... 7

3. Guidelines specific to Microsoft ExpressRoute Interconnect ......................................................... 8

A. Physical connectivity diagram ........................................................................................................ 8

B. Customer Connectivity. .................................................................................................................. 9

C. Q-in-Q and PE-CE Routing ............................................................................................................. 10

D. IP addressing................................................................................................................................. 11

E. VPN Topologies ............................................................................................................................. 11

F. Quality of Service .......................................................................................................................... 12

G. CPE Management ......................................................................................................................... 12

H. Multicast ....................................................................................................................................... 12

I. Azure Expressroute Provisioning Process ...................................................................................... 12

J. Azure Expressroute Modify ........................................................................................................... 17

i. Enable/ Disable Premium Add-on ......................................................................................... 17

ii. Bandwidth Change ................................................................................................................ 17

K. Azure Expressroute Termination Process ..................................................................................... 17

L. Troubleshooting ............................................................................................................................ 19

M. Azure Support Process ................................................................................................................. 20

N. Microsoft Online/ O365 Services.................................................................................................. 22

iii. INTRODUCTION ..................................................................................................................... 22

iv. PRE-REQUISITES .................................................................................................................... 22

v. DESIGN .................................................................................................................................. 22

vi. PROVISIONING STEPS ............................................................................................................ 26

O. ExpressRoute Premium Add-on ................................................................................................... 27

P. References .................................................................................................................................... 29

Q. Annexures..................................................................................................................................... 29

i. Template for IP validation ..................................................................................................... 29

ii. Sample TCL PE configuration ................................................................................................ 30

 1. Introduction
Over the last few years global enterprises large and small have started utilizing Cloud Computing
resources for scaling their IT horse power requirement. These enterprises offload part of their
compute, storage or application requirements to a cloud provider over Internet. However since
Internet is a best effort medium the enterprise does not have total control over the transport to the
cloud provider. Tata Communications with its cloud connect service will offer a more predictable,
secure and SLA backed alternative to the Internet based access. This document describes the
network architecture and the guidelines for provisioning a service over Microsoft Azure.

2. IZO Private Service Architecture:

A. Existing Cloud Delivery Model
As depicted in the figure below, currently Cloud services are delivered over the public internet. For
example if an enterprise wants to subscribe to Google’s cloud Compute IAAS service. The enterprise
signs up online for the required amount of storage , compute resources , OS and Application
requirement and then a virtual machine is provisioned that can be accessed over Internet.

B. IZO Private Service

The IZO Private Service will offer an alternative to the internet based delivery model based on a non-
internet and private SLA backed network.

Key Benefits of IZO Private Service are:

• Guaranteed Quality Of Service
• Latency, Packet Loss and Jitter guarantees
• Routing Predictability
 • Traffic does not traverse multiple Internet backbones and routing does not
dynamically change.
• Traffic stays within a network end to end.
• Availability
• Guaranteed Availability
• Security
• Private Network is not susceptible to DDoS Attacks, Confidentiality, Privacy and
Compliance issues.
• SLAs
• Application Analytics and performance monitoring
• The service will provide advanced application performance monitoring and
reporting capabilities that will help enterprises quickly isolate and troubleshoot
issues.
C. IZO Private Layer-3 Network Architecture
The figure below depicts the layer-3 IZO Private architecture that will be used for Cloud
Interconnection.

The following are the key technical aspects of this model:

• This model is based on Layer-3 VPN NNI. In this model the traffic will flow over the GVPN
Backbone.

• The IZO Private network will connect to NGE network on an ENNI connection to the nearest
NGE equipment and the GVPN ASBR will connect to the nearest NGE equipment on the ENNI
connection. NGE network will provision an EPL service between the two ports.
 • These VLANs will be pre-provisioned or transparently passed and an aggregate bandwidth
will be allocated for all these vlans at the NGE network.

• Layer-3 BGP session will be established between the GVPN ASBR and the Cloud provider’s
Router during customer provisioning. The IP Addressing will be mutually agreed between the
cloud provider and TCL during the customer turn-up.

• PE-CE BGP sessions will be established between the Enterprise Customer’s CPE and the
GVPN PE router.

• Traffic will be prioritized in any QoS CoS based on Customer’s request.

3. Guidelines specific to Microsoft ExpressRoute Interconnect

As described above, the IZO Private will be a special form of an NNI connectivity where the partner
will be a cloud provider where virtual machines will be provisioned dynamically based on Customers
request. Cloud providers have an automated way of provisioning and managing the service and the
specific details will be captured in the following sections.

A. Physical connectivity diagram

The diagram below depicts the high level architecture for IZO Private with Microsoft.

 TCL GVPN PE will connect to nearest NGE device on 1G ports.

 The NGE equipment will connect to the Microsoft Azure platform over 10G links.
 NGE will configure 1G tunnel between the ports connecting Microsoft and TCL and
transparently pass all VLAN’s.
B. Customer Connectivity.
Diagram below provides a logical representation of connectivity between the customer and
Windows Azure via TCL

 Customer will order a circuit to connect to Microsoft Azure Services through connectivity
partner TATA Communications. A “circuit” represents a redundant pair of connections
between the customer and Windows Azure configured in Active-backup configuration.
 Azure services are categorized as Private, Public to represent the IP addressing schemes.
o Windows Azure compute services, namely virtual machines (IaaS) and cloud services
(PaaS) deployed within a virtual network are considered Private.
o Services such as Windows Azure Storage, SQL databases and Websites (to name a
few) are considered Public.
o Services such as O365, Exchange Online, SharePoint Online, Skype for Business, and
CRM Online are Microsoft Online Services or O365 services. Please refer section
(N. Microsoft/O365 Services) for details
 A sub-interface will be created for each service on both the interconnects i.e. 3 sub-interface
per interconnect per customer if a customer subscribes to all 3 services.
 For Private service requirement the Azure facing interface can be placed under the existing
VPN as it is extension his GVPN
 For Public and O365 create a new VRF/VPN as these are hosted on Public IIP’s and NAT is
performed at the CPE.
 Public and O365 interface can be placed under the same VPN as both are on Public IP’s
 BGP sessions will be setup across these 6 sub-interfaces so we can have redundancy and
routes exchanged.
  A remote site in this VPN can now reach the Azure Public/ Private/ O365 services via TCL IZO
Private.

C. Q-in-Q and PE-CE Routing

Diagram below depicts Q-in-Q setup on one of the links.

 TCL will use Q-in-Q VLAN tagging to hand off customers to Windows Azure. Microsoft will
provide the carrier with the port names for the two ports and an STAG to uniquely identify a
customer’s circuit. S-Tag will be unique across the physical link and unique per customer.
 Customer circuit may have 3 pairs of routing session enabled (one pair for public services,
one pair for private services and one pair for O365 services).
o Sub-interface for Private services is identified by a unique CTAG 113
o Sub-interface for public services is identified by a unique CTAG 114
o Sub-interface for O365 services is identified by CTAG 115
 Public and O365 interface can be placed under the same VPN as both are on Public IP’s
 The VLAN IDs are selected by the carrier and passed on to Windows Azure for configuring
routing.
 The 3 C-Tags used will be same for all customers and will be agreed upon by Microsoft and
TCL.
 Customers are required to have a pairs of BGP for every service in order to have a highly
redundant service.
 Local Preference and AS-PATH bgp attributes will be used to make one of the sessions as
primary and the other as a backup.
  Microsoft will use AS 12076 and TCL will use AS 4755
 Microsoft has set prefix limits for every peer, bgp session will be dropped if this limit
exceeds
o 4000 prefixes per BGP session for a Private peer
o 200 prefixes per BGP session for a Public peer
o 200 prefixes per BGP session for a O365 peer
 TCL will set a prefix limit of 2000 on every BGP session for all service types

D. IP addressing
TCL must allocate a /30 CIDR block for assigning IP addresses to interconnect interfaces, these /30 IP
address block must not conflict with the range reserved by the customer for Azure VNET/ on-
premises LAN.

Private Services:
 TCL will allocate IP’s for the Cloud facing interface i.e. 2*/30 Private IP
 TCL will allocate IP’s for the Customer facing interface i.e. atleast 1*/30 Private IP
 TCL will allocate IP’s for CPE management i.e. 1*/32 Private IP

Public Services:
 TCL will allocate IP’s for the Cloud facing interface i.e. 2*/30 Private IP
 TCL will allocate IP’s for the Customer facing interface i.e. atleast 1*/30 Private IP
 TCL will allocate IP’s for CPE management i.e. 1*/32 Private IP
 TCL will allocate Public non-routable IP’s for NAT i.e. 1*/32 Public non-routable IP for user
traffic
 Customer can also bring his own Public IP’s, in this case TCL will not allocate Public IP’s.

O365 Services:
 TCL will allocate Public IP’s for the Cloud facing interface i.e. 2*/30 Public non-routable IP
 TCL will allocate IP’s for the Customer facing interface i.e. atleast 1*/30 Private IP
 TCL will allocate IP’s for CPE management i.e. 1*/32 Private IP
 TCL will allocate Public non-routable IP’s for NAT i.e. 1*/32 Public non-routable IP for user
traffic
 Additional Public non-routable IP’s for Active Directory or any other Application which
requires static NAT will be allocated by TCL on request.
 Customer can also bring his own Public IP’s, in this case TCL will not allocate Public IP’s.

E. VPN Topologies
The IZO Private NNI will be very similar to an Option-A NNI. And any VPN topology (Full Mesh, Hub
and Spoke, Partial Mesh and Extranet) can be supported over the NNI.
F. Quality of Service
 Microsoft supports QOS only for Microsoft Online/ O365 services.
 TCL offers a single class QOS for Public and Private services customers, wherein customers
can opt to place all their traffic on any one COS queue
 For Microsoft Online/ O365 TCL will support multi-queue COS as for any unmanaged
customer. Refer Section (N. Microsoft Online/ O365 Services) for details

G. CPE Management
Since there will be no CPE at the cloud providers, there will be no CPE Management component for
the Cloud facing site, however remaining sites of the VPN can be managed.

H. Multicast
Multicast is not supported by Microsoft

I. Azure Expressroute Provisioning Process

The Snapshots in red are steps performed by customer and Blue by Service provider

1. Customer after signing into his Azure Portal requests a circuit and creates a ServiceKey.
Customer will use a PowerShell cmdlet to make this request.
For example we’ll use TATA as the service provider and will specify a 10 Mbps ExpressRoute circuit
in Hong Kong.

2. Once customer makes a request, it is visible on the Partner Portal under Expressroute tab
3. Click on the correct request to enter the provisioning workflow.
This page gives details of the S tag allocated for this circuit, the location, BW and circuit status.
The S tag allocated here has to be entered into the TCL provisioning systems so that the team
configuring the router interface has the vlans readily available.
NOTE: C vlan is allocated by TCL and has been fixed as 113 for Private Peer and 114 for Public peer

4. Click on start provisioning, enter the configure tab and enter details
Local AS number, IP subnets for the interconnect, C vlans.
NOTE: C vlan is allocated by TCL and has been fixed as 113 for Private Peer and 114 for Public peer
4. Configure the TCL router, Refer the Annexure for Sample configuration

5. BGP comes up and TCL is able to ping the Azure side IP.
Microsoft will start announcing prefixes on the Public peer immediately.
Prefixes on the Private peer will be seen only after customer maps a VNET to this circuit.

6. Come back to provision tab and click finish provisioning, now the state should be
provisioned
7. Customer sees the status of this circuit as Provisioned too

8. Customer to map the Virtual networks in Azure to this circuit.

9. MS will start advertising on the Public Peer once the BGP comes up.

Prefixes will be seen on the Private Peer after customer maps the VNET to the circuit
10. Customer Creates a VM and maps it to a Virtual network

10. This VM is reachable from the Site on TCL GVPN.

By default ICMP is blocked to all VM’s, If you have to test reachability to VM then use RDP/ SSH as
per the endpoint security policy applied by the customer.
J. Azure Expressroute Modify
Customer can modify certain properties of an ExpressRoute circuit without impacting connectivity.

i. Enable/ Disable Premium Add-on

Follow the steps below to enable ER Premium

Follow the steps below to disable ER Premium

ii. Bandwidth Change

Microsoft only supports BW upgrade without disruption.

You cannot reduce the bandwidth of an ExpressRoute circuit without disruption. Downgrading
bandwidth will require you to de-provision the ExpressRoute circuit, and then re-provision a new
ExpressRoute circuit.

Use the following command to upgrade the circuit

K. Azure Expressroute Termination Process

1. Customer sends a request to TCL asking termination. Customer will share the Service-Key
2. Partner logs into the portal finds the circuit for this S-Key and clicks on clear settings for both the
private and public services
3. Then click on Deprovision Circuit Option available on the lower task bar, this will clear the VRF
configured for this customer at Microsoft

4. Now the status of the circuit goes back to Not Provisioned.

At this stage the S vlan allocated to this circuit by Microsoft is not deleted, and is maintained until
customer deletes this circuit.

5. Once customer deletes the circuit, the request is removed from the partner portal and the S vlan
allocated is released
 L. Troubleshooting
Circuit Down:
Connection is the logical circuit assigned to each customer. Here customer link might not be pingable
or BGP down

1. Ping and check if the remote IP is reachable

2. Verify router configuration for the IP’s configured
3. NGE has provisioned a transparent Ethernet link, so circuit issues may not be reported to
NGE
4. Open a case via the Microsoft Premier Support Portal.
5. If BGP is down.
6. Verify the number of routes announced to AWS, it has to be less than 4000 as Microsoft has
prefix limit set to 4000, aggregate routes or announce a default route as per design.
7. Open a support case if issue still persists

Interconnect Down:
Interconnect is the physical link connecting Microsoft and TCL, Interconnect going down means all
customer on this link will be down.

1. TCL and Microsoft are connected via an Ethernet backhaul provided by NGE
2. Check TCL port configuration and status.
3. Contact NGE support as we do now for any backbone/access links and check Ethernet link
status
4. Open a case with Microsoft if issue persist.

Reserved Tiger Microsoft Interconnect

Location Hostname IOR
Port order Identifier
TATA-SGE-06MGR-CIS-1-PRI-
Primary si2-t1a-rt02 Gi 4/0/0 IOR_224073/ IOR_224082
A
Singapore 758972
TATA-SGE-06MGR-CIS-2-SEC-
Secondary si2-t1a-rt01 Gi 8/2/0 IOR_224014/ IOR_224016
A
TATA-HKB-06MGR-CIS-1-PRI-
Primary hk-t1a-rt01 Gi9/0/2 IOR_224123/ IOR_224134
A
HongKong 759061
TATA-HKB-06MGR-CIS-2-SEC-
Secondary hk-t1a-rt08-PE Gi3/0/1 IOR_224148/ IOR_224150
A
TATA-AMB-06GMR-CIS-3-
Primary am-t1a-rt01 Gi 4/1/1 IOR_246120 / IOR_246165
PRI-A
Amsterdam
TATA-AMB-06GMR-CIS-4-
Secondary am-t1a-hrt03 Gi1/1/2 IOR_246209 / IOR_246216
774858 SEC-A
TATA-LON04-06GMR-CIS-3-
Primary ln-t1a-l78x-hpe01 Gi10/1/11 IOR_246245 / IOR_246271
PRI-A
London 774856
TATA-LON04-06GMR-CIS-4-
Secondary ln-t1a-l78x-hpe02 Gi9/2/2 IOR_246272 / IOR_246274
SEC-A
TATA-BOM02-06GMR-CIS-1-
Primary mu-me01-hpe01 (3/1/4) IOR_274899
PRI-A
Mumbai 792796
TATA-BOM02-06GMR-CIS-2-
Secondary mu-me01-hpe02 (3/1/4) IOR_274924
SEC-A
TATA-MAA02-06GMR-CIS-1-
Primary cn-me01-hpe01 (3/1/10) IOR_274934
PRI-A
Chennai 792797
TATA-MAA02-06GMR-CIS-2-
Secondary cn-me01-hpe02 (3/1/10) IOR_274942
SEC-A
 TATA-ASH-06GMR-CIS-3-PRI-
Ashburn Primary ny-t1a-n99x-hpe01 P 3/1/10 IOR_292494
A
800680
TATA-ASH-06GMR-CIS-4-SEC-
Secondary ny-t1a-n99x-hpe02 P 9/1/10 IOR_292505
A
San Jose Primary sc-t1a-s99x-hpe01 P 10/1/10 IOR_292516 TATA-SJC-06GMR-CIS-3-PRI-A
800683 TATA-SJC-06GMR-CIS-4-SEC-
Secondary sc-t1a-s99x-hpe02 P 10/1/10 IOR_292522
A

M. Azure Support Process

1. A support case is opened from the portal: https://premier.microsoft.com
Username: IZOPrivate@tatacommunications.com
Password: TCL_AZURE@1

2. Upon accessing, the first view will be a summary of recent incidents as well as regional contact
information and announcements.

3. To Open a New Incident, Click on the “Submit New Incident” button in the middle of the screen.
Fill the fields as in the below snapshot.
Select sub-topic (Availability, Deployment, Development, Other, Partner Portal, Runtime) based on
the problem and clink NEXT
4. Select severity and describe the problem
Severity Options:
i. Severity C incidents have a response time of 4 business hours
ii. Severity B incidents have a response time of 2 business hours.
iii. Severity A incidents have a response time of 1 business hour and must be submitted over the
phone by referring to TCL Access ID : 001793888

5. Submit after selecting the contact method

N. Microsoft Online/ O365 Services
iii. INTRODUCTION
IZO Private lets you extend your on-premises network to Microsoft Cloud services such as Office 365,
Exchange Online, SharePoint Online, Skype for Business and CRM Online via the ExpressRoute
Microsoft/Office 365 peer.

iv. PRE-REQUISITES
 A valid and active Microsoft Azure account. An Azure subscription is a requirement even if
connectivity is limited to non-Azure Microsoft cloud services, such as Office 365 services.
 An active Office 365 subscription.
 Customer must have an existing business relationship with TCL ie customer should have an
existing GVPN connection.
 Internet connection for DNS resolution and CDN acceleration.
 Traffic destined to O365 services must be from a valid public IPv4 addresses.

v. DESIGN
The Following diagrams explain the Control plane and Data plane setup for typical Azure customer
 New sub-interface will be created on the MS interconnect, towards customer new links or
new sub-interface on existing last-mile will be created.
 Customer requires Internet for DNS and Akamai (MS uses Akamai for application
acceleration)
 A new VRF/VPN for O365/Public service to be created. (Existing VPN can also be used if the
pre-requisites are met)
 IZO private CPE at the customer location will connect to this new VPN with the O365
ExpressRoute circuit at the other side.
 MS will announce global O365 and regional Public service prefixes over this circuit.
 So the customer will now be preferring the more specific prefixes to a default route from
internet or if he has an Internet link with full routes then he has to prefer the IZO
connection.
 User traffic will be PAT’ed at the CPE
 Some applications in Azure will not work with PAT so a Static NAT has to be configured for
such applications eg Active Directory.

 Traffic to O365 will get NAT’ed on the CPE and flow via the O365 VPN to MS
 If customer requires a Public Services connectivity then the new circuit can also be placed in
the same O365 VRF as both are on Public IP’s
 Traffic to Internet will take the default route to Internet.
VLAN

 S-vlan is allocated by Microsoft for every service key generated by the customer.
 C-vlan for O365 services is fixed as 115.

IP Addressing

 TCL will allocate IP’s for the Cloud facing interface i.e. 2*/30 Public non-routable IP
 TCL will allocate IP’s for the Customer facing interface i.e. atleast 1*/30 Private IP
 TCL will allocate IP’s for CPE management i.e. 1*/32 Private IP
 TCL will allocate Public non-routable IP’s for NAT i.e. 1*/32 Public non-routable IP for user
traffic
 Additional Public non-routable IP’s for Active Directory or any other Application which
requires static NAT will be allocated by TCL on request.
 Customer can also bring his own Public IP’s, in this case TCL will not allocate Public IP’s.

NAT requirements for Microsoft/O365 peering

All Traffic to O365 must be initiated from a valid Public IP.

Traffic destined to Microsoft cloud services must be NAT’ed to valid public IPv4 addresses before
they enter the Microsoft network.

 User traffic destined to MS is NAT’ed at the on-premises CPE

 Certain scenarios require Microsoft to initiate connectivity to service endpoints hosted
within your network. A typical example of the scenario would be connectivity to ADFS
servers hosted in your network from Office 365. In such cases, the ADFS server has to be
static NAT’ed at the on-premises CPE.

QOS

QoS requirements apply to the Microsoft peering only

 The following table provides a list of DSCP markings used by Microsoft for Skype for Business
Application.

TRAFFIC TREATMENT (DSCP MARKING) WORKLOADS

CLASS
Voice EF (46) Skype / Lync voice
Interactive AF41 (34) Video
AF21 (18) App sharing
CS3 (24) SIP signaling
Default AF11 (10) File transfer
CS0 (0) Anything else

 Microsoft when sending traffic to TATA will DSCP tag packets based on the workloads.
  For traffic from Customer to Microsoft, customer has to specify port ranges for each of the
workloads and create QOS policies on the client computer. Refer the link
“ https://technet.microsoft.com/en-gb/library/jj205371.aspx ” on how customer can set
Qos.
 Customer can they subscribe for QOS from TATA and select a profile from the 6COS model.

Routing & BGP Communities

 Routing exchange will be over eBGP protocol. EBGP sessions are established between the
MSEEs and TCL routers
 Microsoft will use AS 12076 and TCL 4755
 EBGP will be configured to achieve active/backup using bgp LP and AS-PATH prepend
 MS will accept 200 prefixes per BGP session.
 Default route advertisement over Public/ O365 peering is not permitted
 Once BGP is established MS will announce all the regional O365 routes.
 Microsoft will tag prefixes advertised through public peering and Microsoft peering with
appropriate BGP community values indicating the region the prefixes are hosted in. This will
be available only for customers subscribing to ExpressRoute Premium.

 MS has not enabled bgp community tagging till now. Below are the details of the regional
communities expected from MS:

Geopolitical Microsoft Azure BGP community

ExpressRoute location
Region region value
East US New York 12076:3004
East US 2 12076:3005
West US Santa Clara 12076:3006
US
Central US 12076:3009
North Central US 12076:3007
South Central US 12076:3008
South America Brazil South 12076:3014
North Europe London 12076:3003
Europe
West Europe Amsterdam 12076:3002
East Asia Hong Kong 12076:3010
Asia Pacific
Southeast Asia Singapore 12076:3011
Japan East 12076:3012
Japan
Japan West 12076:3013
Australia East 12076:3015
Australia
Australia Southeast 12076:3016
India India South Chennai 12076:3019
 India West 12076:3018
Mumbai
India Central 12076:3017

 In addition to the above, Microsoft will also tag prefixes based on the service they belong to.
This applies only to the Microsoft peering. MS has not enabled bgp community tagging till
now. The table below provides a mapping of service to BGP community value expected from
MS:

Service BGP community value

Exchange 12076:5010
SharePoint 12076:5020
Skype For Business 12076:5030
CRM Online 12076:5040
Other Office 365 Services 12076:5100
Global prefixes / Anycast 12076:5200
 BGP communities can be used to choose the correct exit interface in cases where customers
have multiple Public/ O365 peers.
 For the latest list of BGP communities refer link: https://azure.microsoft.com/en-
gb/documentation/articles/expressroute-routing/
 For the latest list of Azure regions and ExpressRoute locations refer link:
https://azure.microsoft.com/en-gb/documentation/articles/expressroute-locations/

vi. PROVISIONING STEPS

1. Customer will request ExpressRoute dedicated circuit from Microsoft via his powershell CLI

2. Customer will place an order with cloud type as O365 and service key details.

3. Partner to login to the Partner-portal and locate the service key under the ExpressRoute tab

4. Click on start provisioning, enter the configure tab and enter details
Peer AS number, IP subnets for the interconnect, C vlans.
5. TCL should also enter the prefixes that need to be announced to Azure via the O365 peer.
Microsoft will validate the owner of the Public against the regional routing internet registry (RIR) or
an internet routing registry (IRR)

Scenario 1 TCL provided Public IP announcement:

 Using whois lookup MS will verify if the IP list provided matches the PEER AS.
 Validation is automatic

Scenario 2 Customer Provided Public IP announcement:

 Validation is manual today
 TCL will have to raise a tac case with MS for IP validation, mail should mention the IP, AS,
Owner name. validation will take 48 hrs
 MS is working on automatic validation wherein the portal will have fields to enter AS
number details of the advertised prefixes.
 Refer Annexure for sample tac template for IP validation

6. Complete the provisioning by clicking on the “Finish Provisioning” tab

7. BGP comes up immediately and TCL will start receiving regional O365 prefixes

O. ExpressRoute Premium Add-on

 ExpressRoute Premium is an add-on over the Standard ExpressRoute offering.
 Customer can create a circuit with premium add-on using the cmdlet:
 New-AzureDedicatedCircuit -CircuitName $CircuitName -ServiceProviderName
$ServiceProvider -Bandwidth $Bandwidth -Location $Location -sku Premium
 Or can enable the ExpressRoute premium add-on for your existing circuit using the following
PowerShell cmdlet:
“Set-AzureDedicatedCircuitProperties -ServiceKey "***" -Sku Premium”
 The ExpressRoute Premium add-on provides the following capabilities:

o Increased route limits for Azure private peering from 4,000 routes to 10,000 routes.

o Global connectivity for services. An ExpressRoute circuit created in any region will have
access to resources across any other region in the world. For example, a virtual network
created in West Europe can be accessed through an ExpressRoute circuit provisioned in
India.

o Increased number of VNet links per ExpressRoute circuit from 10 to a larger limit (20-
100), depending on the bandwidth of the circuit.
P. References
https://azure.microsoft.com/en-gb/documentation/services/expressroute/

Q. Annexures
i. Template for IP validation

Subject: IP Validation for Microsoft Peering service key: aef3de7b-001b-449c-b310-4ef8bbd10a33

Content:
Hi Team,
Please validate IP's for the Microsoft Peer, details below:

Service Key: aef3de7b-001b-449c-b310-4ef8bbd10a33

IP details: 180.87.34.1/32, 180.87.34.2/32
AS number: 6453
 ii. Sample TCL PE configuration

Below are the Interface configs for Cisco and Alcatel Lucent routers.
VLAN 555 is S-TAG
VLAN 113 is C-TAG
For Alcatel Lucent Primary router:

port 2/2/9
ethernet
mode access
encap-type qinq
no autonegotiate
exit
no shutdown

interface "Port 2/2/9:555:113" create

description "Cloud connect customer A Private Service"
address x.x.x.x/30
icmp
no mask-reply
no redirects
exit
sap 2/2/9:555.113 create
description "Cloud connect customer A Private Service "
ingress
scheduler-policy "XX"
qos XX
exit
egress
scheduler-policy "XX"
qos XX
exit
exit

bgp
group "AZURE"
family ipv4
as-override
type external
import "IMPORT_AZURE"
export "PE-CE_EXPORT"
local-as 4755
peer-as 12076
disable-communities extended
prefix-limit 2000
neighbor x.x.x.x
exit
exit

policy-statement "IMPORT_AZURE"
entry 10
action accept
local-preference 200
Alcatel Secondary Router

port 2/2/9
ethernet
mode access
encap-type qinq
no autonegotiate
exit
no shutdown

interface "Port 2/2/9:555:113" create

description "Cloud connect customer A Private Service"
address x.x.x.x/30
icmp
no mask-reply
no redirects
exit
sap 2/2/9:555.113 create
description "Cloud connect customer A Private Service "
ingress
scheduler-policy "XX"
qos XX
exit
egress
scheduler-policy "XX"
qos XX
exit
exit

bgp
group "AZURE"
family ipv4
as-override
type external
import "PE-CE_IMPORT"
export "EXPORT_AZURE"
local-as 4755
peer-as 12076
disable-communities extended
prefix-limit 2000
neighbor x.x.x.x
exit
exit

policy-statement "EXPORT_AZURE"
entry 10
action accept
as-path-prepend 4755 1
 For Cisco router

Primary router
ip vrf AZURE
rd 4755:
route-target export 4755:
route-target import 4755:

interface GigabitEthernet9/0/2.555113
description Microsoft Cloud Connect @ Hong Kong - Microsoft Private Services POC
encapsulation dot1Q 555 second-dot1q 113
ip vrf forwarding AZURE
ip address 172.30.100.9 255.255.255.252
no ip redirects
no ip proxy-arp
service-policy input Policy
service-policy output Policy
!
address-family ipv4 vrf AZURE
no synchronization
redistribute connected
neighbor 172.30.100.10 remote-as 12076
neighbor 172.30.100.10 activate
neighbor 172.30.100.10 as-override
neighbor 172.30.100.10 route-map AZURE-IN in
neighbor 172.30.100.10 maximum-prefix 2000
exit-address-family

route-map AZURE-IN permit 10

set local-preference 200
Secondar Router

ip vrf AZURE
rd 4755:
route-target export 4755:
route-target import 4755:

interface GigabitEthernet3/0/1.555113
description Microsoft - Private Service POC
encapsulation dot1Q 555 second-dot1q 113
ip vrf forwarding AZURE
ip address 172.30.100.13 255.255.255.252
no ip redirects
no ip proxy-arp
shutdown
service-policy input Policy
service-policy output Policy
!
address-family ipv4 vrf AZURE
no synchronization
redistribute connected
neighbor 172.30.100.14 remote-as 12076
neighbor 172.30.100.14 activate
neighbor 172.30.100.14 as-override
neighbor 172.30.100.14 route-map AZURE-OUT out
neighbor 172.30.100.14 maximum-prefix 2000
exit-address-family
!
end

route-map AZURE-OUT permit 10

set as-path prepend 4755