Global Cache

Carrier Grade NAT Configuration Guide

December 2017

GGC Partners Only

 Contents
1 Introduction

2 Planning Your Network Topology

3 BGP Configuration

4 Notifying Google

5 Changing Routing for Privately Addressed Users

6 Testing and Diagnostics

6.1 Playing a Test Video
6.2 Verifying your Public IP address
6.3 Verifying your Private IP address
 1
Introduction
This document describes how to configure Google Global Cache (GGC) nodes to operate with your Carrier
Grade NAT (CGN) systems. Once configured, this will allow traffic between the GGC node and private
user IPs to bypass the CGN.

GGC has no requirement that "private IP space" be in the RFC1918 allocation. Any non-internet-routed
space in use in your network can bypass a CGN with this configuration. If appropriate for your usage, we
do recommend leveraging the dedicated CGN space allocated in RFC6598 (100.64.0.0/10).

The process for enabling CGN support for a GGC node has a number of steps. These are described in
detail in the remainder of this document.

1. Planning your network topology

2. BGP configuration
3. Notifying Google
4. Changing routing for privately addressed users
5. Testing and Diagnostics

Please see the 'Google Global Cache - Installation and Operations Guide' 1 for installing a GGC node to
serve general users.
 2
Planning Your Network Topology
The following figure shows a hypothetical network topology used as an example for this document.

Traffic between the users in the private IP space and the general Internet is NAT'ted. A route will be
added to allow traffic between these users and the High Traffic IPs in the GGC sub-net to bypass the
CGN.

Google will provide you with a list of High Traffic IPs for each GGC node.

CGN reference topology

In the route illustration above:

A user with IP 192.168.1.2 can reach a High Traffic IP (e.g. 208.117.227.36) on the GGC servers,
bypassing your CGN. The GGC server will see the user's source IP as 192.168.1.2. It will be able to
send traffic back to 192.168.1.2, bypassing the CGN servers. [red path]
A user with IP 192.168.1.2 can reach all GGC IPs other than the High Traffic IPs (e.g.,
208.117.227.31) through the CGN. The GGC server on 208.117.227.31 will see the user's source IP
NAT'ted as 208.117.8.0/24. It will be able to send traffic back to 192.168.1.2 through the CGN
servers. [green path]
Traffic from the user 192.168.1.2 is directed towards sites on the public Internet through the CGN.
The public Internet will see the user coming from a NAT'ted IP in 208.117.8.0/24. [green path]
Traffic between other users, in publicly routable space, and all GGC server IPs is routed
independently of the CGN. It is seen directly by the GGC servers (and any other server on the public
Internet) with a public non-NAT'ted source address. [blue path]

NOTE: Do not configure private user IP addresses to bypass the CGN, until you have received
confirmation from Google that the GGC nodes are ready for CGN-bypassed traffic. Changing your routing
without prior approval is likely to lead to very poor performance for users. It may even cause complete
failure for some user requests.
 3
BGP Configuration
Accepting traffic from private IP space requires special configuration of the prefixes advertised at GGC
nodes.

We need to know which prefixes are private user IPs, and which prefixes contain public addresses of your
CGN servers. Our systems can then correctly serve these users from all Google data locations.

NOTE: Incorrectly configured community tags will mean Google systems will not be able to distinguish
which user requests originate in private IP space. This will cause traffic to be served from data locations
other than the GGC node.

The following community tags are REQUIRED:

BGP communities for NAT devices & users

Community NAT Configuration Range
15169:12000 Indicates a prefix contains a NAT device. These should be
specified as /24 prefixes
15169:12100 Indicates a prefix is assigned to users who are NATed (it
is either RFC1918, RFC6598, or usurped globally unique
address space)

In the example network, the advertisements would be:

BGP communities example

CIDR Usage Community
208.117.8.0/24 CGN device 15169:12000
192.168.0.0/16 Users routed via CGN 15169:12100
208.0.0.0/8 Users not routed via CGN none

We require all GGC nodes in the same Google Network Location (GNL) 2 to have the same prefixes and
tags advertised to them.

If you have multiple NAT devices in the same GNL, you must ensure the following:

1. All the CGN device prefixes are advertised to all GGC nodes in the GNL, with community
15169:12000.
2. All the private user addresses are advertised to all the GGC nodes in the GNL, with community
15169:12100.
3. You must ensure that CGN bypass can be enabled for all nodes in the GNL, and for all user
prefixes, all at once. This is to ensure that all the private IP addresses have a direct route to all the
GGC nodes.

This means that you cannot reuse private addresses behind multiple NAT devices at the same GNL. You
can reuse private addresses behind NAT devices at different GNLs.

You can configure these community tags now. Until Google verifies CGN support for the node, and you
reconfigure your routing, users will continue to be served via the public IP addresses of your CGN servers.

Changes to your BGP configuration may take up to 2 hours to be seen in all Google systems

For general details about configuring BGP for GGC, please see 'Google Global Cache - Installation and
Operations Guide'. For more detail on use of community tags, please see 'BGP Community Support for
Google Serving'
 4
Notifying Google
Once you have planned your network topology and added community tags to your BGP advertisements,
please inform GGC Operations that the node is ready for CGN support. You should do this by emailing
<ggc@google.com>

Please include the following information:

GGC node names, for all nodes in this GNL 3

Confirmation that you have added community tags to your advertisements, at all nodes in the GNL
Confirmation that you are ready to bypass the CGN for connections to High Traffic IPs
Any special scheduling requirements

Google will verify your prefix advertisements and the GGC node configurations.

We will then provide you with:

A list of High Traffic IPs on the GGC servers, for you to use in your routing configuration
A proposed schedule for final configuration of CGN support on the GGC nodes
 5
Changing Routing for Privately Addressed
Users
Once Google has verified CGN support for the GGC nodes in this GNL, we will let you know that it is ready
for CGN-bypassed traffic.

When we have done this, you can then re-configure your routing, for all nodes in the GNL.

You should ensure that all traffic between private user IPs tagged with 15169:12100, and the High Traffic
IPs on the GGC servers, bypasses the CGN.

Let us know when you have made this change, and we will verify that we see traffic directly from private
user IPs.
 6
Testing and Diagnostics
6.1 Playing a Test Video
Follow the procedure below to play a test video. If the system works as intended, the video traffic should
be served from the GGC node, bypassing the CGN. This procedure is similar to the test procedure in the
'GGC - Installation and Operations Guide'.

Using Google Chrome from a PC in one of the tagged private user IP ranges:

1. Open the 'Developers Tools' (Menu Button > Tools > Developer Tools or Shift+Ctrl+I)
2. Open the 'Network' tab
3. Play a popular video, such as http://www.youtube.com/watch?v=dQw4w9WgXcQ
4. Skip any ads. Ensure that the video starts to play properly.

5. Watch the item 'videoplayback' request in the Network tab. This shows the name of server the video is actually
played from:

Developers Tools in Google Chrome

6. Resolve this server name using nslookup or similar tools:

$ nslookup r3---sn-bjvg2-1gie.c.youtube.com
Server: <your_name_server>
Address: <your_name_server IP>

Non-authoritative answer:
r3---sn-bjvg2-1gie.c.youtube.com
canonical name = r3.sn-bjvg2-1gie.c.youtube.com.
Name: r3.sn-bjvg2-1gie.c.youtube.com
Address: 193.142.125.14

If the resulting address (193.142.125.14 in this example) is an address in the sub-net allocated to the GGC
node, then video is playing correctly from the cache.

NOTE: The base web pages of www.youtube.com may not be served from the cache. These host names
will typically not resolve to the GGC node, and will not bypass the CGN

You can use FireFox as well to perform this test, but you will need to install the FireBug extension.
6.2 Verifying your Public IP address
The following URLs provide information on how Google systems see your IP address. This will assist us in
determining if CGN-bypass is working correctly.

This URL should display the public address of your CGN servers, and the GGC node name.
http://redirector.googlevideo.com/report_mapping

The response should look something like:

208.117.8.1 => yourispname-abc1

6.3 Verifying your Private IP address

You will need to specify one of the High traffic IPs from the list given to you by Google. In this example it
is 208.117.227.36.

This URL should display your private IP: https://208.117.227.36/debug_info

The response should look something like:

client_ip = '192.168.1.2'
private_ip_ranges match: true
url_ip = ''
destination_ip = '208.117.227.36'
date_time = '2013/10/10-21:26:15.424'
validation status code = SIGNATURE_MISMATCH

Check that the response includes a tagged private address for client_ip and has private_ip_ranges
match: true

1. The 'GGC Installation and Operations Guide' is available for download

Dell Poweredge R710 Guide

Dell Poweredge R720xd Guide
↩

2. A Google Network Location (GNL) is a set of GGC nodes serving the same users, with the same failover policy.
GNL's for your network are shown in the Google ISP Portal↩

3. GGC node names are listed in the Google ISP Portal↩