The Keystone of Internet Economy

Version 3.1
Saturday, February 17, 2001

Presentation_ID

1999, Cisco Systems, Inc.

 IXPs

- The keystone to Internet Economy

Why

should I connect to an IXP?

Transit

and Peering

Interconnections
Types

of IXPs & Examples of IXPs

Ciscos
Case

Role

Studies

Technical
Presentation_ID

1999, Cisco Systems, Inc.

- A Short History

Addendum
www.cisco.com

Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

 In many places, ISP would love to

interconnect, but cannot afford large core
routers just to connect to an IXP.
Route reflectors enable ISPs to use
smaller affordable routers.
These routers all interconnect with one
BGP peer connection to a BGP Router
Reflector.
The Route Reflector provides the routing
information

Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

AS 300

Normal
Normal BGP
BGP needs
needs
to
to be
be meshed!
meshed!

Presentation_ID

1999, Cisco Systems, Inc.

AS 100

AS 200

www.cisco.com

Reflector

Router
Router Reflector
Reflector allows
allows
for
for aa BGP
BGP hub
hub (star)
(star)
configuration
configuration

Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

 Solves BGP mesh problem on the IXP

where you need larger routers
Packet forwarding between IXP routers
is not affected.
Normal BGP speakers can co-exist on
the IXP.
Multiple reflectors for redundancy
Easy migration

Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

 Single ANS for the route reflector.

Private AS Number can be used as long
as transit services are forbidden on the
IXP.
Cannot change attribute of
reflected routes.
Multi-lateral Peering Agreement best
fits the route reflector topology.
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

172.16.0.0/19

172.16.96.0/19

B1

A1

N1

AS6500

T1

T1

A2

192.168.1.1

172.16.224.0/19

T3

B2

N2
Cisco
2501

Cisco
2501

Cisco CAT 5000

192.168.1.3

Cisco
7206

192.168.1.2

10M
10M

100M
192.168.1.4

192.168.1.5

10M

100M

Cisco 4500

Router Reflector
Cisco 3620

R1
Presentation_ID

1999, Cisco Systems, Inc.

Services

Z1
www.cisco.com

172.16.0.0/19

172.16.96.0/19

B1

A1

N1

AS6500

T1

T1

A2

192.168.1.1

172.16.224.0/19

T3

B2

N2
Cisco
2501

Cisco
2501

Cisco CAT 5000

192.168.1.3

Cisco
7206

192.168.1.2

10M

iBGP Peers

10M

100M
192.168.1.4

192.168.1.5

10M

100M

Cisco 7206VXR

Router Reflector
Cisco 3620

R1
Presentation_ID

1999, Cisco Systems, Inc.

Services

Z1
www.cisco.com

10

172.16.0.0/19

172.16.96.0/19

172.16.224.0/19

B1

A1

N1

AS6500

T1

T1

A2

T3

B2

N2
IP Traffic

192.168.1.1

Cisco
2501

Cisco
2501

Cisco CAT 5000

192.168.1.3

Cisco
7206

192.168.1.2

10M
10M

100M
192.168.1.4

192.168.1.5

10M

100M

Cisco 7206VXR

Router Reflector
Cisco 3620

R1
Presentation_ID

1999, Cisco Systems, Inc.

Services

Z1
www.cisco.com

11

iBGP Config

Router A2

Router R1 ! Route Reflector

router bgp 65000

router bgp 65000

neighbor 192.168.1.5 remote-as 100

neighbor 192.168.1.1 remote-as 100

neighbor 192.168.1.1 route-reflectorclient

Router B2

neighbor 192.168.1.2 remote-as 100

router bgp 65000

neighbor 192.168.1.5 remote-as 100

neighbor 192.168.1.2 route-reflectorclient

neighbor 192.168.1.3 remote-as 100

Router N2

neighbor 192.168.1.5 route-reflectorclient

router bgp 65000

neighbor 192.168.1.5 remote-as 100
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

12

172.16.0.0/19

172.16.96.0/19

B1

A1

ISP-A

N1

ISP-B

A2

ISP-N

AS65000
iBGP

T1

T1

192.168.1.1

172.16.224.0/19

T3

B2

N2
Cisco
2501

Cisco
2501

Cisco CAT 5000

192.168.1.3

Cisco
7206

192.168.1.2

10M
10M

100M
192.168.1.5

10M

192.168.1.4

100M

Cisco 7206VXR

Router Reflector
Cisco 3620

IXP
Services

R1
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

13

172.16.0.0/19

172.16.96.0/19

ISP-A

ISP-B

eBGP Peer

T1

T1

A2

192.168.1.1

ISP-Bs
AS200
iBGP

B1

A1

172.16.224.0/19

N1

ISP-N

AS65000
iBGP

T3

B2

N2

eBGP Peer

Cisco
2501

Cisco
2501

ISP-Ns
AS300
iBGP

Cisco CAT 5000

192.168.1.3

Cisco
7206

192.168.1.2

10M
10M

100M
192.168.1.5

10M

192.168.1.4

100M

Cisco 7206VXR

Router Reflector
Cisco 3620

IXP
Services

R1
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

14

 Router Reflector Based IXPs have

proven to scale to +60 ISPs.
What are the scaling options?
Direct

Peering with Peers and the

Router Reflector

Replacing

the Router Reflector with a

Router Server

Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

15

172.16.0.0/19

172.16.96.0/19

ISP-A

AS65000
iBGP

T1

A2

B2

Z1

ISP-N

ISP-Z

T3

T3

ISP-Zs
AS300
iBGP

eBGP Peer

N2

eBGP Peer

Cisco
2501

Cisco
2501

172.16.128.0/19

ISP-Ns
AS300
iBGP

N1

ISP-B

T1

192.168.1.1

ISP-Bs
AS200
iBGP

B1

A1

172.16.224.0/19

Cisco CAT 5000

192.168.1.3

Z2
Cisco
7206

192.168.1.6

Cisco
7206

192.168.1.2

10M
10M

100M
192.168.1.5

10M

192.168.1.4

eBGP Peer

100M

Cisco 7206VXR

Router Reflector
Cisco 3620

IXP
Services

R1
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

16

172.16.0.0/19

172.16.96.0/19

ISP-As
A1 AS100

172.16.224.0/19

ISP-Bs
AS200

B1

N1

ISP-Ns
AS300

172.16.128.0/19

Z1

ISP-A

ISP-B

ISP-N

ISP-Z

T1

T1

T3

T3

A2

Cisco
2501

Cisco
2501

192.168.1.1

eBGP Peer

B2

Cisco CAT 5000

N2

192.168.1.3

ISP-Zs
AS400

Z2
Cisco
7206

192.168.1.6

Cisco
7206

192.168.1.2

10M
10M
eBGP Peer

100M
192.168.1.5

192.168.1.4

eBGP Peer

100M

Cisco 7206VXR

IXP
Services

Router Server
Unix w/ Route Server Daemon
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

17

172.16.0.0/19

ISP-B

T1

T1

A2

Cisco
2501

Cisco
2501

Use the
BGP
removeprivate-as
command

AS4558
iBGP

B2

192.168.1.2

Upstream
ISP

172.16.224.0/19

ISP-Bs
AS65000
iBGP

B1

ISP-A

192.168.1.1

ISP with
Private
AS
number

172.16.96.0/19

A1

N-GW

N1

ISP-Ns
AS300
iBGP
Use the
BGP
removeprivate-as
command

ISP-N
T3

N2

192.168.1.3

Cisco CAT 2924XL

Cisco
7206

10M
100M
10M

192.168.1.5

10M

192.168.1.4

100M

Cisco 7206VXR

Router Reflector
Cisco 3620

IXP
Services

R1
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

18

Domestic + International Together

Domestic + International Separate

Internet

National
Gateway

256 kbps

256 kbps

ISP A
2

bp

ISP C

bp

bp

ISP B

ISP A

64

s
bp
K

8
12

8
12

64

s
bp
K

National
Gateway

ISP C

ISP B
4 Mbps

512 Kbps

Internet

2
51

ps
Kb

IXP
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

19

noexport
RR marks prefixes
as BGP Community noexport

Upstream #1

170.10.0.0/20

ISP A

Presentation_ID

Upstream #2

170.10.0.0/20 No-Export

IXP

ISP B

ISP A advertises

ISP Bs GW router

170.10.0.0/20 to the
IXP

sees no-export and

does not advertise
upstream.

1999, Cisco Systems, Inc.

www.cisco.com

20

172.16.224.0/19
N1

N-GW

ISP-Ns
AS300
iBGP

US-GW

Upstream
ISP

The Internet

ISP-N
T3

IXP
AS4558
iBGP

Cisco
2650

N2

Ingress/Egress
Packet and
Route Filtering

192.168.1.3

100M

Cisco CAT 2924XL

192.168.1.5

10M

Router Reflector
Cisco 3620

R1
Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

21

ISP #1
ISP #2
Peer A
IXP-W

A
Peer B
IXP-E

Customer A

Customer C

Customer B

E
F
Customer D

109:200

109:100

Presentation_ID

1999, Cisco Systems, Inc.

www.cisco.com

22