Opportunities and Challenges of Open

Networking

Tobias Jungel, Daniel Fritzsche, Hagen Woesner

BISDN GmbH
 BISDN Who we are:
Founded 2012 in Berlin, Germany

Business so far:
SDN/NFV prototypes for carriers and system integrators

Now:
Early product stage for SDN-WiFi and basebox
 Which problem do we want to solve?

Integration of whitebox switches into OpenStack

Manageable by IT administrators

Switch configuration (VLAN, VXLAN, MPLS, IP, NAT,...)

MLAG, failover, bond interfaces

VLAN assignment per port

Non-goal:
Learn another CLI (I know iproute, do I really need to know more?)
use a separate management system
ssh into each and every switch.
What you want

Hardware Accelerated Routing

Hardware Accelerated Tunneling

Hardware Accelerated NAT

Fast Packet IO between VMs/Container

 What you get (when IT guys do networks)

OpenVSwitch bridges stacked 3 levels high

VM VM VM VM

qbrX qbrY qbrZ qbr

br-int All software,

and really poor performance
~400 kpps
br-tun

vtep vtep vtep vtep vtep

Plus dumb bridges or routers that could do a lot more than L2 forwarding!
 General ASIC pipeline: richly functional

decap: GRE, NVGRE, rate limiting, scheduling

VxLAN, MAC-in-MAC, ... bridging
routing

tunnel L2/L3/
port VLAN ACL QoS modifications port
termination MPLS

content-aware encap VLAN,

decap VLAN, Q-in-Q processing
filter VLAN tunnels
 Control a BCM ASIC

FBOSS, ODL, Floodlight

OpenSwitch ONOS, baseboxd

Fastpath & ICOS, IP Infusion,

OpenNSL OF-DPA
BSN, pica8, Cumulus, ...

Broadcom SDK

ASIC
 Operating system options
So, I have my OCP whitebox switch, what next?

No OPEN (in terms of FREE) access to the SDK/ASIC

 Control plane options for DC integration
Non-SDN control: IP routing
IS-IS or BGP routing
SDN controllers:
ONOS (c.f. CORD)
OpenDayLight (OPNFV et al.)
baseboxd
Intermediate
PCE or BGP speaker, centralized architecture
MPLS Segment routing
 baseboxd

Map physical switch interfaces to Linux as tap interfaces

Send/receive packets via tap interfaces to/from switch

Control switching tables via netlink

OF 1.3 is used as control protocol (with BCM extensions)

https://github.com/bisdn/basebox
 Why netlink?

Simple integration of existing tools:

Ansible/Chef/Puppet/Salt/... Quagga collectd

baseboxd IP BIRD SNMP

OpenFlow Netlink
OF-Config User Space

Kernel Space
nftables Routing Bridging VLAN Interfaces
 Baseboxd architecture

Linux kernel

netlink Tap manager Netlink

baseboxd Adaptation
baseboxd

OpenFlow Switch Abstract Interface

switch
 Linux view bridge abstraction

L2 + VLAN

Configured using a VLAN aware bridge

vlan_aw_br0

vid: 1-5 vid: 3-8 vid: 1-5 vid: 1-20,50

port1 port2 port3 port4

 Centralization and abstraction

BISDN implements domain control via an

baseboxd intermediate OF Controller
OpenFlow CAWR (Capability AWare Routing
(and OF-Config)
controller)
CAWR Implements LLDP, LACP, MLAG
OpenFlow
(and OF-Config)
Controller HA
Making network configuration
OF-Switch 1 OF-Switch n
persistent
CAWR is (currently) closed source
 Why netlink?

Simple integration of existing tools:

Ansible/Chef/Puppet/Salt/... Quagga collectd

baseboxd IP BIRD SNMP

OpenFlow Netlink
OF-Config User Space

Kernel Space
nftables Routing Bridging VLAN Interfaces
 Manual config
# controller (linux):
ip link add type bridge
ip link set bridge0 vlan_filtering 1
ip link set bridge0 up
ip link set port1 master bridge0

# switch: (client_flowtable_dump)
Table ID 10 (VLAN): Retrieving all entries. Max entries = 16384, Current entries = 2.
-- inPort = 1 (Physical) vlanId:mask = 0x0000:0x1fff (VLAN 0) | GoTo = 20 (Termination MAC) newVlanId = 0x1001 (VLAN 1)
| priority = 3 hard_time = 0 idle_time = 0 cookie = 0
-- inPort = 1 (Physical) vlanId:mask = 0x1001:0x1fff (VLAN 1) | GoTo = 20 (Termination MAC) | priority = 3 hard_time = 0
idle_time = 0 cookie = 0
 Manual config
# controller
bridge vlan add vid 2 dev port1

# switch: (client_flowtable_dump)
Table ID 10 (VLAN): Retrieving all entries. Max entries = 16384, Current entries = 2.
-- cut --
-- inPort = 1 (Physical) vlanId:mask = 0x1002:0x1fff (VLAN 2) | GoTo = 20 (Termination MAC) | priority =
3 hard_time = 0 idle_time = 0 cookie = 0

bridge fdb add 68:05:ca:30:63:69 dev port1 master vlan 1

 VLAN aware Bridge networkd
[NetDev] [Match]
Name=switchbr Name=port*
Kind=bridge
[Network]
[Bridge] Bridge=switchbr
DefaultPVID=1
VLANFiltering=1 [BridgeVLAN]
switchbr.netdev PVID=1
[Match] EgressUntagged=1
Name=switchbr VLAN=1-10
port.network
switchbr.network
 Integration into OpenStack

ML2 driver: etcd connector

Optionally, baseboxd controller can run directly on the switch

The option to run baseboxd directly on ofagent makes the

switch
as open source as it gets (because BRCM SDK is still binary
only)
 baseboxd and CAWR

etcd connector
systemd networkd
ML2
Neutron baseboxd
OpenFlow
(and OF-Config)

Compute Server CAWR

OpenFlow
Compute Server (and OF-Config)

Compute Server OF-Switch 1 OF-Switch n

 Sample HA setup
+----------------------------+ +----------------------------+
|controller A | |controller B |
|ACTIVE | |STAND-BY (FAILOVER) |
| +--------------------------------------------------+ |
| |etcd cluster | | | |
| | +-------------+ | | +-------------+ | |
| | | etcd node A | | | | etcd node B | | |
| | +-------------+ | | +-------------+ | |
| +--------------------------------------------------+ |
| | | | | |
| | | | | |
| | | | | |
| +-------v--------+ | | +-------v--------+ |
| | etcd_connector | | | | etcd_connector | |
| +-------+--------+ | | +-------+--------+ |
| | | | | |
| +----v-----+ +-------+ | | +-------+ +----v-----+ |
| | baseboxd | |keep- <------>keep- | | baseboxd | |
| +----+-----+ |alive.d| | | |alive.d| +----+-----+ |
| | +-------+ | | +-------+ | |
| +--v---+ | | | | +--v---+ |
| | CAWR | | | | | | CAWR | |
| +--+---+ + | | + +------+ |
| | | | |
+---------|------------------+ +----------------------------+
v
 Conclusions

Open Networking has come a long way

Still, there is a gap in accessibility of switches from SDN control

plane
ofagent, or OpenNSL binaries are missing in ONL

Controller Northbound Interface discussion still open

Netconf, OpenConfig, RESTconf,

We propose to use Linux API with baseboxd