WHITE P APER Why Your Next Datacenter Network Will Likely Be a Fabric

Sponsored by: Brocade Rohit Mehra September 2012

EXECUTIVE SUMMARY
The use of Ethernet fabrics as the foundation for next-generation datacenter networking has garnered much industry attention over the past year. This isn't surprising given the strong potential role that Ethernet fabrics can play as a key enabler for next-generation datacenters. Driven by advanced virtualization, the datacenter of the future is rapidly becoming a reality today and will be characterized by network use and optimization for both public and private cloud computing. Cloud computing is a force of change in IT organizations across all vertical markets. Private clouds, based on advanced virtualization capabilities, represent one of the key transformational technologies in today's IT environments and, over time, will become the prevailing IT service delivery method. Private clouds take advantage of the capabilities afforded by virtualization to provide unprecedented flexibility in resource allocation and applications. By providing greater levels of automation, orchestration, provisioning, and deployment, private clouds can help organizations become more nimble while reducing operating costs and improving resource utilization. For years, the fast rate of change needed by business has met with friction when faced with the processes and procurement times of most IT organizations. This often causes other internal departments to circumvent IT organizations instead of partnering with them to achieve broader business goals. Dynamic clouds, by contrast, can become the clutch between two gears, allowing for smooth synchronized operations between IT and other departments. Moving to implement a private cloud takes thoughtful planning. To prepare, IT organizations need to make sure that their existing IT and network infrastructure is structured and tuned in such a way as to ensure that a cloud model can deliver on its promises. Because the network is the foundation for cloud-based application transactions taking place across multiple domains both on premise and off premise, it must be optimized for performance, resiliency, and scalability. Traditional networking approaches simply can't address the bandwidth, latency, and scalability requirements demanded by the advanced virtualization capabilities inherent in the private cloud paradigm. At a very basic level, the active/passive path usage and the low per-path utilization of just a few years ago fail quickly with a symphony of applications and application traffic flying around an abstracted and automated cloud environment. Passive devices and links create as much waste as if not more waste than the underutilized server that drove the organization to virtualization in the first place.
www.idc.com Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015

Fortunately, achieving network transformation will be facilitated by another major industry trend that's running in parallel with cloud computing: the rise of Ethernet fabrics. As a way to achieve faster and easier private cloud implementations, Ethernet fabrics can offer benefits directly related to the clearly defined networking requirements that cloud computing imposes, including: Increased performance and scalability Agility/ease of provisioning Automation Full network utilization Improved application performance Increased service velocity This white paper explains how and why Ethernet fabrics can deliver these benefits in today's rapidly evolving IT environments.

CHALLENGES AND TRENDS IN BUILDING A CLOUD-OPTIMIZED NETWORK

Cloud computing is emerging as a way for organizations to dynamically provision services without placing an additional burden on IT staff. By enabling automated provisioning, the cloud model can help IT departments more rapidly scale their compute resources while retaining the flexibility needed to adapt to changing business requirements.

Implementation Plans
In today's IT environments, private clouds represent an extension of the first phase of server virtualization, paving the way for datacenters that are fully virtualized with pooled resources. IDC is forecasting major growth in cloud computing over the next five years. IDC survey data shows rapidly developing interest and initiatives on the part of IT organizations. The IDC CIO Agenda Survey that was conducted in November and December 2011 (see Figure 1) found that investing in cloud services was among the top initiatives cited by decision makers and drew the most responses compared with all other options.

#237004

2012 IDC

FIGURE 1
Top Initiatives for 2012
Q. In 2012, which of the following will be the top 3 IT initiatives at your organization?

Invest in cloud services Consolidation/virtualization

Collaboration tools Big data/business analytics and modeling f or our organizational data Application portf olio consolidation/rationalization Automate systems operations with enterprise management sof tware and tools

Business process management Enhance security/risk management

ERP consolidation

Database consolidation Social media

Smart technologies VoIP/unif ied communications Deliver a range of desktop and mobile devices to end users Enterprise architecture Converged IT

Content management
Open source technologies Video conf erencing/telepresence 0
n = 36 Note: Data shows the percentage of respondents who listed the initiative as a top 3 initiative.
Source: IDC's CIO Agenda Survey, NovemberDecember 2011

10

15 20 25 30 (% of respondents)

35

40

2012 IDC

#237004

An earlier IDC survey found that 44% of IT decision makers are "considering private clouds." Looking at the projected buildouts required to support cloud deployments, IDC expects network infrastructure investments to more than double over the next three years, growing to over $1 billion each for the public cloud segment and the private cloud segment by 2013. IDC's 2012 Cloud Survey shows that private cloud investment plans remain a major priority for IT organizations. As Figure 2 illustrates, 17% of IT budgets are allocated to private cloud today and 22% of IT budgets are expected to be allocated to private cloud in the next 24 months.

FIGURE 2
Cloud Budget Considerations
Q. What percentage of your budget will be allocated to the following areas today and in 24 months?

Traditional IT (internal) we own and manage the hardware, sof tware, and IT staf f

Private cloud architectures (deployed and managed at my company's site)

Outsourced IT/ASP/application management third party owns hardware and runs IT staf f , software ownership is mixed Public cloud services (subscription, delivered over Internet) Private cloud architectures (hosted/managed by third-party provider, dedicated to the use of my company) 0 5 10 15 20 25 30 (% of budgets) 35 40 45

Today 24 months
n = 800 Note: Means include zeros.
Source: IDC's Cloud Survey, 2012

#237004

2012 IDC

The same survey shows that "advanced networking" was ranked third in importance when respondents were asked to prioritize the technology elements deemed essential for implementing private clouds (see Figure 3).

FIGURE 3
Private Cloud: Important Technology Elements
Q. Please rate the importance of the following technology elements that must be present for IT delivery infrastructure to be considered a private cloud.

Server virtualization

Storage virtualization

Advanced networking

Self -service provisioning Automated provisioning of IT resources "on demand" Real-time monitoring of resource utilization Formalized SLAs in place with end users End-to-end application perf ormance monitoring 0 5 10 15 20 (% of respondents) 25

n = 800 Note: Data shows the percentage of respondents who rated the technology element most important.
Source: IDC's Cloud Survey, 2012

2012 IDC

#237004

IDC's 2012 Virtualization Survey asked respondents to identify the primary impetus to rearchitecting the network to support private cloud deployments (see Figure 4). Networking-related issues were among the top 3 items cited by respondents specifically, concerns about having adequate bandwidth and concerns about the network becoming a bottleneck in the provisioning of new services.

FIGURE 4
Private Cloud Network Priorities
Q. What was the main reason you needed to rearchitect the network to support private cloud?

We needed to ensure security between virtual servers We needed more bandwidth to support the virtualized environments
The network became a bottleneck to new service provisioning

We needed simpler/easier/faster provisioning of network devices

We needed to reduce the amount of time to provide ongoing configuration and maintenance

We wanted to migrate to a unified Ethernet fabric (FCoE, iSCSI) We needed to allow for virtual server mobility between physical servers for operational load balancing We needed to allow for virtual server mobility between physical servers for disaster recovery 0 5 10 15 20 25 (% of respondents) 30 35

n = 200
Source: IDC's Virtualization Survey, 2012

Challenges in Implementing Private Clouds

The allure of private cloud computing lies in dynamic service provisioning, the resource allocation private clouds enable, and the "pay as you go" model typical of private clouds. A key enabler is server virtualization, which delivers dramatic efficiencies by decoupling software resources from their physical hardware platforms.

#237004

2012 IDC

Using the network, administrators can quickly create and move virtual machines (VMs) among various compute nodes as required. VMs leverage the available processing, memory, and I/O resources of physical servers to create a flexible pool of available resources. In addition to on-demand resource allocation, the widespread deployment of server virtualization on x86 systems has resulted in major cost and efficiency benefits, including the ability to sharply reduce datacenter footprints. However, deriving the full spectrum of benefits from the private cloud requires both rigorous planning and additional investment on the part of IT organizations. As indicated in Figure 4, a critical area requiring such investment is the enterprise network, which needs to be robust and scalable enough to handle the additional virtualized workloads that will be introduced into the private and/or hybrid cloud. Traditional Ethernet topologies used in the datacenter cannot adequately address the bandwidth, latency, resiliency, and scalability requirements demanded by the private cloud paradigm. In the virtualized datacenter, network connections between all major IT domains need to be fully optimized. However, the traditional approach typically consists of a multitiered hierarchical topology with bandwidth oversubscription employed in every tier. In this arrangement, switches communicate up and down multiple switch "layers."

Spanning Tree Protocol Limitations

While Spanning Tree Protocol (STP) offers loop-free Layer 2 topology, overall bandwidth efficiency is reduced by at least 50% because alternate paths are unavailable. By contrast, Ethernet fabrics do not require overprovisioning because they use a nonblocking multipath topology where all links are active. When a network link failure occurs, STP is slow to reconverge. This causes severe degradation of network performance and can cause a complete loss of frame forwarding capabilities. In an Ethernet fabric in which multiple, equal cost paths exist, the network dynamically adjusts to link failures and rebalances network traffic to ensure there is no loss of frame forwarding capabilities. This can be done at two levels with Layer 2 equal cost multipathing (ECMP) at a multiple node level and Layer 1 trunking between nodes, which, in combination, can provide great efficiency and resiliency. This, however, tends to degrade network performance. In the virtualized environment, organizations can have many applications on a single server, resulting in the traffic between two servers increasing as a factor of how many VMs are on each server. In addition to this concentration of traffic that was once spread over much larger areas, applications themselves become more distributed. This results in a significant amount of traffic between servers and between racks of servers. This effect is often referred to as "east-west" traffic flow. The typical legacy configuration of traffic flowing up and down layers places different pressures and design requirements on the network. The result is that few if any networks that were built even just a few years ago are ready for the east-west traffic flows of today. These traditional hierarchical networks often had many layers forcing application

2012 IDC

#237004

flows across several "hops," a design that introduces latency and impacts application performance. No matter how much efficiency can be gained from reducing the latency of a single device, nothing can provide the benefit of reducing the number of hops. All of these known characteristics of traditional Ethernet networks have the potential to create bottlenecks and can negatively affect the dynamic resource provisioning benefits inherent in the private or hybrid cloud model. By their very nature, these networks contain the same inefficient practices that virtualization was created to fix in the server world.

VM Mobility Issues
As mentioned, the promise of cloud computing is to enable IT to match compute, storage, and network resources to shifting business demands. The ease with which VMs can migrate within the datacenter for load balancing, maintenance, and energy efficiency is a key benefit of server virtualization and therefore private clouds. However, currently, less than 10% of organizations have progressed to a fully virtualized, on-demand datacenter, where virtual workloads continuously and automatically migrate. The network has been one of the barriers to accelerating this IT maturity model. Traditional network architectures limit the scale and mobility of a workload because of the way that networks are partitioned and compartmentalized. Performance, scale, and intelligence specific to each VM are required to meet the availability and resiliency requirements of a highly virtualized datacenter. The movement of VMs adds further stress to the network, and slow or static responses can undercut that mobility. Keeping the network in step with virtualized environments requires low latency to quickly move VMs from one server to another, both within and between datacenters. If the network is bogged down by STP or multitiered Ethernet topologies, the ability of the private cloud to offer dynamic resource provisioning will be constrained.

MAKING THE CASE FOR ETHERNET FABRICS

Virtualization and private cloud computing needs are driving the industry to apply some of the characteristics of Layer 3 networking, such as link-state routing and equal cost multipathing, to Layer 2 in the form of Ethernet fabrics and other virtual network models. This approach improves performance and avoids some of the slowdowns caused by routing table management. The fundamental idea behind an Ethernet fabric is to create a more efficient, costeffective way to move data through a network. Fabric networks are switch based and designed for greater flexibility and simplicity than today's Layer 3 routed networks. They also offer greater scalability than classic hierarchical switched Ethernet networks that rely on STP. Ethernet fabrics involve what could be loosely described as "Layer 2 routing." Building large, fast datacenter networks that can be "routed" based on location-independent Layer 2 MAC addresses has become possible now that memory and processors have grown far more powerful and less expensive than they were when Layer 3 routers were first designed.

#237004

2012 IDC

Performance and Self-Healing

Ethernet fabrics are cloud friendly because they can automatically adjust to changing physical and logical conditions and requirements in the network. Nodes can be added to the fabric on an automatic, nondisruptive basis with rapid reconvergence time. In addition, they can be removed without any operator intervention. Connection paths are designed to form automatically with no configuration required once the switches are connected. Because automation is at the very heart of what a private cloud offers, manual operations need to be minimized as much as possible. If and when a link outage does occur, fabrics can offer self-healing capabilities that automatically redistribute traffic so that business interruption due to performance degradation is avoided. Ethernet fabrics can optimize performance by reducing latency. Whereas STP does not use all links available in the network, Ethernet fabrics keep all links active. By deterministically leveraging multiple paths through the network and continuously determining the most efficient route, the fabric enables high performance while maintaining high availability. In addition, because the fabric keeps all links active, the total number of devices required in service is greatly reduced. This simplifies the network and reduces both capex and opex significantly.

Incremental Implementation and Scalability

Fabrics can be deployed progressively, workload by workload or by node, without completely redesigning a datacenter network. Fabrics that offer this kind of migration capability allow IT organizations to engage in focused experimentation with fabric technologies while minimizing risk and expense. The amount of data managed by the enterprise is projected to grow exponentially over the next few years, putting even greater demands on already overloaded networks. IT managers will need to implement easily scalable networking infrastructures to manage the accelerated growth of data and network traffic. Ethernet fabric solutions combine the best attributes of traditional Layer 2 switching (speed) with those of Layer 3 routing (intelligence and multipath data forwarding) to determine best paths for transmissions and to preserve policies as VMs move around the datacenter.

VM Mobility and Management

With the benefits of full link utilization, deterministic multipathing, and subsecond link reconvergence, Ethernet fabrics optimize VM performance, laying a solid networking foundation for private and hybrid cloud implementation. It is currently fairly simple to move VMs within a blade chassis, but it becomes more difficult as the VMs' sphere of mobility expands across chassis, across racks, and across datacenters. The network, then, needs to be virtualization aware such that port policies follow VMs as they move to different compute nodes.

2012 IDC

#237004

VMs are also becoming a shared management issue across IT domains. Ethernet fabrics provide network operations teams with greater visibility into how VMs are behaving on the network. The sheer number of entities to manage in a virtualized datacenter, where VMs can be provisioned in minutes, grows swiftly and can get out of hand quickly. The flexibility inherent in an Ethernet fabric simplifies the ability to provision and deprovision VMs either on demand or in accordance with business rules and SLAs. The ability to simplify and consolidate both physical cabling and the number of devices in the network will also drive customer adoption of Ethernet fabrics. The overarching mandate is for enterprises to make sure that all components of their virtualized datacenters, including the network, are in sync so that no single component lags and inhibits migration. By migrating to an Ethernet fabric in their datacenters, network managers can achieve their private cloud goals, including reduced infrastructure complexity, increased IT flexibility, and improved workload mobility and performance.

CHALLENGES/OPPORTUNI TIES
Both cloud computing and Ethernet fabrics are still in the earlier stages of the maturity curve as they evolve in parallel. However, they also offer the benefit of incremental implementation as a prelude to future expansion. Going forward, IT organizations will need to better understand the benefits of using flatter, fabric-based networks with distributed intelligence rather than traditional Layer 2 switched or Layer 3 routed networks, given their static nature. As with any new market-ready technology, there is a learning curve with respect to the full spectrum of benefits and challenges. While IT organizations are attracted to the idea of future proofing their network, they need to be convinced that the investment required will indeed effect a lasting network transformation that both extends the ROI cycle and reasonably guarantees that new cloud-based and virtualized applications can be implemented without significant additional investment. It will be important for vendors to educate enterprise buyers with respect to how an Ethernet fabric can seamlessly integrate into their traditional Ethernet switched networks. Prospective buyers need to more fully understand how Ethernet fabric switches can be used as traditional Layer 2 10GbE switches or how Ethernet fabrics can be deployed alongside existing hierarchical Ethernet datacenter topologies.

CONCLUSION
Cloud computing represents the centerpiece of IT transformation today and, in time, will become the prevailing IT service delivery method. To prepare for private cloud computing, IT organizations need to make sure that their existing network infrastructure is optimized for the performance required. Achieving this goal will be made easier by the advent of Ethernet fabrics, which can offer increased performance, scalability, ease of provisioning, and automation and at the same time reduce capex and opex. The goal of

10

#237004

2012 IDC

Ethernet fabrics is to simplify the network by removing layers and reducing the number of physical devices that need to be managed. Cloud-optimized networks will enable resources to move freely through the network because VMs can move seamlessly and transparently to any compute node in the primary datacenter or another datacenter. This increases business agility and allows companies to respond to revenue-generating activities much more quickly. Overall, the industry trend is toward a bigger, flatter network that preserves the low cost and simplicity of classic Ethernet, requires no reconfiguration or change in port configurations when resources move, and uses all paths available for the most efficient transport of VMs and their data. Given these market realities, enterprises seeking to optimize the investments they make in virtualization and cloud technologies should consider migrating to an Ethernet fabric. IDC believes that by migrating to an Ethernet fabric in their datacenters, network managers can reduce infrastructure complexity, increase IT flexibility, and improve workload mobility, enabling private clouds.

Copyright Notice
External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2012 IDC. Reproduction without written permission is completely forbidden.

2012 IDC

#237004

11