Intel Cloud Builders Guide Intel Xeon Processor-based Servers Visual Data Center*

Intel Cloud Builders Guide to Cloud Design and Deployment on Intel Platforms
Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Audience and Purpose

This reference architecture outlines the usage of energy management technologies as part of planning, provisioning, and optimizing strategies in cloud data centers to reduce energy cost. It is intended for data center administrators and enterprise IT professionals who seek Data Center Infrastructure Management solutions to achieve better energy efficiency and power capacity utilization within new or existing data centers. The techniques and results described can be used as a reference to understand the implementation of energy management solutions using hardware and software components illustrated. The reader should be able to develop appropriate energy management solutions based on the design options presented using Visual Data Center* Infrastructure Management Software1 that has Intel Data Center Manager2 as an integrated component, and servers implementing Intel Intelligent Power Node Manager3. Intel Intelligent Power Node Manager is implemented on Intel server chipsets starting with Intel Xeon processor 5500 series platforms4 with significant improvements in the new version, Intel Intelligent Power Node Manager 2.0 being introduced in 2012 on server platforms supporting the Intel Xeon processor E5-2600 product family, which are presented in this paper.

Intel Xeon Processor E5 Product Family Intel Xeon Processor E3-1200 Product Family Intel Xeon Processor 5500 Series Intel Xeon Processor 5600 Series

Visual Data Center

March 2012

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Table of Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Server Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Visual Data Center* Infrastructure Management Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Test-bed Blueprint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Software Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Hardware and Software Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Physical Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Server Setup and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Visual Data Center Software Installation and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Connect Intel Data Center Manager (Intel DCM) to VDC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Add Nodes to VDC through Intel DCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Place Servers into Racks in the 3-D Floor View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Energy Management Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Real Time Server Power Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Real Time Temperature Monitoring and Thermal Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 Generate a Thermal Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 Energy Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Energy Cost and Emission Rate Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22 Carbon Emissions Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Report by Device Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Energy Cost Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 Rack Density Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 Optimize Rack Density/Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 Things to Consider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Architectural Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Future Enhancements in Visual Data Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Power Limiting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Enhanced Device Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Appendix A: Server Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Intel Power Management Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Intel Data Center Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25 Intel Intelligent Power Node Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Executive Summary
The evolution of cloud computing has resulted in highly efficient and carefully optimized data centers with increased server density and capacity that make considerations on energy consumption and utilization extremely critical, along with several other factors that were not as significant in smaller data centers of the past. To support this evolution, Intel works with end users to create an open data center roadmap of usage models that address key IT pain points for more secure, efficient, and simple cloud architectures built on a foundation of transparency. This paper describes a Data Center Infrastructure Management reference architecture based on Visual Data Center * and Intel Power Management solutions via usage models aimed at data center power efficiency and optimal utilization of provisioned power and cooling capacity. The goal of energy management usage models is to optimize productivity per watt in order to reduce total cost of ownership (TCO). Requirements include the capability to monitor power and ambient temperature in real-time at server, rack, zone, and data center levels. This means the ability to monitor and manage aggregated power consumption within a rack, zone, or data center based on available power and cooling resources. In this reference architecture we used servers enabled with Intel Intelligent Power Node Manager and Visual Data Center* (VDC) Infrastructure Management Software that uses Intel Data Center Manager (Intel DCM) to provide data center energy efficiency through real time power and ambient temperature monitoring. Visual Data Center has powerful Data Center Infrastructure Management through three dimensional views of data center and the ability to manage

data center equipment and components. The scope of this document is limited to power management of Intel Intelligent Power Node Manager enabled servers, and we describe the following energy management use cases in detail. We implemented new Intel Xeon processor E5-2600 family based servers, as well as servers based on previous generations to illustrate the use cases. Intel Intelligent Power Node Manager 2.0 is being introduced on server platforms supporting the Intel Xeon processor E5-2600 product family and has been significantly enhanced by utilizing the Running Average Power Limiting capabilities in the Intel Xeon processor E5-2600 product family. 1. Real-time power monitoring to get continuous and actual energy usage visibility at the server level that enables analysis and optimization of power capacity and provisioning in data centers Real time temperature monitoring enabled by servers with Intel Intelligent Power Node Manager, that helps view the thermal map in the data center without additional sensors, and take the necessary corrective actions Energy Reports for power and consumption at server, rack, row, room, and data center levels that can be extended to generate energy cost and carbon footprint reports Rack Density Optimization by monitoring actual power consumed by the servers and provisioning the power budget accordingly rather than using nameplate or de-rated power estimates

pooling of an on-demand, self-managed infrastructure, consumed as a service. This approach extrapolates applications and information from the complexity of underlying infrastructure, so IT can support and enable business value. In concert with Intel and other industry leaders, Visual Data Center helps reduce energy costs in cloud data centers with its innovative Data Center Infrastructure Management solutions. At the core of cloud computing is the ability of the underlying compute, network, and storage infrastructure to act as an efficient, shared resource pool that is dynamically scalable within one data center or across multiple data centers. With this foundation, critical higher-level capabilities such as energy management, guaranteed quality of service, federation, and data center automation are made possible. Intel, along with leaders in software, works to address these new core innovations in Infrastructure as a Service (IaaS). Intel has initiated a program to rapidly enable enterprises and service providers to clarify best practices around design (including reference architectures), deployment, and management. For enterprise IT and cloud service providers who need to utilize their existing data center infrastructure to supply cloud services to their customers, this guide, as part of the Intel Cloud Builders initiative, provides a comprehensive solution overview that covers technical planning and deployment considerations. While server performance-per-watt continues to increase, the energy consumed per server also continues to rise. These advancements enable increasing number of servers and density in modern data centers, making planning and managing of power and cooling resources critically important to ensure efficient utilization of provisioned capacity. In order to realize the vision of

2.

3.

4.

Introduction
Cloud computing is the new model for IT services that has emerged to break the trend of decline in flexibility while increasing in costs. It is an approach to computing that uses the efficient

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

cloud computing, new technologies are needed to address power efficiency and energy management. These will become fundamental to architectures from the microprocessor stage through the application stack. The focus of this paper is energy management and the related usage models. Based on the Environmental Protection Agencys report to the government, in 2006 data centers in the US consumed about 1.5 percent of the nations energy and were poised to double this by 20115. According to market research and consulting firm Pike Research, data centers around the world consumed 201.8 terawatt hours (TWh) in 2010 and energy expenditures reached $23.3 billion. Thats enough electricity to power 19 million average U.S. households. The good news is that, according to Pike Research, the adoption of cloud computing could lead to a 38 percent reduction in worldwide data center energy expenditures by 20206. Companies continue to explore approaches that focus on using existing data center power more efficiently to increase computing capacity, cut power costs, and reduce carbon footprints. Traditionally, organizations have lacked detailed information about actual server power consumption in everyday use. Typically, data center computing capacity has been based on nameplate power, peak server power consumption, or de-rated power loads. In practice however, actual power consumption with real data center workloads is much lower than the ratings. This situation results in over-provisioned data center cooling and power capacity, and increased total cost of ownership (TCO). Better understanding and control over server power consumption allows for more efficient use of existing data center facilities. All of this, applied across tens of thousands of servers, can result in considerable savings.

This paper begins with an overview of server power management and solutions offered by VDC. We then describe various usage models in detail with screenshots of the configuration and test process.

Aggregation of the power consumed at the rack level and management of power within a rack group to ensure that the total power does not exceed the power allocated to a rack. Higher level aggregation and control at the row or data center level to manage a power budget within the average power and cooling resources available. Optimization of productivity per watt through management of power at the server, rack, row, and data center levels to optimize TCO. Application of standards-based power instrumentation solutions available in all servers to allow management for optimal data center efficiency. Extension of instrumentation to enable load balancing or load migration based on power consumption, and close coupled cooling for the management of pooled power and cooling resources.

Server Power Management

In the past, power consumption was an afterthought for server deployment in data centers. Unfortunately, this view persists. For example, in many facilities the utility bill is bundled with the overall building charge, which reduces the visibility of the data center cost. Even though servers have become much more efficient, packaging densities and power have increased much faster. As a result, power and its associated thermal characteristics have become the dominant components of operational costs. Power and thermal challenges in data centers include: Increased total operational costs due to increased power and cooling demands Physical limitations of cooling and power within individual servers, racks, and data center facilities Lack of visibility into actual real-time power consumption of servers and racks Complexity of management components and sub-systems from multiple vendors with incompatible interfaces and management applications These data center management challenges can be translated into the following requirements: Power monitoring and limiting capabilities at all levels of the data center (system, rack identification, and data center). What can be done at an individual server level becomes much more compelling once physical or virtual servers are scaled up significantly.

Visual Data Center* Infrastructure Management Solution

Visual Data Center is an easy to use, centralized data center monitoring and operations management application based on an open integration platform. Through the effective use of Visual Data Center, organizations can improve proactive monitoring and resource management capabilities to improve operational efficiencies in areas such as space, power, and cooling within all of their IT facilities. Visual Data Center functions and capabilities include: Easy-to-use graphical navigation, 3-D visual modeling of facilities and racks, centralization of monitoring and alarming, custom dashboard and trend chart building, port mapping, reporting, capacity management (space, power, and cooling), energy efficiency (PUE), utilization and cost tracking, warranty and service management, asset management (dept owners, location, and tracking), and documentation management.

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Test-bed Blueprint
Intel has worked with Visual Data Center to implement a test bed that features the Visual Data Center Infrastructure Management solution and Dell, SGI, and other Intel processor-based servers that have implemented Intel Intelligent

Power Node Manager Technology. The test bed is intended to provide a flexible environment to simulate those aspects of a commercial data center that are relevant to cloud computing usage models. Visual Data Center software uses the Intel Data Center Manager SDK as an integrated component.

Figure 1: VDC* 3-D visual modeling of facilities and racks

Design Considerations Intel Intelligent Power Node Manager compliant servers as well as a PMBus7 based power supply for real-time power monitoring are required. Software Architecture The following diagrams show the Visual Data Center System Architecture and Software Architecture.

Figure 2: VDC System Architecture

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Visual Data Center is much more than an application - it is an open application framework. The platform can effortlessly talk to SNMP, Modbus, IPMI or any third party data sources. The interface can seamlessly integrate with 2-D, 3-D, and any third party UI plug-in. The software can also work with third party databases/ directories/data sources, such as Active Directory and more. The key Visual Data Center architecture layers include:

1.

Data Collection Layer - This layer is responsible for collecting raw data from IT and facility devices. Using the Visual Data Center Monitor Adapter technology, the system can monitor any device which supports any monitoring protocol. Data Storage Layer - Visual Data Center can reliably store large amounts of data, both real-time and historic, to serve upper layer

modules. The device model database provides complete and up-to-date model information for the end user. 3. Data Analysis Layer - With the built-in, powerful IntelliPass Data Analysis Engine, Visual Data Center can provide users almost endless possibilities for conducting data analysis. Resource Management Layer - This layer provides various core enterprise resource and process management modules. Presentation Layer - Both Smart Racks and Smart Devices revolutionize the way remote access is performed. With Visual Data Center providing real-device looking components with monitoring and controlling capabilities, administrators can visually have full control over the device as if they were in front of the device. The Presentation Layer supports both 2-D and 3-D UIs, as well as third party UI plug-ins.

2.

4.

5.

Figure 3: VDC Software Architecture

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Hardware and Software Description Hardware and software components as specified in the table below were used in the reference architecture test bed. However, wider hardware and software deployment options are supported. Please refer to the Visual Data Center install guides for details. Visual Data Center* Server Visual Data Center Client Virtual Machine Software Workstation Software 2 CPUs, 4GB RAM, 40GB Hard Disk CentOS* 5.5 32 bit Visual Data Center Server Software version 4.2 2 x Intel Xeon processor E7-2860 @ 2.26 GHz, 2GB RAM, 40GB Hard Disk Windows 7 Enterprise* 64-bit Intel Data Center Manager 2.4 Visual Data Center Client Software version 4.2 Server 1 and Server 2 Intel Server Board S2600CP based white box 2x Intel Xeon processor E5-2600 @ 2.7GHz, 32GB RAM, 1TB SATA HDD

Intel Intelligent Power Node Manager 2.0 enabled BMC Card PMBus* enabled power supply Software Server 3 SGI-C2005 Windows Server* 2008 R2 2x Intel Xeon processor 5150 2.6GHz, 4GB RAM, 40GB SATA HDD Intel Intelligent Power Node Manager 1.5 enabled BMC Card PMBus enabled power supply Software Server 4 Dell C6100 RHEL5* 64-bit 2x Intel Xeon processor 5150 2.6GHz, 4GB RAM, 250GB SATA HDD Intel Intelligent Power Node Manager enabled BMC Card PMBus enabled power supply Software Server 5 Intel White Box S5000WB Software Table 1: Hardware and Software description To generate load, workloads like Prime95 and Intel Power Thermal Utility were used. Windows Server 2008 R2 64-bit 2x Intel Xeon processor 5150 2.6GHz, 4GB RAM, 250GB SATA HDD Intel Intelligent Power Node Manager 1.5 enabled BMC Card PMBus enabled power supply Windows Server 2008 R2 64-bit

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Physical Architecture Figure 4 shows the test bed deployment architecture. Visual Data Center Server software is installed on a Visual Machine as shown, and client on a workstation. Five server nodes are used for use case testing that are Intel Intelligent Power Node Manager enabled. Visual Data Center software connects to the nodes out of band via Intel Data Center Manager to monitor power consumption and thermal data. We have also used simulated server nodes to better illustrate some of the usage scenarios.

Figure 4: Physical Architecture of Test Bed Setup

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Server Setup and Configuration Servers have to be set up with operating systems installed and BMC configured as described below. The reader is expected to have the basic knowledge of the server configuration and operating system installation. This will not be explained in detail in this paper. 1. In the BIOS, configure BMC network settings with static or DHCP option as desired, and provide the BMC hostname. We used DHCP. Note the BMC hostname or IP address. Note the user name and password of the BMC user with administrator privileges. Either use root user ensuring administrator privileges are granted, or add another user. Install the operating system, applications, and workloads as desired. Visual Data Center connects to the servers out of band with BMC hostname and log in credentials. There is no agent installed on the server.

Connect Intel Data Center Manager (Intel DCM) to VDC Intel DCM can be installed on the same server as Visual Data Center or on another server. To link Intel DCM to VDC, do the following steps: 1. 2. 3. 4. Log in to the VDC server Run the script vdctools Select option 2 Link with DCM Enter Intel DCM server IP address and host name

The Intel DCM server and VDC server are now linked. Add Nodes to VDC through Intel DCM 1. Start VDC client and log in.

2.

3.

4.

Visual Data Center* Software Installation and Configuration Installation and configure of Visual Data Center software will not be discussed in this document. Please refer to the Visual Data Center installation guide8 for detailed instructions. Configuring Visual Data Center to connect to Intel Data Center Manager, and adding server nodes are explained below. 2. Click on Admin Interface on the menu.

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

3.

Click on Devices, select Device tab, and click the Discover button on the bottom of the page:

10

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

4.

Click on Devices, select the Device tab, and click the Discover button on the bottom of the page.

The latest discovery results should be displayed as:

11

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

5.

Click on Restart to bring up the below screen:

12

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Enter the following values for discovery of Intel DCM supported devices: i. Monitor type should be Intel

ii. Enter IP subnet or IP range or specific IP address, as well as Subnet Mask iii. Enter BMC user name and password iv. Click Add to add to the IP Address Selection v. Repeat steps ii to iv for IP ranges as required. In this case 10.4.2.0 10.4.2.11 and 10.4.10.1 10.4.10.10 vi. Click Start The following screen will be displayed with results:

13

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

6.

Change the device name as appropriate to identify the server. Double click on each device discovered and select the Type, Manufacturer, Product Line, and Model as appropriate as below.

7.

Select the servers that are needed and click Create to add them in the VDC database to be available for adding in the Data Center Infrastructure.

14

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Place Servers into Racks in the 3-D Floor View: 1. Navigate to the 3-D Floor View.

2.

Double click on the rack that you want to add servers to.

15

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

3.

Enter Edit Mode.

4.

Select Search from the left side of the screen and navigate to the device you wish to add.

16

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

5.

Click and hold to drag the device to the desired U position within the rack or chassis as appropriate.

17

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Energy Management Use Cases

Real Time Server Power Monitoring Real time power monitoring at a server level is a critical capability that helps planning, provisioning, and optimizing data center energy and cooling capacity. Visual Data Center combined with Intel DCM can monitor energy usage at real time with a high level of accuracy on servers that implement Intel Intelligent Power Node Manager Technology. Power consumption can be viewed at the server level and aggregated to rack, row, room, and data center levels. The ability of the server to instrument real time power consumption data helps eliminate the need for expensive intelligent power strips in data centers.

In this use case we utilize the data that has been captured to look into the Visual Data Center monitoring capabilities and their usage. 1. Server level power and thermal monitoring

needed. In this case the Intel Server option is selected to view the power data as reported by the server.

The power demand report can be viewed at a server level. To view power and inlet temperature information at a server level, navigate through the 3-D model to the server to be monitored, or use the search capabilities to locate the server. After selecting the server, click Device in the Trend Charts section on the menu, and then click the Trend chart option as

18

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

To view the dashboard, select the Device option in the Alarm Panel, and select the Dashboard called Intel Server to see the dashboard view.

19

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Real Time Temperature Monitoring and Thermal Imaging Servers with Intel Intelligent Power Node Manager with inlet air temperature sensors can report server inlet air temperatures in real time that can be used by Visual Data Center to create a thermal image of the data center. Intel Intelligent Power Node Manager eliminates the need for additional thermal sensors and associated instrumentation to create thermal maps.

The thermal images provide data center managers real-time visualization, allowing them to take corrective action for environment cooling like adjusting set points, moving, adding, or removing vented floor tiles, and relocating equipment. Detailed information rack by rack can be used to plan where new gear can be placed with the lowest impact on cooling requirements.

Generate a Thermal Image 1. Navigate to the 3-D Floor View.

20

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

2.

Click on the Generate Thermal toolbar ribbon button.

Energy Reports Visual Data Center has reporting capabilities as illustrated below for device energy, cost, and power metrics. Each location added to the navigation tree allows for a cost per kilowatt hour definition which is the actual cost of power from the utility for the site. Each device managed in that site will then inherit the cost per kilowatt hour, which allows for energy cost at the device level.

To make PUE calculations easy, Visual Data Center allows users to define the energy profile for any device in the system. Devices listed as Energy Source will be used in the top portion of the PUE calculation for Total Power. Any device listed as IT Device will be used in the bottom portion of the PUE calculation for the IT Device Power. The result is the PUE ratio, which is tracked for all sites in the navigation tree.

Since this method allows for user definition of the IT power, users can configure IT power as the output of the UPS, total power of the PDU, rackmount PDU power totals, or individual IT assets within the rack.

21

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Energy Cost and Emission Rate Configuration Energy cost rates can be configured per building in the location view that would be used for calculation while generating energy cost reports. To configure the energy cost, go to Location Studio and right click on the

Building address, in this case 100 Wall Street to get the following screen. Enter the energy price as applicable per KWhr. VDC uses electricity emission rates published by U.S. Department of Energy9 and similar reports in other geographies which are configured in the VDC server to calculate carbon emission. The emission

rates are periodically updated in the VDC installations online from VDC data center. User can also override these configurations - please refer to VDC documentation for the steps. Some sample energy reports are illustrated below.

Carbon Emissions Report

22

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Report by Device Type

Energy Cost Report

Rack Density Optimization Optimize Rack Density/Usage The collection of real-time power consumption data constitutes an essential capability for power monitoring. Without this data, the best approximation for server power usage comes from the manufacturers specifications. To use the nameplate numbers as a guidepost requires the allowance of a hefty safety margin. To honor the safety margin, in turn, leads to data center power over-provisioning and stranded power that needs to be allocated in case it is needed, but is very unlikely to be used. This situation results in over-provisioned data center power, overcooling of IT equipment, and increased TCO. The availability of power monitoring data allows management by numbers,

which tightly matches servers by power consumption to available data center power. The use case is useful in older data centers under-provisioned for power and in host settings with power quotas in effect. In typical host data centers where the customers are allocated power quotas, the main goal is to optimize the rack utilization so as to place as many servers in a rack as the power budget allows. The following steps can be completed to implement this use case. 1. Monitor power consumption Power consumption of the server or rack should be monitored over a long period, either in production or in a simulated environment generating a load similar to that in production. Monitoring real

production servers is recommended to avoid unplanned power spikes. Duration should be days, weeks, or a quarter depending on the application life cycle scenarios and usage. Record the maximum power demand during the period. 2. Set the power budget for the rack Set the power budget above the determined power consumption with additional allowance, and rack density can be budgeted for this value rather than name plate power or de-rated power. The difference between the total power budget assigned to the rack and the power quota allocated for the rack would provide guidance on how many additional servers of similar power consumption can be added to the rack without overshooting the power quota allocated. Since we will be adding additional servers

23

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

to the rack, the overall performance of the system increases further and stays within the power envelope allocated by the hosting provider. Please refer to the Rack Density optimization case studies by Intel10 and external companies, some of which use power limiting capabilities which may be implemented in Visual Data Center in the future. Continue monitoring the power consumption It is important to continuously monitor the power consumption levels. If the consumption levels are getting closer the power budget frequently, it is advisable to increase the power budget.

Security Communication to servers is done using IPMI protocol secured by user name and password configured in the BIOS. VDC Server-Client Specifications Please refer to VDC Server-Client Specifications12 for VDC installation considerations and details.

Glossary
Intel Intelligent Power Node Manager: Intel Intelligent Power Node Manager resides on Intel Xeon processor 5500 based server (and later) platforms. It provides power and thermal monitoring and policy based power management for an individual server. Capabilities are exposed through standard IPMI interface from supported Baseboard Management Controllers (BMC). This requires an instrumented power supply such as PMBus*. Intel Data Center Manager: Intel Data Center Manager scales Intel Intelligent Power Node Manager functions to racks and groups of servers and enables IT users to benefit from increased rack density, as well as reduced capital and operational expenses. SDK: Software Development Kit QoS: Quality of Service KPI: Key Performance Indicators SLA: Service Level Agreement

Future Enhancements in Visual Data Center*

Power Limiting Server level power limiting capability supported by Intel Intelligent Power Node Manager will be added in future versions of Visual Data Center. This would enable further optimization of server and data center power consumption and planning. Enhanced Device Support As Intel DCM supports more devices for power management, the Visual Data Center integrations with Intel DCM will be updated to maintain tight integration with the Intel DCM platform.

Things to Consider
Architectural and Design Considerations Scalability A single installation of Intel DCM can manage up to 5000 nodes11. For larger implementations, multiple instantiations would be required. Likewise, the Visual Data Center solution is also highly scalable. If multiple Intel DCM server instances are needed to support either multiple geographical locations or large number of devices, Visual Data Center is able to handle the requirements effortlessly by leveraging its distributed Probe architecture. Integration Visual Data Center connects with Intel Intelligent Power Node Manager enabled servers through agentless out of band connectivity, which makes integration easy. For the use cases in this paper, servers should be Intel Intelligent Power Node Manager enabled and should have PMBus compliant instrumented power supply units. VDC also supports devices other than those that are Intel Intelligent Power Node Manager enabled, which is outside the scope of this document.

24

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Appendix A: Server Power Management

Intel Power Management Technologies Microprocessors are possibly the most energy intensive components in servers and have traditionally been the focus of power management strategies. Emergent technologies such as solid state drives have the potential to significantly reduce power consumption and, in the future, management of memory power consumption may be incorporated. Intel Intelligent Power Node Manager and Intel Data Center Manager are designed to address typical data center power requirements such as described above. Intel Data Center Manager (Intel DCM) Intel DCM SDK provides power and thermal monitoring and management for servers, racks, and groups of servers in data centers. Management Console Vendors (ISVs) and System Integrators (SIs) can integrate Intel DCM into their console or command-line applications to provide high value power management features. These technologies enable new power management paradigms and minimize workload performance impact. Intel Intelligent Power Node Manager Intel Intelligent Node Manager is implemented on Intel server chipsets starting with Intel Xeon processor 5500 series platforms with improvements in the new version, Intel Intelligent Power Node Manager 2.0 being introduced in 2012 on server platforms supporting the Intel Xeon processor E5-2600 family. Intel Intelligent Power Node Manager provides power and thermal monitoring and policy based power management for an individual server and is exposed through a standards based IPMI interface13 on supported Baseboard Management Controllers (BMCs). Intel Intelligent Power

Node Manager requires an instrumented power supply that conforms to the PMBus standards7. Some of the key features of Intel Intelligent Power Node Manager include: Real-time power monitoring Platform (server) power limiting Power threshold alerts Smart Ride-Through or SmaRT that increases the servers resiliency to transient faults where the voltage dips for a few cycles Closed Loop System Throttling or CLST that enables optimized platform designs that can continue operating under impaired power supply situations due to power supply unit failures Intel Xeon processors regulate power consumption through voltage and clock frequency scaling. Reduction of the clock frequency reduces power consumption, as does lowering voltage. The scale of reduction is accomplished through a series of discrete steps, each with a specific voltage and frequency. The Intel Xeon processor 5500 series4 can support power steps. These steps are defined under the ACPI14 standard and are colloquially called P-states. P0 is nominally the normal operating state with no power constraints. P1, P2, and so on aggressively increase the power capped states. Voltage and frequency scaling also impacts overall system performance, and therefore will constrain applications. The control range is limited to a few tens of watts per individual microprocessor. This may seem insignificant at the individual microprocessor level, however, when applied to thousands or tens of thousands of microprocessors typically found in a large data center, potential power savings amount to hundreds of kilowatt hours per month. Intel Intelligent Power Node Manager is a chipset extension to the BMC

that supports in-band/out-of-band power monitoring and management at the node (server) level. Intel Intelligent Power Node Manager 2.0 enabled platforms introduced in 2012 bring improved power limiting range and faster response to reach the target power limit through the use of the Running Average Power Limiting (RAPL) interfaces supported in Intel Xeon E52600 processor family. It also allows a finer degree of power control useful for certain applications: it is now possible to monitor and apply power limiting not only on the whole server, but also on the CPU and memory subsystems or domains individually as shown below in figure 5. The power management logic is carried out by a microcontroller embedded in the chipset, the Management Engine or ME, which runs firmware stored in flash memory in the baseboard. ME uses platform power consumption instrumented by PMBus compliant power supplies and RAPL interfaces to control power of the CPU and memory. It also reads inlet air temperature data from the temperature sensors. Utilizing these multiple sensors and controls, Intel Intelligent Power Node Manager provides server platform level power management capabilities that would be meaningful for the users. Intel Intelligent Power Node Manager requires an ACPI compliant operating system to apply power limits when the OS is running.

25

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Figure 5: Intel Intelligent Power Node Manager 2.0 overview

26

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

References
1. VDC* Infrastructure Management Software, http://www. visualdatacenter.com Intel Data Center Manager, http://software.intel.com/sites/ datacentermanager/index.php Intel Intelligent Power Node Manager, http://www.intel.com/ technology/intelligentpower/index. htm Intel Xeon processor 5500 product brief, ftp://download.intel. com/products/processor/xeon/ dc55kprodbrief.pdf EPA Report to Congress on Server and Data Center Energy Efficiency, http://www.energystar.gov/ia/ partners/prod_development/ downloads/EPA_Report_Exec_ Summary_Final.pdf Open Data Center Alliance* Usage: Carbon Foot Print Values, http:// www.opendatacenteralliance.org/ document-sections/category/71docdocs?download=434%3Acarbonfootprint-values PMBus*, http://pmbus.org/specs.html VDC software installation guide, http://visualdatacenter.com/vdcinfo/ VDC_Server_Installation_Guide_ v4_2.pdf U.S. Department of Energy Electricity Emission Factors, http:// www.eia.gov/oiaf/1605/pdf/ Appendix%20F_r071023.pdf

13. Intelligent Platform Management Interface, http://www.intel.com/ design/servers/ipmi/ipmi.htm 14. Advanced Configuration & Power Interface, http://www.acpi.info/

2.

3.

4.

5.

6.

7. 8.

9.

10. Rack Optimization Case Studies http://software.intel.com/sites/ datacentermanager/whitepaper.php 11. Intel DCM Scalability, http://software. intel.com/sites/datacentermanager/ datasheet.php 12. VDC Server- Client Specifications, http://visualdatacenter.com/vdcinfo/ VDC_Server-Client_Specifications_ v4_2.pdf

27

Intel Cloud Builders Guide: Data Center Infrastructure and Energy Management with 3-D Visualization with Visual Data Center*

Disclaimers
Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. See www.intel. com/products/processor_number for details. Intel Virtualization Technology requires a computer system with an enabled Intel processor, BIOS, and virtual machine monitor (VMM). Functionality, performance or other benefits will vary depending on hardware and software configurations. Software applications may not be compatible with all operating systems. Consult your PC manufacturer. For more information, visit http://www.intel.com/go/virtualization +No computer system can provide absolute security under all conditions. Intel Trusted Execution Technology (Intel TXT) requires a computer with Intel Virtualization Technology, an Intel TXTenabled processor, chipset, BIOS, Authenticated Code Modules and an Intel TXT-compatible measured launched environment (MLE). Intel TXT also requires the system to contain a TPM v1.s. For more information, visit http://www.intel.com/technology/security. Requires a system with Intel Turbo Boost Technology. Intel Turbo Boost Technology and Intel Turbo Boost Technology 2.0 are only available on select Intel processors. Consult your PC manufacturer. Performance varies depending on hardware, software, and system configuration. For more information, visit http://www.intel.com/go/turbo Intel Hypter-Threading Technology is available on select Intel Core processors. Requires an Intel HT Technology-enabled system. Consult your PC manufacturer. Performance will vary depending on the specific hardware and software used. For more information including details on which processors support HT Technology, visit http://www.intel.com/info/hyperthreading. INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTELS TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR. Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked reserved or undefined. Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information. The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order. Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or by visiting Intels Web site at www.intel.com. Copyright 2012 Intel Corporation. All rights reserved. Intel, the Intel logo, Xeon, Xeon inside, and Intel Intelligent Power Node Manager are trademarks of Intel Corporation in the U.S. and other countries. *Other names and brands may be claimed as the property of others.