Vous êtes sur la page 1sur 342

These eKit materials are to be used ONLY by you for the express purpose SELF STUDY.

SHARING THE FILE IS STRICTLY PROHIBITED.

D72641
June 2011
Edition 1.0
D71299GC10
Student Guide
Administration
Oracle Exalogic Elastic Cloud

Oracle University and (Oracle Corporation) use only.


Author Copyright 2011, Oracle and/or it affiliates. All rights reserved.

TJ Palazzolo Disclaimer

This document contains proprietary information and is protected by copyright and


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Technical Contributors other intellectual property laws. You may copy and print this document solely for your
own use in an Oracle training course. The document may not be modified or altered
and Reviewers in any way. Except where your use constitutes "fair use" under copyright law, you
James Bayer may not use, share, download, upload, copy, print, display, perform, reproduce,
publish, license, post, transmit, or distribute this document in whole or in part without
Rick Eppstein the express authorization of Oracle.
Mike Palmeter
The information contained in this document is subject to change without notice. If you
Dave Winter find any problems in the document, please report them in writing to: Oracle University,
500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not
warranted to be error-free.
Editors
Restricted Rights Notice
Pj Schemenaur
Richard Wallis

Oracle University and (Oracle Corporation) use only.


If this documentation is delivered to the United States Government or anyone using
the documentation on behalf of the United States Government, the following notice is
Smita Kommini applicable:

U.S. GOVERNMENT RIGHTS


Graphic Designer The U.S. Governments rights to use, modify, reproduce, release, perform, display, or
Satish Bettegowda disclose these training materials are restricted by the terms of the applicable Oracle
license agreement and/or the applicable U.S. Government contract.

Publishers Trademark Notice

Shaik Basha Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names
may be trademarks of their respective owners.
Srividya Rameshkumar
Contents
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1 Course Overview
Course Objectives 1-2
Target Audience 1-3
Introductions 1-4
Course Schedule 1-5
Course Practices 1-7

Oracle University and (Oracle Corporation) use only.


Classroom Guidelines 1-8
For More Information 1-9
Oracle by Example (OBE) 1-10
Related Training 1-11

2 The Exalogic Solution


Objectives 2-2
Why Data Centers? 2-3
Data Center Challenges 2-4
What Is Exalogic? 2-5
Why Exalogic? 2-7
What Is InfiniBand? 2-8
An Engineered Solution 2-9
Java Performance 2-10
High Availability 2-11
Multi-Tenancy 2-12
Software Compatibility 2-13
Selecting an Operating System 2-14
Licensing and Support Overview 2-15
Quiz 2-16
Summary 2-18

3 Hardware Components
Objectives 3-2
Exalogic Machine Topology 3-3
Compute Nodes 3-4
Compute Node: Front View 3-5
Compute Node: Rear View 3-6

iii
Compute Node Memory 3-7
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Storage Appliance 3-8


Disk Caching 3-9
Storage Appliance: Front View 3-10
Storage Appliance: Rear View 3-11
InfiniBand (IB) Switches 3-12
Gateway Switch Hardware Views 3-13
IB Host Channel Adapter (HCA) 3-14
Management Switch 3-15
Scaling Your Data Center 3-16

Oracle University and (Oracle Corporation) use only.


Scaling Your Data Center 3-17
Hardware Package Details 3-18
Power Options 3-19
Hardware Maintenance 3-20
Quiz 3-21
Summary 3-25

4 Initial Machine Configuration


Objectives 4-2
Power-On Procedure 4-3
Hardware Configuration Process 4-4
Exalogic Online Checklists 4-5
Example: Access Route Checklist 4-6
Site Requirements 4-7
Exalogic Networks 4-8
Network Configuration Worksheets 4-9
Default Addresses for a Full Rack 4-10
Default Host Names 4-11
Installation Spreadsheet 4-12
Exalogic Configuration Utility 4-13
ECU Internals 4-14
InfiniBand Networking Concepts 4-15
Bonded Interfaces 4-16
Default Network Configuration 4-17
Exalogic Diagnostic Utilities 4-18
Utility Examples 4-19
Distributed Command Line Interface (DCLI) 4-20
DCLI Examples 4-21
Quiz 4-22

iv
Summary 4-25
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Practice 4-1: Connect to a Compute Node 4-26

5 Fusion Middleware Concepts


Objectives 5-2
Middleware Concepts 5-3
Oracle Fusion Middleware (FMW) Review 5-4
WebLogic Server (WLS) Overview 5-5
Java EE Deployment: Example 5-6
WLS Architecture Overview 5-7

Oracle University and (Oracle Corporation) use only.


WLS Domain Contents 5-8
WLS Machines and Compute Nodes 5-9
Node Manager Concepts 5-10
Transaction Recovery 5-11
Message Recovery 5-12
WLS Cluster Architecture 5-13
WLS Exalogic Topology: Example 5-14
WLS Exalogic Capacity Planning 5-15
Quiz 5-16
Summary 5-19

6 Storage Configuration
Objectives 6-2
ZFS Appliance Capabilities 6-3
Administrative Tasks 6-4
Getting Started 6-5
Accessing the Browser User Interface (BUI) 6-6
Viewing Appliance Status 6-7
Viewing Network Settings 6-9
Some Supported Services 6-10
Configuring File System Services 6-11
Identifying Hardware Components 6-12
Replacing Failed Hardware 6-14
Performing Failover Actions 6-15
Creating Administrative Users 6-16
Roles and Authorizations 6-17
Shared Storage Concepts 6-18
Default Storage Configuration 6-19
Creating a Pool 6-20
Pool Data Profiles 6-21

v
Creating a Project 6-22
Creating a Share 6-24
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Overriding Project Defaults 6-25


Setting Quotas 6-26
Restricting Share Access 6-27
Configuration Management 6-28
Mounting an NFS Share 6-29
Storage Command-Line Interface (CLI) 6-30
Some CLI Commands 6-31
CLI Examples 6-32
FMW Storage Topology 6-33

Oracle University and (Oracle Corporation) use only.


FMW Storage: Example 6-34
Quiz 6-35
Summary 6-38
Practice 6-1 Initialize Shared File Systems 6-39

7 Network Configuration
Objectives 7-2
Exalogic Networks: Review 7-3
IB Networking Concepts: Review 7-4
Default Compute Node Network 7-5
Default Storage Appliance Network 7-6
Quarter-Rack Network Topology 7-7
Half-Rack Network Topology 7-8
Full-Rack Network Topology 7-9
Multi-Rack Topology 7-10
Transceivers and Cables 7-11
Administration Tasks 7-12
Status LEDs 7-13
Replacing Failed Hardware 7-14
IB Switch Command-Line Interface (CLI) 7-15
Monitoring the Hardware 7-16
Monitoring Connection Status 7-17
Running IB Diagnostics 7-18
Subnet Manager (SM) Concepts 7-19
Subnet Manager Recommendations 7-20
Configuring the Subnet Manager 7-21
Fabric Monitor Interface 7-22
Fabric Monitor Examples 7-23
Monitoring IB Connections 7-24
Monitoring Gateway Connections 7-25

vi
Multiple Addresses Versus Multiple Ports 7-26
Configuring a Virtual Address 7-27
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Network Configuration and Multi-Tenancy 7-28


Virtual LAN (VLAN) 7-29
Configuring a VLAN 7-30
Subnet Partitioning 7-31
Quiz 7-32
Summary 7-35
Practice 7-1 Configure Network Settings 7-36

8 Creating FMW Domains

Oracle University and (Oracle Corporation) use only.


Objectives 8-2
WebLogic Concepts: Review 8-3
WebLogic Communication Types 8-4
Installation and Configuration Process 8-5
Shared or Dedicated Installations? 8-6
Available WebLogic Installers 8-7
Installation Overview 8-8
Domain Templates 8-9
Configuration Wizard: Overview 8-10
Shared or Copied Domains? 8-13
Migrating Existing Domains 8-14
Pack and Unpack Tools 8-15
Starting the Administration Server 8-16
WebLogic Console: Overview 8-17
Adding a New Server 8-18
Updating Machine Membership 8-19
Updating Cluster Membership 8-20
Quiz 8-21
Summary 8-23
Practice 8-1 Initialize a WebLogic Domain 8-24

9 Optimizing FMW Domains on Exalogic


Objectives 9-2
Additional Configuration Tasks 9-3
Default WebLogic Networking 9-4
Isolating Client Communication 9-5
Isolating Cluster Communication 9-6
Session Replication: Overview 9-7
Network Channels 9-8
Creating a Channel 9-9

vii
Some Channel Protocols 9-11
Creating a Cluster Replication Channel 9-12
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Using SDP for Replication 9-13


Using Multiple Ports for Replication 9-14
Other Replication Optimizations 9-15
Other WebLogic Optimizations 9-16
Enabling Other Optimizations 9-17
Licensing Review 9-18
Quiz 9-19
Summary 9-21
Practice 9-1 Configure Network Channels and Optimizations 9-22

Oracle University and (Oracle Corporation) use only.


10 Starting and Migrating Servers
Objectives 10-2
Node Manager: Review 10-3
Node Manager: Architecture 10-4
Node Manager: Configuration Tasks 10-5
Recommended NM Storage Configuration 10-6
Basic Node Manager Properties 10-7
Enrolling a NM with a Domain 10-8
Starting Servers from the Console 10-9
Starting Servers from the Command Line 10-10
Starting NM on Machine Startup 10-11
WebLogic Logs 10-12
Practice 10-1 Start Servers with the Node Manager 10-13
Whole-Server Migration 10-14
Automatic Server Migration: Example 10-15
Leasing Types 10-16
Migration Configuration Tasks 10-17
Node Manager Network Properties 10-18
Configuring Cluster Leasing 10-19
Database Leasing Schema 10-20
Candidate Machines 10-21
Machine Failback 10-22
Manual Server Migration 10-23
Migrating a Server 10-24
Pinned Server Resources 10-25
Recommended Storage Configuration for JMS and Transaction Recovery 10-26
Configuring a Server's Transaction Store 10-27
Creating a JMS Persistent Store 10-28
Quiz 10-29

viii
Summary 10-32
Practice 10-2 Migrate Failed Servers 10-33
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

11 Exalogic Database Connectivity


Objectives 11-2
Ethernet Database Connectivity 11-3
Oracle Real Application Clusters (RAC): Overview 11-4
Oracle Exadata: Overview 11-5
Direct InfiniBand Database Connectivity 11-6
Exalogic Quarter Rack and Exadata Quarter Rack 11-7
Exalogic Full Rack and Exadata Full Rack 11-8

Oracle University and (Oracle Corporation) use only.


JDBC: Overview 11-9
WebLogic Data Source: Overview 11-10
Active GridLink for RAC 11-11
GridLink and FCF 11-12
GridLink and Services 11-13
GridLink and Single Client Access Name (SCAN) 11-14
Creating a GridLink Data Source 11-15
Enabling SDP for Exadata Connectivity 11-16
Multi Data Source: Overview 11-17
Multi Data Source: Architecture 11-18
Licensing Review 11-19
Quiz 11-20
Summary 11-22
Recorded Demonstration: Active GridLink 11-23

12 Additional FMW Infrastructure


Objectives 12-2
Coherence: Overview 12-3
Coherence: Architecture 12-4
Role of Coherence Applications 12-5
Basic Coherence Configuration Tasks 12-6
Creating a Coherence Cluster 12-7
Creating a Coherence Server 12-8
Starting Coherence Servers 12-10
Coherence*Web: Overview 12-11
WLS Cluster: Review 12-12
Proxy Plug-Ins 12-13
Recommended Proxy Topology 12-14
Obtaining and Using Plug-Ins 12-15
Oracle Web Tier: Overview 12-16

ix
OHS Architecture 12-17
Oracle Process Manager and Notification Server (OPMN) 12-18
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

OPMNCTL: Examples 12-19


Basic Plug-In Parameters 12-20
OHS Plug-In: Example 12-21
Quiz 12-22
Summary 12-24
Practice 12-1: Configure a Cluster Proxy 12-25

13 Managing Hardware with ILOM


Objectives 13-2

Oracle University and (Oracle Corporation) use only.


Integrated Lights Out Manager (ILOM) 13-3
ILOM Tasks 13-4
Sideband Management 13-5
Accessing the Browser Interface 13-6
Management Settings 13-7
Adding Administrative Users 13-8
ILOM Roles 13-9
Powering the Device On or Off 13-10
Updating Device Firmware 13-11
ILOM Glossary 13-12
Using Indicator LEDs 13-13
Viewing Sensor Readings 13-14
Managing Power Consumption 13-15
Analyzing Power Consumption 13-16
Alert Concepts 13-17
Configuring Alerts 13-18
Command-Line Interface (CLI) Example 13-19
Connecting to the Host OS 13-20
Quiz 13-21
Summary 13-23
Practice 13-1: Use Compute Node ILOM 13-24

x
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Course Overview

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle University and (Oracle Corporation) use only.


Course Objectives
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

After completing this course, you should be able to:


Locate and use checklists and worksheets to plan an
installation
Describe Exalogic's default storage and network setup
Perform basic storage administration and maintenance
tasks

Oracle University and (Oracle Corporation) use only.


Configure a compute node's network to support Fusion
Middleware
Create and optimize a WebLogic domain on Exalogic
Start, kill, and migrate servers by using the Node Manager
View device status and health with Oracle Integrated
Lights Out Manager (ILOM)

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 1 - 2


Target Audience
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

This course is intended for data center administrators.


There are no prerequisite skills, but attendees will benefit
from having a fundamental knowledge of:
The UNIX command line
Basic UNIX network and storage concepts
The Oracle Fusion Middleware product stack

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Target Audience
If you are concerned whether your background and experience match that of the target
audience, ask the instructor.

Oracle Exalogic Elastic Cloud Administration 1 - 3


Introductions
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Introduce yourself.
Tell us about:
Your company and role
Any previous experience with Sun hardware
Any previous Fusion Middleware product experience

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 1 - 4


Course Schedule
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Day Lesson

1 AM 1. Course Overview
2. The Exalogic Solution
3. Hardware Components
PM 4. Initial Machine Configuration
5: Fusion Middleware Concepts

Oracle University and (Oracle Corporation) use only.


6: Storage Configuration
2 AM 6: Storage Configuration continued
7: Network Configuration
PM 8: Creating FMW Domains
9: Optimizing FMW Domains for Exalogic

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Course Schedule
The course schedule might vary depending on the pace of the class. The schedule shown is
an estimate. The instructor may provide updates.

Oracle Exalogic Elastic Cloud Administration 1 - 5


Course Schedule
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Day Lesson

3 AM 10: Starting and Migrating Servers

PM 11: Exalogic Database Connectivity


12: Additional FMW Infrastructure
13: Managing Hardware with ILOM

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Course Schedule (continued)


The course schedule might vary depending on the pace of the class. The schedule shown is
an estimate. The instructor may provide updates.

Oracle Exalogic Elastic Cloud Administration 1 - 6


Course Practices
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Each topic is reinforced with a hands-on practice.


Some practices include a scripted solution to aid any
students who fall behind.
Students:
Share a single Exalogic machine and file system
Are each assigned a pair of compute nodes

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 1 - 7


Classroom Guidelines
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The instructor starts each session at the scheduled time.


Do ask questions, but be respectful of the current topic and
the interests of other students.
Ensure that cell phones and pagers are silent.

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Classroom Guidelines
These guidelines enable you to get maximum benefit from the course.

Oracle Exalogic Elastic Cloud Administration 1 - 8


For More Information
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Topic Website
Education and training http://education.oracle.com
Product documentation http://www.oracle.com/technology/documentation
Product downloads http://www.oracle.com/technology/software
Product articles http://www.oracle.com/technology/pub/articles

Oracle University and (Oracle Corporation) use only.


Product support http://www.oracle.com/support
Product forums http://forums.oracle.com
Product tutorials and demos Go to otn.oracle.com and click Learning Library.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

For More Information


These materials are not intended to be a complete reference for all hardware and software
products associated with Oracle Exalogic. After you complete the course, Oracle provides a
variety of resources that you can use to obtain additional information.

Oracle Exalogic Elastic Cloud Administration 1 - 9


Oracle by Example (OBE)
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

OBEs are free online step-by-step tutorials that cover specific


product features.

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle by Example (OBE)


The Oracle by Example (OBE) series provides hands-on, step-by-step instructions on how to
implement various technology solutions to business problems. OBE solutions are built for
practical real-world situations, enabling you to gain valuable hands-on experience as well as
use the presented solutions as the foundation for production implementation, dramatically
reducing time to deployment.

Oracle Exalogic Elastic Cloud Administration 1 - 10


Related Training
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Course Title
Sun Fire Server Installation and Administration
Sun Storage 7000 Administration
Oracle WebLogic Server: Administration Essentials
Oracle WebLogic Server: Advanced Administration

Oracle University and (Oracle Corporation) use only.


Oracle WebLogic Server: Monitor and Tune Performance
Oracle WebLogic Server: Diagnostics and Troubleshooting
Oracle Coherence: Administer and Troubleshoot Clusters
Exadata and Database Machine Administration Workshop
Oracle Enterprise Manager Grid Control: Essentials
Sun Ops Center Administration

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Related Training
Note that some of the courses listed in the slide are available as traditional or virtual
classroom training, while others are self-paced online offerings (eStudies).

Oracle Exalogic Elastic Cloud Administration 1 - 11


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle University and (Oracle Corporation) use only.


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The Exalogic Solution

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle University and (Oracle Corporation) use only.


Objectives
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

After completing this lesson, you should be able to:


Describe the challenges faced by data center
administrators
List the major features of Exalogic
Discuss the software supported by the Exalogic hardware

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 2 - 2


Why Data Centers?
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Traditional distributed applications suffer from:


Heterogeneous and unreliable hardware
Uneven performance and resource utilization
High-latency networks
Complex integration solutions
High labor costs for deployment and maintenance

Oracle University and (Oracle Corporation) use only.


The data center model strives to solve these problems by
using a centralized infrastructure.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Why Data Centers?


For most enterprise IT organizations, years of innovation, expansion, and acquisition have
resulted in sprawling infrastructure that stretches the limits of manageability. Though the
individual IT systems and applications in service are often well considered and expertly
implemented, the sheer scale of the ongoing IT investment itself has emerged as the
dominant concern. Even when the latest technologies, open standards, vendors, and
architectural practices, such as service-oriented architecture (SOA), have been used
pervasively, most enterprises now find themselves with too many platforms, too many
technologies, too many domains of expertise, and too many vendors to coordinate and
manage.
In response, a number of technologies and practices have become staples for large
enterprises, ranging from virtualization and centralized storage to enterprise-wide
standardization of software and hardware. This transformation typically involves adoption of a
more centralized, automated, and elastic infrastructure and regime. Perhaps, most
importantly, these engineered systems must not be monolithic. It must be possible for
enterprises to implement them over time, at a pace determined by real business needs and
investment timelines.

Oracle Exalogic Elastic Cloud Administration 2 - 3


Data Center Challenges
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Common concerns include:


Meeting performance and availability requirements
The consolidation of resources to minimize floor space and
power costs
Keeping applications isolated from one another, to address
maintenance and security concerns

Oracle University and (Oracle Corporation) use only.


Keeping the system manageable
Scaling the infrastructure as the business grows

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Data Center Challenges


The need for enterprise IT organizations to provide next-generation cloud features such as
elastic capacity while meeting ever more demanding performance and reliability requirements
is driving demand for a new approach to infrastructure. Whether data-intensive or processing-
intensive, whether homogeneous or highly heterogeneous, the key to success is hardware
and software engineered together for performance, reliability, and scale. Building or using
custom, special-purpose systems for different applications is wasteful and expensive.
The most common model for application isolation involves multiple IP subnetting, in which the
most mission-critical applications are assigned their own IP subnets. In this model, some
subnets may also contain applications that have less stringent or otherwise different resource
requirements. Other subnets may contain multiple applications, such as those dedicated to a
given department or line of business, or even used for application testing and development.

Oracle Exalogic Elastic Cloud Administration 2 - 4


What Is Exalogic?
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle University and (Oracle Corporation) use only.


CPUs, storage, Optimized Oracle
network software

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

What Is Exalogic?
Exalogic is an integrated hardware and software system designed to provide a complete
platform for a wide range of application types and widely varied workloads. Exalogic is
intended for large-scale, performance-sensitive, mission-critical application deployments. It
combines Oracle Fusion Middleware and Sun hardware to enable a high degree of isolation
between concurrently deployed applications, which have varied security, reliability, and
performance requirements. Exalogic enables customers to develop a single environment that
can support end-to-end consolidation of their entire applications portfolio.
Exalogic hardware is preassembled and delivered in standard 19 42U rack configurations.
The main hardware components of a single Exalogic machine Full Rack are the following:
30 Sun Fire X4170 M2 compute nodes
One dual controller Sun ZFS Storage 7320 appliance with 20 disks
Four Sun Network QDR InfiniBand Gateway Switches
One Sun Datacenter InfiniBand Switch 36
One 48-port Cisco Catalyst 4948 Ethernet management switch
Two redundant 24 kVA power distribution units

Oracle Exalogic Elastic Cloud Administration 2 - 5


What Is Exalogic?
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle Exalogic hardware:


Combines Sun storage and servers by using a high-speed
InfiniBand network
Supports up to 360 cores and 2.8 TB of RAM per rack
installation
Is specifically engineered to host Oracle Fusion

Oracle University and (Oracle Corporation) use only.


Middleware software
Results in huge performance gains for WebLogic
applications
Supports Enterprise Manager for centralized configuration,
monitoring, and diagnostics

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

What Is Exalogic? (continued)


You can connect up to eight Exalogic machines, or a combination of Exalogic machines and
Oracle Exadata database machines, together without the need for any external switches. If
more than eight racks are required to be connected on the same InfiniBand fabric, Oracle
offers a choice of several high-capacity data center switches, which enable the creation of
Exalogic clouds comprising hundreds of racks and tens of thousands of processors.
Exalogic is designed to fully leverage an internal InfiniBand fabric that connects all of the
processing, storage, memory, and external network interfaces within an Exalogic machine to
form a single, large computing device. Each Exalogic machine is connected to the customer's
data center networks via 10 GB Ethernet (external traffic) and 1 GB Ethernet (management
traffic) interfaces.
Oracle Enterprise Manager Grid Control with Oracle WebLogic Server Management Pack
Enterprise Edition's capabilities include Exalogic specific management tools to monitor the
Oracle software deployed in the Exalogic environment. If using Solaris as your Exalogic
operating system, you can also utilize Oracle Enterprise Manager Ops Center to provide
configuration and management capabilities for the Exalogic hardware components.

Oracle Exalogic Elastic Cloud Administration 2 - 6


Why Exalogic?
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The Exalogic solution:


Is rigorously tested and certified for Fusion Middleware
Provides the best Java performance for the cost
Is factory assembled and then configured to your
specifications
Supports horizontal scalability by linking racks together

Oracle University and (Oracle Corporation) use only.


through InfiniBand
Facilitates multiple levels of application isolation
Proactively works with the Oracle support network
Can be directly linked to your existing Oracle Exadata
infrastructure

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Why Exalogic?
The combination of Oracle Exalogic software and hardware results in substantial performance
gains for Java based applications running on WebLogic Server and other Oracle Fusion
Middleware technologies. In addition to performance, the fact that Exalogic hardware and
software have been engineered together also means that customers are required to do far
less in setting up and running Exalogic than for alternative environments.
InfiniBand is fundamental to the Exalogic Elastic Cloud system. In addition to providing an
extremely fast, high-throughput interconnect between all of the hardware units within a
deployment, it also provides extreme scale, application isolation, and elasticity. Traditional
approaches to growing a data centers compute capacity involve either vertically scaling
individual computers or networking together many individual computers by using common
technologies such as Ethernet. By contrast, the lossless switched InfiniBand I/O fabric (on
which the Exalogic system is based) connects all configurations together, essentially forming
a single large computer.
An Exalogic cloud is a resource pool that can be dynamically subdivided into secure units of
capacity. InfiniBand supports partitions, in which communication between endpoints on the
I/O fabric is strictly controlled. Individual compute nodes, or even specific I/O devices, may be
grouped into logical partitions within which communication is allowed.

Oracle Exalogic Elastic Cloud Administration 2 - 7


What Is InfiniBand?
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Distributed systems communicate with one another to:


Separate application tiers (Web, business logic, caching)
Provide high availability (clusters)
InfiniBand:
Is a standard network architecture designed for very high
speeds between nodes in close proximity to each other

Oracle University and (Oracle Corporation) use only.


Acts as a fabric to connect CPUs and other devices
Is ideal for supercomputers and data centers
Delivers 40 GB (4x faster than high speed Ethernet)
Supports direct application connections that bypass the OS
Is interoperable with Ethernet for external communication

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

What Is InfiniBand?
Moving data between applications over a traditional network can consume a lot of time and
drain precious server resources. With traditional network technologies, data exchanges
traverse the operating systems on both the source and destination servers, resulting in
excessive application latency due to operating system calls, buffer copies, and interrupts.
InfiniBand, which today delivers 40 GB per second connectivity with application-to-application
latency as low as 1 microsecond, has become a dominant fabric for high-performance
enterprise clusters. Its ultra-low latency and near-zero CPU utilization for remote data
transfers make InfiniBand ideal for high-performance clustered applications.
InfiniBand also provides a direct channel from the source application to the destination
application, bypassing the operating systems on both servers. InfiniBands channel
architecture eliminates the need for OS intervention in network and storage communication.
This frees server memory bandwidth and CPU cycles for application processing.
In addition to carrying all InfiniBand traffic, the Sun Network QDR InfiniBand Gateway Switch
enables all InfiniBand attached servers to connect to an Ethernet LAN by using standard
Ethernet semantics. No application modifications are required for applications written to use
standard Ethernet.

Oracle Exalogic Elastic Cloud Administration 2 - 8


An Engineered Solution
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Exalogic hardware and software are specifically


engineered and tested to work together out of the box:
Servers, storage, network, OS, drivers, middleware
Easy to replicate and scale
Custom data center solutions:
Require significant time to plan, implement, and test

Oracle University and (Oracle Corporation) use only.


Must be built one component at a time
Result in compatibility issues
Require complex ordering and patching procedures

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Engineered Solution
An engineered system combines network, storage, processing, and software into a pre-
optimized package. An engineered approach enables IT to cut deployment time from months
to hours, reduces errors, and enables IT personnel and resources to be allocated to higher-
value work. It also reduces the overall time and costs associated with planning, procurement,
installation, and testing. Exalogic provides a repeatable model for building out data center
systems.

Oracle Exalogic Elastic Cloud Administration 2 - 9


Java Performance
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Java is one of the most pervasive technologies in todays


enterprise applications.
Oracles Java EE platform is now optimized for Exalogic
specific hardware capabilities, including:
JRockit and Hotspot Java Virtual Machines (JVMs)
WebLogic Server

Oracle University and (Oracle Corporation) use only.


Oracle benchmarks show significant performance gains
against comparable hardware:
5 to 10 times faster response times
50% more throughput

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Java Performance
Exalogic has been designed from the ground up to provide the ideal environment for
enterprise Java applications and Java based infrastructure. Oracles entire Fusion Middleware
portfolio is optimized for deployment on Exalogic. Oracle Exalogic Elastic Cloud software
includes a number of optimizations and enhancements made to the core products within
Oracle WebLogic Suite, which includes Oracle WebLogic Server, Oracle Coherence, Oracle
JRockit, and Oracle HotSpot. The combination of Oracle Exalogic software and hardware
results in substantial performance gains for Java based applications running on WebLogic
Server and other Oracle Fusion Middleware technologies.

Oracle Exalogic Elastic Cloud Administration 2 - 10


High Availability
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Hardware redundancy at all levels, including:


Processors
Disks
Power and cooling
Network switches and cabling
Software features include:

Oracle University and (Oracle Corporation) use only.


Bonded network interfaces
WebLogic clustering and automatic server migration
Active GridLink database connections

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

High Availability
Hardware components within an Exalogic machine use multiple physical InfiniBand (IB) ports
and switches for high availability. Each InfiniBand adapter has two ports, is assigned a single
IP address, and supports active/passive bonding. The active port of the adapter connects to
an IB switch and the passive port connects to another switch. If one fails, traffic is
transparently failed over to the other.
Clustering is configuring a group of WebLogic Servers to work together to provide client
access to the services offered by the servers in the cluster. The cluster appears to a client as
one instance, whether the client is a Web client or a Java application. By replicating the
services provided by one instance, an enterprise system achieves a fail-safe and scalable
environment. Scalability is achieved by balancing the load of incoming requests across the
servers in the cluster.
When a migratable WebLogic Server becomes unavailable for any reason (for example, if it
hangs or loses network connectivity, or if its host machine fails), migration is automatic. Upon
failure, the server is automatically restarted on the same machine if possible. If the server
cannot be restarted on the machine where it failed, it is migrated to another machine. In
addition, an administrator can manually initiate migration of a server instance.

Oracle Exalogic Elastic Cloud Administration 2 - 11


Multi-Tenancy
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Data centers must be able to:


Host applications from multiple lines of business or even
multiple businesses
Balance density with security
Exalogic supports isolation at multiple levels:
Physical network layer (InfiniBand partitions, VLANs)

Oracle University and (Oracle Corporation) use only.


Transport network layer (IP subnets)
Storage access control
Solaris zones
Future Exalogic versions will also support
additional hardware virtualization solutions.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Multi-Tenancy
By default, the Exalogic machine includes a single InfiniBand partition at the rack level. All
Exalogic compute nodes and the storage appliance are full members of this default partition.
The Exalogic network hardware allows you to configure multiple partitions to restrict traffic
between specific IB connections. However, Oracle currently only supports the use of the
default partition to help ensure compatibility between multiple Exalogic and/or Exadata racks.
The Exalogic network hardware does support virtual LANs (VLANs) to achieve a similar result
to partitions. Finally, you can limit access to the network by using multiple subnet masks. The
subnet masks will be enforced at the OS level, not by the network hardware.
If you are using the Solaris OS on Exalogic, you can also utilize the zones feature. Zones act
as completely isolated, virtual operating systems within a single OS instance. Each zone has
its own node name, virtual network interfaces, and storage assigned to it.
You can limit access to an Exalogic shared file system by using both standard UNIX file
permissions and IP- or DNS-based client exceptions. When a client attempts access, its
access will be granted according to the first exception in the list that matches the client (or, if
no such exception exists, according to the global share mode).

Oracle Exalogic Elastic Cloud Administration 2 - 12


Software Compatibility
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

On Exalogic hardware, Oracle currently supports:


Oracle Linux 5.5 and Solaris 11 operating systems
Any applications already certified on these platforms
Fusion Middleware 11.1.1.4 (WebLogic Server 10.3.4) is
optimized to take advantage of Exalogic hardware,
including the InfiniBand network.

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Software Compatibility
In addition to unique support for Java applications and Oracle Fusion Middleware, Exalogic
also provides users with a choice of Oracle Linux or Oracle Solaris operating systems.
Exalogic is 100% compatible with all standard Oracle Linux 5 and Solaris 11 applications, and
no special certification for Exalogic is required. In other words, all Oracle applications that are
certified for the appropriate releases of Oracle Linux and Solaris are supported on Exalogic.
The Exalogic specific optimizations for Fusion Middleware are certified only against Exalogic.
They are not supported on other non-Exalogic hardware, and regardless of whether the
hardware utilizes InfiniBand. Currently, Exalogic is not certified to run virtualization platforms
such as Oracle VM or VMware, although this is expected to change in the future.

Oracle Exalogic Elastic Cloud Administration 2 - 13


Selecting an Operating System
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

At the factory:
Both Linux and Solaris base images are pre-installed on the
storage appliance
The compute nodes are configured to run Linux by default
The latest base images are also available on
edelivery.oracle.com.

Oracle University and (Oracle Corporation) use only.


Refer to the Machine Guide for detailed instructions on re-
imaging the compute nodes with Solaris.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Selecting an Operating System


During manufacturing, Exalogic compute nodes are pre-installed with the Oracle Linux
operating system. The Exalogic Machine Guide documentation describes how to re-image
Exalogic compute nodes from Oracle Linux to Oracle Solaris. The imaging process is
performed using a laptop that is connected to your Exalogic machine rack. It utilizes a Preboot
Execution Environment (PXE) server architecture. The overall tasks are:
1. Install Oracle VirtualBox on the laptop.
2. Connect the laptop to the Exalogic management network.
3. Run the Solaris Automated Installer (AI) VM image on the laptop.
4. Start the AI and DHCP servers within the AI VM.
5. Configure a list of compute node MAC addresses that the AI server will accept.
6. Access each compute nodes ILOM interface, configure it to boot using PXE, and reboot
it. During the first boot, the Oracle Solaris operating system is installed on the compute
node. During the second boot, the first run service executes and sets up the Exalogic
machine environment, including network, tuning, and so on. After the third boot, you can
start using the compute node.

Oracle Exalogic Elastic Cloud Administration 2 - 14


Licensing and Support Overview
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Hardware:
Exalogic racks and supporting hardware are sold in one of
three specific configurations, depending on initial capacity
needs.
To provide quality support, racks can be expanded and
combined only in specific ways.

Oracle University and (Oracle Corporation) use only.


Low-level hardware customization is not supported.
Software:
A license for Exalogic Elastic Cloud Software permits the
use of Exalogic specific-features in WebLogic Suite.
WebLogic Suite is a separate license and a prerequisite.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Licensing and Support Overview


Exalogic Elastic Cloud Software is not a binary distribution like WebLogic Server. It is simply
the name given to the use of WebLogic Server's features that optimize its performance on
Exalogic. These features are disabled by default. Like most Oracle software products, both
WebLogic Suite and Exalogic Elastic Cloud Software are licensed per processor.

Oracle Exalogic Elastic Cloud Administration 2 - 15


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

What type of network fabric is used to connect Exalogic


hardware components?
a. Ethernet
b. iWARP
c. VIA
d. InfiniBand

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: d

Oracle Exalogic Elastic Cloud Administration 2 - 16


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Which two licenses are required to use Fusion Middleware's


optimizations for Exalogic?
a. WebLogic Suite
b. Enterprise Manager Grid Control
c. Exalogic Elastic Cloud Software
d. E-Business Suite

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: a, c

Oracle Exalogic Elastic Cloud Administration 2 - 17


Summary
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

In this lesson, you should have learned how to:


Describe the challenges faced by data center
administrators
List the major features of Exalogic
Discuss the software supported by the Exalogic hardware

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 2 - 18


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Hardware Components

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle University and (Oracle Corporation) use only.


Objectives
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

After completing this lesson, you should be able to:


Describe the components of an Exalogic machine and their
relationships
Discuss the hardware specifications for an Exalogic
compute node
Discuss the hardware specifications for the Exalogic

Oracle University and (Oracle Corporation) use only.


storage appliance
Compare the available hardware packages
Discuss opportunities for scaling your Exalogic data center

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 3 - 2


Exalogic Machine Topology
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Clients

Rack
Compute Node

Client Compute Node Mgmt


Network Network

Oracle University and (Oracle Corporation) use only.


10 Gb 40 Gb

InfiniBand Mgmt 1 Gb
Storage Appliance
Switches Switch
40 Gb

IB
Network
Power Distribution Units (PDUs)

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Exalogic Machine Topology


Exalogic hardware is preassembled and delivered in standard 19 42U rack configurations.
Each Exalogic configuration is a unit of elastic cloud capacity balanced for compute-intensive
workloads. Each Exalogic configuration contains a number of hot-swappable compute nodes
along with a clustered, high-performance disk storage subsystem. The hardware also
includes a high-bandwidth InfiniBand fabric to connect every individual component within the
configuration as well as to externally connect additional Exalogic or Exadata Database
Machine racks. In addition, each configuration includes multiple 10 Gb Ethernet ports for
integration with the data center's service network, along with 1 Gb Ethernet ports used for
integration with the data centers management network. All Exalogic configurations are fully
redundant at every level and are designed with no single point of failure.
All device management ports are connected to your local data center management network
by using a Cisco Catalyst 4948 switch, which is a built-in component of an Exalogic rack. This
switch offers 48 ports of wire-speed 10/100/1000BASE-T with 4 alternative wired ports that
can accommodate optional 1000BASE-X Small Form-Factor Pluggable (SFP) optics.
Reliability and serviceability are delivered with optional internal AC or DC 1 + 1 hot-
swappable power supplies and a hot-swappable fan tray with redundant fans.

Oracle Exalogic Elastic Cloud Administration 3 - 3


Compute Nodes
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Dual processor
Redundant power and cooling
Redundant solid state boot disks
InfiniBand host channel adapter

Oracle University and (Oracle Corporation) use only.


Sun Fire X4170 M2
2.93 GHz Intel Xeon Cores 12 (6 per CPU)
1333 MHz RAM 96 Gb
FlashFire Solid State Disk 32 GB (64 Gb raw)
InfiniBand Ports 2
Gigabit (GB) Ethernet Ports 4

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Compute Nodes
Data centers often standardize on a server that can run a wide range of business applications
and can be managed easily. The versatility of the Sun Fire X4170 M2 server makes it a
perfect fit for this, because this server balances compute power, memory capacity, and I/O
capability in a compact and energy-efficient, 1-rack unit (RU) enclosure. This is a server that
is ideal for middleware workloads, system administration, and application development.
The Sun Fire X4170 M2 server includes two Intel Xeon Processor 5600 series, each with six
processing cores. This processor includes an energy management feature that reduces
power consumption when workload decreases. In addition, the server has built-in hardware
and software power management capabilities to further enhance power efficiency.
Each compute node contains two solid-state disks (SSDs), which host the operating system
images used to boot the node, act as high-performance local swap space, and also provide
storage for any diagnostic data generated by the system during a failure. Mirroring for the two
solid-state disk drives is accomplished via a RAID controller.
The Sun Fire X4170 M2 server supports up to 18 double data rate (DDR3) memory dual inline
memory module (DIMMs) slots, which can be populated with 4 GB and 8 GB DIMMs.
However, within an Exalogic configuration, Oracle supports only the twelve 8 GB DIMMs that
are installed at the factory.

Oracle Exalogic Elastic Cloud Administration 3 - 4


Compute Node: Front View
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

32 GB SATA Solid
State Disk (2) USB (2)

Oracle University and (Oracle Corporation) use only.


Power LED Empty (6)
Fault LEDs

Power
Button

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Compute Node: Front View


Although an Exalogic compute node provides bays to support up to 8 SSD or PCI disks, only
two of these are used by default. Each contains a 32 GB SSD disk, which is kept
synchronized by using a RAID controller.
The default contents of the compute node disk include an Oracle Linux image. No separate
OS installation is required. If you prefer to use Solaris instead, a Solaris image is available on
the storage appliance.
Although the compute node's local disk has available space, its sole intended purpose is to be
a high-speed boot disk. Oracle recommends that customers do not use it to store applications
and other data. Instead, they can use the shared storage appliance for all non-OS artifacts.

Oracle Exalogic Elastic Cloud Administration 3 - 5


Compute Node: Rear View
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Redundant Power Serial PCI Express InfiniBand


Supply (2) Management Port 2.0 Slot (3) Port (2)
Video

Oracle University and (Oracle Corporation) use only.


USB (2)
Status LEDs 1 Gb Ethernet 1 Gb Network Interface
Management NIC Cards (NICs) (4)

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Compute Node: Rear View


The Sun Fire X4170 M2 server offers hot-swappable and redundant RAID-enabled disks,
cooling fans, and power supply units.
InfiniBand connectivity is accomplished via the dual-port host controller adapter (HCA). This is
a quad data rate (QDR) PCI-E card, which takes advantage of the latest InfiniBand rates.
System management is simplified with Oracle Integrated Lights Out Manager (ILOM), which
comes standard in every Sun Fire X4170 M2 server. ILOM centralizes system management
locally or remotely to ease system configuration, software provisioning, and updates. It
provides a consistent interface across the entire x86 product line.

Oracle Exalogic Elastic Cloud Administration 3 - 6


Compute Node Memory
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Each compute node has 18 DIMM slots and supports up to


144 GB RAM (8 GB per slot).
However, the memory is fixed at 96 GB to optimize the
clock speed at 1333 MHz.
Oracle does not support modifying the default hardware
configuration.

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Compute Node Memory


DIMM = dual inline memory module
Each CPU supports three channels, each of which supports three DIMMs, making a total of
18 (2 x 3 x 3).

Oracle Exalogic Elastic Cloud Administration 3 - 7


Storage Appliance
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Two redundant nodes (heads)


SAS array of twenty hard disks
(mirrored, two hot spares)
Solid state cache disks
InfiniBand connectivity
Web administration console

Oracle University and (Oracle Corporation) use only.


Sun ZFS Storage Appliance 7320
2.40 GHz Intel Xeon Cores 4 per head
1333 MHz RAM 12 per head
Serial Attached SCSI (SAS) Disk 18 TB (40 TB raw)
Solid State Read Cache 2 TB per head
Solid State Write Cache 72 GB

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Exalogic Storage
The Exalogic ZFS Storage 7320 system consists of two storage controllers or heads in a
high-availability cluster configuration, along with one disk shelf. The 7320 controller base
configuration includes two CPUs, 4 Gb Ethernet ports, redundant power supplies, an
InfiniBand network controller, and dual-port SAS HBA for storage expansion. The CPUs are
Intel Xeon 5600 series, 2.40 GHz, and contain four core processors each (a total of eight
cores). The clustered configuration simply uses two servers and a cluster card in each server
for a heartbeat connection between them.
All user-accessible storage is provided by one Sun disk shelf that is external to the two
servers. Solid-state drives are used as a high-performance write cache known as LogZilla or
ZFS intent log (ZIL) devices, and are in place of 4 of the 24 drives in the disk shelf. The
remaining 20 drives are available for storage.
The RAID function is performed by the software. RAID-1 is recommended, but other options
are available. In the default configuration, there are twenty 2 TB disks (40 TB in total space)
but two of them are used as spares. Therefore, there is 36 TB total disk space or 18 TB of
replicated disk space.

Oracle Exalogic Elastic Cloud Administration 3 - 8


Disk Caching
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The storage appliance utilizes high-speed solid state disks to:


Cache frequently read files
Write files as a background task

Storage Node
Compute File write

Oracle University and (Oracle Corporation) use only.


Node File read Compute
Read Cache
Node

Write Cache/Buffer

Disk Array

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Disk Caching
The Exalogic storage appliance transparently executes writes to low-latency SSD media so
that writes can be quickly acknowledged, allowing the application to continue processing.
Then, it automatically flushes the data to high-capacity drives as a background task.
Another type of SSD media acts as a cache to reduce read latency, and the appliance also
transparently manages the process of copying frequently accessed data into this cache to
seamlessly satisfy read requests from clients.

Oracle Exalogic Elastic Cloud Administration 3 - 9


Storage Appliance: Front View
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Nodes (2) 512 GB Solid State 512 GB Boot


(Active and Standby) Read Cache (4 each) Disks (2 each)

Oracle University and (Oracle Corporation) use only.


18 GB Solid State
Write Cache (4)

2 TB Hard Disk
Drive (20)

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Storage Appliance: Front View


Within each of the two Exalogic storage controllers, there are four 512 GB solid state disks,
which are used as a read cache by ZFS. There are also two redundant hard disks that are
used to boot the appliances operating system.
On the disk shelf, four of the drive bays are occupied by 18 GB solid state disks, which serve
as a write cache for the storage appliance. The remaining 20 drives are conventional 2 TB
SAS hard disk drives.

Oracle Exalogic Elastic Cloud Administration 3 - 10


Storage Appliance: Rear View
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Cluster interconnect SAS Host Bus


InfiniBand ports
for failover Adapter (HBA)

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Storage Appliance: Rear View


All inter-head communication consists of one or more messages transmitted over one of the
three cluster I/O links provided by the Clustron hardware. This device offers two low-speed
serial links and one Ethernet link. The use of serial links allows for greater reliability. Ethernet
links may not be serviced quickly enough by a system under extremely heavy load. All three
links are formed by using ordinary straight-through gigabit Ethernet cables. The cables
connecting the serial connections must be used to connect opposing sockets on the two
connectors, effectively providing a crossover.
Clustered heads never communicate using external service or administration network
interfaces, and the interconnects form a secure private network. Messages fall into two
general categories: regular heartbeats used to detect the failure of a remote head, and higher-
level traffic associated with the resource manager and the cluster management subsystem.
The SAS HBA and the JBOD designs both include two four-channel mini-SAS connectors for
the A path and B path. Both the A and B paths can access all of the disk drives which
provides connectivity to all of the disks from either path.

Oracle Exalogic Elastic Cloud Administration 3 - 11


InfiniBand (IB) Switches
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

InfiniBand Quad Data Rate (QDR) ports


10 Gb Ethernet gateway ports
Redundant power and cooling
Embedded management module
LEDs for box status and for each
connection status

Oracle University and (Oracle Corporation) use only.


NM2-GW NM2-36P
Spine (used only to
Type Gateway (leaf)
connect multiple racks)
40 Gb IB Ports 32 36
10 Gb Ethernet
8 0
Ports

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

InfiniBand Switches
Within Exalogic, the InfiniBand gateway switches provide a shared, high-speed network for
application clusters that comprise compute nodes and the shared storage appliance. The
gateway switch features a standard 1U data center form factor, thirty-two 40 Gb InfiniBand
ports, eight 10 Gb Ethernet ports, and the Sun ILOM management interface.
Oracle's converged fabric leverages the properties of the InfiniBand architecture to enable
high performance in demanding, clustered data center environments. The fabric supports the
creation of logically isolated network partitions as well as advanced features for traffic
isolation and quality of service (QoS) management. QDR InfiniBand delivers 40 Gb of raw
bandwidth. The InfiniBand base rate is 10 Gb per second. QDR-capable products deliver four
times the base rate.
Deploying a converged fabric minimizes the cost and complexity of building and operating
scalable systems by reducing the number of adapters, cables, and switches to install,
manage, and maintain. LAN traffic carried over Oracle's converged data center fabric uses
familiar Ethernet semantics. Furthermore, the LAN ports on the Sun Network QDR InfiniBand
Gateway Switch appear as endpoints to the LAN, thus avoiding potential interoperability
issues with network management tools and methods.

Oracle Exalogic Elastic Cloud Administration 3 - 12


Gateway Switch Hardware Views
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Ethernet Port (8)


Ethernet Four over each IB port
Management Port (2)
REAR

InfiniBand Port (32) Unused Port (2)

Oracle University and (Oracle Corporation) use only.


Redundant Power
Supply (2)
FRONT

Redundant Fans
(3; 5 max.)

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Switch Hardware Views


Within the Sun Network QDR InfiniBand Gateway Switch, the embedded fabric manager is
enabled to support active/standby dual manager configurations, ensuring a seamless
migration of the fabric management services in the event of a module failure. The switch is
also provisioned with redundant power and cooling for high availability.
The front panel of the Sun Network QDR InfiniBand Gateway Switch has 36 QSFP
connectors. The connectors are each dedicated to InfiniBand or Gateway functions as follows:
The leftmost 32 ports support QDR InfiniBand connectivity.
The two upper rightmost ports support Ethernet connectivity.
The two lower rightmost ports are covered with a plug marked Do Not Remove. No
connections should be made to these ports.
The switch has two physical ports dedicated to Ethernet functionality. These ports can support
up to four 10 GB Ethernet connections each. Oracle provides splitter cables, which fan out a
single Quad Small Form Pluggable (QSFP) switch port to four optical fiber LC connectors. A
QSFP transceiver is available to connect the splitter cable to the switch.

Oracle Exalogic Elastic Cloud Administration 3 - 13


IB Host Channel Adapter (HCA)
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Compute and storage nodes use two physical IB ports and


switches for redundancy (active/passive).
A special bond network adapter on the OS makes these
two interfaces appear as a single interface to applications.

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

IB Host Channel Adapter (HCA)


Oracle's family of Sun PCIe Quad Data Rate (QDR) InfiniBand Host Channel Adapters
(HCAs) gives you better price/performance for both throughput and low latency. They are
designed for servers supporting the PCI Express 2.0 specification with either the low-profile or
modular hot-pluggable PCI Express (PCIe) ExpressModule form factors. The adapters
support two IB ports for connecting IB traffic (QSFP connectors); each port supports 10
Gb/sec traffic on SDR, 20 Gb/sec traffic on DDR, and 40 Gb/sec on QDR.

Oracle Exalogic Elastic Cloud Administration 3 - 14


Management Switch
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

All device management ports are


connected to a dedicated 1 Gb switch.
NET0 on compute nodes and storage
appliance
Isolate management and monitoring
tasks from client traffic.

Oracle University and (Oracle Corporation) use only.


The switch includes redundant power
and cooling.

Cisco Catalyst 4948


1 Gb Ethernet Ports 48

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Management Switch
The Cisco Catalyst 4948 offers 48 ports of wire-speed 10/100/1000BASE-T with four
alternative wired ports that can accommodate optional 1000BASE-X Small Form-Factor
Pluggable (SFP) optics. Exceptional reliability and serviceability are delivered with optional
internal AC or DC 1+1 hot-swappable power supplies and a hot-swappable fan tray with
redundant fans.
The Cisco Catalyst 4948 includes a single, dedicated 10/100 console port and a single,
dedicated 10/100 management port for offline disaster recovery. Remote in-band
management is available with the Simple Network Management Protocol (SNMP), telnet
client, Bootstrap Protocol (BOOTP), and Trivial File Transfer Protocol (TFTP). Support for
local or remote out-of-band management is delivered through a terminal or modem attached
to the console interface. The management port helps enable the Cisco Catalyst 4948 to
reload a new image from a TFTP server.

Oracle Exalogic Elastic Cloud Administration 3 - 15


Scaling Your Data Center
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Multi-Rack
Full Rack
Quarter Half Rack
Rack

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Scaling Your Data Center


InfiniBand is fundamental to the Exalogic Elastic Cloud system. In addition to providing an
extremely fast, high-throughput interconnect between all of the hardware units within a
deployment, it also provides extreme scale, application isolation, and elasticity. Traditional
approaches to growing a data centers compute capacity involve either vertically scaling
individual computers or networking many individual computers by using common technologies
such as Ethernet. By contrast, the lossless switched InfiniBand I/O fabric (on which the
Exalogic system is based) connects all configurations, essentially forming a single large
computer.
Exalogic systems scale horizontally, meaning that there is no degradation of system
performance as the size of the cloud increases. It is equally important that an Exalogic cloud
is a resource pool that can be dynamically subdivided into secure units of capacity. The
underpinnings of this capability are in the design of InfiniBand itself, which was designed for
precisely this use. InfiniBand supports partitions, in which communication between endpoints
on the I/O fabric is strictly controlled within the fabric switches. Individual compute nodes, or
even specific I/O devices, may be grouped into logical partitions, within which communication
is allowed.

Oracle Exalogic Elastic Cloud Administration 3 - 16


Scaling Your Data Center
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Grow your data center based on these three supported


configurations (quarter rack, half rack, and full rack).
Half and full racks include unused spine InfiniBand
switches to link racks together.
Connect from two to eight full racks without any external
InfiniBand switches.

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Scaling Your Data Center (continued)


It is possible to connect as many as eight full racks of Exalogic machines (or any combination
of Exalogic and Exadata configurations) without the need for any external switches. In cases
where more than eight racks of Exalogic or Exadata hardware are required, Oracle offers a
choice of several high-capacity data center switches. This approach supports the creation of
Exalogic clouds that can comprise hundreds of racks and tens of thousands of processors.

Oracle Exalogic Elastic Cloud Administration 3 - 17


Hardware Package Details
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Metric Quarter Rack Half Rack Full Rack


Compute Nodes 8 16 30
Node Cores 96 192 360
Node RAM 768 GB 1.5 TB 2.8 TB

Oracle University and (Oracle Corporation) use only.


Storage Appliance 1 1 1
Disk Storage 40 TB 40 TB 40 TB
IB Gateway Switches 2 2 4
IB Spine Switches 0 1 1
Approx. Weight (lbs.) 1000 1500 2100

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Hardware Package Details


Quarter-rack configurations contain eight compute nodes, one storage appliance that contains
an active-passive pair of servers along with disks, two InfiniBand (IB) gateways, and a Cisco
management switch.
Half-rack configurations contain 16 compute nodes, one storage appliance, two IB gateways,
one IB spine switch, and a management switch.
Full-rack configurations contain 30 compute nodes, one storage appliance, four IB gateways,
one IB spine switch, and a management switch.
Note that the Exalogic shipping pallet, ramps, and corrugated packaging add approximately
150 pounds to the values mentioned here.

Oracle Exalogic Elastic Cloud Administration 3 - 18


Power Options
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

A full rack includes (and requires) two redundant 24-kVA


power distribution units (PDUs).
Three-phase high and low options
No single-phase option
Half racks and quarter racks are available with 15-kVA or
24-kVA PDUs.

Oracle University and (Oracle Corporation) use only.


PDU power consumption can be monitored from an LCD
display or from the racks management network.
40A
30A
20A
10A
0A
Phase 1 Phase 2 Phase 3
16.1 14.5 14.1

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Power Options
To protect your Exalogic machine from such disturbances, you should have a dedicated
power distribution system, power-conditioning equipment, as well as lightning arresters or
power cables to protect from electrical storms. The cabinets for the Exalogic machine are
shipped with grounding-type power cords (three-wire). Always connect the cords to grounded
power outlets. For additional grounding, attach a chassis earth ground cable to the Exalogic
machine. The additional ground point enables electrical current leakage to dissipate more
efficiently.
Each rack has two pre-installed power distribution units (PDUs). Different types of PDUs
accept different types of power sources and use different numbers of power outlets. You must
specify the type of PDU that is correct for your data center.
If you upgrade a quarter rack or half rack that was originally ordered with a 15-kVA PDU to a
full rack, you must also upgrade to a 24-kVA PDU.

Oracle Exalogic Elastic Cloud Administration 3 - 19


Hardware Maintenance
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The Exalogic Machine Guide includes a Replacement


Units section that lists part numbers for:
Compute node SSDs, fans, power supplies, memory, and
PCI-E cards
Storage appliance SSDs, HDDs, fans, power supplies, and
PCI-E cards

Oracle University and (Oracle Corporation) use only.


Switch fans and power supplies
IB and Ethernet cables of various lengths
Refer to the service manuals of individual products for
detailed component replacement steps.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 3 - 20


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Name three supported Exalogic rack configurations.


a. Admin
b. Half
c. Basic
d. Quarter

Oracle University and (Oracle Corporation) use only.


e. Full

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: b, d, e

Oracle Exalogic Elastic Cloud Administration 3 - 21


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Which two types of storage are available to an Exalogic


compute node?
a. Shared disks
b. Gateway disk
c. Solid State disk
d. QDR disk

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: a, c

Oracle Exalogic Elastic Cloud Administration 3 - 22


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

How many hard disks does the Exalogic storage appliance


contain?
a. 10
b. 15
c. 18
d. 20

Oracle University and (Oracle Corporation) use only.


e. 40

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: d

Oracle Exalogic Elastic Cloud Administration 3 - 23


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

What type of InfiniBand switch is used to connect multiple


racks?
a. Subnet
b. Gateway
c. Spine
d. Management

Oracle University and (Oracle Corporation) use only.


e. Quarter

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: c

Oracle Exalogic Elastic Cloud Administration 3 - 24


Summary
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

In this lesson, you should have learned how to:


Describe the components of an Exalogic machine and their
relationships
Discuss the hardware specifications for an Exalogic
compute node
Discuss the hardware specifications for the Exalogic

Oracle University and (Oracle Corporation) use only.


storage appliance
Compare the available hardware packages
Discuss opportunities for scaling your Exalogic data center

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 3 - 25


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle University and (Oracle Corporation) use only.


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.


Initial Machine Configuration

Oracle University and (Oracle Corporation) use only.


Objectives
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

After completing this lesson, you should be able to:


Locate and use checklists and worksheets to plan an
installation
Explain how Oracle initializes a rack based on your input
Describe the Exalogic default network setup
List some basic diagnostic utilities

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 4 - 2


Power-On Procedure
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1. Ensure that all power cables are connected:


Single phase: three cables from each PDU
Three phase: one cable from each PDU
2. Ensure that all six PDU circuit breakers are turned on.
3. Wait for all components to start in standby mode (power
LEDs blink).

Oracle University and (Oracle Corporation) use only.


4. Press the power button on both storage appliance heads
and verify that power LEDs no longer blink.
5. Press the power button on each compute node
and verify that power LEDs no longer blink.
6. Verify that power LEDs are lit on the switches.
7. Remotely reboot components by using ILOM.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Power-On Procedure
For a newly installed Exalogic rack, unfasten the power cord cable ties. The ties are for
shipping only and are no longer needed. Then route the power cords to the facility receptacles
either above the rack or below the flooring, and secure the power cords in bundles. Lastly,
plug the PDU power cord connectors into the facility receptacles. The 24 kVA PDU has four
plugs, and the 15 kVA PDU has two plugs
Switch on the six power distribution unit (PDU) circuit breakers located on the rear of the main
PDU (PDU A) inside the Exalogic machine. The circuit breakers are on the rear of the
Exalogic machine cabinet. Press the ON (|) side of the toggle switch. Wait three to five
minutes for all Oracle Integrated Lights Out Manager (ILOM) service processors to boot.
Next, open the front cabinet door. Verify that server standby power is on for compute nodes,
InfiniBand gateway switches, and storage appliance in the Exalogic machine. When standby
power is distributed to the chassis, the service processor (SP) powers on and starts up to
manage the system. The main power is supplied for the remainder of the chassis and fans
when the power button on the front of the server chassis is pressed. The power button is
disabled while the SP is starting. After the SP has started, the power button is enabled and
the system is placed in standby power mode.

Oracle Exalogic Elastic Cloud Administration 4 - 3


Hardware Configuration Process
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1. The customer completes checklists and worksheets.


2. Oracle delivers, installs, connects, and tests the rack(s).
3. Oracle runs the Exalogic Configuration Utility (ECU) to
initialize network and other settings.
4. The customer uses ILOM and OS tools to further
customize the default network configuration, if needed.

Oracle University and (Oracle Corporation) use only.


5. The customer configures projects and shares on the
storage appliance.

1 2 3

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Hardware Configuration Process


Initial installation and startup (step 2 in the slide) includes the following substeps:
1. Ensure that each of the three main power cords is connected.
2. Switch on the six power distribution unit (PDU) circuit breakers located on the rear of the
main PDU (PDU A) inside the Exalogic machine. The circuit breakers are on the rear of
the Exalogic machine cabinet.
3. Wait three to five minutes for all ILOM service processors to boot.
4. Open the front cabinet door. Verify that server standby power is on for compute nodes,
InfiniBand gateway switches, and the Sun ZFS Storage 7320 appliance in the Exalogic
machine. The Power/OK LED on the front panel blinks green in a pattern of 0.1 seconds
on, 2.9 seconds off.
5. Verify that the main power is on for each compute node. In main power mode, the
Power/OK LED on the front panel blinks in a pattern of one second on, one second off
while the system BIOS starts. After the BIOS initialization is completed and the
operating system begins to start, the Power/OK LED is illuminated and remains a steady
green.
6. Press the soft switches located on the front of the two storage heads in the Sun ZFS
Storage 7320 appliance. Wait three to five minutes for it to initiate NFS services.

Oracle Exalogic Elastic Cloud Administration 4 - 4


Exalogic Online Checklists
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The documentation includes a very thorough set of checklists


to ensure that the site is prepared for an Exalogic machine:
System Components Checklist
Data Center Room Checklist
Data Center Environmental Checklist
Access Route Checklist

Oracle University and (Oracle Corporation) use only.


Facility Power Checklist
Power Checklist
Safety Checklist
Logistics Checklist
Network Specification Checklist

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Exalogic Online Checklists


The System Components Checklist contains questions pertaining to the number of racks
being installed; if they are full, half, or quarter racks; and whether this system will be
connected to another Exalogic machine and, if so, whether the racks are adjacent to each
other.
The Data Center Room Checklist includes questions pertaining to the location of the Exalogic
machine, maintenance access requirements, position of rack in regard to air flow, raised floor
support, and floor-to-ceiling height.
Inside the Data Center Environmental Checklist expect to see questions pertaining to meeting
temperature and humidity requirements, floor layouts and ventilation requirements, and
general air conditioning and airflow requirements.
The Access Route Checklist contains questions that involve whether the doors and entryways
meet height and width requirements (including the unpacked unit), if the ramps or inclined
angles are in the permitted range, whether the pallet jack supports the device weight, and
whether the elevators width, height, and weight are supported.

Oracle Exalogic Elastic Cloud Administration 4 - 5


Example: Access Route Checklist
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Has the access route been checked for clearances of the


packaged equipment?
Do all the doors and entryways conform to the width and
height requirements for transportation, including the width
of the unpacked unit?
Do all the doors meet the height requirement of 87 inches

Oracle University and (Oracle Corporation) use only.


for packaged delivery?
Are there any stairs or ramps in the moving path for the
new hardware?
If a pallet jack is to be used, have you confirmed whether
the pallet jack supports the device weight?

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Example: Access Route Checklist


The Facility Power Checklist contains questions involving power configurations, single-phase
or three-phase power, the quantity of power outlets, confirming proper socket receptacles,
deciding on optional ground cables, confirming that suitable circuit breakers are installed, and
determining if a UPS is going to be used.
The Power Checklist contains questions pertaining to the minimum required power sources,
the location of power outlets, and the power frequency.
The Safety Checklist includes questions involving the location of emergency power shutoff,
the fire protection system, and antistatic flooring.
The Logistics Checklist contains questions involving data center personnel; their security or
access control; where the delivery, unpacking, and staging area is going to be; the moving
personnel; whether there are any restrictions on the delivery truck; and the delivery dock
access.
The Network Specification Checklist contains questions that pertain to reserving unused IP
addresses, registering the IP addresses with DNS, and running and labeling network cables.

Oracle Exalogic Elastic Cloud Administration 4 - 6


Site Requirements
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The documentation includes a comprehensive list of site


requirements.

Component Full Rack Half Rack Quarter Rack

17.575 kW 10.897 kW 7.206 kW


Power

Oracle University and (Oracle Corporation) use only.


(18.5 kVA) (11.47 kVA) (7.585 kVA)
63124 BTU/hour 39137 BTU/hr 25881 BTU/hr
Cooling
(66533 kJ/hr) (41250 kJ/hr) (27278 kJ/hr)
Physical 1998 mm (h) x 600 mm (w) x 1200 mm (d)
Dimensions 78.66 in (h) x 23.62 in (w) x 47.24 in (d)
966.6 kg 679.5 kg 491.2 kg
Weight
(2131 lbs.) (1498 lbs.) (1083 lbs.)
IP Addresses 98 54 30

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Site Requirements
The operating temperature range is 15 degrees Celsius to 32 degrees Celsius (59 degrees
Fahrenheit to 89.6 degrees Fahrenheit). The humidity range is 10 percent to 90 percent
relative humidity, noncondensing.
The minimum ceiling height for the cabinet is 2300 mm (90 inches), measured from the true
floor or raised floor, whichever is higher. An additional 914 mm (36 inches) is for top
clearance. The space above the cabinet and its surroundings must not restrict the movement
of cool air between the air conditioner and the cabinet, or the movement of hot air coming out
of the top of the cabinet. Oracle recommends that the Exalogic machine be installed on raised
flooring. The site floor and the raised flooring must be able to support the total weight of the
Exalogic machine.
The cabinets for the Exalogic machine are shipped with grounding-type power cords (three-
wire). Always connect the cords to grounded power outlets. Because different grounding
methods are used, depending on location, check the grounding type, and refer to
documentation such as IEC documents for the correct grounding method.
Always provide adequate space in front of and behind the rack to allow for proper ventilation.
Do not obstruct the front or rear of the rack with other equipment or objects.

Oracle Exalogic Elastic Cloud Administration 4 - 7


Exalogic Networks
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Network Speed Description


Provides access to compute nodes via
Client 10 Gb
intranet/Internet
Is used to monitor and administer components
Management 1 Gb Provides access to ILOM and other device

Oracle University and (Oracle Corporation) use only.


management interfaces
Is a private, nonroutable network
Connects racks, along with all rack components
Private
40 Gb Is used by compute nodes to access shared
InfiniBand
storage
Is used for internal cluster communication

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Exalogic Networks
Exalogic machine includes compute nodes and the Sun ZFS Storage 7320 appliance, as well
as equipment to connect the compute nodes to your network. The network connections allow
the servers to be administered remotely, enable clients to connect to the compute nodes, and
enable client access to the storage appliance. Additional configuration, such as defining
multiple virtual local area networks (VLANs) or enabling routing, may be required for the
switches to operate properly in your environment. This type of configuration is beyond the
scope of the installation service.
There are up to five networks for Exalogic machine. Each network must be on a distinct and
separate subnet from the others. The Exalogic management network connects to your
existing management network, and is used for administrative work for all components of
Exalogic machine. It connects ILOM, compute nodes, server heads in the storage appliance,
and switches connected to the Ethernet switch in the Exalogic machine rack. This
management network is in a single subnet. Do not use the management network interface
(ETH0/NET0) on compute nodes for client or application network traffic. Cabling or
configuration changes to these interfaces on Exalogic compute nodes is not permitted.

Oracle Exalogic Elastic Cloud Administration 4 - 8


Network Configuration Worksheets
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle can initialize the network settings of Exalogic


machine components based on customer input.
Complete the worksheets found in the documentation.

Worksheet Inputs

Oracle University and (Oracle Corporation) use only.


Domain name
General Domain Name Service (DNS) server location
Network Time Protocol (NTP) server location
Starting IP address
Client
Subnet mask, gateway address
Starting IP address
Management
Subnet mask, gateway address
Starting IP address
Private
Subnet mask, gateway address

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Network Configuration Worksheets


The Exalogic machine requires a large number of host names and IP addresses during initial
configuration. The number of IP addresses required for a particular network, such as the
management network, depends on the type of system. The network configuration (such as
host names and IP addresses) used during installation is generated from information you
supply to your Oracle technical representative in the completed configuration worksheet.
Default bonded IP addresses for the private InfiniBand network are assigned to Exalogic
compute nodes during manufacturing. Because these addresses are nonroutable and not
accessible externally, no customer worksheets are required. However, your Oracle
representative can reconfigure these IP addresses by using the Oracle ECU tool, if desired.

Oracle Exalogic Elastic Cloud Administration 4 - 9


Default Addresses for a Full Rack
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Network Component Addresses


Client Compute Node 192.168.1.1-14, 17-32
Storage Appliance 192.168.1.15-16
Management Compute Node 192.168.1.101-114, 117-132
Storage Appliance 192.168.1.115-116

Oracle University and (Oracle Corporation) use only.


IB Switch 192.168.1.201-205
Management Switch 192.168.1.200
PDU 192.168.1.210-211
Private Compute Node 192.168.10.1-14, 17-32
Storage Appliance 192.168.10.15-16

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Default Addresses for a Full Rack


Refer to the documentation for the default addresses assigned to full, half, and quarter
Exalogic racks during manufacturing.

Oracle Exalogic Elastic Cloud Administration 4 - 10


Default Host Names
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Host names are derived from the assigned rack name (el01,
el02, el03, by default).

Network Component Sample Hostname


Client Compute Node el01cn01

Oracle University and (Oracle Corporation) use only.


Storage Appliance el01sn01
Management Compute Node el01cn01-c
Storage Appliance el01sn01-c
IB Switch el01sw-ib01
InfiniBand Compute Node el01cn01-priv
Storage Appliance el01sn-priv

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Default Hostnames
The Exalogic machine name is used to derive the host names for the network interfaces of all
components. For example, an Exalogic machine name of el01 will result in compute node
host names of el01cn01, el01cn02, el01cn03, and so on. In a multi-rack configuration, each
Exalogic machine name should be unique and identify the rack. Oracle recommends using
el01 for the first Exalogic rack, el02 for the second, el03 for the third, and so on.

Oracle Exalogic Elastic Cloud Administration 4 - 11


Installation Spreadsheet
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle produces a spreadsheet with your desired network


configuration.

Oracle University and (Oracle Corporation) use only.


Management IPs

Private IPs

Client IPs

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Installation Spreadsheet
The Exalogic installation spreadsheet is divided into a Configuration tab and a Preview tab.
Entries made on the Configuration tab prepopulate entries on the Preview tab. Entries on the
Preview tab are used as the basis for the configuration that will be generated and used with
ECU. You can also manually edit the Preview values for any fine tuning, although this is not
recommended.
Enter configuration parameters in the General section of the spreadsheet, including the
machine name, and base names for storage and compute nodes. These values are used as
prefixes for host names that will be generated. Enter the domain name, region, and time zone,
as applicable. The Customer Name field determines the location to which the configuration
file will be saved.
Enter configuration parameters in the Network section of the spreadsheet. In this section, you
enter the list of DNS Name Servers, NTP Servers, Search Domains, and Default Gateway.
The start IP Address (Network), Subnet Mask, and Gateway will be the same across all
NET0, ILOM, BOND0, and BOND1 interfaces. As you enter values in these fields, the
corresponding fields on the Preview tab are populated with values.

Oracle Exalogic Elastic Cloud Administration 4 - 12


Exalogic Configuration Utility (ECU)
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle ECU (formerly OneCommand):


Includes a set of scripts and utilities that perform the initial
configuration of the Exalogic machine
Is intended to be run by an Oracle representative from a
compute node
Uses data from the spreadsheet so that all components

Oracle University and (Oracle Corporation) use only.


match your specification
Can be used only on a machine in its original factory
configuration

Spreadsheet Macro Input File Script

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Exalogic Configuration Utility (ECU)


After the Exalogic machine is installed in your data center and powered on, your Oracle
representative will optionally collect information about your specific network configuration
requirements, IP address ranges, host names to be assigned to each compute node, and so
on. This information is entered in a spreadsheet. The data entered in the spreadsheet
provides input to the Oracle Exalogic machine configuration tools, such as the Oracle ECU
tool. This tool includes a set of scripts and utilities that perform the initial configuration of the
Exalogic machine and is run by your Oracle representative on-site. For ECU to work, the
machine must be in a state as configured in the factory. It is not intended to be a tool for
maintaining or re-imaging a rack.
The spreadsheet is an Oracle OpenOffice document with macros that detect events to
prepopulate fields, perform validation, and finally generate an ECU configuration file. The
ECU tools and spreadsheet are available for download from the Oracle E-Delivery site. The
package is named Configuration Utilities for Exalogic.

Oracle Exalogic Elastic Cloud Administration 4 - 13


ECU Internals
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1. Ping all components by using their factory network


settings.
2. Set up SSH on all compute nodes and distribute keys.
3. Push files to compute nodes.
4. Verify hardware, firmware, and operating system.
5. Test IB topology and connectivity.

Oracle University and (Oracle Corporation) use only.


6. Update the network settings on all compute nodes.
7. Update the network settings on all IB switches and storage
nodes.
8. Ping all components by using their new network settings.
9. Retest IB topology and connectivity.
10. Configure NTP on all nodes.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

ECU Internals
If you are running ECU for the first time, it generates a file representing the factory
configuration of the rack. Review the exalogic_current.conf and
exalogic_pending.conf files to ensure that they are accurate. Verify that the current
configuration has the correct number of nodes and switches and the numbers match with the
pending configuration. Verify IP addresses, host names, and such network parameters in both
files.
ECU verifies ILOM for all compute nodes listed in the exalogic_current.conf file. The
master node is omitted from this process because it cannot ping its own ILOM when sideband
management is used. ECU also verifies entries corresponding to the storage heads in the
exalogic_current.conf file and validates that their corresponding ILOMs can be pinged
as well.
ECU verifies that each Sun Network QDR InfiniBand Gateway Switch (NM2-GW) is running a
Subnet Manager (SM). If a switch is not running an SM, an SM is dynamically started on that
switch. It also runs the verify-topology utility, which validates the network cabling.
ECU distributes keys to each compute node and allows subsequent steps to operate more
efficiently by not prompting for a password. After this step runs, accessing remote compute
nodes via SSH will not require a password.

Oracle Exalogic Elastic Cloud Administration 4 - 14


InfiniBand Networking Concepts
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Term Definition Description


IPoIB IP over InfiniBand Applications connected by an IB fabric communicate
by using standard IP address semantics.

SDP Socket Direct Protocol Applications communicate directly with the IB fabric,
bypassing the operating system's TCP/IP stack.

Oracle University and (Oracle Corporation) use only.


EoIB Ethernet over Applications within an IB fabric communicate with
InfiniBand external Ethernet networks.
vNIC Virtual Network Software that emulates an Ethernet NIC on the IB
Interface Card network

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

InfiniBand Networking Concepts


IP over InfiniBand (IPoIB) enables applications on separate devices to communicate with
each other over a private InfiniBand fabric by using native IB protocols and without the
overhead of Ethernet. For example, a compute node on one Exalogic rack may communicate
with a database on an Exadata rack. However, applications must support the SDP protocol to
utilize IPoIB instead of the default TCP/IP stack of the host operating system.
The InfiniBand switches also act as gateways to connect to external Ethernet networks. They
support eight 10 Gb Ethernet ports. Exalogic compute nodes can communicate through these
ports by using Ethernet over InfiniBand (EoIB). Each port is represented on the compute
nodes as a vNIC. This allows that node's IB connection to appear like any other Ethernet NIC
to both the operating system and to the external Ethernet network.

Oracle Exalogic Elastic Cloud Administration 4 - 15


Bonded Interfaces
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Compute and storage nodes use two physical IB ports and


switches for redundancy (active/passive).
A special network adapter makes these two interfaces
appear as a single interface to applications:
Linux: Bonded interface
Solaris: IP multi-path (IPMP) group

Oracle University and (Oracle Corporation) use only.


Different bonded interfaces are used for IPoIB and EoIB.

IPoIB, SDP
bond0
Switch1 IB0
Apps
EoIB vNIC0
Switch2 IB1
bond1
vNIC1
Compute Node

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Bonded Interfaces
The Exalogic compute nodes and storage appliance include a Host Channel Adapter (HCA)
to communicate with an InfiniBand network. The HCA consists of two ports, IB0 and IB1.
These raw IB connections are represented on the operating system as the interfaces ib0 and
ib1. The operating system and applications are generally not intended to use these interfaces
directly.
InfiniBand switches (both leaf and spine switches) are configured to automatically separate
the IPoIB traffic and the EoIB traffic. After the InfiniBand gateway switches are connected to
the Exalogic compute nodes, the following bonded interfaces are defined:
bond0 for IPoIB traffic and for using SDP
bond1 for EoIB traffic (uses two vNICs to communicate through ib0 and ib1)
Each HCA has an IP address, and active/passive bonding is configured. The active port of the
HCA connects to an IB switch and the passive port of the HCA connects to another switch. If
one connection fails, traffic is transparently failed over to the other.
Solaris IPMP provides the same functionality as bonded Interfaces on Oracle Linux. Oracle
recommends that you name your IPMP groups bond0 and bond1 to keep the terminology
consistent with Oracle Linux.

Oracle Exalogic Elastic Cloud Administration 4 - 16


Default Network Configuration
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Private IB
Network

IPoIB, Compute Node


SDP
IB IB0 bond0

Oracle University and (Oracle Corporation) use only.


Switches IB1 bond1 eth0
EoIB

NET3 NET2 NET1 NET0 MGMT

Client Ethernet
Network
Mgmt Mgmt
Network Switch

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Default Network Configuration


By default, each Exalogic compute node is configured with one bonded EoIB interface for one
external LAN (client network), and is named bond1.
The Cisco Ethernet management switch is connected to the NET0 port of compute nodes, the
NET0 port of the storage appliance, and also the management ports of the InfiniBand
gateway switches. On the compute nodes, this connection is represented on the operating
system by an eth network interface, such as eth0. The compute nodes are configured at the
time of manufacturing to use sideband management only. Therefore, the MGMT (or ILOM)
port is not connected, but ILOM is accessible from NET0.
Two physical IB ports (0A and 1A) on each switch are used to host the eight virtual Ethernet
ports. For example, port 0A comprises 0A-ETH-0, 0A-ETH-1, 0A-ETH-2, and 0A-ETH-3.
Splitter cables are used to route each of these two IB ports to existing Ethernet networks.
The number of Ethernet ports used in Exalogic deployment depends on your specific
bandwidth requirements (how many 10 Gb ports can be shared per compute node) and on
your specific LAN/VLAN connection requirements. You can configure additional EoIB network
interfaces for connecting to additional LANs, as required.

Oracle Exalogic Elastic Cloud Administration 4 - 17


Exalogic Diagnostic Utilities
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The base compute node OS image includes


/opt/exalogic.tools.
You can use these tools to:
Verify IB configuration for full, half, and quarter racks
Verify IB network connectivity to other components
Determine which IB ports are currently active

Oracle University and (Oracle Corporation) use only.


Verify compute node hardware and firmware
Verify BIOS settings and OS packages
Additional third-party IB tools are found at /usr/sbin.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 4 - 18


Utility Examples
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

# check_ibports
...
Port 1 is in the ACTIVE state

# verify-topology t quarterrack
Quarter rack expected to have 2 internal switches but has 1
... [FAILURE]

Oracle University and (Oracle Corporation) use only.


Are any hosts connected to spine switch... [SUCCESS]
[ERROR] node05 appears to be disconnected from the fabric

# CheckSWProfile
[SUCCESS]....Has supported operating system
[SUCCCES]....Has supported processor
[SUCCESS]....Kernel is at the supported version
[SUCCESS]....Has supported kernel architecture
[SUCCESS]....Software is at the supported profile

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Utility Examples
The check_ibports tool determines which of the two ports on the local IB Host Channel
Adapter (HCA) is active.
The verify-topology tool validates the expected cabling and connectivity for a full, half, or
quarter rack configuration.
The CheckSWProfile tool validates the current compute nodes operating system, kernel,
software packages, and hardware architecture for supported levels.

Oracle Exalogic Elastic Cloud Administration 4 - 19


Distributed Command Line Interface (DCLI)
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The dcli utility:


Enables you to execute a command on multiple nodes
from a single location
Can be used to create folders, change permissions, mount
file systems, run diagnostics, and so on
Can also upload the file to execute onto each node

Oracle University and (Oracle Corporation) use only.


Prompts for each node's credentials unless SSH user
equivalence is configured or the SSH key is exchanged

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Distributed Command Line Interface (DCLI)


The dcli utility is part of the base OS image and is found at /opt/exalogic.tools. You
can use the DCLI utility to execute commands or scripts in parallel across a defined set of
host names. It simplifies any operations that must be run across a subset or all compute
nodes. Configuration of SSH user equivalency across all cells is an important prerequisite for
optimizing the use of DCLI. For convenience, the tool also provides the -k option to automate
the distribution of SSH private keys. Internally the Oracle ECU tool utilizes DCLI to perform
several of its tasks.
DCLI also captures any output from each compute node and prints it to the shell from which
DCLI was run. The output from each individual compute node is prefixed by that particular
compute node's host name.

Oracle Exalogic Elastic Cloud Administration 4 - 20


DCLI Examples
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Configure SSH user equivalence

setup-ssh H el01cn02 P mypassword


setup-ssh H el01cn03 P mypassword
...

Oracle University and (Oracle Corporation) use only.


Execute a command on the specified nodes

dcli c el01cn02,el01cn03 "mount a t nfs"

Upload and execute a script on nodes defined in an external file

dcli g nodes.txt x configMailServer.sh

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

DCLI Examples
To configure SSH user equivalence between the host from which DCLI will be run and a
target host, use the setup-ssh utility, which is found in the same location as DCLI.
To give DCLI a list of hosts on which to run the specified command, supply a comma-
separated list with the -c option or supply the location of a file that lists the host names with
the -g option. Then either give a command to run, surrounded by quotation marks, or provide
the name of a script file with the x argument. The script file will be uploaded to the users
home folder on each target host before being executed.

Oracle Exalogic Elastic Cloud Administration 4 - 21


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

What tool is used by Oracle at a customer site to initialize an


Exalogic machine?
a. EME
b. ECU
c. ERU
d. ECE

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: b

Oracle Exalogic Elastic Cloud Administration 4 - 22


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

A full rack weighs twice as much as a half rack.


a. True
b. False

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: b

Oracle Exalogic Elastic Cloud Administration 4 - 23


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Which of the following is not a default network associated with


the Exalogic machine?
a. Management network
b. Client Access Network
c. Template network
d. InfiniBand private network

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: c

Oracle Exalogic Elastic Cloud Administration 4 - 24


Summary
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

In this lesson, you should have learned how to:


Explain how Oracle initializes an Exalogic machine based
on your input
Describe the Exalogic default network and storage setup
List some of the capabilities of ILOM
Customize the Exalogic storage configuration

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 4 - 25


Practice 4-1:
Connect to a Compute Node
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

This practice covers the following topics:


Remotely accessing a compute node's operating system
Inspecting the default network and file system settings
Locating and using simple diagnostic tools
Executing a command on multiple compute nodes

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 4 - 26


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.


Fusion Middleware Concepts

Oracle University and (Oracle Corporation) use only.


Objectives
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

After completing this lesson, you should be able to:


Describe a typical Java EE deployment
Discuss the WebLogic domain architecture
Explain the purpose of the WebLogic Node Manager
Describe the WebLogic recovery services

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 5 - 2


Middleware Concepts
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Middleware:
Extends the traditional client/server software architecture
Is the software that sits between clients, business
applications, and business data
Should be invisible to users
Is a key component of distributed enterprise systems

Oracle University and (Oracle Corporation) use only.


Enables dynamic, service-oriented architectures (SOA)
Enables you to focus more on your custom business
requirements and less on your infrastructure

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Middleware Concepts
Middleware is the software that connects software components or enterprise applications.
Middleware is the software layer that lies between the operating system and the applications
on each side of a distributed computer network. Typically, it supports complex, distributed
business software applications. It is especially integral to information technology that is based
on web services and service-oriented architecture (SOA). Middleware makes application
development easier by providing common programming abstractions, by masking application
heterogeneity and the distribution of the underlying hardware and operating systems, and by
hiding low-level programming details.
Because of the continued growth and use of network-based applications by businesses,
middleware technologies have become increasingly important. Companies and organizations
are now building enterprise-wide information systems by integrating previously independent
applications with new software developments. Middleware can provide uniform, standard, and
high-level interfaces to application developers and integrators so that applications can easily
be composed, reused, ported, and made to interoperate.

Oracle Exalogic Elastic Cloud Administration 5 - 3


Oracle Fusion Middleware (FMW) Review
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Most components of FMW:


Are written in Java and require a JVM
Require a Java EE application server such as WebLogic
Server

Oracle University and (Oracle Corporation) use only.


Web Identity Content
SOA ...
Center Mgmt Mgmt

WebLogic Server Coherence

Java Virtual Machine (JVM) WebTier

OS/Hardware

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Fusion Middleware (FMW) Review


FMW products are organized into two general categories.
Java components: Deployed as one or more Java EE applications and a set of
resources. Java components are deployed to an Oracle WebLogic Server domain as
part of a domain template. Examples of Java components are the Oracle SOA Suite and
Oracle WebCenter components.
System components: Not deployed as Java applications. Instead, a system
component is managed by the Oracle Process Manager and Notification server
(OPMN). A Java component and a system component are peers. Examples of FMW
system components are Oracle HTTP Server, Oracle Web Cache, Oracle Internet
Directory, and Oracle Business Intelligence.

Oracle Exalogic Elastic Cloud Administration 5 - 4


WebLogic Server (WLS) Overview
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

WebLogic Server:
Is a Java application
Hosts Java EE applications
Can be administered from a web-based console or a
scripting interface
Provides clustering services for load distribution and high

Oracle University and (Oracle Corporation) use only.


availability
Includes a robust and high-performance enterprise
messaging infrastructure
Offers an extensible security realm for authentication,
authorization, credential mapping, and so on

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WebLogic Server (WLS) Overview


Oracle WebLogic Server is a scalable, enterprise-ready Java Platform, Enterprise Edition
(Java EE) application server. The WebLogic Server infrastructure supports the deployment of
many types of distributed applications and is an ideal foundation for building applications
based on SOA. In addition to the Java EE implementation, WebLogic Server enables
enterprises to deploy mission-critical applications in a robust, secure, highly available, and
scalable environment. These features enable enterprises to configure clusters of WebLogic
Server instances to distribute load, and provide extra capacity in case of hardware or other
failures. New diagnostic tools enable system administrators to monitor and tune the
performance of deployed applications and the WebLogic Server environment itself. Extensive
security features protect access to services, keep enterprise data secure, and prevent
malicious attacks.
WebLogic Server provides many tools for system administrators to help with these tasks,
including a browser-based Administration Console, the WebLogic Scripting Tool (WLST), a
scripting language for automation of WebLogic system administration tasks based on Jython.
WLS also supports SNMP.

Oracle Exalogic Elastic Cloud Administration 5 - 5


Java EE Deployment: Example
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Web
Application
Server

EJB
Application

Oracle University and (Oracle Corporation) use only.


Persistence
Server
Web Service
XML
Application
Server EJB
Message Queue
Application

Server Server

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Java EE Deployment: Example


Users interact with a web application by using a browser. The web application is responsible
for rendering the website and for capturing user input through buttons, forms, links, and so on.
The web application itself may contain all of the necessary business logic to perform the tasks
that users request, such as retrieving the details of a product, processing an order, or filing an
insurance claim. But instead the web application may access Enterprise JavaBeans (EJB)
applications to perform the business logic. These EJB applications may be located on the
same server as the web application or on a different server.
Java EE containers like WebLogic Server provide additional services for EJB, including
persistence. For example, WebLogic could be responsible for writing a newly placed order to
a relational database. Similarly, WLS could update multiple databases within a single
transaction. WebLogic also supports the Java Message Service (JMS) to enable reliable,
asynchronous communication between applications.
If you want to expose business logic in a standard way to other applications, both within your
organization and beyond, use a web service application. XML-based web services can be
accessed by both Java and non-Java applications and are also a cornerstone for SOA.

Oracle Exalogic Elastic Cloud Administration 5 - 6


WLS Architecture Overview
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Server processes are grouped into logical domains.


Each domain has an administration server that is used to
start, configure, manage, and monitor other servers.

Machine Machine Machine

Oracle University and (Oracle Corporation) use only.


Domain B
Domain A Domain A Domain B
Admin
Server Server Server
Admin
Server Server
Server

Product Product
Installation Product Installation Installation

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WLS Architecture Overview


An Oracle WebLogic Server administration domain is a logically related group of Oracle
WebLogic Server resources. Domains include a special Oracle WebLogic Server instance
called the Administration Server, which is the central point from which you configure and
manage all resources in the domain. Usually, you configure a domain to include additional
Oracle WebLogic Server instances called managed servers. You deploy web applications,
EJB, web services, and other resources onto the managed servers and use the
Administration Server for configuration and management purposes only.
You can use a single Oracle WebLogic Server installation to create and run multiple domains,
or you can use multiple installations to run a single domain. How you organize your Oracle
WebLogic Server installations into domains depends on your business needs. You can define
multiple domains based on different system administrators responsibilities, application
boundaries, or geographical locations of the machines on which servers run. Conversely, you
might decide to use a single domain to centralize all Oracle WebLogic Server administration
activities.
For development or test environments, you can create a simple domain that consists of a
single server instance. This single instance acts as an Administration Server and hosts the
applications that you are developing.

Oracle Exalogic Elastic Cloud Administration 5 - 7


WLS Domain Contents
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

A domain is a directory structure that contains:


Configuration files managed by the admin server
Server start scripts
LDAP files
Each server in the domain creates a separate subdirectory
that contains:

Oracle University and (Oracle Corporation) use only.


Log files
MyDomain
Recovery files
bin

config

servers

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WLS Domain Contents


The domain directory must have Write privileges, as well as Read privileges, for the domain to
function properly, even if no changes are made to the configuration after it is created.
The bin directory contains scripts that are used in the process of starting and stopping the
Administration Server and the managed servers in the domain. These scripts are generally
provided as .sh files for UNIX and .cmd files for Windows.
Each domain describes its configuration in an XML document that is located in the domain's
configuration directory. At run time, each Oracle WebLogic Server instance in a given domain
creates an in-memory representation of the configuration described in this document. The
central configuration file for a domain is DOMAIN_NAME/config/config.xml. This file
specifies the name of the domain and the configuration of each server instance, cluster,
resource, and service in the domain. The file includes references to additional XML files that
are stored in subdirectories of the DOMAIN_NAME/config directory.
The servers/SERVER_NAME/logs directory holds logs and diagnostic information. The
servers/SERVER_NAME/store directory holds WebLogic persistent stores, which are used
for failover and recovery of transactions and messages. For each persistent store, there is a
separate subdirectory.

Oracle Exalogic Elastic Cloud Administration 5 - 8


WLS Machines and Compute Nodes
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

WLS allows you to define logical machines that:


Denote hardware boundaries
Are assigned one or more servers
Help WLS make failover and migration decisions
Typically correspond to compute nodes on Exalogic

Oracle University and (Oracle Corporation) use only.


Compute Node 1 Compute Node 2

Domain
Machine 1 Machine 2
Server Server
Server Server

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WLS Machines and Compute Nodes


WLS machine definitions identify a particular physical piece of hardware. A machine is used
to indicate which servers are colocated on the same hardware. WLS cannot simply use host
names or IP addresses to accurately determine this, so machines are explicitly defined. This
information is used by the WLS Node Manager when restarting or migrating a failed server,
and also by clustered servers when selecting the best location for storing replicated data.
Note that the WLS machine name does not need to be a valid host name or listen address. It
is just a unique logical identifier. However, to avoid confusion, Oracle recommends that you
give a machine the same name as the compute node it is intended to represent. Also avoid
confusing the concept of a WLS machine with an Exalogic machine, which refers to the
physical rack containing multiple compute nodes.

Oracle Exalogic Elastic Cloud Administration 5 - 9


Node Manager Concepts
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Node Manager:
Is a process that accepts remote commands to start, stop,
or suspend servers on the same machine
Monitors server availability and restarts failed ones
Can be used to migrate servers on a failed machine to
another machine

Oracle University and (Oracle Corporation) use only.


Machine
Server
Node Start/stop
Manager Monitor/
restart Server

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Node Manager Concepts


Server instances in a WebLogic Server production environment are often distributed across
multiple domains, machines, and geographic locations. Node Manager is a WebLogic Server
utility that enables you to start, shut down, and restart Administration Server and Managed
Server instances from a remote location. Although Node Manager is optional, it is
recommended if your WebLogic Server environment hosts applications with high-availability
requirements. A Node Manager process is not associated with a specific WebLogic domain
but with a machine. You can use the same Node Manager process to control server instances
in any WebLogic Server domain, as long as the server instances reside on the same machine
as the Node Manager process.
If a server instance that was started by using Node Manager fails, Node Manager
automatically restarts it. If Node Manager fails or is explicitly shut down, upon restart, it
determines the server instances that were under its control when it exited. Node Manager can
restart any failed server instances as needed.
The Administration Console can be used to issue commands to node managers running on
remote machines. WLST (in offline mode) also serves as a Node Manager command-line
interface that can run in the absence of a running Administration Server.

Oracle Exalogic Elastic Cloud Administration 5 - 10


Transaction Recovery
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Applications can use WebLogic to orchestrate distributed


transactions that involve multiple databases, messages,
and so on.
Transactions ensure that all operations either succeed or
fail.
WebLogic logs in-progress transactions to the file system.

Oracle University and (Oracle Corporation) use only.


WebLogic uses these logs to recover from system crashes
or network failures.

Server

TLOGs

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Transaction Recovery
WebLogic Server supports distributed transactions and the two-phase commit protocol for
enterprise applications. A distributed transaction is a transaction that updates multiple
resource managers (such as databases) in a coordinated manner. In contrast, a local
transaction begins and commits the transaction to a single resource manager that internally
coordinates API calls; there is no transaction manager. The two-phase commit protocol is a
method of coordinating a single transaction across two or more resource managers. It
guarantees data integrity by ensuring that transactional updates are committed in all of the
participating databases, or are fully rolled back out of all the databases, reverting to the state
prior to the start of the transaction. In other words, either all the participating databases are
updated, or none are updated.
WLS executes a recovery procedure when failures occur. It determines which transactions
were active in the machine at the time of the crash, and then determines whether the
transaction should be rolled back or committed. Because this service is designed to gracefully
handle transaction recovery after a crash, Oracle recommends that you attempt to restart a
crashed server and allow the service to handle incomplete transactions. When moving
transaction log records after a server failure, make all transaction log records available on the
new machine before starting the server there.

Oracle Exalogic Elastic Cloud Administration 5 - 11


Message Recovery
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

WebLogic records in-progress JMS messages to either:


The file system (default)
A database
If WebLogic and/or a client fails, the server can recover
and not lose messages.
Only messages that the application or administrator flags

Oracle University and (Oracle Corporation) use only.


as persistent are saved.

Server
Producer Message Server Consumer

Store

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Message Recovery
An enterprise messaging system enables applications to communicate with one another
through the exchange of messages. A message is a request, report, and/or event that
contains information needed to coordinate communication between different applications.
Java Message Service (JMS) servers reside on a WebLogic Server instance, host a defined
set of JMS modules, and also define any associated persistent storage for recovery.
A persistent message is guaranteed to be delivered once-and-only-once. The message
cannot be lost due to a JMS provider failure and it must not be delivered twice. It is not
considered sent until it has been safely written to a file or database. Non-persistent messages
are not stored. They are guaranteed to be delivered at-most-once, unless there is a JMS
provider failure, in which case messages may be lost, and must not be delivered twice.
For storing persistent messages, you can simply use the host server's default persistent file
store, which requires no configuration on your part. However, you can also create a dedicated
file-based store or JDBC store for JMS. A file store maintains subsystem data, such as
persistent JMS messages and durable subscribers, in a group of files in a directory. A JDBC
store maintains subsystem data in a relational database.

Oracle Exalogic Elastic Cloud Administration 5 - 12


WLS Cluster Architecture
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

A WLS cluster supports additional features:


To provide high availability for applications and services
To perform load balancing and failover
That are transparent to both applications and clients

Domain

Oracle University and (Oracle Corporation) use only.


Hardware or
software Cluster

Server 1
Web client Proxy
Server 2
EJB client
JMS client Server n

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WLS Cluster Architecture


Clustering is configuring a group of WebLogic Servers to work together to provide client
access to the services offered by the servers in the cluster. The cluster appears to a client as
one instance, whether the client is a web client or a Java application. By replicating the
services provided by one instance, an enterprise system achieves a fail-safe and scalable
environment. Scalability is achieved by balancing the load of incoming requests across the
servers in the cluster.
WebLogic Server provides clustering support for web applications by replicating the HTTP
session state of clients. You can balance the web application load across a cluster by using a
WebLogic Server proxy plug-in or the external load-balancing hardware.
Failover for EJB objects is accomplished by using the objects replica-aware stub. When a
client makes a call through a replica-aware stub to a service that fails, the stub detects the
failure and retries the call on another replica.
WLS also supports clustering for JMS applications. Applications that use a distributed
destination are more highly available than applications that use stand-alone destinations
because WebLogic JMS provides load balancing and failover for the members of a distributed
destination in a cluster.

Oracle Exalogic Elastic Cloud Administration 5 - 13


WLS Exalogic Topology: Example
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Compute Node A
Admin Server Domain
Server 1 Server 2 Server 3

Server 4 Server 5 Server 6


Node
Manager

Oracle University and (Oracle Corporation) use only.


Compute Node B
Server 7 Server 8 Server 9

Server10 Server11 Server12

Admin
Cluster Server Node
Manager

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WLS Exalogic Topology: Example


The example in the slide shows a single WebLogic domain that spans two Exalogic compute
nodes. Each compute node or machine has a single running node manager to support all
WebLogic (and optionally Coherence) processes on that compute node. The domain defines
a WebLogic administration server and 12 managed servers. The managed servers are all
members of a single WebLogic cluster and are distributed across the two compute nodes to
provide high availability. However, remember that to completely utilize the capacity of a
compute node, you should try running one server for each core, or 24 servers for two compute
nodes. For the example system in the slide, however, 12 servers provided sufficient
performance.

Oracle Exalogic Elastic Cloud Administration 5 - 14


WLS Exalogic Capacity Planning
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Exalogic was designed to host one WLS process on each


CPU core:
A full rack can support up to 360 WLS processes.
A half rack can support up to 180 WLS processes.
Ignore admin servers and node managers, which do not
process significant loads.

Oracle University and (Oracle Corporation) use only.


If CPU usage is low and RAM is available, add more WLS
processes.
Most Java EE applications are I/O driven, but very CPU-
intensive WLS processes may benefit from multiple cores.
As always, you should determine optimal performance
settings for your WLS cluster experimentally.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WLS Exalogic Capacity Planning


An instance of WLS running a typical Web application on a 64-bit platform will utilize about 4
to 6 GB of RAM under peak load. However, a single Exalogic compute node has 96 GB of
RAM. Therefore, it is likely that even if you run one WLS instance per compute node core, you
will still have ample RAM to run additional WLS instances if CPU utilization is not high.
The most commonly tuned performance characteristic of any Java process is the heap size.
The heap represents the amount of RAM that is available to the JVM for storing objects. The
default heap size for WLS is 512 MB, but larger settings may increase performance and
throughput. You can override the default setting by editing setDomainEnv.sh or by defining
an environment variable named USER_MEM_ARGS.

Oracle Exalogic Elastic Cloud Administration 5 - 15


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Which is not a component of a WebLogic domain?


a. Machine
b. Proxy
c. Cluster
d. Server

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: b

Oracle Exalogic Elastic Cloud Administration 5 - 16


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Which WLS feature enables you to start servers remotely?


a. Cluster Manager
b. Process Manager
c. Build Manager
d. Node Manager

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: d

Oracle Exalogic Elastic Cloud Administration 5 - 17


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Name two types of WebLogic services that are logged for


failover and recovery.
a. Beans
b. Requests
c. Messages
d. Applications

Oracle University and (Oracle Corporation) use only.


e. Transactions

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: c, e

Oracle Exalogic Elastic Cloud Administration 5 - 18


Summary
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

In this lesson, you should have learned how to:


Describe a typical Java EE deployment
Discuss the WebLogic domain architecture
Explain the purpose of the WebLogic Node Manager
Describe the WebLogic recovery services

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 5 - 19


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle University and (Oracle Corporation) use only.


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Storage Configuration

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle University and (Oracle Corporation) use only.


Objectives
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

After completing this lesson, you should be able to:


List the capabilities of the storage appliance
Perform basic administration and maintenance tasks
Configure appliance access services
Describe Exalogics default storage setup

Oracle University and (Oracle Corporation) use only.


Create pools, projects, and shares
Mount a shared file system by using NFS
Administer the appliance from the command line
Describe the recommended configuration for FMW

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 6 - 2


ZFS Appliance Capabilities
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Key features of the Exalogic storage appliance include:


Browser, command-line, and scripting interfaces
Redundancy at all levels and for all paths
Low-cost SAS and SATA hard disks
Integrated SSDs for read and write caching

Oracle University and (Oracle Corporation) use only.


Low-impact data compression and virus scanning
Support for a variety of client protocols
Template-driven configuration
Customizable alerts and analytics
Generation of support tickets

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

ZFS Appliance Capabilities


The Sun ZFS Storage 7000 family of products provides efficient file and block data services to
clients over a network, and a rich set of data services that can be applied to the data stored
on the system. For example, the appliance can optionally compress data before writing to the
storage pool. This allows for much greater storage utilization at the expense of increased
CPU utilization.
The ZFS Hybrid Storage Pool is composed of optional Flash-memory devices for acceleration
of reads and writes; low-power, high-capacity disks; and DRAM memory, all managed
transparently as a single data hierarchy.
To maximize the availability of your data in production, Sun ZFS Storage appliances include a
complete end-to-end architecture for data integrity, including redundancies at every level of
the stack. Key features include:
Predictive self-healing and diagnosis of all system hardware failures: CPUs, DRAM, I/O
cards, disks, fans, power supplies
Multiple I/O paths between the controller and disk shelves
Phone Home service to report telemetry to Oracle support during all software and
hardware issues

Oracle Exalogic Elastic Cloud Administration 6 - 3


Administrative Tasks
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Core administrative tasks for the appliance include:


Determine system status and health
Identify and replace failed hardware
Configure access protocols
Configure storage pools and shares

Oracle University and (Oracle Corporation) use only.


Configure users and access rights
Back up or restore your configuration settings
Monitor system load and performance

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Administrative Tasks
Analytics provides real-time graphs of various statistics, which can be saved for later viewing.
About a dozen high-level statistics are provided, such as NFSv3 operations/sec, which can
then be customized to provide lower-level details. Groups of viewed statistics can be saved as
worksheets for future reference.

Oracle Exalogic Elastic Cloud Administration 6 - 4


Getting Started
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1. Power on the appliance.


2. (Optional) Configure initial IP addresses, DNS locations,
and root credentials by using OneCommand.
3. Connect by using SSH and enter start /SP/console.
4. Verify or update the configuration settings.
5. Continue the initial setup steps by using the browser or

Oracle University and (Oracle Corporation) use only.


command-line interfaces.
6. Verify or update the configuration settings a final time.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Getting Started
All stand-alone controllers should have at least one NIC port configured as a management
interface. Select the Allow Admin option in the browser user interface (BUI) to enable BUI
connections on port 215 and CLI connections on SSH port 22. Gather the following
information in preparation for configuring an Ethernet interface on the storage controller: IP
address, IP netmask, host name, DNS domain name, DNS server IP address, and gateway IP
address.
You can access the system by connecting an administrative client to the serial management
port on the controller. Alternatively, if a Dynamic Host Configuration Protocol (DHCP) server is
on the network, you can access the system by connecting the network management port to
your network. For a serial connection, use telnet. For an Ethernet connection, use SSH. At the
command prompt, type start /SP/console. After you configure the primary interface,
configure the remaining system parameters by using the BUI from any client on the same
network.
Note that if you are installing a cluster, configure only one controller initially. The software
propagates the configuration to the peer controller during cluster initialization. After the cluster
is initialized, you can administer the system from either storage controller. However, do not
attempt initial configuration on both controllers independently.

Oracle Exalogic Elastic Cloud Administration 6 - 5


Accessing the Browser User Interface (BUI)
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The default port is 215 and the default account is root.


A help menu is also available at /wiki.

Oracle University and (Oracle Corporation) use only.


Main menu

Power off or
reboot appliance.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Accessing the Browser User Interface (BUI)


The BUI is the graphical tool for administration of the appliance. The BUI provides an intuitive
environment for administration tasks, visualizing concepts, and analyzing performance data.
Direct your browser to the system by using either the IP address or host name that you
assigned to the NET-0 port during initial configuration as follows: https://ipaddress:215 or
https://hostname:215. The login screen appears. Internet Explorer 6 and earlier versions have
known issues and are unsupported, and login will not complete.
The online help linked in the top right of the BUI is context-sensitive. For every top-level and
second-level screen in the BUI, the associated help page appears when you click the Help
button.
The masthead contains several interface elements for navigation and notification, as well as
primary functionality. At the left, from top to bottom, are the company logo, a hardware model
badge, and hardware power-off and restart buttons. Across the right, again from top to
bottom, are login identification, logout, help, and main navigation. The title bar appears below
the masthead and provides local navigation and functions that vary depending on the current
view.
System alerts appear in the masthead as they are triggered. If multiple alerts are triggered
sequentially, refer to the list of recent alerts found on the Dashboard screen or the full log
available on the Maintenance > Logs screen.

Oracle Exalogic Elastic Cloud Administration 6 - 6


Viewing Appliance Status
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Select a pool. 1
2

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Viewing Appliance Status


The Status > Dashboard screen provides a view of storage, memory, services, hardware,
activity, and recent alerts. The Status > Settings screen enables you to change the graphs
that appear on the dashboard and to customize the threshold settings associated with the
weather icons shown for each graph on the dashboard.
The activity area of the dashboard shows graphs of eight performance statistics by default.
The statistical average is plotted in blue and the maximum appears in light gray. To go to the
Analytics worksheet for an activity, click one of the four graphs (day, hour, minute, second) for
the statistic that you want to evaluate. To view the average for each graph, move the cursor
over a graph and the average appears in the tooltip. The weather icon in the upper left
provides a report of activity according to thresholds that you can customize for each statistic.
The weather icon is intended to grab your attention when something is unusually busy or idle.
The vertical scale of all graphs is printed at the top right, and all graphs are scaled to this
same height. The height is calculated from the selected graph (plus a margin). The height will
rescale based on activity in the selected graph, with the exception of utilization graphs, which
have a fixed height of 100%.

Oracle Exalogic Elastic Cloud Administration 6 - 7


Viewing Appliance Status
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Service
status

Disk usage

Click for more details

Memory

Oracle University and (Oracle Corporation) use only.


usage

Hardware
availability

Click to
view full log.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Viewing Appliance Status (continued)


The Usage area of the dashboard provides a summary of your storage pool and main
memory usage. The pool name may be clicked to change the pool that is displayed on the
status screen, should multiple pools be configured on the appliance. The name of the pool
appears at the top right of the Usage area. To the left is a pie chart of used and available
space. To go to the Shares screen for the pool, click the Storage pie chart. To the left is a pie
chart that shows memory usage by component. To go to the Analytics worksheet for dynamic
memory usage broken down by application name, click the Memory pie chart.
The Services area of the dashboard shows the status of services on the appliance, with a light
icon to indicate the status of each service. Most services will be green, indicating that the
service is online, or gray, indicating that the service is disabled.
The Hardware area of the dashboard shows a status overview of the hardware components
on the appliance, including CPUs, disks, and fans. If there is a known fault, an amber icon is
displayed. To go to the Hardware Maintenance screen for a detailed look at hardware state,
click the name of a hardware component.
The Recent Alerts section shows the last four appliance alerts. Click the box to go to the
Maintenance Logs screen to examine all recent alerts in detail.

Oracle Exalogic Elastic Cloud Administration 6 - 8


Viewing Network Settings
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1
2

3
Ethernet
port status
View/edit addresses

Oracle University and (Oracle Corporation) use only.


4

IB port status

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Viewing Network Settings


The networking configuration features enable you to create a variety of advanced networking
setups from your physical network ports, including link aggregations, virtual LANs (VLANs), IB
partitions, and IP multipathing (IPMP) groups. You can then define any number of IPv4 and
IPv6 addresses for these abstractions, for use in connecting to the various data services on
the system.
Devices are created by the system to represent the available network or InfiniBand ports.
They have no configuration settings of their own. Data links are required to complete the
network configuration, whether they apply specific settings to the network devices or not.
Interfaces assign IP addresses to the data links.
The Configuration page is shown by default, and lists Devices, Datalinks, and Interfaces,
along with buttons for administration. The Devices list shows link status on the right, as well
as an icon to reflect the state of the network port. To configure an IP address on a network
device, first create a data link, and then create an interface to use that data link. You can use
the + icon for this task, but the interface also supports drag-and-drop. For example, you can
configure a data link and drag it onto the Interfaces table.
The Addresses page shows a summary table of the current network configuration. The
Routing page provides configuration of the IP routing table and associated properties.

Oracle Exalogic Elastic Cloud Administration 6 - 9


Some Supported Services
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Category Service Description


Access SSH Command-line admin interface
Data NFS File system access via NFS v3 or v4 protocols
FTP File system access via FTP or FTPS protocols
SFTP File system access via SFTP protocol
HTTP File system access via WebDAV over HTTP/HTTPS

Oracle University and (Oracle Corporation) use only.


SMB File system access via SMB protocol
Replication Replicate data to another remote storage appliance.
Migration Move new and existing NFS clients to another server.
Virus Scan Scan updated files and quarantine any infected ones.
Directory LDAP Authenticate users via an external LDAP directory.
AD Authenticate users via an MS Active Directory server.
System SNMP System access by using SNMP

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Some Supported Services


The Secure Shell (SSH ) service allows users to log in to the appliance CLI and perform most
of the same administrative actions that can be performed in the BUI. The SSH service can
also be used as means of executing automated scripts from a remote host.
Network file system (NFS) is an industry-standard protocol to share files over a network. NFS
versions 2, 3, and 4 are supported. Setting the NFS minimum and maximum versions to the
same value will cause the appliance to communicate only with clients who are using that
version. This may be useful if you find an issue with one NFS version or the other (such as
the performance characteristics of that NFS version with your workload), and want to force
clients to use only the version that works best.
The FTP service allows file system access from FTP clients. Anonymous logins are not
allowed, and users must authenticate. The SSH File Transfer Protocol (SFTP) service allows
file system access from SFTP clients. The SFTP service uses a nonstandard port number for
connections to the appliance. This is to avoid conflicts with administrative SSH connections to
port 22. By default, the SFTP port is 218 and must be specified on the SFTP client before
connecting.
The HTTP service provides access to file systems by using the HTTP and HTTPS protocols
and the HTTP extension, Web-based Distributed Authoring and Versioning (WebDAV).

Oracle Exalogic Elastic Cloud Administration 6 - 10


Configuring File System Services
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1
2

Oracle University and (Oracle Corporation) use only.


Enable, disable, or
restart service.

4
Configure
service settings.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Configuring File System Services


1. Click Configuration.
2. Click Services.
3. Locate the list of Data Services. Select a service to edit its configuration. You can also
enable and disable the service at the appliance level.
4. Edit the available settings for the selected service.
NFS settings include:
- Minimum/maximum supported version: Controls which versions of NFS are
supported
- Grace period: Seconds that all clients have to reclaim locks after an appliance
reboot. During this period, the NFS service only processes reclaims of old locks.
All other requests for service must wait until the grace period is over.
FTP settings include:
- Maximum # of connections: Set this to cover the anticipated number of
concurrent users. By default this is 30, and 0 implies unlimited.
- Default login root: The default is / and points to the top of the shares hierarchy.

Oracle Exalogic Elastic Cloud Administration 6 - 11


Identifying Hardware Components
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1
2 Power off
or restart.
3
el01stor01

Oracle University and (Oracle Corporation) use only.


Turn on
locator LED.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Identifying Hardware Components


The Maintenance > Hardware screen of the BUI provides visual representations of the
physical system components, enabling you to identify and locate hardware components
visually and verify their status. A green icon indicates a good status, amber indicates a fault,
and gray indicates a component that is not present.
The main hardware page lists the system chassis, a summary of its contents, and any
attached disk shelves (on supported systems). This provides an overview of the hardware
present on the system, as well as controls to locate, reset, or power off the system.
A thumbnail of the controller is presented at the left. Clicking the thumbnail or the Show
Details link takes you to a detailed view of the chassis and is identical to clicking the right-
pointing arrow at the top left of the view.

Oracle Exalogic Elastic Cloud Administration 6 - 12


Identifying Hardware Components
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Location of selected
component
Change
perspective.

Oracle University and (Oracle Corporation) use only.


5

View specifications.

Take disk offline. Turn on disk


Component status locator LEDs.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Identifying Hardware Components (continued)


To view the chassis details, click the arrow icon. At the left is a set of images describing the
chassis. If there are multiple views, you can switch between them by clicking the name of the
view above the image. For each view, faulted components will be highlighted in red. In
addition, the currently selected component will be highlighted in the image. Clicking a
component within the image selects the corresponding component in the list to the right.
A tab is present for each component type in the following list. Each component type has a
state icon, which will be orange if there is a faulted component of the given type. Clicking the
information icon while over a row or double-clicking a row opens a dialog box with detailed
information about the component. The information displayed in the list depends on the
component type, but typically includes the manufacturer, model, part number, serial number,
revision number, and size/speed.
Disk components also provide additional icons to toggle the locate indicator for the disk or to
take the disk offline. This option is available only for disks that are part of a configured storage
pool (including the system pool). Taking a disk offline prevents the system from reading or
writing to it. Faulted devices are automatically avoided.

Oracle Exalogic Elastic Cloud Administration 6 - 13


Replacing Failed Hardware
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

See the Sun ZFS Storage Appliance Customer Service


Manual for detailed procedures about disks, power
supplies, fans, and so on.
For example, to replace a drive on the disk shelf:
1. Take the disk offline.
2. Press the release button on the front of the disk.

Oracle University and (Oracle Corporation) use only.


3. Pull the ejection lever and remove the old disk.
4. Insert the new disk and verify that the LED indicator turns
green.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Replacing Failed Hardware


The shelf drives are hot swappable and can be replaced without removing power from the
shelf. The replacement drive must be of the same capacity and type as the drive to be
replaced. To avoid possible data loss when removing nonfaulted drives, label each drive with
the number of the slot from which it was removed and reinstall each drive into the same slot.
Faulted drives are indicated by an amber LED.
The power supplies are hot-swappable and can be replaced without removing power from the
system. The power supplies can produce a high-energy hazard and should be replaced only
by instructed individuals with authorized access to the equipment. Failed power supplies are
indicated by an amber LED on the rear panel. To replace a power supply:
1. Ensure that the power supply on/off switch is in the off position.
2. Disconnect the power cord tie strap from the power cord and unplug the power cord
from the power supply.
3. Using your thumb and forefinger, unscrew both ejection screws until loose and swing
the ejection arms out until they are fully open.
4. Pull the power supply out of the chassis, being careful not to damage the circuit board
connector extending from the back of the power supply.
5. With the ejection arms fully open, slide the new power supply into the chassis slot.

Oracle Exalogic Elastic Cloud Administration 6 - 14


Performing Failover Actions
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

In Exalogic, storage heads are in an active/passive state


by default.
You can manually fail over to the other storage head for
maintenance purposes.

Oracle University and (Oracle Corporation) use only.


2

3
Active head Passive head
Cluster ports
are connected
el01stor01 el01stor02

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Performing Failover Actions


All communication between storage heads is transmitted over one of the three cluster I/O
links provided by the Clustron hardware. This device offers two lower-speed serial links and
one Ethernet link. The use of serial links allows for greater reliability. Ethernet links may not
be serviced quickly enough by a system under extremely heavy load.
Normally, heartbeat messages are sent by each head on all three cluster I/O links at 50 ms
intervals. Failure to receive any message is considered link failure after 200 ms (serial links)
or 500 ms (Ethernet links). If all three links have failed, the peer is assumed to have failed.
Takeover arbitration will be performed.
The Configuration > Cluster view provides a graphical overview of the status of the cluster
card, the cluster head node states, and all of the resources. Thumbnail pictures are displayed
for each head and for each cluster card connection. A solid line connects a link when that link
is connected and active, and the line disappears if that connection is broken or while the other
system is restarting/rebooting.
The Setup and Unconfig buttons are used to initialize the cluster configuration or to revert a
cluster back to stand-alone operation. The Failback button makes the currently selected
storage head the new owner (active) within the cluster. The Takeover button is similar to
Failback but also forces the other storage head to reboot.

Oracle Exalogic Elastic Cloud Administration 6 - 15


Creating Administrative Users
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1
2

3
4

Oracle University and (Oracle Corporation) use only.


5

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Creating Administrative Users


1. Click Configuration.
2. Click Users.
3. Add a new user.
4. Users can either be Directory or Local. Local users have all account and role information
saved on the appliance. Directory users exist in an external LDAP or NIS repository,
while their role information is still saved on the appliance. In addition to a username and
password, these other optional user fields are also available:
- Kiosk User: A kiosk user will not be able to access the appliance via the CLI.
- Kiosk Screen: A specific screen (such as Dashboard) that this kiosk user is
restricted to
5. Users are granted privileges by assigning custom roles to them. A role is a collection of
privileges that can be assigned to users. It may be desirable to create administrator and
operator roles with different authorization levels. Staff members may be assigned any
role that is suitable for their needs without assigning unnecessary privileges. An
Exceptions screen is also available. These authorizations are excluded from those
normally available due to the selected roles.

Oracle Exalogic Elastic Cloud Administration 6 - 16


Roles and Authorizations
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Select

Oracle University and (Oracle Corporation) use only.


2 authorization type

Add authorizations
to role

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Roles and Authorizations


Authorizations allow users to perform specific tasks, such as creating shares, rebooting the
appliance, and updating the system software. Authorizations are grouped into Scopes, and
each scope may have a set of optional filters to narrow the scope of the authorization. For
example, rather than an authorization to restart all services, a filter can be used so that this
authorization can restart the HTTP service only.
1. From the same Configuration > Users screen, add a new Role.
2. Browse the scopes in the BUI to see which other authorizations exist. Available scopes
include Alerts, Analytics, Clustering, Hardware, Networking, Projects/Shares, Roles,
Services, and Users.

Oracle Exalogic Elastic Cloud Administration 6 - 17


Shared Storage Concepts
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Term Definition

Pool High availability and replication characteristics for a collection of disks

Project Default administrative and file system settings for a collection of shares

Share A file system mount point, access protocols, access rights, and other
settings

Oracle University and (Oracle Corporation) use only.


Pool

Project Project

Share Share Share Share

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Shared Storage Concepts


The appliance is based on the ZFS file system. ZFS groups the underlying storage devices
into pools. Shared file systems then allocate disk space from these pools. Before creating file
systems, you must first configure storage on the appliance. After a storage pool is configured,
you do not have to statically size file systems, although this behavior can be achieved by
using quotas and reservations.
Each storage node can have any number of pools, and each pool can be assigned ownership
independently in a cluster. While an arbitrary number of pools is supported, creating multiple
pools with the same redundancy characteristics owned by the same cluster head is not
advised. Doing so results in poor performance, suboptimal allocation of resources, artificial
partitioning of storage, and additional administrative complexity.
The storage appliance exports file systems as shares, which are managed in this section of
the appliance. All file systems are grouped into projects. A project defines a common
administrative control point for managing shares. All shares within a project can share
common settings, and quotas can be enforced at the project level in addition to the share
level. Projects can also be used solely for grouping logically related shares, so their common
attributes (such as accumulated space) can be accessed from a single point.

Oracle Exalogic Elastic Cloud Administration 6 - 18


Default Storage Configuration
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The OneCommand tool:


Defines a single pool that spans 18 of the 20 disks
Defines common shares as well as shares dedicated for
each compute node
Enables NFS access on all shares

Oracle University and (Oracle Corporation) use only.


exalogic

common

node1 ... node30


general patches

general dumps general dumps

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Default Storage Configuration


The default storage configuration is performed at the time of manufacture. A single storage
pool is configured. Active-passive clustering for the server heads is configured. Data is
mirrored, which yields a highly reliable and high-performing system.
Two exclusive NFS shares are available for each of the Exalogic compute nodes: one for
crash dumps, and another for general purposes. You can implement access control for these
shares based on your requirements. Two common NFS shares are also available and are
intended to be accessed by all compute nodes. One of the shares is intended for patches and
another for general purposes.

Oracle Exalogic Elastic Cloud Administration 6 - 19


Creating a Pool
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Multiple pools are recommended only when you require


drastically different levels of redundancy or performance.

1
2

Oracle University and (Oracle Corporation) use only.


3

el01stor01

Data profile for the


default pool

Disk allocation for


the default pool

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Creating a Pool
Each node can have any number of pools, and each pool can be assigned ownership
independently in a cluster. While an arbitrary number of pools is supported, creating multiple
pools with the same redundancy characteristics owned by the same cluster head is not
advised. Doing so will result in poor performance, suboptimal allocation of resources, artificial
partitioning of storage, and additional administrative complexity.
When allocating raw storage to pools, keep in mind that filling pools completely results in
significantly reduced performance, especially when writing to shares. These effects typically
become noticeable after the pool exceeds 80% full, and can be significant when the pool
exceeds 90% full. Therefore, best results are obtained by over-provisioning by approximately
20%.
After the task of creating a new pool is started, the setup falls into two different phases:
verification and configuration. The verification phase enables you to verify that all storage is
attached and functioning, and allocate disks within chassis. In a stand-alone system, this
presents a list of all available storage and drive types, with the ability to change the number of
disks to allocate to the new pool. By default, the maximum number of disks is allocated, but
this number can be reduced in anticipation of creating multiple pools.

Oracle Exalogic Elastic Cloud Administration 6 - 20


Pool Data Profiles
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Profile Description

Striped No redundancy

Mirrored Data is always copied to another disk.


Reduces usable space by 1/2

Triple-Mirrored Data is always copied to 2 additional disks.

Oracle University and (Oracle Corporation) use only.


Reduces usable space by 2/3

Single-Parity RAID For every 3 disks, there is a single parity disk for recovery.
Reduces usable space by 1/4

Double-Parity RAID For every 3 disks, there are 2 parity disks for recovery.
Reduces usable space by 2/5

Triple-Parity RAID For every n disks, there are 3 parity disks for recovery.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Pool Data Profiles


After verification is completed, the next step involves choosing a storage profile that reflects
the RAS and performance goals of your setup. The set of possible profiles presented depends
on your available storage.
Striped: Data is striped across disks, with no redundancy whatsoever. Although this
maximizes both performance and capacity, it comes at great cost: A single disk failure
results in data loss.
Mirrored: Data is mirrored, reducing capacity by half but yielding a highly reliable and
high-performing system. This is recommended when space is considered ample but
performance is at a premium.
Triple-Mirrored: Data is triply mirrored, reducing capacity by one-third, but yielding a
very highly reliable and high-performing system. This configuration is intended for
situations in which maximum performance, and availability are required while capacity is
much less important. Compared with a two-way mirror, a three-way mirror adds
additional protection against disk failures and latent disk failures, in particular during
reconstruction for a previous failure.
Single-Parity RAID: Each stripe is kept to three data disks and a single parity disk.
Narrow stripe widths offer better random read performance than the wider stripe,
double-parity configuration, but they do not have quite the capacity cost of the mirrored
option.
Oracle Exalogic Elastic Cloud Administration 6 - 21
Creating a Project
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1 5
2
6
6
Base path for share
3 mount points

Oracle University and (Oracle Corporation) use only.


4

Default settings for


this project's shares

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Creating a Project
1. Click Shares in the top menu.
2. Click Projects on the submenu.
3. A list of all projects is displayed. Click the plus (+) button to create a new project.
4. Supply the project with a name. By default, this name is used as the base mount point
for this project.
5. Click the General tab for this new project.
6. Configure the default settings for any shares that will be members of this project
including:
- Quota: Sets a maximum limit on the total amount of space consumed by all file
systems and within the project.
- Read-only: The contents of a read-only file system cannot be modified, regardless
of any protocol settings. This setting does not affect the ability to rename, destroy,
or change properties of the file system.

Oracle Exalogic Elastic Cloud Administration 6 - 22


Creating a Project
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Default owner and


UNIX permissions

Oracle University and (Oracle Corporation) use only.


Service is
enabled.

Default settings for


each protocol

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Creating a Project (continued)


User: The owner of the root directory. This can be specified as a user ID or username.
For UNIX-based NFS access, this can be changed from the client by using the chown
command.
Group: The group of the root directory. This can be specified as a group ID or group
name. For UNIX-based NFS access, this can be changed from the client by using the
chgrp command.
Permissions: Standard UNIX permissions (owner/group/everyone, read/write/execute)
for the root directory. For UNIX-based NFS access, this can be changed from the client
by using the chmod command. Selecting permissions is done by clicking individual
boxes. Alternatively, clicking the label (user, group, or other) selects (or deselects)
all permissions within the label.
Share Mode: Determines whether shares in this project are available for reading only,
for reading and writing, or neither.
Anonymous User Mapping: Unless the root option is in effect for a particular client,
the root user on that client is treated as an unknown user, and all attempts by that user
to access the share's files will be treated as attempts by a user with this UID.

Oracle Exalogic Elastic Cloud Administration 6 - 23


Creating a Share
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1 2 4

Oracle University and (Oracle Corporation) use only.


Default mount point:
/export/CService1/<name>

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Creating a Share
Shares that are part of a project can either have local settings for properties, or they can
inherit their settings from the parent project. By default, shares inherit all properties from the
project. If a property is changed on a project, all shares that inherit that property are updated
to reflect the new value. When inherited, all properties have the same value as the parent
project, with the exception of the mount point. When inherited, the mount point is the
concatenation of the project setting and the share name.
1. Access a project.
2. Click the Shares menu option.
3. Click the plus (+) button to define a new shared file system.
4. Give the share a name. By default, this name is used as part of the mount point. Edit
other share settings such as:
- Data Migration Source: If set, this file system is actively shadowing an existing
file system, either locally or over NFS.
- Case sensitivity: It controls whether directory lookups are case-sensitive or not
case-sensitive.

Oracle Exalogic Elastic Cloud Administration 6 - 24


Overriding Project Defaults
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1 2

Clear to override
project values.

Oracle University and (Oracle Corporation) use only.


3

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Overriding Project Defaults


Every file system on the appliance must be given a unique mount point which serves as the
access point for the file system data. Projects can be given mount points, but these serve only
as a tool to manage the namespace by using inherited properties. Projects are never mounted
and do not export data over any protocol.
The project/share combination is a unique identifier for a share within a pool. Multiple projects
can contain shares with the same name, but a single project cannot contain shares with the
same name. A single project can contain both file systems, and they share the same
namespace.
Shares that are part of a project can either have local settings for properties, or they can
inherit their settings from the parent project. By default, shares inherit all properties from the
project. If a property is changed on a project, all shares that inherit that property are updated
to reflect the new value. When inherited, all properties have the same value as the parent
project, with the exception of the mount-point and SMB properties. When inherited, these
properties concatenate the project setting with their own share name.
To override the default properties defined in the parent project, deselect the Inherit from
project check box.

Oracle Exalogic Elastic Cloud Administration 6 - 25


Setting Quotas
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

You can restrict the maximum size of:


All shares within a project
A specific share

Oracle University and (Oracle Corporation) use only.


1

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Setting Quotas
The appliance leverages a pooled storage model where all file systems share common space.
By default, file systems never have an explicit size assigned to them, and only take up as
much space as they need. This approach provides maximum flexibility and simplicity of
management in an environment when users are generally trusted to do the right thing. A
stricter environment, where user's data usage is monitored and/or restricted, requires more
careful management. A quota represents a limit on the amount of space that can be
consumed by any particular entity. This can be based on file system, project, user, or group,
and is independent of any current space usage. Clients attempting to write new data will get
an error message when the file system is full, either because of a quota or because the
storage pool is out of space.
A reservation, on the other hand, represents a guarantee of space for a particular project or
file system. This takes available space away from the rest of the pool without increasing the
actual space consumed by the file system. This setting cannot be applied to users and
groups. The traditional notion of a statically sized file system can be created by setting a
quota and reservation to the same value. By default, a reservation includes all snapshots of a
file system. If, instead, the Include Snapshots property is not set, then the reservation only
applies to the immediate data of the file system.

Oracle Exalogic Elastic Cloud Administration 6 - 26


Restricting Share Access
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

You can limit access to a share by configuring:


Standard UNIX file permissions
NFS exceptions based on client IP address or DNS

Oracle University and (Oracle Corporation) use only.


2

Starting IP and
CIDR bit mask

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Restricting Share Access


Exceptions to the overall sharing modes may be defined for clients or collections of clients.
When a client attempts access, its access will be granted according to the first exception in
the list that matches the client (or, if no such exception exists, according to the global share
modes defined previously). These client collections may be defined by using one of three
types:
Host(FQDN) or Netgroup: A single client whose IP address resolves to the specified
fully qualified name, or a netgroup that contains fully qualified names to which a client's
IP address resolves. An example is exl01cn01.mydomain.com.
DNS Domain: All clients whose IP addresses resolve to a fully qualified name ending in
this suffix. An example is mydomain.com.
Network: All clients whose IP addresses are within the specified IP subnet, expressed
in CIDR notation. An example is 192.168.20.0/22.
For each specified client or collection of clients, you will then express two parameters:
whether the client will be permitted read-only or read-write access to the share, and whether
the root user on the client will be treated as the root user (if selected) or the unknown user.

Oracle Exalogic Elastic Cloud Administration 6 - 27


Configuration Management
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Use this menu to:


Update or roll back the appliance software
Back up the current appliance configuration
Restore the initial configuration or a previously saved one
Generate a diagnostic bundle and upload it to Oracle
support

Oracle University and (Oracle Corporation) use only.


1
2

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Configuration Management
Each update may be supplied with new firmware or updates to external resources. In general,
these updates are backward-compatible and applied automatically without user intervention.
There are exceptions, however, for nonreversible updates. These updates involve updating a
resource external to the system software in a way that is incompatible with older software
releases. After the update is applied, rolling back to previous versions will result in undefined
behavior. For these updates, you will always be given an explicit option to apply them
automatically during upgrade or apply them after the fact.
Following the application of a software upgrade, any hardware for which the upgrade includes
newer versions of firmware will be upgraded. There are several types of devices for which
firmware upgrades may be made available; each has distinct characteristics.
The appliance can generate support bundles containing system configuration information and
core files for use by remote support in debugging system failures. Support bundles are
generated automatically in response to faults if the Phone Home service is enabled.
Administrators can also manually generate and upload a support bundle. To facilitate this, the
appliance must be connected to the Internet, either directly or through the web proxy
configured on the Phone Home service screen.

Oracle Exalogic Elastic Cloud Administration 6 - 28


Mounting an NFS Share
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

/etc/fstab (Linux):
1

el01stor01:/export/CService1/apps /u01/apps nfs


rw,bg,hard,nointr,rsize=131072,wsize=131072,tcp,vers=3

/etc/vfstab (Solaris):

Oracle University and (Oracle Corporation) use only.


el01stor01:/export/CService1/apps /u01/apps nfs -yes
rw,bg,hard,nointr,rsize=131072,wsize=131072,proto=tcp,
vers=3

Recommended settings

2
> mount /u01/apps

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Mounting an NFS Share


UNIX NFS mount options include:
bg: If the first NFS mount attempt times out, retry the mount in the background. After a
mount operation is moved to the background, all subsequent mounts on the same NFS
server will be run in the background immediately, without first attempting the mount. A
missing mount point is treated as a timeout, to allow for nested NFS mounts.
hard/soft: Specifies whether the program that is using a file via an NFS connection
should stop and wait (hard) for the server to come back online (if the host serving the
exported file system is unavailable) or whether it should report an error (soft). Using soft
mounts is not recommended, as they can generate I/O errors in very congested
networks or when using a very busy server.
intr: Allows NFS requests to be interrupted if the server fails or cannot be reached
rsize/wsize: These settings speed up NFS communication for reads (rsize) and writes
(wsize) by setting a larger data block size, in bytes, to be transferred at one time. For
NFSv2 or NFSv3, the default values for both parameters is set to 8192. For NFSv4, the
default values for both parameters is set to 32768.
tcp/udp: Specifies that the NFS mount uses the TCP or UDP protocol

Oracle Exalogic Elastic Cloud Administration 6 - 29


Storage Command-Line Interface (CLI)
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Use SSH to connect to the appliance.


Tab completion is supported.
Supply a batch of commands by redirecting input to a file.
Configuration entities and commands are organized into
groups or contexts, similar to the BUI.

Oracle University and (Oracle Corporation) use only.


configuration maintenance shares status analytics

storage system Project1 storage

services hardware Project2 services

users activity
ShareA
roles alerts

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Storage Command-Line Interface (CLI)


The CLI is designed to mirror the capabilities of the BUI, while also providing a powerful
scripting environment for performing repetitive tasks. The appliance presents a command-line
interface available through either the serial console or SSH. To administer the appliance, you
must log in as root. When you log in, the CLI presents you with a prompt that consists of the
host name, followed by a colon, followed by a greater-than sign.
Tab completion is used extensively. If you are not sure what to enter in a particular context,
pressing the Tab key provides you with possible options. Throughout the documentation,
pressing Tab is presented as the word tab in bold italic. Help is always available: The help
command provides context-specific help. Help on a particular topic is available by specifying
the topic as an argument to help. An example is help commands. Available topics are
displayed by tab-completing the help command, or by typing help topics.
A central principle in the CLI is the context in which commands are executed. The context
dictates which elements of the system can be managed, and which commands are available.
Contexts have a tree structure in which contexts may themselves contain nested contexts and
the structure generally mirrors that of the views in the BUI. Note that the prompt changes to
reflect the context, with the context provided between the colon (:) and the greater-than sign
(>) in the prompt.

Oracle Exalogic Elastic Cloud Administration 6 - 30


Some CLI Commands
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Command Description

help Print a list of commands that can be used in current context.


<context> Navigate to the given context.
select <entity> Navigate to the given entity (project, file system, and so on).
destroy <entity> Delete the given entity.

Oracle University and (Oracle Corporation) use only.


<entity> <name> Create a new entity (for example, project Project1).
show Print a list of properties, configuration entities, and child
contexts in the current context.
get Print all properties of the current entity, or a specific property.
set <prop>=<value> Update the given property of the current entity.
commit/revert Save or undo your modifications.
done Return to the parent context.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Some CLI Commands


Some child contexts are dynamic in that they correspond not to fixed views in the browser, but
rather to dynamic entities that have been created by either the user or the system. To
navigate to these contexts, use the select command followed by the name of the dynamic
context. The names of the dynamic contexts contained within a given context are shown by
using the list command.
Once in a context, execute context-specific commands. For example, to obtain the current
user's preferences, execute the get command from the configuration preferences context. The
set command will set a property to a specified value, with the property name and its value
separated by an equal sign (=).
The text uncommitted in the prompt denotes that this an uncommitted context. An
uncommitted entity is committed via the commit command. Any attempt to navigate away from
the uncommitted context will prompt for confirmation. When committing an uncommitted
entity, the properties associated with the new entity will be validated, and an error will be
generated if the entity cannot be created. For example, the creation of a new threshold alert
requires the specification of a statistic name. Failure to set this results in an error.

Oracle Exalogic Elastic Cloud Administration 6 - 31


CLI Examples
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Create a new project and share:

shares
project CService1
set mountpoint=/export/CService1
set compression=gzip
commit
select CService1

Oracle University and (Oracle Corporation) use only.


filesystem apps
set readonly=true
commit

Monitor appliance health:

status
storage
show
hardware
show

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

CLI Examples
To return to the previous context, use the done command. Similarly, to navigate to a parent
context, use the cd command. Inspired by the classic UNIX command, cd takes an argument
of .. to denote moving to the parent context.

Oracle Exalogic Elastic Cloud Administration 6 - 32


FMW Storage Topology
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle recommends that you:


Configure storage projects:
For each separate FMW installation
To isolate data for different business units or domains
Create project shares for:
Product installations and binaries

Oracle University and (Oracle Corporation) use only.


WebLogic domains
WebLogic Node Manager configurations
WebLogic persistent stores

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

FMW Storage Topology


If your Exalogic machine will host applications that require different versions of FMW
products, Oracle recommends that you create a separate project on the storage appliance for
each version and then make them available to all or a subset of the compute nodes. For most
FMW products, including WLS, compute nodes must be granted only read access to these
projects. Each product installation will be a separately configured share within the project.
Oracle recommends that, whenever possible, you use the shared storage appliance to store
products, configuration files, logs, and applications, and that you not use the compute node's
local flash storage device. A node's local device is intended primarily as a boot device.
The WLS persistent store service provides a built-in, high-performance storage solution for
WLS subsystems that require persistence for recovery and migration purposes. For example,
it can store persistent JMS messages or transaction logs. The persistent store uses a file-
based store by default, but database stores are also supported. Both types can survive a
process crash or hardware power failure without losing any committed changes. To support
automatic failover, these stores must be available to all candidate servers (usually all
members of a cluster) in a domain.

Oracle Exalogic Elastic Cloud Administration 6 - 33


FMW Storage: Example
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

exalogic

FMW11gR1 HR

domains
Middleware

Oracle University and (Oracle Corporation) use only.


/hrdomain
/wlserver_1034
/jrockit_160
nodemanager

recovery
/jms
/tlog

apps

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 6 - 34


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Which of the following is not a task that can be performed with


the appliance BUI?
a. Illuminate an LED.
b. Monitor disk performance.
c. Back up the configuration.
d. Order replacement cables.

Oracle University and (Oracle Corporation) use only.


e. Assign an authorization to a role.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: d

Oracle Exalogic Elastic Cloud Administration 6 - 35


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Which of the following represents the entity hierarchy used by


the appliance?
a. Pool, Project, Share
b. Port, Pool, Share
c. Share, Port, Pool
d. Share, Pool, Project

Oracle University and (Oracle Corporation) use only.


e. Pool, Share, Project

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: a

Oracle Exalogic Elastic Cloud Administration 6 - 36


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Name three services that clients can use to access a shared


file system.
a. FTP
b. NFS
c. SOAP
d. SCSI

Oracle University and (Oracle Corporation) use only.


e. WebDAV

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: a, b, e

Oracle Exalogic Elastic Cloud Administration 6 - 37


Summary
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

In this lesson, you should have learned how to:


List the capabilities of the storage appliance
Perform basic administration and maintenance tasks
Configure appliance access services
Describe Exalogic's default storage setup

Oracle University and (Oracle Corporation) use only.


Create pools, projects, and shares
Mount a shared file system by using NFS
Administer the appliance from the command line
Describe the recommended configuration for FMW

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 6 - 38


Practice 6-1
Initialize Shared File Systems
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

This practice covers the following topics:


Creating storage projects and shares
Configuring NFS access for a share
Mounting shares on a local file system

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 6 - 39


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle University and (Oracle Corporation) use only.


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Network Configuration

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle University and (Oracle Corporation) use only.


Objectives
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

After completing this lesson, you should be able to:


Describe the network topology for various rack
configurations
Identify switch LEDs and replace failed hardware
Access an InfiniBand switch from the browser or command
line

Oracle University and (Oracle Corporation) use only.


Describe the role of the InfiniBand subnet manager
Configure a compute node's network to support FMW

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 7 - 2


Exalogic Networks: Review
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Network Speed Description


Provides access to compute nodes via
Client 10 Gb
intranet/Internet
Is used to monitor and administer components
Management 1 Gb Provides access to ILOM and other device

Oracle University and (Oracle Corporation) use only.


management interfaces
Is a private, nonroutable network
Connects racks, along with all rack components
Private
40 Gb Is used by compute nodes to access shared
InfiniBand
storage
Is used for internal cluster communication

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Exalogic Networks: Review


An Exalogic machine includes compute nodes, a Sun ZFS Storage 7320 appliance, as well as
equipment to connect the compute nodes to your network. The network connections allow the
servers to be administered remotely, enable clients to connect to the compute nodes, and
enable client access to the storage appliance. Additional configuration, such as defining
multiple virtual local area networks (VLANs) or enabling routing, may be required for the
switches to operate properly in your environment and is beyond the scope of the installation
service.
There are up to five networks for an Exalogic machine. Each network must be on a distinct
and separate subnet from the others. The Exalogic management network connects to your
existing management network and is used for administrative work for all components of the
Exalogic machine. It connects ILOM, compute nodes, server heads in the storage appliance,
and switches connected to the Ethernet switch in the Exalogic machine rack. This
management network is in a single subnet. Do not use the management network interface
(ETH0/NET0) on compute nodes for client or application network traffic. Cabling or
configuration changes to these interfaces on Exalogic compute nodes is not permitted.

Oracle Exalogic Elastic Cloud Administration 7 - 3


IB Networking Concepts: Review
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Term Definition Description


IPoIB IP over InfiniBand Applications connected by an IB fabric communicate
using standard IP address semantics.

SDP Socket Direct Protocol Applications communicate directly with the IB fabric,
bypassing the operating system's TCP/IP stack.

Oracle University and (Oracle Corporation) use only.


EoIB Ethernet over Applications within an IB fabric communicate with
InfiniBand external Ethernet networks.
vNIC Virtual Network Software that emulates an Ethernet NIC on the IB
Interface Card network

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

IB Networking Concepts: Review


IP over InfiniBand (IPoIB) enables applications on separate devices to communicate with
each other over a private InfiniBand fabric by using native IB protocols and without the
overhead of Ethernet. For example, a compute node on one Exalogic rack may communicate
with a database on an Exadata rack. However, applications must support the SDP protocol to
utilize IPoIB instead of the default TCP/IP stack of the host operating system.
The InfiniBand switches also act as gateways to connect to external Ethernet networks. They
support eight 10-GB Ethernet ports. Exalogic compute nodes can communicate through these
ports by using Ethernet over InfiniBand (EoIB). Each port is represented on the compute
nodes as a vNIC. This representation enables that node's IB connection to appear like any
other Ethernet NIC to both the operating system and the external Ethernet network.
Up to eight external physical Ethernet networks can be connected to the IB switches. But you
can also define multiple VLANs for each of these Ethernet ports. By default, no VLANs are
configured.

Oracle Exalogic Elastic Cloud Administration 7 - 4


Default Compute Node Network
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Private IB
Network

IPoIB, Compute Node


SDP
IB IB0 bond0

Oracle University and (Oracle Corporation) use only.


Switches IB1 bond1 eth0
EoIB

NET3 NET2 NET1 NET0 MGMT

ILOM Ethernet
Client
Network
Mgmt Mgmt
Network Switch

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Default Compute Node Network


By default, each Exalogic compute node is configured with one bonded EoIB interface for one
external LAN (client network), and is named bond1. The Cisco Ethernet management switch
is connected to the NET0 port of compute nodes, the NET0 port of the storage appliance, and
also the management ports of the InfiniBand gateway switches. On the compute nodes, this
connection is represented on the operating system by an eth network interface, such as
eth0. The compute nodes are configured at the time of manufacturing to use sideband
management only. Therefore, the MGMT (or ILOM) port is not connected, but ILOM is
accessible from NET0.

Oracle Exalogic Elastic Cloud Administration 7 - 5


Default Storage Appliance Network
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Private IB
Network

NFS
Storage Heads
IPoIB IB0
IB
Switches IB1

Oracle University and (Oracle Corporation) use only.


NET3 NET2 NET1 NET0 MGMT

BUI, ILOM Ethernet

Mgmt Mgmt
Network Switch

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Default Storage Appliance Network


The Cisco Ethernet management switch is connected to the NET0 port of the storage
appliance. Each storage nodes ILOM interface is accessible from NET0, along with the
storage appliance browser user interface (BUI). Note that the storage appliance is not
configured as part of the Exalogic client network, and is only intended to be accessed by other
nodes within the same data center.

Oracle Exalogic Elastic Cloud Administration 7 - 6


Quarter-Rack Network Topology
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

10 Gb Network
1-8 1-8
Redundant links 16 unused
10 Gb IB ports
Redundant
switches
6
NM2-GW #1 NM2-GW #2

Oracle University and (Oracle Corporation) use only.


40 Gb

2 Storage 2
Heads 12
IB0 (active) IB1 (passive)
8 Compute 8
Nodes 18

Not shown: mgmt network

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Quarter-Rack Network Topology


NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch (leaf switches). Half of
the compute nodes are connected with their active ports to the first gateway switch and their
passive ports to the second gateway switch. The remaining half of the compute nodes are
connected with their active ports to the second gateway switch and their passive ports to the
first gateway switch. This connection is for high availability and load distribution purposes.
Exalogic machine quarter racks do not contain the Sun Datacenter InfiniBand Switch 36.
Applications running on compute nodes communicate with applications on other compute
nodes using this InfiniBand network. Exalogic machines can be connected to an external
network, including a standard database hosted on a machine outside of the Exalogic
machine, via the InfiniBand-to-10 Gb Ethernet gateways using Ethernet over InfiniBand
(EoIB). Each Exalogic machine configuration includes at least two such gateways, which also
act as InfiniBand switches connecting all compute nodes and the Sun ZFS Storage 7320
appliance within the Exalogic machine.

Oracle Exalogic Elastic Cloud Administration 7 - 7


Half-Rack Network Topology
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

10 Gb Network
1-8 1-8
8 unused
IB ports

6
NM2-GW #1 NM2-GW #2

Oracle University and (Oracle Corporation) use only.


2 Storage 2
Heads 12
IB0 IB1
8 Compute 8
Nodes 18
IB1 IB0
8 Compute 8
Nodes 916

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Half-Rack Network Topology


The Sun Datacenter InfiniBand Switch 36, referred to as a spine switch, is present in a half
rack but not connected in a single rack configuration.

Oracle Exalogic Elastic Cloud Administration 7 - 8


Full-Rack Network Topology
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

10 Gb Network
1-8 1-8 1-8 1-8

4 6 4
NM2-GW #3 NM2-GW #1 NM2-GW #2 NM2-GW #4
2 Storage 2
Heads 1-2
All four switches No unused

Oracle University and (Oracle Corporation) use only.


are interconnected. 8 Compute 8 IB ports
Nodes 18
8 Compute 8
Nodes 916
7 Compute 7
Nodes 1723
7 Compute 7
Nodes 2430

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Full-Rack Network Topology


The spine switch is also present in a full rack but not connected in a single rack configuration.

Oracle Exalogic Elastic Cloud Administration 7 - 9


Multi-Rack Topology
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Refer to the Exalogic Multirack Cabling Guide for details.

Six unused
NM2-36 Full Rack IB ports
4
Four unused
IB ports
NM2-GW NM2-GW

Oracle University and (Oracle Corporation) use only.


NM2-GW NM2-GW 6 6

6 6 NM2-GW NM2-GW
NM2-GW NM2-GW
4

NM2-36
Full Rack

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Multi-Rack Topology
Sun Network QDR InfiniBand Gateway Switch (NM2-GW) 4x4 (four switches in each Exalogic
machine full rack) cross-connects are re-cabled to their respective local spine switches (Sun
Datacenter InfiniBand Switch 36 (NM2-36). In addition, each leaf switch is connected to the
other racks spine switch. For example, leaf switches in rack 1 are connected to the spine
switch in rack 2.
Before you can set up the multi-rack cabling, you must first shut down the affected Exalogic
machines and Oracle Exadata Database Machines. Consider the operational impact of
shutting down the systems, and take appropriate action. Also be careful when you handle
InfiniBand cables. InfiniBand cables are very fragile. The cable ends might break off if the
ends are dropped or mishandled in any way.
A spare cable bundle is provided with Exalogic machines. Extract the spare InfiniBand cables
from the bundle. You need to use the InfiniBand cables only. Restore spare Ethernet cables
to the original spares location. You do not use the Ethernet cables.
Re-label all inter-rack cables from leaf switches to spine switches. Re-label each cable to
identify the correct locations at both cable ends, so that cables can be connected correctly
during future maintenance.
Refer to the Exalogic Machine Multirack Cabling Guide for more detailed instructions.

Oracle Exalogic Elastic Cloud Administration 7 - 10


Transceivers and Cables
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

IB switches utilize transceivers to support different types of


cables and connectors.
Gateway switches also use special splitter cables:
One IB QSFP connector
Four LC SFP+ connectors (fiber-optic)
Multiple 10 Gb connections required to match IB bandwidth

Oracle University and (Oracle Corporation) use only.


Refer to the Exalogic Machine Guide for a list of available
transceivers and cables of various lengths.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Transceivers and Cables


Generally speaking, IB switches support both pass-through and splitter cables, and both
optical and copper. Copper IB cables are typically limited to 5 meters in length, while optical
cables can reach 30 meters or more.
Cable paths should be as short as possible. When the length of a cable path has been
calculated, select the shortest cable to satisfy the length requirement. Also keep in mind that
bending cables and bundling cables together both increase the required lengths of cables.
The gateway switches support the following types of transceivers:
10 Gb SFP+SR (LC MMF connector)
QSFP (MTP MMF connector)
The gateway also supports QSFP optical splitter cables in lengths of 10, 20, and 30 meters.
The QSFP transceivers are installed by default in the gateway switches of your Exalogic
machine. The splitter cables and SFP+ transceivers for 10 Gb data center connectivity should
be ordered separately.

Oracle Exalogic Elastic Cloud Administration 7 - 11


Administration Tasks
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Core administrative tasks for the Exalogic network include the


following:
Determine switch status and health.
Identify and replace failed switch hardware.
Verify the desired IB network topology.
Identify and troubleshoot connectivity issues.

Oracle University and (Oracle Corporation) use only.


Configure the IB subnet manager.
Configure virtual IP addresses on compute nodes.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 7 - 12


Status LEDs
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Front

DC power OK
Power supply fault

AC power OK
Fan fault

Oracle University and (Oracle Corporation) use only.


Link status
Locator

Switch fault

Switch OK

Rear

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Status LEDs
Status LEDs are used on many components of the switch chassis as a means of indicating
the components state. The power supply status LEDs and fan status LEDs are located on the
front of the switch chassis:
The power OK LED is green. When it is on, a 12V DC voltage is present.
The power attention/fault LED is amber. When it is on, a fault has been detected on the
power supply and power has been shut down.
The power AC OK LED is green. When it is on, AC power is present.
The fan attention/fault LED is amber. When it is on, there is a problem with the fan.
The chassis status LEDs, network management status LEDs, and link status LEDs are
located on the rear of the switch chassis:
The switch locator LED is white. When it is flashing, the switch is indentifying itself. The
locator LED can be enabled remotely using the browser or command-line management
interfaces.
The switch attention/fault LED is amber. When it is on, a networking fault has been
detected on the switch.
The switch OK LED is green. When it is off, the switch is off or is initializing. When it is
on, the gateway is functional and no faults are present.

Oracle Exalogic Elastic Cloud Administration 7 - 13


Replacing Failed Hardware
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

See the Sun Network QDR InfiniBand Gateway Switch:


Service Manual for detailed procedures about power
supplies, fans, and so on.
For example, to replace a fan on the switch:
1. Loosen the screw on the top-right side.
2. Use the handle to remove the old fan.

Oracle University and (Oracle Corporation) use only.


3. Insert the new fan and verify that the fault LED turns off.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Replacing Failed Hardware


To replace a switch power supply:
1. Locate the power supply to be removed, using alert or locator LEDs.
2. Press and hold the release tab to the left and pull the handle of the power supply.
Continue to pull the handle of the power supply to remove it from the chassis.
3. Verify that the slot where the power supply installs is clean and free of debris.
4. Orient the power supply with the status LEDs on the left and the release tab on the right.
Slide the power supply into the open slot, pushing on the handle.
5. When the power supply seats, push firmly so that the release tab clicks to secure the
power supply into the chassis.
6. Power on the power supply.

Oracle Exalogic Elastic Cloud Administration 7 - 14


IB Switch Command-Line Interface (CLI)
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Use SSH to connect to a switch via the management


network.
Log in as root/welcome1 and run the password command
to modify the default.
Use the help command to see a list of available
commands.

Oracle University and (Oracle Corporation) use only.


The interface also supports tab completion.

View all commands:

# help all

View hardware diagnostic commands:

# help diag

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

IB Switch Command-Line Interface (CLI)


The management controller uses a simplified Linux OS and file system. After you log in to the
root account, the shell prompt (#) appears, and you can enter shell commands.
The hardware commands are a means of monitoring the switch hardware and working with
gateway ports. Only the root user of the management controller can run all of the hardware
commands
The InfiniBand commands are a means of monitoring and controlling many aspects of the
InfiniBand fabric. These commands are also run from the management controller, which is
also the host of a Subnet Manager internal to the gateway. Only the root user of the
management controller can run the InfiniBand commands.
With the help command, you can either supply the specific name of a command or a class of
commands. The available classes include general, diag, ibdiag, sm, and other.

Oracle Exalogic Elastic Cloud Administration 7 - 15


Monitoring the Hardware
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

# version
SUN DCS gw version: 1.1.2-2
Build time: Sep 24 2010 10:32:29
FPGA version: 0x33
SP board info:
Manufacturing Date: 2010.06.03
Serial Number: "NCD5A0025"
...

Oracle University and (Oracle Corporation) use only.


# showunhealthy
OK - No unhealthy sensors

# env_test
...
Voltage test returned OK
...
Temperature test returned OK
...
FAN test returned OK
...

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Monitoring the Hardware


In time, newer or updated gateway firmware might become available. The version hardware
command shows the hardware and software versions, as well as date information for the
gateway and management controller.
The showunhealthy hardware command shows a list of switch components that appear to
have a problem. Unlike the env_test command, this command displays messages for only
those components that have failed testing.
The env_test hardware command performs a series of hardware and environmental tests of
the gateway. Its output provides voltage and temperature values, pass-fail results, and error
messages. This command is essentially a combination of the following other commands:
checkpower
checkvoltages
showtemps
getfanspeed
connector
checkboot

Oracle Exalogic Elastic Cloud Administration 7 - 16


Monitoring Connection Status
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

# listlinkup Cable present


Connector 0A Not present No cable but no link
...
Connector 5A Present <-> Switch Port 30 down (Enabled)
Connector 6A Present <-> Switch Port 35 down (Enabled)
Connector 7A Present <-> Switch Port 32 up (Enabled)
...
Cable present
Connector 0A-ETH Present
and link is up
Bridge-0 Port 0A-ETH-1 (Bridge-0-2) up (Enabled)

Oracle University and (Oracle Corporation) use only.


Bridge-0 Port 0A-ETH-2 (Bridge-0-2) down (Enabled)
Bridge-0 Port 0A-ETH-3 (Bridge-0-1) down (Enabled)
Bridge-0 Port 0A-ETH-4 (Bridge-0-1) down (Enabled)

Hosts found on Gateway


this IB network splitter cable
# ibhosts
Ca: ... ports 2 "el01cn01 EL-C 192.168.10.8 HCA-1"
Ca: ... ports 2 "el01cn02 EL-C 192.168.10.7 HCA-1"
...

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Monitoring Connection Status


The listlinkup hardware command lists the presence of links and the up-down state of the
associated ports on the switch chip.
The ibhosts IB command is a script that discovers the InfiniBand fabric topology or uses the
existing topology file to extract the channel adapter nodes.
A related IB command is ibportstate, which queries the logical and physical state of a
specific InfiniBand port. The command can return the link width and speed of a switch port, as
well as enabling, disabling, or resetting the port. The command can also set the link speed of
any InfiniBand port.

Oracle Exalogic Elastic Cloud Administration 7 - 17


Running IB Diagnostics
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

# ibdiagnet
Discovering ... 13 nodes (1 Switches & 12 CA-s) discovered.
...
No bad Links (with logical state = INIT) were found
...
Subnet: IPv4 PKey:0x0001 QKey:0x00000b1b MTU:2048Byte ...
No members found for group
...
# ibnetdiscover

Oracle University and (Oracle Corporation) use only.


Topology file: generated on Sat Feb 5 02:31:53 2011
...
Ca2 ... "el01cn0101 EL-C 192.168.10.8 HCA-1"
[1](21280001a0a4b9) "S-002128547ca2c0a0"[18]
# lid 15 lmc 0 "SUN IB QDR GW switch 100.140.50.5" lid 6 4xQDR
...
# ibcheckerrors
...
Summary: 13 nodes checked, 0 bad nodes found
28 ports checked, 0 ports have errors beyond threshold

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Running IB Diagnostics
The ibdiagnet command performs a collection of tests on the InfiniBand fabric by using
directed route packets and extracts all the available information regarding connectivity. The
command also checks for duplicate node/port GUIDs in the InfiniBand fabric along with
suspected bad links, and displays any issues to standard output. It also generates several
output files, including:
ibdiagnet.lst: List of all the nodes, ports, and links in the fabric
ibdiagnet.fdbs: Dump of the unicast forwarding tables of the fabric switches
ibdiagnet.sm: List of all the Subnet Manager (state and priority) in the fabric
The ibnetdiscover command discovers the InfiniBand network topology and generates a
human-readable topology file. It displays all nodes, node types, node descriptions, links, port
numbers, port LIDs, and GUIDs.
The ibcheckerrors command uses the topology file that was previously generated by
ibnetdiscover to scan the InfiniBand fabric, validate the connectivity as described in the
topology file, and report errors as indicated by the port counters.

Oracle Exalogic Elastic Cloud Administration 7 - 18


Subnet Manager (SM) Concepts
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

The IB subnet manager process has these responsibilities:


Discovering and monitoring the network topology
Assigning IDs to each endpoint on the network
Calculating and programming route and partition tables
When multiple switches run the SM, the following occurs:
One switch is designated as the master and the others as

Oracle University and (Oracle Corporation) use only.


standby for failover.
Each switch can be assigned a numeric priority (015).

Switch A Switch B
SM Master SM
Switch A
SM

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Subnet Manager (SM) Concepts


The Exalogic subnet manager (SM) manages all operational characteristics of the InfiniBand
network, such as discovering the network topology and calculating forwarding tables. The
InfiniBand network typically has more than one SM, but only one SM is active at a time. The
active SM is the master SM; others are standby SMs. If the master SM shuts down or fails, a
standby SM automatically becomes the master SM.
If you are connecting a single Exalogic machine to a single Exadata Database Machine,
ensure that an SM runs on all of the InfiniBand switches (leaf switches and spine switches).
You must configure the master SM to run on a spine switch only. If you are connecting
multiple Exalogic machines to one or more Exadata Database Machines, only spine switches
and Top of Rack (ToR) switches should run SM. You must configure the master SM to run on
a ToR switch.

Oracle Exalogic Elastic Cloud Administration 7 - 19


Subnet Manager Recommendations
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Topology Spine Switches Gateway Switches

Stand-alone Rack (default) Not used Enable SM


Set to priority 5
Two Full or Half Racks Enable master SM Disable SM
Set to priority 8

Oracle University and (Oracle Corporation) use only.


One Full/Half + One Quarter Enable SM Enable master SM

Three or More Racks Enable master SM Disable SM


Set to priority 8

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Subnet Manager Recommendations


For a single Exalogic machine, gateway (leaf) switches should run the subnet manager (SM).
You must set the Subnet Manager priority for each of the gateway switches to 5.
For two Exalogic machines, the master SM should run on a spine switch for half-rack/full-rack
configurations and on a leaf switch for configurations involving quarter racks. For
configurations not including a quarter rack, you must set the SM priority on the spine switch to
8. In addition, you must disable the SM on the gateway switches in multi-rack configurations.
For three or more Exalogic machines, the SM should run on the spine switch only. You should
disable SM on other switches. You must set the SM priority on the spine switch to 8.

Oracle Exalogic Elastic Cloud Administration 7 - 20


Configuring the Subnet Manager
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

(Re)start the subnet manager:

# disablesm
# enablesm

View the current master subnet manager:

# getmaster
Local SM enabled and running ...

Oracle University and (Oracle Corporation) use only.


Master SubnetManager on sm lid 6 sm guid ...:
SUN IB QDR GW switch 100.150.50.5

View the current priority:

# setsmpriority list
smpriority 5 ...

Update the priority:

# setsmpriority 8

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Configuring the Subnet Manager


If you want to quickly determine the local Subnet Manager's priority and state, you can use
the sminfo command. A state value of 3 indicates a master SM.
The getmaster command displays the host name or IP address of the switch in the network
where the master SM is running.
You are required to relocate the master SM from a leaf switch to the spine switch when you
are connecting more than one Exalogic machine. Relocating the master SM does not affect
the availability of the InfiniBand network. You can perform this task while normal workload is
running. When you use the disablesm command on a switch that is currently hosting the
master SM, the master SM is automatically relocated to another switch in the network.

Oracle Exalogic Elastic Cloud Administration 7 - 21


Fabric Monitor Interface
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1 Access the switch


from a browser.
Main menu
3

Oracle University and (Oracle Corporation) use only.


Hardware and firmware
specifications
Similar functionality to the CLI

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Fabric Monitor Interface


By using the Fabric Monitor, you can visually monitor the status of the gateway, the switch
hardware, and the connectors through a web-based interface. The Fabric Monitor is
accessible from the ILOM web interface of the switch.
1. Access the ILOM browser interface on the management network. Then log in as the root
account of the switch (root/welcome1, by default).
2. Click the Switch/Fabric Monitoring Tools tab and then click Launch Fabric Monitor.
3. The top portion of the interface displays an interactive graphic representing the rear
LEDs and ports on the switch. The bottom portion of the interface gives detailed
hardware and IB information about the switch. This information is organized into a series
of tabs.
Overall, the browser interface provides equivalent functionality to that of the switch command-
line interface, although it is generally read-only. Hidden on the right side of the Fabric Monitor
is the control panel. Clicking the More link opens the control panel and legend panels. The
control panel has five check boxes with which you can select the types of data that you want
the interface to poll. By default, all data is collected.

Oracle Exalogic Elastic Cloud Administration 7 - 22


Fabric Monitor Examples
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Monitor the
hardware.

Oracle University and (Oracle Corporation) use only.


View the subnet
manager settings.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Fabric Monitor Examples


The System Info tab displays status information regarding the switch hardware, including
basic information about the management controller, firmware version and build date, FRU IDs
for the chassis and power supplies, and status of the power supplies and fans.
The Sensor Info tab displays the latest hardware sensor readings for the switchs power
supplies and fans, including the current voltage and temperature values.
The IB Performance tab displays the status and available bandwidth of the switch ports by
using a table format. By clicking on a column heading, the information in the table is sorted
according to that column heading, either in ascending or descending order.
The IB Port Map tab displays information about peer devices attached to the switch by using a
table format.
The Subnet Manager tab displays the current SM settings for this switch, including whether or
not it is the master SM, along with the priority of the SM.

Oracle Exalogic Elastic Cloud Administration 7 - 23


Monitoring IB Connections
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1
Active link LED status
Not connected with errors
2

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Monitoring IB Connections
The rear panel diagram displays the presence of connectors and their status by using a
graphic. The diagram displays the management controllers IP address, the connector
receptacles, and their respective connector names. When a cable is attached to a receptacle,
a connection is made. That connection is displayed in the diagram as a gray rectangle, with
three or four smaller indicators. Moving the mouse cursor over an indicator, clicking an
indicator, or clicking a connection opens a window that provides additional information about
that indicator or connection.
In the rear panel diagram, there are 32 InfiniBand receptacles displayed, labeled 0A to 15A
and 0B to 15B. When a connector is physically present in an InfiniBand receptacle, the
receptacle changes from a black rectangle to a gray rectangle with three indicators. Moving
the mouse cursor over an indicator that is orange or red opens a small window that provides
the reason for the respective state. A center indicator is orange when the link is at a speed
slower than QDR, such as SDR or DDR. A right indicator is red when there are significant
errors (symbol, recovery, and so on) on the link.
Clicking a gray InfiniBand connector opens a window that displays connector FRU, port state,
error, and statistical information for that connection.

Oracle Exalogic Elastic Cloud Administration 7 - 24


Monitoring Gateway Connections
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Oracle University and (Oracle Corporation) use only.


2

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Monitoring Gateway Connections


The rear panel diagram displays four gateway receptacles, labeled 0A, 1A, 0B, and 1B. When
a connector is physically present in a gateway receptacle, the receptacle changes from a
black rectangle to a gray rectangle with four indicators. Each indicator represents one of the
four possible ports available at the connection. The rectangles left of the gateway connection
are the BX indicators, which display the status of the internal switch hardware connections.
Moving the mouse cursor over a BX indicator opens a small window that provides information
about the BridgeX port. If the indicator is red, then the window displays a reason for the
respective state.
Clicking a gray gateway connector opens a window that displays connector FRU and port
information for that connection. At the top of the window is the connector name. There are two
parts of the window, the cable FRU ID information on the left, and a smaller status pane for
the ports on the right. Clicking a tab displays that ports information, including the physical
state, logical state, protocol, and any configured vNICs.

Oracle Exalogic Elastic Cloud Administration 7 - 25


Multiple Addresses Versus Multiple Ports
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

To run multiple WebLogic Servers on a single compute


node, each type of communication must have a unique
combination of IP/port.
Oracle recommends assigning each server:
Unique floating/virtual IP addresses
The same port numbers

Oracle University and (Oracle Corporation) use only.


This approach is also required when using the server
migration feature.

Machine Machine

192.168.1.1:7003 ServerA OR
192.168.1.1:7003 ServerA
192.168.1.1:7005 ServerB 192.168.1.2:7003 ServerB

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Multiple Addresses Versus Multiple Ports


Using Oracles recommended enterprise deployment configuration, create and assign
individual floating IP addresses for each of the WebLogic Managed Servers and for the
Administration Server. Floating IP addresses are IP addresses that may be re-assigned
between compute nodes in the cluster. For example, if a compute node fails or goes down,
then WebLogic Managed Servers running on one compute node can be migrated to another
compute node. When the WebLogic Node Manager is set up, it manages the registration and
removal of the floating IP addresses. This overall process is referred to as WebLogic whole
server migration.

Oracle Exalogic Elastic Cloud Administration 7 - 26


Configuring a Virtual Address
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Add an IP address to the bond0 (IPoIB) interface:

ifconfig bond0:1 192.168.100.100 netmask 255.255.255.0 up

(or ip addr add 192.168.100.100/24 dev bond0)

Update the routing tables of other hosts on this network:

Oracle University and (Oracle Corporation) use only.


arping U -I bond0 192.168.100.100

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Configuring a Virtual Address


ifconfig [interface] [address] netmask [mask] up/down
arping [ -AbDfhqUV] [ -c count] [ -w deadline] [ -s source] [-I
interface] [destination]
Common uses for ifconfig include setting an interface's IP address and subnet mask, and
disabling or enabling a given interface. At boot time, the operating system initializes its
network interfaces with shell scripts that call ifconfig. Other floating IP addresses can be
dynamically added and removed after the OS has booted, but these will no longer be present
after the server is restarted.
With logical network interfaces, you can configure a physical interface such as eth0 to have
many different IP addresses, even IP addresses that are in different IP classes. Logical
interfaces consist of the physical units name as well as a logical name, separated by a colon.
For example, eth0:1, eth0:2, and so on.
The arping command is used to discover hosts on a network. The program tests whether a
given IP address is in use on the local network, and can get additional information about the
device using that address. It is analogous in function to ping but operates at layer 2 (Link)
instead of layer 3 (Internet). Include the -U option to send a broadcast message by using the
interface specified by the -I option. The -q option disables all command output (quiet). The -
c option indicates the number of packets to attempt to send and wait for a reply.
Oracle Exalogic Elastic Cloud Administration 7 - 27
Network Configuration and Multi-Tenancy
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Optionally configure the Exalogic network to:


Allow a single LAN to appear to be multiple LANs
Isolate traffic for different groups of compute nodes
Restrict the databases and other external systems that
specific compute nodes can access

Oracle University and (Oracle Corporation) use only.


Network Layer Technique

Physical Virtual LAN (VLAN)

Internet IP Subnet Partitioning

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Network Configuration and Multi-Tenancy


VLANs are essentially layer 2 entities, compared with IP subnets, which are layer 3. In an
environment employing VLANs, a one-to-one relationship often exists between VLANs and IP
subnets, although it is possible to have multiple subnets on one VLAN or have one subnet
spread across multiple VLANs.
Creating a VLAN might be necessary on an Exalogic switch if you wish to connect to a
database whose network is already configured with a VLAN. In addition, VLANs might be a
useful technique for isolating database traffic within the data center or across data centers.

Oracle Exalogic Elastic Cloud Administration 7 - 28


Virtual LAN (VLAN)
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

A VLAN:
Groups together physical network connections that are
allowed to communicate with one another
Enables remote hosts to communicate as if they were on a
single LAN
Is registered on a specific port of a switch

Oracle University and (Oracle Corporation) use only.


Is assigned a numeric ID

Switch
Port1: VLAN10
Port2: VLAN10
Port3: VLAN20
Port4: VLAN30

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Virtual LAN (VLAN)


Static VLANs are essentially port-based VLANs. They are created by assigning ports to a
VLAN. As a device enters the network, the device automatically assumes the VLAN of the
port. If the device later changes physical ports and needs access to the same VLAN, the
network administrator must manually make a port-to-VLAN assignment for the new
connection. No traffic is permitted across different VLANs.

Oracle Exalogic Elastic Cloud Administration 7 - 29


Configuring a VLAN
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Display all VLANs on this switch:

# showvlan

Connector VLN PKEY


-----------------------
1A-ETH-2 3 1234

Oracle University and (Oracle Corporation) use only.


Assign a VLAN to a switch port:

# createvlan 1A-ETH-3 -VLAN 3 PKEY default

Switch VLAN ID IB partition ID


connector ID

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Configuring a VLAN
You can use the showvlan command to identify which VLANs are associated with the IB
gateway 10 Gb Ethernet connectors. The command lists the connectors on the left, and the
associated VLANs and respective partition keys on the right.
When you create a VLAN, you create a mapping between a gateway connector, a VLAN
identifier, and a partition key (0000-7fff). By default on Exalogic only one partition is available,
so only one partition key is valid. You can create VLANs with the createvlan command. If a
VNIC will be associated to a VLAN, you must create the VLAN before the VNIC.

Oracle Exalogic Elastic Cloud Administration 7 - 30


Subnet Partitioning
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

IP subnets:
Enable hosts on the same local network to be isolated
from one another
Are configured and enforced at the OS level by using
subnet masks

Oracle University and (Oracle Corporation) use only.


Exalogic

NodeA NodeB NodeC


192.168.1.1 192.168.1.2 192.168.1.10
255.255.255.248 255.255.255.248 255.255.255.248

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Subnet Partitioning
When planning an Exalogic deployment across a collection of compute nodes, use a subnet
mask that provides a suitable IP address range to cover all of these servers on the target
group of compute nodes. In some scenarios, the application from one department may require
communication with the application of another department, each of which is hosted on
separate compute nodes. To enable the applications to communicate, you must set up
another IP subnet in which all target compute nodes are members.

Oracle Exalogic Elastic Cloud Administration 7 - 31


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

How many InfiniBand switches are included in a full rack?


a. Two
b. Three
c. Five
d. Six

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: c

Oracle Exalogic Elastic Cloud Administration 7 - 32


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Which of the following is not a task that can be performed with


the switch CLI?
a. Check LED status.
b. Configure a VLAN.
c. Create a virtual IP.
d. Set the SM priority.

Oracle University and (Oracle Corporation) use only.


e. View the network topology.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: c

Oracle Exalogic Elastic Cloud Administration 7 - 33


Quiz
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

How many physical Ethernet connections does a single IB


gateway switch support?
a. Two
b. Four
c. Six
d. Eight

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Answer: d

Oracle Exalogic Elastic Cloud Administration 7 - 34


Summary
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

In this lesson, you should have learned how to:


Describe the network topology for various rack
configurations
Identify switch LEDs and replace failed hardware
Access an InfiniBand switch from the browser or command
line

Oracle University and (Oracle Corporation) use only.


Describe the role of the InfiniBand subnet manager
Configure a compute node's network to support FMW

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 7 - 35


Practice 7-1
Configure Network Settings
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

This practice covers the following topics:


Locating your assigned virtual IP addresses
Configuring virtual IP addresses on a compute node

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 7 - 36


These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Creating FMW Domains

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle University and (Oracle Corporation) use only.


Objectives
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

After completing this lesson, you should be able to:


List the tasks involved in creating a domain
Install WebLogic Server on Exalogic
Create a WebLogic domain on Exalogic
Create and share domains from the command line

Oracle University and (Oracle Corporation) use only.


Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Oracle Exalogic Elastic Cloud Administration 8 - 2


WebLogic Concepts: Review
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Machine Machine Machine

/domainB /domainB
/domainA /domainA
Admin
Server
Server Server
Admin
Server Server Server

Oracle University and (Oracle Corporation) use only.


Server
Cluster Cluster

Node Node
Node Manager
Manager Manager

Product Installation

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WebLogic Concepts: Review


An Oracle WebLogic Server administration domain is a logically related group of Oracle
WebLogic Server resources. Domains include a special Oracle WebLogic Server instance
called the Administration Server, which is the central point from which you configure and
manage all resources in the domain. Usually, you configure a domain to include additional
Oracle WebLogic Server instances called managed servers. You deploy web applications,
EJBs, web services, and other resources onto the managed servers and use the
Administration Server for configuration and management purposes only.
You can use a single Oracle WebLogic Server installation to create and run multiple domains,
or you can use multiple installations to run a single domain. How you organize your Oracle
WebLogic Server installations into domains depends on your business needs. You can define
multiple domains based on different system administrators responsibilities, application
boundaries, or geographical locations of the machines on which servers run. Conversely, you
might decide to use a single domain to centralize all Oracle WebLogic Server administration
activities.
For development or test environments, you can create a simple domain that consists of a
single server instance. This single instance acts as an Administration Server and hosts the
applications that you are developing.

Oracle Exalogic Elastic Cloud Administration 8 - 3


WebLogic Communication Types
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Type Description Preferred Network


Web (HTTP) Client Access to web or web service Client or Private
applications (internal web service)
Java (T3) Client Access to JMS or EJB applications Client or Private
Admin Interactions between admin and Private
managed servers in a domain

Oracle University and (Oracle Corporation) use only.


Admin Console Access to the WebLogic Client
administration console
Cluster Heartbeat Periodic messages between cluster Private
members to determine availability
Cluster Replication Exchange of user data between Private
cluster members for high availability
Database Outgoing JDBC communication Client or Private
(Exadata)

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

WebLogic Communication Types


If you want to expose business logic in a standard way to other applications, both within your
organization and beyond, use a web service application. XML-based web services can be
accessed by both Java and non-Java applications and are also a cornerstone for service-
oriented applications (SOA). If the web service is accessible by other remote systems and
external partners, it must be made available on the Exalogic client network. If instead the web
service is simply a reusable business logic component that is accessed by other applications
within the same data center, the Exalogic private IB network may be a better candidate.
WebLogic applications often communicate with external database systems. In Java EE, this is
accomplished using Java Database Connectivity (JDBC). Most databases are hosted outside
of Exalogic and, therefore, outside of the private IB network. In these cases, WebLogics
JDBC configuration settings would specify the Exalogic client network. However, if your
database is hosted on an Oracle Exadata machine and is connected to the same IB network
as your Exalogic machine, JDBC communication should use that network instead.

Oracle Exalogic Elastic Cloud Administration 7 - 4


Installation and Configuration Process
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

1. Run the WLS installer and place it onto shared storage.


2. Run other FMW installers if necessary.
3. Run the FMW Configuration Wizard to define a new
domain:
a) Select the appropriate template or templates.
b) Place the domain onto shared storage.

Oracle University and (Oracle Corporation) use only.


c) Define initial servers, clusters, and machines.
d) Use bond0 (private) addresses for default server
communication.
4. Start the administration server for the domain.
5. Use the WLS console or scripting tool to make additional
modifications.

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Installation and Configuration Process


Install the Oracle WebLogic product binaries on one of the shares in the Sun ZFS Storage
7320 appliance. The share must be accessible by all compute nodes that are going to run this
version of WebLogic.
Ensure that you do not run the Oracle WebLogic installer as the root user. Instead, log in as a
non-root user who has write privileges to the appropriate shares mounted on the Sun ZFS
Storage 7320 appliance. Similarly, do not create domains or run server processes as root.
The product installation includes a set of predefined domain and extension templates. This
set includes the base WebLogic Server domain template and various extension templates that
enable you to add component features and samples to the base domain.
The WebLogic web-based administration console runs on a domains administration server.
Using this tool, you can dynamically create or delete servers, modify the settings of existing
servers, add servers to a cluster, or remove servers from a cluster. You can also use the
console to create JDBC database connections and deploy Java EE applications.

Oracle Exalogic Elastic Cloud Administration 8 - 5


Shared or Dedicated Installations?
These eKit materials are to be used ONLY by you for the express purpose SELF STUDY. SHARING THE FILE IS STRICTLY PROHIBITED.

Sharing a single FMW installation across domains:


Requires fewer product binaries/patches to maintain
Enforces consistency throughout the organization
Dedicating an installation for each domain:
Provides greater management flexibility
Enables you to maintain product binaries/patches

Oracle University and (Oracle Corporation) use only.


independently and on different schedules

Domain 1
FMW Installation 1
Domain 2
FMW Installation