Exadata 2014 Implementation Specialist Bootcamp Hands v3

Exadata 2014 Implementation Specialist Boot Camp
Exadata 2014 Implementation Specialist Bootcamp

Exercises
Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any
material, code, or functionality, and should not be relied upon in making purchasing decisions.
The development, release, and timing of any features or functionality described for Oracles
products remains at the sole discretion of Oracle.
Oracle Training Materials Usage Agreement

Use of this Site (Site) or Materials constitutes agreement with the following terms and conditions:
1. Oracle Corporation (Oracle) is pleased to allow its business partner (Partner) to download and copy the
information, documents, and the online training courses (collectively, Materials") found on this Site. The use of the
Materials is restricted to the non-commercial, internal training of the Partners employees only. The Materials may not
be used for training, promotion, or sales to customers or other partners or third parties.
2. All the Materials are trademarks of Oracle and are proprietary information of Oracle. Partner or other third party at
no time has any right to resell, redistribute or create derivative works from the Materials.
3. Oracle disclaims any warranties or representations as to the accuracy or completeness of any Materials. Materials
are provided "as is" without warranty of any kind, either express or implied, including without limitation warranties of
merchantability, fitness for a particular purpose, and non-infringement.
4. Under no circumstances shall Oracle or the Oracle Authorized Delivery Partner be liable for any loss, damage,
liability or expense incurred or suffered which is claimed to have resulted from use of this Site of Materials. As a
condition of use of the Materials, Partner agrees to indemnify Oracle from and against any and all actions, claims,
losses, damages, liabilities and expenses (including reasonable attorneys' fees) arising out of Partners use of the
Materials.
5. Reference materials including but not limited to those identified in the Boot Camp manifest cannot be redistributed in
any format without Oracle written consent.
For Oracle employees and authorized partners only. Do not distribute to third parties.
2014 Oracle Corporation Proprietary and Confidential Page 1 of 87
Table of Contents
CONNECT TO THE ENVIRONMENT 4
TASK 1: CONNECT TO THE ENVIRONMENT: 5
TASK 2: CONNECT TO THE EXADATA STORAGE CELLS 6
TASK 3: CHECK CELL ENVIRONMENT 7
CREATE DISKS AND ASM DISKGROUPS 9

TASK 1: CREATE CELL DISKS 9
TASK 2: CREATE GRID DISKS AND PARTITION THEM 12
CREATE DATABASE 17
TASK 1: CONNECT AND VERIFY THE CELLS TO THE DATABASE ENVIRONMENT 17
TASK 2: CREATE ASM DISKGROUPS 19
TASK 3: CREATE A DATABASE USING THE EXADATA STORAGE CELLS (ASM) 25
SMART SCAN 36
TASK 1: LOAD DATA INTO THE DATABASE 36
TASK 2: SMART SCAN AND QUERY OFFLOAD 37
BULK LOADING LAB 42

TASK 1: SETUP A DATABASE FILESYSTEM 42
TASK 2: MOUNT THE DATABASE FILESYSTEM 44
TASK 3: LOAD DATA INTO EXTERNAL TABLES 45
TASK 4: UNMOUNT THE DATABASE FILESYSTEM 49
COMPRESSION 50
TASK 1: COMPRESSION 50
REPROVISION DISKS 52
TASK 1: VERIFY DISK SETUP 52
TASK 2: DROP GRIDDISKS 53
TASK 3: RECREATE GRIDDISKS 54
DCLI AND OTHER MONITORING 56

TASK 1: WORK WITH METRICS IN THE EXADATA STORAGE CELL 56
TASK 2: DCLI 60
IORM 62
TASK 1: IO RESOURCE MANAGER (IORM) 62
SUPPLEMENTAL EXERCISES 66
TASK 1: SECURITY 66
TASK 2: FLASH CACHE 75
TASK 3: STORAGE INDEXES 76

TASK 4: INVISIBLE INDEXES 82
TROUBLESHOOTING 86
CONNECTING TO VNC (MAC) 86
SSH ISSUES 86
FIXING IP ISSUES 86
CELLS 86
DBS 87
CELLDISK ERROR 87
RESTART CELLD PROCESS 87
HOW TO START ASM IF ITS DOWN 87
Lab A Connect to the Environment

Objective: Verify connection and setup and create a database 30 minutes
Instructions:
In this practice you will learn how to use the laboratory environment that supports all the practices in this boot camp.
You will also create a database that uses the Exadata storage cells.
The laboratory environment for this course consists of three virtual machines configured to provide a database server
and two Exadata storage cells. To access the three virtual machines you will establish a VNC session to the database
server. From there you will create terminal sessions as required to connect to the Exadata storage cells.
Database Server (db)
Exadata Storage Cell (cell1)

Exadata Storage Cell (cell2)
Assumptions:
You must have been provided with the IP address for the hands on systems by your instructor. Fill in the
tables below with the IP addresses.
You have installed VNC viewer (or equivalent software) on your laptop or the provided machine.
Task 1: Connect to the environment:

Connect your laptop (or provided machine) to the network supplied by your instructor. The instructor will provide you
with the IP address for your storage cells. The storage cells can also be accessed using their hostnames from the
database server. Fill in the IP Address information in the table below.
Depending on the location of the boot camp you may have to connect to the student network using SSL enterprise
VPN solution. The instructor will provide more detailed information if necessary.
Array SSL Username Password
Hostname IP address
db
cell1
cell2
Username Password
root oracle
celladmin oracle
oracle oracle
Task 2: Connect to the Exadata Storage Cells

1. Open a terminal session in the VNC session to the database server. To connect to the server start the already
installed VNC client and connect to the IP address using port 50 (screen size 1024x768) or port 51 (screen
size 1400x1050).
2. You are now connected as the oracle user on the db node.

3. Execute the following commands to check their availability. Note that you may see additional messages
relating to server identities. Answer yes if you are prompted to acknowledge server authenticity. The password
for the celladmin user is oracle.
[oracle@db ~]$ ssh celladmin@cell1 date
celladmin@cell1's password:
Wed Sep 28 11:07:09 EDT 2011
[oracle@db ~]$ ssh celladmin@cell2 date
Wed Sep 28 11:07:18 EDT 2011
As a result you should see the result of the date command on the screen. The date will be different than what
shown above.
Task 3: Check Cell Environment

1. Using the terminal session establish a connection to your first Exadata cell (cell1) using the celladmin
user (password oracle). Confirm that you are connected to the cell and execute the following cellcli
commands and then exit the session
[oracle@db ~]$ ssh celladmin@cell1
Last login: Tue Sep 13 12:18:31 2011 from db.exadata.local
[celladmin@cell1 ~]$ cellcli -e list cell
cell1 online
[celladmin@cell1 ~]$ cellcli -e list cell detail
name: cell1
bmcType: absent
cellVersion: OSS_11.2.0.1.0_LINUX.X64_091102
cpuCount: 1
fanCount: 1/1
fanStatus: normal
id: f8d9b4d1-6d33-4a60-87cb-992f5e6c3cce
interconnectCount: 1
interconnect1: eth0
iormBoost: 0.0
ipaddress1: 10.216.212.38/24
kernelVersion: 2.6.18-164.9.1.0.1.el5xen
makeModel: Fake hardware
metricHistoryDays: 7
offloadEfficiency: 1.0
powerCount: 1/1
powerStatus: normal
status: online
temperatureReading: 0.0
temperatureStatus: normal
upTime: 36 days, 22:01
cellsrvStatus: running
msStatus: running
rsStatus: running
[celladmin@cell1 ~]$ exit
Connection to cell1 closed.
[oracle@db ~]$
2. Using the terminal session establish a connection to your second Exadata cell (cell2) using the celladmin
user. Confirm that you are connected to the cell and execute the following cellcli commands and then exit
the session
Last login: Tue Sep 13 12:21:36 2011 from db.exadata.local
[celladmin@cell2 ~]$ cellcli -e list cell
cell2 online
[celladmin@cell2 ~]$ cellcli -e list cell detail
name: cell2
bmcType: absent
cellVersion: OSS_11.2.0.1.0_LINUX.X64_091102
cpuCount: 1
fanCount: 1/1
fanStatus: normal
id: c679d044-1f81-4e1e-8dc3-eecd31099645
interconnectCount: 1
interconnect1: eth0
iormBoost: 0.0
ipaddress1: 10.216.212.39/24
kernelVersion: 2.6.18-164.9.1.0.1.el5xen
makeModel: Fake hardware
metricHistoryDays: 7
offloadEfficiency: 1.0
powerCount: 1/1
powerStatus: normal
status: online
temperatureReading: 0.0
temperatureStatus: normal
upTime: 36 days, 22:05
cellsrvStatus: running
msStatus: running
rsStatus: running
[celladmin@cell2 ~]$ exit
[oracle@db ~]$
You should now have a general feeling for the cellcli command environment. In the next chapters we will go a bit
deeper into cellcli commands and start building out our Exadata environment. We will also use the cellcli
command shell.
Do not proceed to the next section until you have clearance from
your instructor.
Lab B Create Disks and ASM Diskgroups

Objective: Verify connection and setup and create a database 30 minutes
Task 1: Create cell disks

Your two cells currently consist of 12 physical disks. This task will create 12 cell disks on both Exadata storage cells.
1. Establish a terminal connection to the cell1 Exadata cell as the celladmin user.
2. Launch the Exadata cell command-line interface (CellCLI) using the command cellcli. The Efficiency ratio
displayed is the ratio between data accessed by the storage cell and data processed by the database for the
cell. Larger values mean more offloading to the cell. The value will be different in your environment.
Last login: Wed Sep 28 12:25:29 2011 from db.exadata.local
[celladmin@cell1 ~]$ cellcli
CellCLI: Release 11.2.1.2.0 - Production on Wed Sep 28 12:30:08 EDT 2011
Copyright (c) 2007, 2009, Oracle. All rights reserved.

Cell Efficiency Ratio: 1
CellCLI>
3. Before we can use the Exadata storage cell disks, we need to create celldisks (stamp the disks with Exadata
information for example) and Griddisks (partitioning, naming etc.). First we create the celldisks.
An Exadata Storage cell contains 12 hard disks. Typically, cell disks and grid disks are created on each hard
disk so that data can be spread evenly across the cell. Use the following command to list the LUNs (disks) that
currently do not contain cell disks. The hard disk is specified because we only want hard disks and not flash
disks displayed.
CellCLI> list lun where disktype = harddisk
/opt/oracle/cell11.2.1.2.0_LINUX.X64_091102/disks/raw/disk01
/opt/oracle/cell11.2.1.2.0_LINUX.X64_091102/disks/raw/disk01 normal
CellCLI>
4. Create cell disks on all available hard disks; that is, the hard disks that dont already contain cell disks. We
specify the hard disk section because we only want hard disks (not flash disks) to be used in these celldisks.
CellCLI> create celldisk all harddisk
CellDisk CD_disk01_cell1 successfully created
CellCLI>
5. Exit your CellCLI session but keep your terminal session open.
CellCLI> exit
quitting
[celladmin@cell1 ~]$
6. Establish a new terminal connection to the cell2 Exadata cell as the celladmin user.

CellCLI>
8. Before we can use the Exadata storage cell disks, we need to create cell disks (stamp the disks with Exadata
information for example) and Griddisks (partitioning, naming etc.). First we create the cell disks.
An Exadata Storage cell contains 12 hard disks. Typically, cell disks and grid disks are created on each hard
disk so that data can be spread evenly across the cell. Use the following command to list the LUNs (disks) that
currently do not contain cell disks. The hard disk is specified because we only want hard disks and not flash
disks displayed.
CellCLI> list lun where disktype = harddisk

CellCLI>
9. Create cell disks on all available hard disks; that is, the hard disks that dont already contain cell disks. We
specify the hard disk section because we only want hard disks (not flash disks) to be used in these celldisks.
CellCLI>
Note that the default cell disk naming convention used in this virtualized environment is slightly different to a native
Exadata cell.
Task 2: Create Grid disks and partition them
Now that the cell disks have been created, you will create two Griddisks on each of the 24 cell disks.
1. Establish a terminal connection to the cell1 Exadata cell as the celladmin user.

CellCLI>
3. The official guideline is to have two partitions on every griddisk. The fastest partition for the production data
and the slowest partition for archive logging etc. When you are creating the Griddisk, you can specify sizes
and naming. Note that the first Griddisk you create will be on the fastest part of the disk. The second will be on
a slower part and so on.
From the CellCLI, list the celldisks with free space.
CellCLI> list celldisk attributes name,freespace where freespace != 0
CD_disk01_cell1 448M
CellCLI>
4. Create a set of grid disks on all the available hard disk based cell disks. Set the size of each grid disk to
250MB.
CellCLI> create griddisk all harddisk prefix=data, size=250M
GridDisk data_CD_disk01_cell1 successfully created
CellCLI>
5. Create another set of grid disks on all the available hard disk based cell disks, This time do not set a size for
these grid disks so they can consume all of the remaining unallocated free space.
CellCLI> create griddisk all harddisk prefix=archive
GridDisk archive_CD_disk01_cell1 successfully created
CellCLI>
6. List all the grid disks and some specific attributes. Can you see the grid disks you created? How big are they?
The value UNUSED for ASMModeStatus indicates that the newly created grid disks have not yet been
consumed by ASM.
CellCLI> list griddisk attributes name, size, ASMModeStatus
archive_CD_disk01_cell1 208M UNUSED

data_CD_disk01_cell1 240M UNUSED
CellCLI>
7. Exit your CellCLI session but keep your terminal session open. You should remain connected to cell1 as the
celladmin user.
CellCLI> exit
quitting
8. We have to do the same steps on the other Exadata cell. Establish a terminal connection to the cell2
Exadata cell as the celladmin user.
9. Launch the Exadata cell command-line interface (CellCLI) using the cellcli command. The Efficiency ratio

CellCLI>
10. The official guideline is to have two partitions on every griddisk. The fastest partition for the production data
and the slowest partition for archive logging etc. When you are creating the Griddisk, you can specify sizes
and naming. Note that the first Griddisk you create will be on the fastest part of the disk. The second will be on
a slower part and so on.
From the CellCLI, list the celldisks with free space.
CellCLI> list celldisk attributes name, freespace where freespace != 0

CellCLI>
11. Create a set of grid disks on all the available hard disk based cell disks. Set the size of each grid disk to
250MB.
CellCLI> create griddisk all harddisk prefix=data, size=250M
CellCLI>
12. Create another set of grid disks on all the available hard disk based cell disks, This time do not set a size for
these grid disks so they can consume all of the remaining unallocated free space.
CellCLI> create griddisk all harddisk prefix=archive
CellCLI>
13. List all the grid disks and some specific attributes. Can you see the grid disks you created? How big are they?
The value UNUSED for ASMModeStatus indicates that the newly created grid disks have not yet been
consumed by ASM.
CellCLI> list griddisk attributes name, size, ASMMOdeStatus
CellCLI>
14. Exit your CellCLI session but keep your terminal session open. You should remain connected to cell2 as the
celladmin user.
CellCLI> exit
quitting
,,,,,,
24
your instructor.
Lab C Create Database

Objective: Create a database on new Exadata ASM diskgroups 60 minutes
Task 1: Connect and verify the cells to the database environment

1. Start a new terminal session (or reuse the one connected to the db host) and view the file
/etc/oracle/cell/network-config/cellinit.ora. The cellinit.ora file contains the database
server IP address that connects to the storage network. This file is host specific, so the IP Address will be
specific to each database server. The IP address is specified in Classless Inter-Domain Routing (CDR) format.
Verify this by looking into the /etc/hosts file.
[oracle@db ~]$ cat /etc/oracle/cell/network-config/cellinit.ora
ipaddress1=10.216.212.37/24
[oracle@db ~]$ grep db /etc/hosts
10.216.212.37 db.exadata.local db
[oracle@db ~]$
2. View the file /etc/oracle/cell/network-config/cellip.ora. The cellip.ora file contains the IP
addresses (or hosts) that are used by storage cells to send data to the database server host. The IP
addresses correspond to the bonded InfiniBand interface on the cells however in our virtualized environment
that is not true. Verify that this file have the correct entries in the format of cell=cellX and that there are
two entries present.
[oracle@db ~]$ cat /etc/oracle/cell/network-config/cellip.ora
cell="cell1"
cell="cell2"
[oracle@db ~]$
3. Verify that the ASM instance is running on the db node by executing the command
ps ef | grep +ASM
[oracle@db ~]$ ps -ef | grep +ASM
oracle 2545 1 0 Oct26 ? 00:00:00 asm_pmon_+ASM
oracle 2547 1 0 Oct26 ? 00:00:00 asm_vktm_+ASM
oracle 2551 1 0 Oct26 ? 00:00:00 asm_gen0_+ASM
oracle 2553 1 0 Oct26 ? 00:00:00 asm_diag_+ASM
oracle 2555 1 0 Oct26 ? 00:00:00 asm_psp0_+ASM
oracle 2557 1 0 Oct26 ? 00:00:01 asm_dia0_+ASM
oracle 2559 1 0 Oct26 ? 00:00:00 asm_mman_+ASM
oracle 2561 1 0 Oct26 ? 00:00:00 asm_dbw0_+ASM
oracle 2563 1 0 Oct26 ? 00:00:00 asm_lgwr_+ASM
oracle 2565 1 0 Oct26 ? 00:00:00 asm_ckpt_+ASM
oracle 2567 1 0 Oct26 ? 00:00:00 asm_smon_+ASM
oracle 2569 1 0 Oct26 ? 00:00:00 asm_rbal_+ASM
oracle 2571 1 0 Oct26 ? 00:00:00 asm_gmon_+ASM
oracle 2573 1 0 Oct26 ? 00:00:00 asm_mmon_+ASM

oracle 2575 1 0 Oct26 ? 00:00:00 asm_mmnl_+ASM
oracle 2577 1 0 Oct26 ? 00:00:00 asm_xdmg_+ASM
oracle 2585 1 0 Oct26 ? 00:00:00 asm_dskm_+ASM
oracle 13785 13765 0 11:43 pts/2 00:00:00 grep +ASM
[oracle@db ~]$
4. Set the environment variables to reflect the +ASM Instance already running. This has to be done as the
oracle user.
[oracle@db ~]$ . oraenv
ORACLE_SID = [oracle] ? +ASM
The Oracle base for ORACLE_HOME=/u01/oracle/product/11.2.0/grid is /u01/oracle
5. Verify that the cells are visible using the kfod command.
[oracle@db ~]$ kfod
-------------------------------------------------------------------------
Disk Size Path User Group
=========================================================================
1: 208 Mb o/cell1/archive_CD_disk01_cell1 <unknown> <unknown>
13: 240 Mb o/cell1/data_CD_disk01_cell1 <unknown> <unknown>



--------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
==========================================================================
+ASM /u01/oracle/product/11.2.0/grid
[oracle@db ~]$
Task 2: Create ASM diskgroups
1. The ASM Instance has already been installed for you. You need to create 2 diskgroups based on the grid
disks you have created in earlier tasks. Set the environment variables to the GRID_HOME.
ORACLE_SID = [+ASM] ? +ASM
[oracle@db ~]$
2. There are 2 ways of creating a diskgroup for ASM. One is using the GUI asmca, one is using SQL*Plus. First
we create a diskgroup using SQL*Plus. Login to the ASM environment:

[oracle@db ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.1.0 Production on Wed Sep 28 14:24:45 2011
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Automatic Storage Management option
SQL>
3. Verify that all disks are visible to ASM. You should be able to see all disks offered by the Exadata storage cell.
Note the path as this will be used when we create the ASM diskgroups.
SQL> SELECT path, label, header_status FROM v$asm_disk ORDER BY path;
PATH LABEL HEADER_STATU

----------------------------------- ------------------------- ---------
o/cell1/archive_CD_disk01_cell1 ARCHIVE_CD_DISK01_CELL1 CANDIDATE
o/cell1/data_CD_disk01_cell1 DATA_CD_DISK01_CELL1 CANDIDATE



48 rows selected.
SQL>
4. Create a new ASM diskgroup consuming the hard drive based grid disks. Take care to use the correct DISK
search string so that you consume the correct disks. This step can also be done using the asmca utility, this is
the command line way of creating the diskgroup. Be patient, this might take a short while. (You can also
execute the following script: /home/oracle/scripts/7-4.sql if you dont want to type in the statement
below).
SQL> CREATE diskgroup DATA normal redundancy disk 'o/*/data*'
attribute 'compatible.rdbms' = '11.2.0.0.0',
'compatible.asm' = '11.2.0.0.0',
'cell.smart_scan_capable' = 'TRUE',
'au_size' = '4M';
Diskgroup created
SQL>
5. Execute the following query (or execute the SQL script /home/oracle/scripts/7-5.sql) to check if the
diskgroup has been created properly in ASM. Pay special attention to the failgroups and note how the grid
disks from each different Exadata cell are automatically grouped into separate failure groups.
SQL> SELECT dg.name,
d.name,
d.failgroup,
d.state
FROM v$asm_disk d,
v$asm_diskgroup dg
WHERE d.group_number = dg.group_number
AND dg.name = 'DATA';
NAME NAME FAILGROUP STATE

------------------------- ------------------------------ --------- -------
DATA DATA_CD_DISK07_CELL2 CELL2 NORMAL

24 rows selected.
SQL>
6. Examine your newly created diskgroup using the following query

SQL> SELECT name, total_mb, state FROM v$asm_diskgroup WHERE name = 'DATA';
NAME TOTAL_MB STATE

------------------------------ ---------- -----------
DATA 5760 MOUNTED
SQL>
7. Exit your SQL*plus session.
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit
Production
[oracle@db ~]$
8. Start the GUI for ASM using the asmca command.
[oracle@db ~]$ asmca
9. You will see a similar screen. It shows you the diskgroup DATA that you already have created.
10. Click Create to create a new ASM diskgroup
11. Name the diskgroup ARCHIVE and keep the Normal Redundancy (default). Click on Show Eligible and click
on the box next to Disk Path. Please note the Failure Group information; it is by default filled with the name of
the cell.
12. Click on the Show Advanced Options Button and change the Database Compatibility to11.2.0.0.0. And then
click OK.
13. Your diskgroup will now be created. Be patient, this can take a little while.
14. Click OK. Now are presented with a screen that shows you both the diskgroups you have created; DATA and
ARCHIVE.
15. Exit the ASM GUI tool.
Task 3: Create a database using the Exadata Storage Cells (ASM)

We are going to create a databse using the 2 new diskgroups (DATA and ARCHIVE) created earlier. The tablespaces
will reside in the +DATA ASM diskgroup, the Flash Recovery will reside in the +ARCHIVE ASM diskgroup.
1. Establish a terminal session to the db host as the oracle user and set the environment variables correctly.
Note than the input to the ORACLE_SID is dbhome_1 because there is an entry in /etc/oratab that is named
dbhome_1 which points to the correct ORACLE_HOME. IMPORTANT: Using ORCL will not work.
ORACLE_SID = [+ASM] ? dbhome_1
The Oracle base for ORACLE_HOME=/u01/oracle/product/11.2.0/dbhome_1 is /u01/oracle
[oracle@db ~]$
2. Start the Database Creation Assistant (dbca)
[oracle@db ~]$ dbca
3. You will be presented with the following screen. Click Next to start the assistant.
4. Select the Create Database option and click Next.
5. Select the Data Warehosue template and click Next.
6. Name the database ORCL and click Next.
7. Accept the default values and click Next.
8. Select Use the Same Administrative Password for All Accounts and use the password Oracle99.
9. Select Automatic Storage Management (ASM) from the Storage Type dropdown list. Click the Browse button
and select +DATA. Click next and enter the password oracle for the ASMSNMP credentials.
A pop-up window will be shown asking for the ASMSNMP password. Fill in the dialog box with the password oracle.
10. Specify the Flash Recovery Area (+ARCHIVE) by clicking on the Browse button and change the size to 2100 M
Bytes. Do not enable archiving (this is a must in a production environment but not in our lab environment).
11. Ignore the warning message about the size of the Flash Recovery Area by clicking yes.
12. Check the Sample Schemas check box and click next.
13. Change the Memory Size (SGA and PGA) to 400MB and click Next.
14. Accept the default values and click Next.
15. Check the Create Database checkbox and click Finish.
16. Verify that the configuration looks correct and click Ok.
17. Database creation starts. This can take about 20 minutes depending on the load of the test environment we
are using.
18. Make sure that the database is working by creating a new user and a table. The database you just created is
now using ASM and Exadata Storage Cells.
19. When dbca is finished click Exit.
20. Set the environment variables to the Grid (ASM) installation

ORACLE_SID = [dbhome_1] ? +ASM
21. Execute the following command and verify that all of the listed series are online on db.
[oracle@db ~]$ crsctl stat res -w "TARGET = ONLINE" -t

--------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------
ora.ARCHIVE.dg
ONLINE ONLINE db
ora.DATA.dg
ONLINE ONLINE db
ora.LISTENER.lsnr
ONLINE ONLINE db
ora.asm
ONLINE ONLINE db Started
--------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE db
ora.diskmon
1 ONLINE ONLINE db
ora.orcl.db
1 ONLINE ONLINE db Open
[oracle@db ~]$
Do not proceed to the next section until you have clearance from your
instructor.
Lab D Smart Scan

Objective: Use Smart Scan to access data in Exadata 20 minutes
Task 1: Load data into the database

1. Connect to the database using SQL*Plus as the SYS or SYSTEM user (password Oracle99).

ORACLE_SID = [+ASM] ? ORCL
[oracle@db ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.1.0 Production on Tue Oct 11 11:18:47 2011
Connected to:
With the Partitioning, Automatic Storage Management, OLAP, Data Mining
and Real Application Testing options
SQL>
2. Create a tablespace that can hold the data we will load
SQL> CREATE TABLESPACE tbs_1 datafile '+DATA';
Tablespace created.
SQL>
3. Create a user called test with the password test and grant the user the dba role (not recommended in a real
world situation but for environment it is fine).
SQL> CREATE USER test IDENTIFIED BY test DEFAULT TABLESPACE TBS_1;
User created.
SQL> GRANT DBA TO test;
Grant succeeded.
SQL> exit
Production
[oracle@db ~]$
4. Exit SQL*plus and then login as the newly created user, test (password test). Once logged in as the user
test we will create a table called LARGE_TABLE. When the table is created exit SQL*Plus.
[oracle@db ~]$ cd /home/oracle

[oracle@db ~]$ sqlplus test/test @large_table.sql
Connected to:
Table created.
SQL> exit
Production
5. Load some demo data into the database
[oracle@db ~]$ sqlldr userid=test/test direct=true rows=100000 control=large_table.ldr
SQL*Loader: Release 11.2.0.1.0 - Production on Tue Oct 11 11:26:08 2011
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Save data point reached - logical record count 100000.

Load completed - logical record count 1000000.

[oracle@db ~]$
Task 2: Smart Scan and Query Offload

In this task we will run some queries that do block I/O on the table we created. We will see how the Exadata cell does
offloading to increase performance.
To force a select query to do only block I/O we will set the parameter cell_offload_oprocessing to FALSE. Once the
cell offload processing is disabled predicate filtering is not offloaded to the Exadata cells and only block I/O code path
is executed.
1. Connect to the database using SQL*Plus as the test user (password test).
[oracle@db ~]$ sqlplus test/test
Connected to:
SQL>
2. Verify the cell parameters by running show parameter cell. The cell_offload_processing parameter should be
set to true.
SQL> show parameter cell;
NAME TYPE VALUE

------------------------------------ ----------- -----------------------
cell_offload_compaction string ADAPTIVE
cell_offload_decryption boolean TRUE
cell_offload_parameters string
cell_offload_plan_display string AUTO
cell_offload_processing boolean TRUE
cell_partition_large_extents string DEFAULT
SQL>
3. Set the cell_offload_processing parameter to FALSE and verify that the parameter is set to FALSE by running
show parameter cell.
SQL> ALTER session SET cell_offload_processing=FALSE;
Session altered.
NAME TYPE VALUE

------------------------------------ ----------- -----------------------
cell_offload_processing boolean FALSE
SQL>
4. Set another parameter (hidden and should not be used in production systems) called
_serial_direct_read to TRUE. This way we will force every query to use a direct read (and therefore
BYPASSES/USES the Exadata smart scan options). To set this parameter the user must have DBA privileges
(which our test user do).
SQL> ALTER session SET "_serial_direct_read" = TRUE;
Session altered.
SQL>
5. Run a select query on the table we created and loaded earlier
SQL> SELECT id FROM large_table WHERE id > 999000;
999500
999501
999502
999503
1000 rows selected.
6. View the statistics for the query by executing the following query (or execute the script
/home/oracle/scripts/2-6.sql). We will see the following information (save the query as we will use it
again):
Physical read_bytes (column A) is the same as IO_Interconnect_bytes (column B). This means that all
physical reads on the cell where retruned to the database. There was no smart scanning or offloading on
the cell
io_cell_offload_eligible_bytes (column C) and io_cell_offload_returned_bytes (column D) are both 0
which supports our findings.
SQL> SELECT sql_id,

physical_read_bytes A,
io_interconnect_bytes B,
io_cell_offload_eligible_bytes C,
io_cell_offload_returned_bytes D
FROM v$sql
WHERE sql_text LIKE '%from_large_table%'
AND sql_text NOT LIKE '%v$sql%';
SQL_ID A B C D
------------- ---------- ---------- ---------- ----------
297mg96w2afa7 130555904 130555904 0 0
7. We will now turn on the parameter cell_offload_processing and the hidden parameter _serial_direct_read (set
to TRUE). Even if those parameters have been enabled, not all queries get offloaded for a predicate filtering.
Only the queries that qualifty for predicate filtering (and direct read) are offloaded, other still go through the
block I/O code path.
SQL> ALTER session SET cell_offload_processing = TRUE;
Session altered.
SQL> ALTER session SET "_serial_direct_read" = TRUE;
Session altered.
NAME TYPE VALUE

------------------------------------ ----------- ------------------------
cell_offload_processing boolean TRUE
SQL>
8. Run the same statement as we ran earlier
SQL> SELECT id FROM large_table WHERE id > 999000;
999500
999501
999502
999503
1000 rows selected.
9. Verify the statistics again by execute the following statement (or run the script /home/oracle/scripts/2-
6.sql). Note that youll see the statistics from our first run too. We will see the following information (save the
query as we will use it again):
io_cell_offload_eligible_bytes (column C) and io_cell_offload_returned_bytes (column D) are both non-
zero in this case, which means this query was eligible for offload filtering and that a certain amount of data
was actually offloaded. If you devide the column D and C, you can see that in this case less than 1% of the
data was returned although we did a full table scan.
SQL> SELECT sql_id,

physical_read_bytes A,
io_interconnect_bytes B,
io_cell_offload_eligible_bytes C,
io_cell_offload_returned_bytes D
FROM v$sql
WHERE sql_text LIKE '%from_large_table%'
AND sql_text NOT LIKE '%v$sql%';
SQL_ID A B C D
------------- ---------- ---------- ---------- ----------
297mg96w2afa7 127926272 28320 127918080 20128
SQL>
10. Exit SQL*plus
SQL> exit
Production
[oracle@db ~]$
your instructor.
Lab E Bulk Loading Lab

Objective: Load data into an Exadata database using
45 minutes
DBFS and External tables
Task 1: Setup a Database filesystem

In this task you will be loading the Exadata Database
Machine using a database filesystem (DBFS) and use this
to stage a CSV formatted file. You will then create a table
using external tables. Note that in this environment you are
configuring DBFS in the same database as the one holding
your data. This is not recommended, Oracle recommends
that you create a separate dedicated database instance for
DBFS.
1. Open a terminal window (or use an already open

one) and make sure that you are the oracle user.
2. Connect to the database. Login in as the database

administer

ORACLE_SID = [oracle] ? ORCL
SQL*Plus: Release 11.2.0.1.0 Production on Tue Feb 14 04:37:32 2012
Connected to:
SQL>
3. Create a new tablespace for the database file system (DBFS) using the DBFS ASM diskgroup you created
earlier.
SQL> CREATE bigfile TABLESPACE dbfs datafile '+DATA' size 128M;
Tablespace created.
SQL>
4. Create a new database user to support DBFS and grant the database user the required system privileges and
roles
SQL> CREATE USER dbfs IDENTIFIED BY welcome quota unlimited ON dbfs;
User created.
SQL> GRANT CREATE session, CREATE TABLE, CREATE PROCEDURE, dbfs_role TO dbfs;
Grant succeeded.
SQL>
5. Exit the SQL*Plus session.
SQL> exit
Production
[oracle@db ~]$
6. Create a directory named DBFS in your home directory.
[oracle@db ~]$ mkdir $HOME/DBFS

[oracle@db ~]$
7. Change the home directory to $ORACLE_HOME/rdbms/admin
[oracle@db ~]$ cd $ORACLE_HOME/rdbms/admin

[oracle@db admin]$
8. Connect to the database. Login as the newly created user
[oracle@db admin]$ sqlplus dbfs/welcome
Connected to:
SQL>
9. Run the dbfs_create_filesystem_advanced.sql script file to create the database objects for your DBFS store.
The first parameter (dbfs) specififes the tablespace where the DBFS is created. The second parameter
(mydbfs) specifices the name of the DBFS store. The last four parameters specified various parameters inside
the DBFS store.
SQL> @dbfs_create_filesystem_advanced.sql dbfs mydbfs nocompress nodeduplicate

nonencrypt non-partition
SQL> SET FEEDBACK 1
SQL> SET NUMWIDTH 10
SQL> SET LINESIZE 80
SQL> SET TRIMSPOOL ON
SQL> SET TAB OFF
SQL> SET PAGESIZE 100
SQL> undefine fs_dedup

SQL> undefine fs_encrypt
SQL> undefine fs_partition
SQL>
SQL> exit;
Production
[oracle@db admin]$
10. Change to your home directory
[oracle@db admin]$ cd $HOME

[oracle@db ~]$
11. Create a file named dbfs_pwd.txt which contains the password for the DBFS database user.
[oracle@db ~]$ echo welcome > dbfs_pwd.txt

[oracle@db ~]$
Task 2: Mount the Database Filesystem

1. Set LD_LIBRARY_PATH to include the fuse libraries and then launch the DBFS client (dbfs_client). This will
mount the database file system (it will take a few minutes as we are running on an VM, type carefully, note
there is no space between allow_other, the comma, and direct_io)
[oracle@db mydbfs]$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib64

[oracle@db ~]$ nohup $ORACLE_HOME/bin/dbfs_client dbfs@ORCL -pass-through-fuse-options
o allow_other,direct_io /home/oracle/DBFS < dbfs_pwd.txt &
nohup: appending output to `nohup.out'
2. Verify that the dbfs_client process is running
[oracle@db mydbfs]$ ps -ef | grep dbfs_client

oracle 17900 15616 0 05:39 pts/2 00:00:00
/u01/oracle/product/11.2.0/dbhome_1/bin/dbfs_client dbfs@ORCL -o
allow_other,direct_io,trace_level=5 /home/oracle/DBFS
oracle 18021 13575 0 05:40 pts/0 00:00:00 grep dbfs_client
[oracle@db mydbfs]$
3. Execute the df command. The database file system should be mounted.
[oracle@db mydbfs]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda2 4089720 2941056 1148664 72% /
/dev/xvda1 31077 16980 12493 58% /boot
/dev/xvdb1 8768128 7757848 564880 94% /u01
tmpfs 2097152 420592 1676560 21% /dev/shm
dbfs 448 25 423 6% /home/oracle/DBFS
[oracle@db mydbfs]$
Task 3: Load data into External Tables

1. Copy the CSV file you are going to load into the database into the database file system you have created and
check the file. This will be out staging are for loading data into the database.
[oracle~]$ cp $HOME/scripts/customers.csv $HOME/DBFS/mydbfs/

[oracle@db ~]$ ls -l $HOME/DBFS/mydbfs
total 7376
-rw-r--r-- 1 oracle oinstall 7552705 Feb 14 20:47 customers.csv
[oracle@db ~]$
2. Use the head command to list the first lines of the file
[oracle@db ~]$ head $HOME/DBFS/mydbfs/customers.csv

55,"Bruce","Lange","hi","INDIA",1999,"Bruce.Lange@oracle.com",166
74,"Claude","Brown","d","SWITZERLAND",4737,"Claude.Brown@oracle.com",163
81,"Max","Capshaw","us","AMERICA",650,"Max.Capshaw@oracle.com",165
97,"Roy","Kazan","zhs","CHINA",2707,"Roy.Kazan@oracle.com",156
111,"Robert","Young","i","ITALY",4895,"Robert.Young@oracle.com",169
146,"Ridley","Schneider","th","THAILAND",3775,"Ridley.Schneider@oracle.com",168
245,"Matthias","Russell","i","ITALY",2839,"Matthias.Russell@oracle.com",158
252,"Edward","Hoskins","ja","JAPAN",2469,"Edward.Hoskins@oracle.com",153
262,"Grace","Sen","zhs","CHINA",500,"Grace.Sen@oracle.com",162
281,"Gtz","Peckinpah","th","THAILAND",1128,"Gtz.Peckinpah@oracle.com",151
[oracle@db ~]$
3. Now will you create an external table to reference the data. Connect to the database as a database
administrator
Connected to:
SQL>
4. Create a directory object that points to your DBFS staging directory
SQL> CREATE directory dbfs_stage AS '/home/oracle/DBFS/mydbfs';
Directory created.
SQL>
5. Grant read and write permissions on your newly created directory object to the test user.
SQL> GRANT read, write ON directory dbfs_stage TO test;
Grant succeeded.
SQL>
6. Connect as the test user
SQL> connect test/test

Connected
SQL>
7. Create an external table which references the data in your DBFS CSV data file by executing the following
statement as the test user (or run the /home/oracle/scripts/8-22.sql).
SQL> CREATE TABLE dbfs_customers

(
customer_id NUMBER(12),
cust_first_name VARCHAR2(30),
cust_last_name VARCHAR2(30),
nls_language VARCHAR2(3),
nls_territory VARCHAR2(30),
credit_limit NUMBER(9,2),
cust_email VARCHAR2(100),
account_mgr_id NUMBER(6)
)
organization external
(
type oracle_loader
default directory dbfs_stage
access parameters
(
records delimited by newline
badfile dbfs_stage:'custxt%a_%p.bad'
logfile dbfs_stage:'custxt%a_%p.log'
fields terminated by ',' optionally enclosed by '"'

missing field values are null
(
customer_id, cust_first_name, cust_last_name, nls_language,
nls_territory, credit_limit, cust_email, account_mgr_id
)
)
location ('customers.csv')
)
parallel
reject limit unlimited;
Table created.
SQL>
8. Configure you session to display execution plans and run the following query to display the number of records
in the external table. Note the parallel execution and the EXTERNAL TABLE ACCESS FULL.
SQL> SET autotrace ON explain;

SQL> SELECT COUNT(*) FROM dbfs_customers;
COUNT(*)
----------
100000
Execution Plan
----------------------------------------------------------
Plan hash value: 2705389078
-----------------------------------------------------------------------------
|Id |Operation | Name | Rows | Cost (%CPU)|Time |
TQ |IN-OUT|PQ Distrib|
-----------------------------------------------------------------------------|
0|SELECT STATEMENT | | 1| 16 (0)|00:00:01|
| | |
| 1| SORT AGGREGATE | | 1| | |
| | |
| 2| PX COORDINATOR | | | | |
| | |
| 3| PX SEND QC (RANDOM) | :TQ10000 | 1| | |
Q1,00| P->S |QC (RAND) |
| 4| SORT AGGREGATE | | 1| | |
Q1,00| PCWP | |
| 5| PX BLOCK ITERATOR | | 8168| 16 (0)|00:00:01|
Q1,00| PCWC | |
| 6| EXTERNAL TABLE ACCESS FULL| DBFS_CUSTOMERS| 8168| 16 (0)|00:00:01|
Q1,00| PCWP | |
-----------------------------------------------------------------------------
9. Turn off automatic query plan display
SQL> SET autotrace OFF;

SQL>
10. Use CTAS (Create Table As Select) command to load the external table data contained in the CSV file into a
new regular table in the database
SQL> CREATE TABLE customers AS SELECT * FROM dbfs_customers;
Table created.
SQL>
11. Verify the numbers of records in the database are the same as the number of rows in the CSV file.
SQL> SELECT COUNT(*) FROM customers;
COUNT(*)
----------
100000
12. Exit your SQL*Plus session and list the content of the DBFS directory.
SQL> exit
Production
[oracle@db ~]$ ls -l $HOME/DBFS/mydbfs
total 7388
-rw-r--r-- 1 oracle oinstall 6 Feb 14 21:29 andy.txt
-rw-r--r-- 1 oracle oinstall 7552705 Feb 14 20:47 customers.csv
-rw-r--r-- 1 oracle dba 1377 Feb 14 21:55 custxt000_13640.log
[oracle@db ~]$
13. Examine the content of the log files (note that the filename will be different than the sample below):
[oracle@db ~]$ cat $HOME/DBFS/mydbfs/custxt000_14113.log
LOG file opened at 02/14/12 22:00:25
Field Definitions for table DBFS_CUSTOMERS

Record format DELIMITED BY NEWLINE
Data in file has same endianness as the platform
Rows with all null fields are accepted
Fields in Data Source:
CUSTOMER_ID CHAR (255)

Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
Task 4: Unmount the Database filesystem
1. Change to your home directory
[oracle@db ~]$ cd $HOME
2. Use the fusermount to unmount the DBFS file system.
[oracle@db ~]$ fusermount -u /home/oracle/DBFS
3. Verify that the DBFS is no longer mounted.
[oracle@db ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda2 4089720 2949316 1140404 73% /
/dev/xvda1 31077 16980 12493 58% /boot
/dev/xvdb1 8768128 7775696 547032 94% /u01
tmpfs 2097152 424700 1672452 21% /dev/shm
[oracle@db ~]$
4. Unmounting the DBFS file system terminated the dbfs_client process. Verify that the process no longer
running.
[oracle@db ~]$ ps -ef | grep dbfs_client

oracle 14978 9529 0 22:10 pts/3 00:00:00 grep dbfs_client
[oracle@db ~]$
5. What will happen if you try to query the external table after the DBFS is unmounted? Try!
your instructor.
Lab F Compression
Objective: Create tables using the multiple levels of
25 minutes
compression available in Exadata
Task 1: Compression
To demonstrate the value of compression with Exadata, we are going to create a number of tables, each with different
form of compression and then compare the result.
1. Establish a connection to the database using the user test (password test).

ORACLE_SID = [ORCL] ? ORCL
SQL*Plus: Release 11.2.0.1.0 Production on Fri Oct 21 13:14:31 2011
Connected to:
SQL>
2. Execute the following statements to create four tables with different compressions. The elapsed time will vary.
SQL> SET timing ON

SQL> CREATE TABLE large_uncompressed AS SELECT * FROM large_table;
Table created.
Elapsed: 00:00:28.73
SQL> CREATE TABLE large_normal_compress compress AS SELECT * FROM large_table;
Table created.
Elapsed: 00:00:20.58
SQL> CREATE TABLE large_ehcc_query compress FOR query low AS SELECT * FROM large_table;
Table created.
Elapsed: 00:00:06.56
SQL> CREATE TABLE large_ehcc_archive compress FOR archive high AS SELECT * FROM
large_table;
Table created.
Elapsed: 00:00:39.35
SQL>
3. Verify the tables that they have the right compression
SQL> SELECT table_name,

compression,
compress_for
FROM user_tables
WHERE table_name LIKE 'LARGE%';
TABLE_NAME COMPRESS COMPRESS_FOR

------------------------------ -------- ------------
LARGE_EHCC_ARCHIVE ENABLED ARCHIVE HIGH
LARGE_EHCC_QUERY ENABLED QUERY LOW
LARGE_NORMAL_COMPRESS ENABLED BASIC
LARGE_TABLE DISABLED
LARGE_UNCOMPRESSED DISABLED
Elapsed: 00:00:00.14
SQL>
4. Run the following query to compare the sizes of the tables (or run the script /home/oracle/scripts/4-
4.sql). You can calculate the compression ratio achieved by using the formula: Compression Ratio =
Uncompressed Size/ Compressed Size.
SQL> col segment_name format a25

SQL> col MB format 999.9
SQL> SELECT segment_name, SUM (bytes)/1024/1024 MB FROM user_segments
WHERE segment_name LIKE 'LARGE%'GROUP BY segment_name ORDER BY 2;
SEGMENT_NAME MB
------------------------------ ----------
LARGE_NORMAL_COMPRESS 72.0
LARGE_EHCC_ARCHIVE 8.0
LARGE_EHCC_QUERY 22.0
LARGE_TABLE 128.0
LARGE_UNCOMPRESSED 128.0
Elapsed: 00:00:00.02
SQL>
your instructor.
Lab G Reprovision Disks

Objective: Drop disks in Exadata to demonstrate the
25 minutes
rebalancing capability of ASM
Task 1: Verify Disk Setup

In this exercise we are going to drop the first disk of the Exadata Storage Cell cell1. First we need to connect to the cell
and collect some data
1. Connect to cell1 as the celladmin user and start the CellCLI command. The Efficiency ratio displayed is the
ratio between data accessed by the storage cell and data processed by the database for the cell. Larger
values mean more offloading to the cell. The value will be different in your environment.

Last login: Fri Oct 21 16:07:29 2011 from db.exadata.local
CellCLI: Release 11.2.1.2.0 - Production on Fri Oct 21 16:17:49 EDT 2011

CellCLI>
2. List the physical disks
CellCLI> list physicaldisk

/opt/oracle/cell11.2.1.2.0_LINUX.X64_091102/disks/raw/FLASH01
/opt/oracle/cell11.2.1.2.0_LINUX.X64_091102/disks/raw/FLASH01 normal
/opt/oracle/cell11.2.1.2.0_LINUX.X64_091102/disks/raw/FLASH02
CellCLI>
3. We will drop /opt/oracle/cell/disks/raw/disk01. Check which celldisk is attached to our physicaldisk
CellCLI> list lun attributes name, celldisk -

> where physicaldrives = -
> "/opt/oracle/cell11.2.1.2.0_LINUX.X64_091102/disks/raw/disk01"
CD_disk01_cell1
CellCLI>
4. Check which griddisks are based on this celldisk
CellCLI> list griddisk where celldisk="CD_disk01_cell1"

archive_CD_disk01_cell1 active
data_CD_disk01_cell1 active
CellCLI>
Task 2: Drop Griddisks

5. There are 2 griddisks based on this celldisk
Archive_CD_disk01_cell1
Data_CD_disk01_cell1
Drop the ASM disks (same as the griddisks) in ASM. To do this, connect to ASM on the DB node

Connected to:
SQL>
6. Execute the following commands. Note that the drop disk command will return control to you immediately. In
the background data will be moved from the two griddisks to other locations. Only if the disks are empty they
will be released from ASM (this can take a while)
SQL>
SQL> alter diskgroup DATA drop disk data_CD_disk01_cell1;
Diskgroup altered.
SQL> alter diskgroup ARCHIVE drop disk archive_CD_disk01_cell1;
Diskgroup altered.
SQL>
7. After the completion of the drop disk command on the ASM diskgroups we can drop the disks on the Exadata
cell (cell1). Open a session to cell1 as the celladmin user and start the CellCLI utility. The Efficiency ratio


CellCLI>
8. Execute the following commands to take the disk out of the configuration and can be replaced or setup for
another diskgroup.
CellCLI> drop griddisk archive_CD_disk01_cell1

GridDisk archive_CD_disk01_cell1 successfully dropped
If you get the following error you need to wait until ASM has rebalanced the data on the disk you want to drop.
Or you can and the attribute force with the drop command (drop griddisk data_CD_disk01_cell1
force)
CellCLI> drop griddisk data_CD_disk01_cell1
CELL-02549: Grid disk is in use and FORCE is not specified for the operation.
Task 3: Recreate Griddisks

9. Now we can create a new griddisk. Start CellCLI and execute the following commands
CellCLI> create griddisk data_CD_new_disk01_cell1 celldisk=CD_DISK01_cell1, size=250M

GridDisk data_CD_new_disk01_cell1 successfully created
CellCLI> create griddisk archive_CD_new_disk01_cell1 celldisk=CD_disk01_cell1

GridDisk archive_CD_new_disk01_cell1 successfully created
CellCLI>
10. Check the result
CellCLI> list griddisk

archive_CD_disk02_cell1 active
archive_CD_new_disk01_cell1 active
data_CD_disk02_cell1 active
data_CD_new_disk01_cell1 active
CellCLI>
11. The new griddisks are available for ASM immediately. Login into the ASM instance and execute the following
command to add disks back into the diskgroups

ORACLE_SID = [+ASM] +ASM
Connected to:
SQL> alter diskgroup DATA add disk 'o/*/data_CD_new_disk01_cell1' rebalance power 9;
Diskgroup altered.
SQL> alter diskgroup ARCHIVE add disk 'o/*/archive_CD_new_disk01_cell1' rebalance power

9;
Diskgroup altered.
SQL>
your instructor.
Lab H DCLI and Other Monitoring

Objective: Logging and monitoring of an Exadata Storage
30 minutes
Cell and DCLI
Task 1: Work with metrics in the Exadata Storage Cell

1. Login into one of the Exadata Storage Cells as the celladmin user (password oracle) and list the metrics
available using the CellCLI utility. The Efficiency ratio displayed is the ratio between data accessed by the
storage cell and data processed by the database for the cell. Larger values mean more offloading to the cell.
The value will be different in your environment.


CellCLI> list metricdefinition detail
name: N_NIC_TRANS_SEC
description: "Total number of IO packets transmitted by
interconnects per second"
metricType: Rate
objectType: CELL
unit: packets/sec
name: N_RDMA_RETRY_TM
description: "Latency of the retry actions during RDMA transmission
to a particular host"
metricType: Cumulative
objectType: HOST_INTERCONNECT
unit: ms
CellCLI>
2. The list is very long so we can filter out the metrics we would like to see
CellCLI> list metricdefinition attributes name, description where objecttype="cell"

CL_CPUT "Cell CPU Utilization is the percentage of time over
the previous minute that the system CPUs were not idle (from /proc/stat)."
CL_FANS "Number of working fans on the cell"
CL_MEMUT "Percentage of total physical memory on the cell that
is currently used"
CL_RUNQ "Average number (over the preceding minute) of

processes in the Linux run queue marked running or uninterruptible (from
/proc/loadavg)."
CL_TEMP "Temperature (Celsius) of the server, provided by the
BMC"
N_NIC_NW "Number of non-working interconnects"
N_NIC_RCV_SEC "Total number of IO packets received by interconnects
per second"
N_NIC_TRANS_SEC "Total number of IO packets transmitted by
interconnects per second"
CellCLI>
3. Create a list of all current r values available.
CellCLI> list metriccurrent detail
name: N_NIC_TRANS_SEC
alertState: normal
collectionTime: 2011-10-21T15:55:52-04:00
metricObjectName: cell1
metricType: Rate
metricValue: 15.5 packets/sec
objectType: CELL
name: N_RDMA_RETRY_TM
alertState: normal
collectionTime: 2011-10-21T15:55:16-04:00
metricObjectName: db.exadata.local
metricType: Cumulative
metricValue: 0 ms
objectType: HOST_INTERCONNECT
CellCLI>
4. Restrict the list by using a where clause
CellCLI> list metriccurrent attributes name, metricvalue where objecttype="cell"

CL_CPUT 0.7 %
CL_FANS 1
CL_MEMUT 38 %
CL_RUNQ 0.0
CL_TEMP 0.0 C
N_NIC_NW 0
N_NIC_RCV_SEC 44.0 packets/sec
N_NIC_TRANS_SEC 12.3 packets/sec
CellCLI>
5. Create list of all alertdefinitions available
CellCLI>list alertdefinition detail
name: Stateful_HardwareAlert
alertShortName: Hardware
alertSource: Hardware
alertType: Stateful
description: "Hardware Stateful Alert"
metricName:
name: Stateful_SoftwareAlert
alertShortName: Software
alertSource: Software
alertType: Stateful
description: "Software Stateful Alert"
metricName:
CellCLI>
6. Setup an threshold for latency rate metric CT_IO_WT_LG_RQ to generate stateful alerts for ASM category, if
latency per large IO request is greater than 10ms (wrning), or greater than 15ms (critical)
CellCLI> list metriccurrent where name = "CT_IO_WT_LG_RQ"

CT_IO_WT_LG_RQ OTHER 0.0 ms/request
CT_IO_WT_LG_RQ _ASM_ 0.0 ms/request
CellCLI> create threshold CT_IO_WT_LG_RQ._ASM comparison = ">", warning=10, critical=15

Threshold CT_IO_WT_LG_RQ._ASM successfully created
CellCLI>
7. List the threshold
CellCLI> list threshold detail

name: CT_IO_WT_LG_RQ._ASM
comparison: >
critical: 15.0
warning: 10.0
CellCLI>
8. To check the status of an alert use the following command. There shouldnt be any alerts.
CellCLI> list alerthistory
9. All runtime logging on the cells is done using the Advanced Diagnostic Repository, just as for the database.
Login into one of the cells as the celladmin user (password oracle)

10. Start the ADR command line interface
[celladmin@cell1 ~]$ adrci
ADRCI: Release 11.2.0.1.0 - Production on Fri Oct 21 16:07:58 2011
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
No ADR base is set

adrci>
11. Set the ADR Base
adrci> set base /opt/oracle/cell/log

adrci>
12. Display the homes for this repository
adrci> show home

ADR Homes:
diag/asm/user_root/host_712035098_76
diag/asm/cell/cell1
adrci>
13. Set a home and show the logfile for the cell
adrci> set home diag/asm/cell/cell1

adrci> show alert
14. Exit the alert log by pressing the Escape key and type q! (regular vi command)
ADR Home = /opt/oracle/cell/log/diag/asm/cell/cell1:

*************************************************************************
Output the results to file: /tmp/alert_14160_4720_cell1_1.ado
adrci>
15. Show any incidents in this cell. There should not be any incidents
adrci> show incidents
ADR Home = /opt/oracle/cell/log/diag/asm/cell/cell1:

*************************************************************************
0 rows fetched
adrci>
16. Exit the ADR utility
adrci> exit
Task 2: DCLI
1. Start a new terminal session on the DB node and become root (password oracle). Make sure you are on
the home directory of root (/root)
[oracle@db ~]$ su - root

Password:
[root@db ~]# cd /root
[root@db ~]#
2. Copy the dcli script from cell1 to the DB node. The dcli script is located in /opt/oracle/cell/cellsrv/bin
[root@db ~]# scp cell1:/opt/oracle/cell/cellsrv/bin/dcli .

root@cell1's password:
dcli 100% 33KB 33.2KB/s 00:00
[root@db ~]#
3. Create a new file in the home directory of root that contains information about the cells
[root@db ~]# echo cell1 > /root/mycells

[root@db ~]# echo cell2 >> /root/mycells
[root@db ~]# cat /root/mycells
cell1
cell2
[root@db ~]#
4. Create user equivalence keys on the targets (cells).
[root@db ~]# ./dcli -k -g mycells

cell1: ssh key added
cell2: ssh key added
[root@db ~]#
5. List the status of the cells specified in your file mycells
[root@db ~]# ./dcli -g mycells 'cellcli -e list cell '

cell1: cell1 online
cell2: cell2 online
[root@db ~]#
6. List all physical disks on all cells using hostnames instead of the file where you have specified the cells
[root@db ~]# ./dcli -c cell1,cell2 'cellcli -e list physicaldisk'

cell1: /opt/oracle/cell11.2.1.2.0_LINUX.X64_091102/disks/raw/FLASH01
cell1: /opt/oracle/cell11.2.1.2.0_LINUX.X64_091102/disks/raw/disk01
[root@db ~]#
your instructor.
Lab I IORM
Objective: Perform resource management tasks in the
30 minutes
Exadata Database
Task 1: IO Resource Manager (IORM)

All the following commands in this task are to be run on the Exadata Storage Cell.
1. Connect to cell1 from the database node using the celladmin user (password oracle)

Last login: Wed Oct 12 11:48:23 2011 from db.exadata.local
2. Start CellCLI. The Efficiency ratio displayed is the ratio between data accessed by the storage cell and data
processed by the database for the cell. Larger values mean more offloading to the cell. The value will be
different in your environment.


CellCLI>
3. Check the status of the IO Resource Manager Plans by executing the following command.
CellCLI> list iormplan detail

name: cell1_IORMPLAN
catPlan:
dbPlan:
status: inactive
CellCLI>
4. Establish a connection from the DB node to the cell2 Exadata Storage Cell in a new terminal window.

Last login: Wed Oct 12 11:49:16 2011 from db.exadata.local
5. Start CellCLI. The Efficiency ratio displayed is the ratio between data accessed by the storage cell and data


CellCLI>
6. Check the status of the IO Resource Manager Plans by executing the following command:

catPlan:
dbPlan:
status: inactive
CellCLI>
7. IORM has to be activated individually on every Exadata Storage Cell separately. This can be done by using
the dcli command utility. As we only have 2 cells we will use the CellCLI command on both cells. Execute thw
following command on cell1 and verify the status
CellCLI> alter iormplan active

IORMPLAN successfully altered

catPlan:
dbPlan:
status: active
CellCLI>
8. Execute the same thing on cell2
CellCLI> alter iormplan active


catPlan:
dbPlan:
status: active
CellCLI>
9. Enable an inter-database resource plan, execute the following command in the cellcli interface for cell1
CellCLI> alter iormplan dbplan=((name=ORCL, level=1, allocation=100), -

> (name=PROD, level=2, allocation=100), -
> (name=other, level=3, allocation=100))
10. Verify that the plan has been set correctly for cell1

catPlan:
dbPlan: name=ORCL,level=1,allocation=100
name=PROD,level=2,allocation=100
name=other,level=3,allocation=100
status: active
CellCLI>
11. Logout out of CellCLI and check the alert log file for the cell1. The command should be on one line the
hyphen is there for display purposes only.
[celladmin@cell1 ~]$ tail -/opt/oracle/celllog/diag/asm/cell/cell1/trace/alert.log

Fri Oct 21 13:04:28 2011
I/O Resource Manager: Setting default inter-database plan
I/O Resource Manager: Setting default category plan
I/O Resource Manager: Setting default inter-database plan
Fri Oct 21 13:07:44 2011
I/O Resource Manager: Setting inter-database plan
I/O Resource Manager enabled
12. Before continuing with the exercises, disable the IORM plan on both cells (cell1 and cell2)
CellCLI> alter iormplan inactive

CellCLI>
Lab J Supplemental Exercises

Objective: If time permits, do the following exercises 60 minutes
Task 1: Security
1. Establish a connection to the Exadata Storage Cell cell1 and show the security attributes and status of a
griddisk (in this case data_CD_disk01_cell1). Verify that the AvailableTo: filed is empty as we havent defined
any security rules yet). When done close the connection to the Exadata Storage Cell cell1. The Efficiency ratio


CellCLI> list griddisk data_CD_disk01_cell1 detail

name: data_CD_disk01_cell1
availableTo:
cellDisk: CD_disk01_cell1
comment:
creationTime: 2011-09-28T12:56:54-04:00
diskType: HardDisk
errorCount: 0
id: 00000132-b0f7-ac9b-0000-000000000000
offset: 48M
size: 240M
status: active
CellCLI>
2. We will now setup ASM scoped security. To be able to accomplish this we need to stop the database and the
ASM instance.

[oracle@db ~]$ srvctl stop database -d ORCL
[oracle@db ~]$
ORACLE_SID = [ORCL] ? +ASM
Connected to:
SQL> shutdown immediate

ASM diskgroups dismounted
ASM instance shutdown
SQL> exit
Production
[oracle@db ~]$
3. Verify that the database and ASM instance is not running
[oracle@db ~]$ srvctl status database -d ORCL

Database is not running.
[oracle@db ~]$ srvctl status asm
ASM is not running.
[oracle@db ~]$
4. Establish a connection to the Exadata Storage Cell cell1 as the celladmin user (password oracle) and
start CellCLI. The Efficiency ratio displayed is the ratio between data accessed by the storage cell and data


CellCLI>
5. We are now going to create a random key (you can create your own by yourself or you can have the Exadata
cell create one for you). Please note that the following command has to be executed on only one cell and the
result needs to be used on both cells. Also, the value of the key will be different than shown below.
CellCLI> create key

1d8a1a561ad4f03a3b99d32e465f7c2
CellCLI>
6. Assign the key to the +ASM instance on cell1
CellCLI> assign key for +ASM='1d8a1a561ad4f03a3b99d32e465f7c2'

Key for +ASM successfully created
CellCLI>
7. Establish a connection to the Exadata Storage Cell cell2 as the celladmin user (password oracle) and
start CellCLI. The Efficiency ratio displayed is the ratio between data accessed by the storage cell and data


CellCLI>
8. Assign the key to the +ASM instance on cell2
CellCLI> assign key for +ASM='1d8a1a561ad4f03a3b99d32e465f7c2'

Key for +ASM successfully created
CellCLI>
9. If you misplace the key, you can always use get it by executing the following command
CellCLI> list key

+ASM 1d8a1a561ad4f03a3b99d32e465f7c2
CellCLI>
10. Now assign the +ASM database to the griddisks on cell1
CellCLI> alter griddisk all availableTo='+ASM'

GridDisk archive_CD_disk01_cell1 successfully altered
GridDisk data_CD_disk01_cell1 successfully altered

CellCLI>
11. Assign the +ASM database to the griddisks on cell2
CellCLI> alter griddisk all availableTo='+ASM'


CellCLI>
12. Now try to start the ASM instance on the DB system. ASM will report an insufficient number of disks for both
diskgroups because we limited the access to the disks but we didnt link the +ASM instance with the key we
used (there are more details in the alert.log).

Connected to an idle instance.
SQL> startup
ASM instance started
Total System Global Area 283930624 bytes

Fixed Size 2212656 bytes
Variable Size 256552144 bytes
ASM Cache 25165824 bytes
ORA-15032: not all alterations performed
ORA-15017: diskgroup "DATA" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA"
ORA-15017: diskgroup "ARCHIVE" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup
"ARCHIVE"
SQL>
13. Shutdown the ASM instance so we can add the key

ORA-15100: invalid or missing diskgroup name

SQL>
14. On the DB node, become the root user (password oracle) and change the prompt to the directory where the
key for the ASM instance is stored and create the file cellkey.ora that contains the key for the ASM instance.
[oracle@db ~]$ su - root

Password:
[root@db ~]# cd /etc/oracle/cell/network-config/
[root@db network-config]# echo key=1d8a1a561ad4f03a3b99d32e465f7c2 > cellkey.ora
[root@db network-config]# echo asm=+ASM >> cellkey.ora
[root@db network-config]# chown oracle:dba cellkey.ora
[root@db network-config]# chmod 660 cellkey.ora
[root@db network-config]# cat cellkey.ora
key=1d8a1a561ad4f03a3b99d32e465f7c2
asm=+ASM
[root@db network-config]#
15. Exit the root user session and start the ASM instance. The ASM instance will report the diskgroups as
mounted.

SQL> startup

ASM diskgroups mounted
SQL>
16. Exit the SQL*Plus Session.

17. We will now implement Database scoped security. Establish a connection to one of the cells and create a new
key that we will use for the database. Note that the key will be different than displayed below. The Efficiency
ratio displayed is the ratio between data accessed by the storage cell and data processed by the database for
the cell. Larger values mean more offloading to the cell. The value will be different in your environment.


CellCLI> create key

2cba467959845acb84a0ffa39270e924
CellCLI>
18. Connect to the Exadata Storage Cell cell1 and assign the key for the ORCL instance (DB instance) on cell1
using the CellCLI command.
CellCLI> assign key for ORCL='2cba467959845acb84a0ffa39270e924'

Key for ORCL successfully altered
CellCLI>
19. Connect to the Exadata Storage Cell cell2 and assign the key for the ORCL instance (DB instance) on cell2
using the CellCLI command.
CellCLI> assign key for ORCL='2cba467959845acb84a0ffa39270e924'

Key for ORCL successfully created
CellCLI>
20. Assign the ORCL database to the griddisks on the cell1. Make sure to include any current values because
altering the field overwrites the previous values
CellCLI> alter griddisk all availableTo='+ASM,ORCL'


CellCLI>
21. Verify that the availableTo field has the correct information on a griddisk

availableTo: +ASM,ORCL
comment:
creationTime: 2011-09-28T12:56:54-04:00
diskType: HardDisk
errorCount: 0
id: 00000132-b0f7-ac9b-0000-000000000000
offset: 48M
size: 240M
status: active
CellCLI>
22. Assign the ORCL database to the griddisks on the cell2. Make sure to include any current values because
altering the field overwrites the previous values
CellCLI> alter griddisk all availableTo='+ASM,ORCL'


CellCLI>
23. Verify that the availableTo field has the correct information on a griddisk

availableTo: +ASM,ORCL
comment:
creationTime: 2011-09-28T13:57:02-04:00
diskType: HardDisk
errorCount: 0
id: 00000132-b12e-b733-0000-000000000000
offset: 48M
size: 240M
status: active
CellCLI>
24. Start the ORCL instance on the DB node. The database will start without a problem as no key has been
defined yet on the database level.

[oracle@db ~]$ srvctl start database -d ORCL
Database is running.
[oracle@db ~]$
25. Shutdown the database and the ASM instance so we can define a key for the database.

Connected to:

SQL> quit
Production
[oracle@db ~]$
26. On the DB system, create the directory that will hold the keyfile. Do NOT use the $ORACLE_BASE/admin
directory!

[oracle@db db ~]$ mkdir -p $ORACLE_HOME/admin/ORCL/pfile
[oracle@db db ~]$ cd $ORACLE_HOME/admin/ORCL/pfile
27. Create a dummy keyfile, without the correct key (so we can verify that security works).
[oracle@db pfile]$ echo key=12345 > cellkey.ora

[oracle@db pfile]$ echo asm=+ASM >> cellkey.ora
[oracle@db pfile]$ chmod 660 cellkey.ora
[oracle@db pfile]$ cat cellkey.ora
key=12345
asm=+ASM
[oracle@db pfile]$
28. Start the database again, this will fail as we didnt provide the system with the right key.
[oracle@db pfile]$ . oraenv

[oracle@db pfile]$ sqlplus / as sysasm
SQL> startup

SQL> quit
Production
[oracle@db pfile]$ srvctl start database -d ORCL
PRCR-1079 : Failed to start resource ora.orcl.db
ORA-00205: error in identifying control file, check alert log for more info
CRS-2674: Start of 'ora.orcl.db' on 'db' failed
[oracle@db pfile]$
29. Shutdown the database again and re-create the cellkey.ora file but this time use the proper values.

[oracle@db ~]$ cd $ORACLE_HOME/admin/ORCL/pfile
[oracle@db pfile]$ srvctl stop database -d ORCL
PRCC-1016 : ORCL was already stopped
[oracle@db pfile]$ echo key=2cba467959845acb84a0ffa39270e924 > cellkey.ora
[oracle@db pfile]$ echo asm=+ASM >> cellkey.ora
[oracle@db pfile]$ chmod 600 cellkey.ora
[oracle@db pfile]$ cat cellkey.ora
key=2cba467959845acb84a0ffa39270e924
asm=+ASM
[oracle@db pfile]$
30. Start the database and check the status of the database.
[oracle@db pfile]$ srvctl start database -d ORCL

[oracle@db pfile]$ srvctl status database -d ORCL
[oracle@db pfile]$
Task 2: Flash Cache

Flashcache can be maintained automatically or you can specify on table, partition and tablespace level that certain
objects should be kept in the flash cache.
1. Login into the Exadata Storage cell cell1 (password oracle) and look what objects are currently in the
Flash Cache using the CellCLI command. The Efficiency ratio displayed is the ratio between data accessed by
the storage cell and data processed by the database for the cell. Larger values mean more offloading to the
cell. The value will be different in your environment.


CellCLI> list flashcachecontent detail
cachedKeepSize: 0
cachedSize: 401408
dbID: 2539498073
hitCount: 0
missCount: 0
objectNumber: 76669
tableSpaceNumber: 7
cachedKeepSize: 0
cachedSize: 1343488
dbID: 2539498073
hitCount: 1
missCount: 0
objectNumber: 76670
tableSpaceNumber: 7
CellCLI>
2. The result is a long list of objects which are currently in the flash cache, The attribute objectNumber refers to
the object_id in user_objects and by querying this table you can see exactly which objects are stored in flash
cache. Login to the database as the SYS user and take one of the numbers and execute the following query to
see which object it is.
SQL> select object_name, object_type from dba_objects where object_id = 76670;
OBJECT_NAME
----------------------------------------------------------------OBJECT_TYPE
-------------------
LARGE_EHCC_QUERY
TABLE
SQL>
Task 3: Storage Indexes

While we were using the Exadata Cells, another feature was automatically maintained; Storage Indexes.
1. Login into the database as the user test (password test) and execute the following query to see the current
statistics for storage indexes (or execute the script /home/oracle/scripts/7-1.sql).

[oracle@db pfile]$ sqlplus test/test
Connected to:
SQL> col MB format 9,999.9

SQL> SELECT a.name, b.value/1024/1024 MB
FROM v$sysstat a, v$mystat b
WHERE a.statistic# = b.statistic#
AND (a.name IN ('physical read total bytes', 'physical write total bytes', 'cell IO
uncompressed bytes') OR a.name LIKE 'cell phy%');
NAME MB
---------------------------------------------------------------- ---------
physical read total bytes 122.0
physical write total bytes .0
cell physical IO interconnect bytes .0
cell physical IO bytes saved during optimized file creation .0
cell physical IO bytes saved during optimized RMAN file restore .0
cell physical IO bytes eligible for predicate offload 121.9
cell physical IO bytes saved by storage index 121.0
cell physical IO interconnect bytes returned by smart scan .0
cell IO uncompressed bytes .9
9 rows selected.
SQL>
2. To see the benefits of storage indexes and query offloading we need to clear the storage indexes. Start by
stopping the database and ASM instance on the db node (if you dont do this the database might be corrupted
as we are simulating the Exadata environment in virtual machines).

[oracle@db ~]$
Connected to:

SQL> exit
Production
[oracle@db ~]$
3. Verify that the database and ASM instance is not running before proceeding to the next step.

Database is not running.
[oracle@db ~]$ srvctl status asm
ASM is not running.
[oracle@db ~]$
4. To see the benefits of storage indexes and query offloading we need to clear the storage indexes. As storage
indexes are only cleared when the cell is offline, we will stop and start the cells. Login into cell1 as the root
user (password oracle) and execute the following commands and logout of the cell.
[oracle@db pfile]$ ssh root@cell1

Last login: Thu Aug 25 10:14:53 2011 from db.exadata.local
[root@cell1 ~]# cellcli

CellCLI> alter cell shutdown services all
Stopping the RS, CELLSRV, and MS services...

The SHUTDOWN of services was successful.
CellCLI> alter cell startup services all
Starting the RS, CELLSRV, and MS services...

Getting the state of RS services...
running
Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
Starting MS services...
The STARTUP of MS services was successful.
CellCLI> exit
quitting
[root@cell1 ~]# exit

logout

[oracle@db pfile]$
5. Logon to cell2 as the root user and execute the same commands. Do NOT do this until the first cell is up and
running again. The Efficiency ratio displayed is the ratio between data accessed by the storage cell and data
[oracle@db pfile]$ ssh root@cell2

Last login: Thu Aug 25 10:14:59 2011 from db.exadata.local
[root@cell2 ~]# cellcli

CellCLI> alter cell shutdown services all
Stopping the RS, CELLSRV, and MS services...

The SHUTDOWN of services was successful.
CellCLI> alter cell startup services all
Starting the RS, CELLSRV, and MS services...

Getting the state of RS services...
running
Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
Starting MS services...
The STARTUP of MS services was successful.
CellCLI> exit
quitting
[root@cell2 ~]# exit

logout

[oracle@db pfile]$
6. Startup the ASM instance.

SQL> startup

SQL>
7. Exit the SQL*Plus Session and start the database instance. Verify that the database is running.

[oracle@db ~]$ srvctl start database -d ORCL
[oracle@db ~]$
8. Login to the database as the test user (password test) and check the statistics (or execute the script
/home/oracle/scripts/7-1.sql). The cell physical IO bytes saved by storage index should be close to
0

NAME MB
--------------------------------------------------------------- ----------
physical read total bytes .296875
physical write total bytes 0
cell physical IO bytes saved during optimized file creation 0
cell physical IO bytes saved during optimized RMAN file restore 0

cell physical IO bytes eligible for predicate offload 0
cell physical IO bytes saved by storage index 0
cell physical IO interconnect bytes returned by smart scan 0
cell IO uncompressed bytes 0
9 rows selected.
SQL>
9. Run a query with timing on and check the statistics. Write down the result on how long the query took.
SQL> set timing on

SQL> select count(*) from large_table where id between 100 and 200;
COUNT(*)
----------
101
Elapsed: 00:00:01.44
10. Execute the same query again and note the timing. As you can see the second execution was a lot faster. This
is because we are using the offloading capabilities and storage indexes.
SQL> select count(*) from large_table where id between 100 and 200;
COUNT(*)
----------
101
Elapsed: 00:00:00.02
11. Check the statistics for the query offloading by executing the following query (or run the script
/home/oracle/scripts/7-1.sql).

NAME MB
--------------------------------------------------------------- ----------
physical read total bytes 244.3125
physical write total bytes 0
cell physical IO bytes saved during optimized file creation 0
cell physical IO bytes saved during optimized RMAN file restore 0
cell physical IO bytes eligible for predicate offload 243.984375
cell physical IO bytes saved by storage index 121.070313
cell physical IO interconnect bytes returned by smart scan .021774292
cell IO uncompressed bytes 122.914063
9 rows selected.
Elapsed: 00:00:00.02
SQL>
Task 4: Invisible Indexes

In this task you will make an index invisible so you can see the effects of removing an index on your queries and how
this effects offloading the execution of the query.
1. Open a terminal window (or use an already open one) and make sure that you are the oracle user.
2. Connect to the database as the test user.

Connected to:
SQL>
3. Create an index on the customers table
SQL> CREATE INDEX cust_id_idx ON customers (customer_id);

Index created.
SQL>
4. The object USER_INDEXES has a new column named VISIBILITY to indicate whether the index is visible or
not.
SQL> SELECT index_name, visibility FROM user_indexes WHERE index_name = 'CUST_ID_IDX';
INDEX_NAME VISIBILIT
------------------------------ ---------
CUST_ID_IDX VISIBLE
SQL>
5. Create the statistics for the TEST schema
SQL> EXECUTE DBMS_STATS.GATHER_SCHEMA_STATS('TEST',DBMS_STATS.AUTO_SAMPLE_SIZE);
PL/SQL procedure successfully completed.
SQL>
6. Configure your session to display execution plans.
SQL> SET autotrace ON explain
System altered.
SQL>
7. Run the following query. Note that the execution plan uses an index range scan on the CUSTOMERS_ID_IDX.
SQL> SELECT AVG(credit_limit) FROM customers WHERE customer_id BETWEEN 1000 AND 1500;
AVG(CREDIT_LIMIT)
-----------------
2728.14516
Elapsed: 00:00:00.01
Execution Plan
----------------------------------------------------------
--------------------------------------------------------------------------------
------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)

| Time |
--------------------------------------------------------------------------------
------------
| 0 | SELECT STATEMENT | | 1 | 10 | 20 (0)

| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 10 |
| |
| 2 | TABLE ACCESS BY INDEX ROWID| CUSTOMERS | 50 | 500 | 20 (0)

| 00:00:01 |
|* 3 | INDEX RANGE SCAN | CUST_ID_IDX | 50 | | 2 (0)

| 00:00:01 |
--------------------------------------------------------------------------------
------------
Predicate Information (identified by operation id):

---------------------------------------------------
3 - access("CUSTOMER_ID">=1000 AND "CUSTOMER_ID"<=1500)
SQL>
8. Make the CUSTOMER_ID_IDX invisible.
SQL> ALTER INDEX cust_id_idx invisible;
Index altered.
SQL>
9. Verify that the index is invisible.
SQL> SELECT index_name, visibility FROM user_indexes WHERE index_name = 'CUST_ID_IDX';
INDEX_NAME VISIBILIT
------------------------------ ---------
CUST_ID_IDX INVISIBLE
SQL>
10. Run the same query again as in step 7. Notice that an Exadata Smart Scan is used rather than an index range
scan.
SQL> SELECT AVG(credit_limit) FROM customers WHERE customer_id BETWEEN 1000 AND 1500;
AVG(CREDIT_LIMIT)
-----------------
2728.14516
Execution Plan
----------------------------------------------------------
--------------------------------------------------------------------------------
--------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Ti

me |
--------------------------------------------------------------------------------
--------
| 0 | SELECT STATEMENT | | 1 | 10 | 264 (1)| 00

:00:04 |
| 1 | SORT AGGREGATE | | 1 | 10 | |
|
|* 2 | TABLE ACCESS STORAGE FULL| CUSTOMERS | 50 | 500 | 264 (1)| 00

:00:04 |
--------------------------------------------------------------------------------
--------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage("CUSTOMER_ID"<=1500 AND "CUSTOMER_ID">=1000)

filter("CUSTOMER_ID"<=1500 AND "CUSTOMER_ID">=1000)
11. Exit your SQL*Plus session
SQL> exit
Production
[oracle@db ~]$
Appendix Troubleshooting
Objective: Resolve issues with the environment
Connecting to VNC (MAC)
To connect to VNC using MAC, use Chicken of the Sea VNC and connect to VNC running on port 50.
Display: 50
Password: oracle
SSH Issues
Delete /home/oracle/.ssh/known_hosts file (or rename it) on the offending server
Fixing IP Issues
Cells
1. Login as root on both cells
2. Stop the cell service
a. service celld stop
3. Get the ipaddress of the server
a. ifconfig
4. Edit the network file
a. vi /etc/sysconfig/network-scripts/ifcfg_eth0
IPADDR=10.216.#.#
NETMASK=255.255.252.0
BOOTPROTO=static
5. Restart the network service
a. service network restart
6. Redploy the Exadata services
a. sh $OSS_SCRIPTS_HOME/unix/setup_dynamicDeploy
7. Startup the Exadata services
a. cellcli e alter cell startup services rs
b. cellcli e alter cell startup services ms
c. cellcli e create cell interconect1=eth0
DBS
1. Login as root
2. Get the ip address
a. ifconfig
3. Edit nework
a. vi /etc/sysconfig/network-scripts/ifcfg_eth0
IPADDR=10.216.#.#
NETMASK=255.255.252.0
BOOTPROTO=static
4. service network restart
Celldisk Error
CellCLI> create celldisk all
No cell disks qualified for this create operation
CellCLI> drop celldisk ALL HARDDISK force

CellDisk CD_disk01_cell2 successfully dropped
..

..
Restart Celld Process
/etc/rc.d/init.d/celld status|stop | start
How to start ASM if its down

1. cd $ORACLE_HOME/bin
2. ./crsctl start resource ora.cssd
3. .crs_stat t
4. sqlpplus / as sysasm
5. startup

Exadata 2014 Implementation Specialist Bootcamp Hands v3

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Exadata 2014 Implementation Specialist Bootcamp Hands v3

Transféré par

Droits d'auteur :

Formats disponibles

Exadata 2014 Implementation Specialist Boot Camp

Exadata 2014 Implementation Specialist Bootcamp

Safe Harbor Statement

Oracle Training Materials Usage Agreement

CREATE DISKS AND ASM DISKGROUPS 9

BULK LOADING LAB 42

DCLI AND OTHER MONITORING 56

TASK 3: STORAGE INDEXES 76

Lab A Connect to the Environment

Database Server (db)

Exadata Storage Cell (cell1)

Task 1: Connect to the environment:

Array SSL Username Password

Task 2: Connect to the Exadata Storage Cells

2. You are now connected as the oracle user on the db node.

Task 3: Check Cell Environment

Lab B Create Disks and ASM Diskgroups

Task 1: Create cell disks

Copyright (c) 2007, 2009, Oracle. All rights reserved.

Copyright (c) 2007, 2009, Oracle. All rights reserved.

CellCLI> list lun where disktype = harddisk

Task 2: Create Grid disks and partition them

Copyright (c) 2007, 2009, Oracle. All rights reserved.

archive_CD_disk05_cell1 208M UNUSED

Copyright (c) 2007, 2009, Oracle. All rights reserved.

CellCLI> list celldisk attributes name, freespace where freespace != 0

Lab C Create Database

Task 1: Connect and verify the cells to the database environment

oracle 2573 1 0 Oct26 ? 00:00:00 asm_mmon_+ASM

13: 240 Mb o/cell1/data_CD_disk01_cell1 <unknown> <unknown>

25: 208 Mb o/cell2/archive_CD_disk01_cell2 <unknown> <unknown>

37: 240 Mb o/cell2/data_CD_disk01_cell2 <unknown> <unknown>

Task 2: Create ASM diskgroups

we create a diskgroup using SQL*Plus. Login to the ASM environment:

SQL*Plus: Release 11.2.0.1.0 Production on Wed Sep 28 14:24:45 2011

Copyright (c) 1982, 2009, Oracle. All rights reserved.

PATH LABEL HEADER_STATU

o/cell1/data_CD_disk01_cell1 DATA_CD_DISK01_CELL1 CANDIDATE

o/cell2/archive_CD_disk01_cell2 ARCHIVE_CD_DISK01_CELL2 CANDIDATE

o/cell2/data_CD_disk01_cell2 DATA_CD_DISK01_CELL2 CANDIDATE

NAME NAME FAILGROUP STATE

DATA DATA_CD_DISK09_CELL1 CELL1 NORMAL

DATA DATA_CD_DISK08_CELL1 CELL1 NORMAL

6. Examine your newly created diskgroup using the following query

NAME TOTAL_MB STATE

10. Click Create to create a new ASM diskgroup

15. Exit the ASM GUI tool.

Task 3: Create a database using the Exadata Storage Cells (ASM)

4. Select the Create Database option and click Next.

5. Select the Data Warehosue template and click Next.

6. Name the database ORCL and click Next.

7. Accept the default values and click Next.

14. Accept the default values and click Next.

15. Check the Create Database checkbox and click Finish.

19. When dbca is finished click Exit.

20. Set the environment variables to the Grid (ASM) installation

[oracle@db ~]$ . oraenv

[oracle@db ~]$ crsctl stat res -w "TARGET = ONLINE" -t

Lab D Smart Scan

Task 1: Load data into the database

[oracle@db ~]$ . oraenv

SQL*Plus: Release 11.2.0.1.0 Production on Tue Oct 11 11:18:47 2011

Copyright (c) 1982, 2009, Oracle. All rights reserved.

2. Create a tablespace that can hold the data we will load