Académique Documents
Professionnel Documents
Culture Documents
Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance
$ SOLUTION VERIFIED - Updated July 20 2017 at 9:55 PM - English ()
Environment
Red Hat Enterprise Linux 7
Red Hat Enterprise Linux 6
Red Hat Enterprise Linux 5
Red Hat Enterprise Linux 4
Oracle 9g
Oracle 10g
Oracle 11g
Oracle 12c
Issue
How can I tune my Red Hat Enterprise Linux system for Oracle 10g/11g?
High memory consumption on Oracle database system
High CPU consumption on Oracle hosts
How should I congure swap memory for an Oracle database?
Slower than expected IO performance running Oracle on RHEL
What is the maximum number of huge pages I should be using per system?
Resolution
IMPORTANT: This article is not an authoritative source of information on how to congure Red Hat Enterprise Linux for use with Oracle products, as
these products are not shipped by Red Hat. This is just an example of how the conguration may look like and is provided just for the convenience
only. The authoritative source of such information is the Oracle Installation Guide or Oracle OS Tuning Guide for your version of Oracle product,
such a documentation could be found on Oracle Metalink site, namely:
RHEL4 32 bit (x86):
Note 303859.1 - Requirements for Installing Oracle 9iR2 on RHEL 4
Note 392940.1 - Requirements for Installing Oracle 10.1.0.x RDBMS on RHEL 4 x86 platform
Note 343431.1 - Requirements for Installing Oracle 10gR2 RDBMS on RHEL 4 x86 platform
Note 430653.1 - Requirements for Installing Oracle 11gR1 32-bit on RHEL 4
Note 880211.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 4 x86
RHEL4 64 bit (x86-64):
Note 353529.1 - Requirements for Installing Oracle 9iR2 64-bit on RHEL 4 x86-64 (AMD64/EM64T)
Note 390900.1 - Requirements for Installing Oracle 10g (10.1.0.x) RDBMS on RHEL 4 on AMD64/EM64T (Linux x86-64)
Note 339510.1 - Requirements for Installing Oracle 10gR2 RDBMS on RHEL 4 on AMD64/EM64T
Note 437123.1 - Requirements for Installing Oracle 11gR1 RDBMS on RHEL 4 on AMD64/EM64T
Note 880942.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 4 on AMD64/EM64T
RHEL5 32 bit (x86):
Note 419646.1 - Requirements For Installing Oracle 10gR2 On RHEL 5 (x86)
Note 438765.1 - Requirements for Installing Oracle 11gR1 32bit RDBMS on RHEL 5
Note 880936.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 5 on 32-bit x86
RHEL5 64 bit (x86-64):
Note 421308.1 - Requirements For Installing Oracle10gR2 On RHEL 5 (x86_64)
Note 438766.1 - Requirements for Installing Oracle 11gR1 RDBMS on RHEL 5 on AMD64/EM64T
Note 880989.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 5 on AMD64/EM64T
Note 1529433.1 - Requirements for Installing Oracle Database 12.1 on RHEL5 or OL5 64-bit (x86-64)
RHEL6 64 bit (x86-64):
Note 1441282.1 - Requirements for Installing Oracle 11gR2 RDBMS on RHEL6 or OL6 64-bit (x86-64)
Note 1529864.1 - Requirements for Installing Oracle Database 12.1 on RHEL6 or OL6 64-bit (x86-64)
https://access.redhat.com/solutions/39188 1/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
Master Note of Linux OS Requirements for Database Server
(https://blogs.oracle.com/db/entry/master_note_of_linux_os_requirements_for_database_server)
Where can I nd an Oracle Tuning Guide for Red Hat Enterprise Linux? (https://access.redhat.com/solutions/26368)
Automatic conguration
Red Hat provides a RHEL Tuner for Oracle (https://access.redhat.com/labs/rheltfo/) tool to help tune your Red Hat Enterprise Linux for Oracle. The tool
incorporates the information included in this document but makes it easier to generate valid and support-recommended congurations. You can leave
feedback on the tool at RHEL Tuner for Oracle (https://access.redhat.com/labsinfo/rheltfo).
Manual conguration
Note: All the numbers mentioned in this article are not generic and have been shown to have a positive effect on database workloads resident on
Red Hat Enterprise Linux. That being said, the specic values depend on user environments and may require further adjustment.
Swapping for Oracle is not ideal and should be avoided as much as possible. The following tunable will tune the kernel to swap less aggressively.
vm.swappiness=10
For example if a system has 1000 pages of memory and dirty_background_ratio is set to 3% , writeback will begin when 30 pages have been dirtied.
vm.dirty_background_ratio=3
If it is set to 40% on a 1000 page system, a process dirtying pages will be made to wait once the 400th page is dirtied.This mechanism will, thus, slow the
dirtying of pages while the system catches up.
vm.dirty_ratio=40
vm.dirty_expire_centisecs=500
How often pdush is activated to clean dirty pages (in hundreds of a second):
vm.dirty_writeback_centisecs=100
HugePages
2. Calculate the recommended number of HugePages using the formula below, using Hugepagesize and the System Global Area (SGA) values from
Oracle. See Oracle documentation for more about SGA and where to nd those values. Note that the Program Global Area (PGA) does not use huge
pages.
The following is an example. Values were are all converted to kilobytes for calculation:
https://access.redhat.com/solutions/39188 2/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
Hugepagesize = 2048 KB
20 GB SGA = 20971520 KB
vm.nr_hugepages=10240
vm.hugetlb_shm_group=<insert oracle group ID here>
4. Once the above setting has been tested and shown to provide the performance required, it is recommended to move HugePage allocation to the kernel
line in /boot/grub/grub.conf as a boot option. This allows for earlier and cleaner allocation of HugePages:
If Hugepages are not set in kernel, they are dynamically allocated by setting the value in nr_hugepages.
When the system is under memory pressure, the huge pages cannot be swapped.
5. Change the Oracle database conguration to use the Huge Pages. Contact Oracle support if assistance is needed
Note: In order for Oracle database to use Huge Pages in Red Hat Enterprise Linux 4, 5 or 6, you may also need to increase the ulimit parameter
"memlock" for the oracle user in /etc/security/limits.conf. The memlock setting is specied in KB and must match the memory size of the number of
Huge Pages that Oracle should be able to allocate. So, if the Oracle database should be able to use 512 Huge Pages, then memlock must be set to at
least (512 * Hugepagesize), which on a default system would be 1048576 KB (512*2048).
Shared Memory
SHMALL is the maximum total amount of shared memory pages that can be used system-wide. Hence, SHMALL should always be at least
ceil(SHMMAX/PAGE_SIZE) .
SHMMAX is the maximum size in bytes of a single shared memory segment that can be created.
page=$(getconf PAGE_SIZE)
Calculate 75% of the total memory in the system in pages for SHMALL :
Set the SHMMAX and the SHMALL values in the /etc/sysctl.conf le:
Set the maximum number of shared memory segments with SHMMNI in the /etc/sysctl.conf le:
https://access.redhat.com/solutions/39188 3/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
The rst value, SEMMSL, is the maximum number of semaphores per semaphore set
The second value, SEMMNS, denes the total number of semaphores for the system
The third value, SEMOPM, denes the maximum number of semaphore operations per semaphore call
The last value, SEMMNI, denes the number of entire semaphore sets for the system
#vi /etc/security/limits.conf
#<domain> <type> <item> <value>
oracle hard nofile 10000
Disabling transparent hugepages (THP) recommended just for Red Hat Enterprise Linux 6
OR
# tuned-adm off
Note: The tuned-adm command will revert all your settings to what they were before tuned started and disable the tuning services from running at boot.
As an alternative, a customized tuned prole can be created, taking over the old settings and only disabling THP. Refer to kbase Disabling transparent
hugepages (THP) on Red Hat Enterprise Linux 6 is not taking effect. (https://access.redhat.com/site/solutions/422283) for details.
# reboot
I/O scheduler
The default CFQ I/O scheduler is appropriate for most workloads, but does not offer optimal performance for database environments. Instead, one
of the following IO schedulers should be used:
The deadline scheduler is recommended for physical systems (https://access.redhat.com/solutions/54164)
The noop scheduler is recommended for virtual systems (https://access.redhat.com/solutions/5427)
Reference
* 2016 - Deploying Oracle RAC Database 12c on Red Hat Enterprise Linux 7 - Best Practices (https://access.redhat.com/articles/1357883)
* 2016 - Deploying Oracle Database 12c on Red Hat Enterprise Linux 7- Recommended Practices (https://access.redhat.com/articles/1282303)
* 2014 - Red Hat Enterprise Linux 7 - Using Tuned for Tuning an Oracle Workload (https://access.redhat.com/videos/898563)
* 2014 - Deploying Oracle Database 12c on Red Hat Enterprise Linux 6 (https://access.redhat.com/articles/725843)
https://access.redhat.com/solutions/39188 4/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
* 2013 - Deploying Oracle RAC 11g R2 Database on Red Hat Enterprise Linux 6 - Best Practices (https://access.redhat.com/site/articles/479093)
* 2013 - Deploying Oracle Database 11g R2 on Red Hat Enterprise Linux 6 - Best Practices (https://access.redhat.com/articles/395013)
* 2008 - Deploying Oracle 10g RHEL 5 (https://access.redhat.com/site/articles/215753)
Tags labs_rheltfo (/taxonomy/tags/labsrheltfo) oracle (/tags/oracle) performance (/tags/performance) performance_tuning (/tags/performancetuning) rhel_4 (/tags/rhel_4)
This solution is part of Red Hats fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while
supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited
form.
33 Comments
17 October 2011 2:55 PM (https://access.redhat.com/solutions/39188#comment-334603)
30 Points How often pdush is activated to clean dirty pages (hundreths of a.... vm.dirty_writeback_centisecs=100
Reply (/Ajax_comments/Reply/39188/334603)
Paul Qu (/user/1135473)
(/user/1135473)
If use Huge Page, no need for other vm.* parameters tuning as Huge Page memory does not swap.
NEWBIE
https://access.redhat.com/solutions/39188 5/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
Yogesh Babar (/user/10283)
Hello,
(/user/10283)
RED HAT Thanks for catching this, indeed it should be /etc/sysctl.conf. I have corrected the article.
PRO
729
Reply (/Ajax_comments/Reply/39188/436023)
Points
COMMUNITY
MEMBER
Reply (/Ajax_comments/Reply/39188/499703)
57 Points
EXPERT That is a great question. I don't know of any way that non-Red Hat associates can see what changed.
1003
Points I took a look at the changes. The only substantive changes were the recommendations relating to kernel.sem.
Reply (/Ajax_comments/Reply/39188/563153)
ACTIVE You are right on the PGA does not use huge pages.The goal of setting up huge pages is to provide the correct number of pages to handle the
CONTRIBUTOR
running shared memory segments used by the SGA. An easy way to calculate the number of huge pages without using the
182 Points
hugepages_settings.sh script created by Oracle is to take the size of your SGA in kilobytes and divide that by the huge page size from the
OS. In RHEL, the page size is 2048 kb. So the calculation would be: SGA in kilobytes / 2048 (huge page size). Once you have this size, you
should add a few additional huge pages just to ensure all of the SGA can t in large pages. After you have set this piece up, you also need to
set the memlock parameter within your limits.d conf le located in /etc/security/limits.d/ directory). Setting memlock allows the oracle user
to lock a certain amount of memory from physical RAM that isn't swapped out. The value is expressed in kilobytes and is important from the
https://access.redhat.com/solutions/39188 6/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
Oracle perspective because it provides the oracle user permission to use huge pages. So the next question might be how do I know if Oracle
is using the huge pages? Well you can check cat /proc/meminfo and look at the hugepages total and hugepages free and see how the value
goes down when you start your Oracle db instance. The other place to look is in the Oracle alert log. Within the alert log search for the word
'Large' and you should see something like:
In this example the key line was: Total System Global Area in large pages = 19 GB (100%) as it shows that the Total SGA is in large pages.
As a shameless plug check out my Deploying Oracle 11gR2 on RHEL6 reference architectures at:
https://access.redhat.com/site/articles/395013 and my Deploying Oracle 11gR2 RAC on RHEL6 at:
https://access.redhat.com/site/articles/479093
Roger
Reply (/Ajax_comments/Reply/39188/691593)
ACTIVE
CONTRIBUTOR Reply (/Ajax_comments/Reply/39188/691603)
182 Points
Reply (/Ajax_comments/Reply/39188/814223)
https://access.redhat.com/solutions/39188 7/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
COMMUNITY
MEMBER I'm interested too to know why the author of the article (good for the other aspects) take in account also PGA and number of oracle
41 Points processes.
Regards
Reply (/Ajax_comments/Reply/39188/818063)
Reply (/Ajax_comments/Reply/39188/693103)
ACTIVE I'm not sure why the log shows the memlock value as unlimited, because I denitely have a value set. I don't think there is a great risk of
CONTRIBUTOR
Oracle getting out of hand and xing more memory than its SGA. Oracle is just looking to see if large pages are available and if it can
182 Points
accommodate all the memory of the SGA into large pages. As for the value set for memlock, I'd set it to slightly larger than what you believe
will be your largest SGA at any point in time. This way all your use cases with a smaller SGA are taken care of.
With regards to the dirty_ratio, the reason I increased the value from what you normally see as 15 to 80 is based on my understanding of
dirty ratio. Dirty ratio denes the maximum percentage of total memory that can be lled with dirty pages before user processes are forced
to write dirty buffers themselves during their time slice instead of being allowed to do more writes. Note that all processes are blocked for
writes when this happens, not just the one that lled the write buffers. This can cause what is perceived as an unfair behavior where a single
proces can hog all the I/O on a system. However, an application that can handle their writes being blocked altogether might benet from
decreasing the value.
The biggest thing I want to point out about kernel parameters in general is that there is no perfect answer. Every workload is different and
requires performance testing. These values should just be seen as a starting point for your workload. The behavior of this parameter to my
knowledge is the same for RHEL5 and RHEL6 so it would apply to both distributions.
The vm.dirty_background_bytes and vm.dirty_bytes is something I'm looking into further but have not tested personally. Due to that, I'm
sticking to the use of ratios until I'm able to do some performance testing around the difference between using ratios vs bytes parameters.
Hope that helps and if you have any other questions feel free to ask :)
Take care,
Roger
Reply (/Ajax_comments/Reply/39188/693353)
vm.huge_tlb_shm_group = gid
Reply (/Ajax_comments/Reply/39188/726223)
Gi Kim (/user/2014383)
(/user/2014383) 75% of System Memory in bytes
NEWBIE mem=$(free|grep Mem|awk '{print $2}') # KiB
17 Points totmem=$(echo "$mem1024"|bc) # B
max=$(echo "$totmem75/100"|bc) # B
HugePages in KiB
huge=$(grep Hugepagesize /proc/meminfo|awk '{print $2}') # KiB
all=$(echo "$max/$huge"|bc)
So in the above calculation you have B / KiB .. i think $huge should also be put into the same metrics (Bytes).. (times $huge by 1024)
Reply (/Ajax_comments/Reply/39188/795873)
Gi Kim (/user/2014383)
(/user/2014383)
kernel.shmall is measured in "pages". In the summit notes and this guide it is assumed the pages is 2MiB (that is why you divide by
NEWBIE 'hugepagesize'). When i set the value in /etc/sysctl.conf and use ipcs -l to conrm the new size i nd that the O/S is multiplying the value in a
17 Points page size of 4KiB rather than 2MiB.
Any guidance on how to set and check shared memory values when hugepages is in use?
Reply (/Ajax_comments/Reply/39188/795913)
Why we reserve hugepages for PGA and for (20 KB * # of Oracle processes running) ?
Oracle stated that hugepages are used only for SGA(shared memory).
Reply (/Ajax_comments/Reply/39188/814143)
https://access.redhat.com/solutions/39188 9/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
Ameya Sathe (/user/3903963)
The solution does not mention the environment of Red Hat Enterprise Linux 7. Is it not veried?
(/user/3903963)
RED HAT Reply (/Ajax_comments/Reply/39188/888653)
COMMUNITY
MEMBER
27 Points
Reply (/Ajax_comments/Reply/39188/915923)
52 Points By the same token, why not integrate this article in this document, once all the parameters are validated?
Reply (/Ajax_comments/Reply/39188/915933)
HugePages in KiB
huge=$(grep Hugepagesize /proc/meminfo|awk '{print $2}') # KiB
all=$(echo "$max/$huge"|bc)
So in the above calculation you have B / KiB .. i think $huge should also be put into the same metrics (Bytes).. (times $huge by 1024)"
Reply (/Ajax_comments/Reply/39188/970763)
https://access.redhat.com/solutions/39188 10/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
Jose Gonzalez Goni (/user/9815943)
The calculation for shmall and shmmax is buggy. It seems that the free command does not always show the full number of bytes. i.e:
(/user/9815943)
NEWBIE #free -h
15 Points total used free shared buffers cached
Mem: 19G 18G 826M 4.9G 397M 16G
-/+ buffers/cache: 1.9G 17G
Swap: 49G 0B 49G
#free -b
total used free shared buffers cached
Mem: 2095443148 2008875008 865681408 5300641792 416825344 1759333171
-/+ buffers/cache: 2078593024 1887583846
Swap: 5368289280 0 5368289280
In this case, 19GB = 20954431488 bytes but free -b shows 2095443148 bytes (note the missing 8 at the end).
memkb=$(grep MemTotal /proc/meminfo | awk '{print $2}') mem=$(echo "$memkb * 1024" | bc)
Thanks
Reply (/Ajax_comments/Reply/39188/992693)
https://access.redhat.com/solutions/39188 11/12
8/9/2017 Tuning Red Hat Enterprise Linux for Oracle and Oracle RAC performance - Red Hat Customer Portal
https://access.redhat.com/solutions/39188 12/12