Advanced Technical Support, eServer, pSeries

Virtual Memory Management Tuning Parameter: lru_file_repage

Barry J. Saad Harold R. Lee

ATS Performance Analysis

6/1/2006

2005 IBM Corporation

Advanced Technical Support - pSeries

VMM Tunable: lru_file_repage

Purpose: Consider the re-page count when determining what type of memory to steal. Range: 1 (default) use computational and file memory repage counters when determining what type of memory to steal. 0 steal only file-memory When to Tuning: Protect computational memory pages - application maintains its own cache Monitoring with vmstat, if you have activity in the sr scan and fr free columns, which indicates the lrud is running Monitoring with vmstat, if you have activity in the pi and po columns, which indicating paging activity to page space. Remember this is a hint to VMM and other conditions can change the determination of what memory to steal; therefore, dont forget the other VMM tuning parameters (most notable: set maxclient and maxperm to low values and set strict_maxperm and strict_maxclient to soft limits).
Steal only file-memory
Yes

S Ig et t no he re v re alu pa e ge ot co 0 un ts

lru_file_repage Se tt he Co val ns ue i de t o rr 1( ep de ag fau ing lt)

to

to

Is the computationalmemory repage greater than the file-memory repage

No

Steal either computational or file memory

ATS Performance Analysis

6/1/2006

2005 IBM Corporation

Advanced Technical Support - pSeries

Virtual Memory Manager Tunable Parameter: lru_file_repage

The lru_file_repage parameter was introduced at ML4 of AIX5L 5.2 and ML1 of AIX5L 5.3. The parameter provides a hint to the VMM about whether re-page counts should be considered when determining what type of memory to steal. The process starts when the VMM needs memory and the lrud process starts running to steal memory. During the process the VMM will make a determination to steal any memory type or limit the stealing to file cache memory. This determination is made based on the setting of a number of memory manager parameters and kernel counters, including:
Memory Manager Tuning Parameters Kernel Counters maxperm and strict_maxclient numperm minperm numclient maxclient and strict_maxclient file re-page lru_file_repage working re-page

This discussion will focus in on how the setting of lru_file_repage influences the memory steal selection determination. What is a repage? A page fault is consider to either a new page fault or a repage fault. A repage fault occurs when a page that is known to have been referenced recently is referenced again, and is not found in memory because the page has been replaced. In a sense, a repage can be viewed as a failure in the page selection algorithm in an ideal world you would not have any repage faults. To classify a page fault as new or repage, the VMM maintains a repage history buffer and maintains two counters that estimate computational-memory repaging and file-memory repaging. The repaging rates are multiplied by 90% each time the page-replacement algorithm runs, so that they reflect recent repaging activity more strongly than historical repaging activity.

ATS Performance Analysis

6/1/2006

2005 IBM Corporation

Advanced Technical Support - pSeries

Tuning Options By setting lru_file_repage = 0 youre telling the VMM that your preference is to steal file-memory pages only. By setting lru_file_repage = 1, which is the default, youre telling the VMM that your preference is to make the determination based on the repage counters, as shown in the figure. Now, a number of other conditions can exist that would change your preference, including: numperm less than minperm -- stealing either computational or file memory numperm greater than maxperm -- stealing filememory only numclient greater than maxclient steal filememory only Finally, a number of other boundaries conditions can change the determination as well when the lrud determines that forward progress to its defined page stealing goal isnt being achieved. When does this value need to be tuned? The reason to change the default value is that youve decided that you need to protect computational memory, and the system is running under the large memory model. The large memory model says that the computational memory requirements is less than 75% of real memory. The easier way to calculate the computational memory requirement is to a run a svmon command when the system is fully loaded.
# svmon size memory pg space 4 233472 131072 inuse 227544 13005 6/1/2006 2005 IBM Corporation free 5928 pin 42518 virtual 84350 computational % = (84350/233472) * 100 = 36%
No

S Ig et t no he re v re alu pa e ge ot co 0 un ts

lru_file_repage Se tt he Co val ns ue i de t o rr 1( ep de ag fau ing lt)

to

to

Steal only file-memory

Yes

Is the computationalmemory repage greater than the file-memory repage

Steal either computational or file memory

ATS Performance Analysis

Advanced Technical Support - pSeries

In general, when the system is running an application that maintains its own cache you should alter this value and set lru_file_repage to 0. Of course, if the page replacement daemon lrud never runs then changing this value will have no impact system behavior. The easier way to tell if the lrud is running use the vmstat command if you note values in the sr and fr column then the lrud is or has been running and it may be appropriate to set the value. Now, when you run vmstat and note paging activity to page space (i.e., value in the pi and po) setting the value of lru_file_repage to 0 should stop that behavior. If it doesnt then you know one of the other conditions is impacting the selection process and tuning the other VMM parameters is required to get the desired system results. Finally, as with all tunable parameter changes, you should note the behavior of the system before and after the change- in this case I would use vmstat. If performance improves leave the parameter as set. If performance degrades run the parameter to the original value. Remember, it is not uncommon to pass the optimal point in the tuning process, when you do - dial back the last set of changes and take a baseline snapshot using the the vmo a or vmo -L command.

ATS Performance Analysis

6/1/2006

2005 IBM Corporation