White Paper: Using Query Hints for Top SQL Performance

Author: Randy Cunningham

Introduction
Ask a diverse sample of Oracle professionals for their opinion concerning query hints, and their responses will tend to fall into one of three groupings: 1) What are hints all about, anyway? 2) Hints are one of the coolest features in Oracle, and they have rescued my career. 3) Hints are evil, and cause far more problems than they solve. Never, never use hints! In reality, hints can be detrimental to the performance and scalability of a database. Also, hints can be useful for achieving an optimally performing database. The key to success is understanding when, where, why and how hints should be used.

What is a hint?
A hint is a special syntax of a comment in a SQL statement block that specifies an instruction, or hint, to Oracles cost-based optimizer (CBO). The optimizer uses hints to influence or constrain its choice of an execution plan. Hints may appear in SQL originating from SQL*Plus, PL/SQL, Java, Toad, ODBC, C++, Cobol or any application with an Oracle SQL interface. Hints are specified within a statement block using either style of comment that Oracle supports: {DELETE|INSERT|SELECT|UPDATE} /*+ hint [text] [hint[text]]... */ or {DELETE|INSERT|SELECT|UPDATE} --+ hint [text] [hint[text]]... where: DELETE, INSERT, SELECT, or UPDATE is a query keyword that begins a statement block. Comments containing hints must appear immediately following these keywords. + is a plus sign that causes Oracle to interpret the comment as a list of hints. The plus sign must follow immediately after the comment delimiter (no space is permitted). hint is a validly constructed hint. The space between the plus sign and the hint is optional, and is recommended in PL/SQL prior to Oracle9i. If the comment contains multiple hints, then separate the hints by at least one space. text is other commenting text that can be interspersed with the hints.

What do hints do?

Hints enable you to influence goals, behaviors, methods and choices for which the optimizer ordinarily exercises defaults. Default optimizer behavior is determined by information such as statistics on the objects in the query, system statistics, session parameters and database parameters. Over 100 hints are categorized according to how they influence the CBO.
Optimization goals and approaches

Ordinarily the optimizer will choose a query plan designed to provide the best overall throughput and efficiency. However, quickest response might be a more important goal than overall throughput for an application such as a scrollable browser. You can bias the optimizer to do this with a more index-intensive plan by specifying the FIRST_ROWS hint. For example: SELECT /*+FIRST_ROWS*/ item_class, item_no, uom, description, qty FROM item_table JOIN item_availability

USING (item_no) WHERE item_class = :INPUT_CLASS AND qty > 0; The hints that influence optimization goals and approaches are: ALL_ROWS, FIRST_ROWS(n), CHOOSE and RULE.
Access Methods

You can recommend the use of a specific index, an index processing method, or even a full table scan, as a method for accessing a particular table that you specify. Influencing access methods is quite possibly the most prevalent use of hints, and it is replete with pitfalls. Note the use of an indexing hint in the subquery: SELECT domain_name, domain_owner FROM spammers m WHERE effdt = (SELECT /*+INDEX_DESC(s uce_effdt) */ MAX(effdt) -- retrieve only the latest FROM spammers s WHERE s.domain_name = m.domain_name); Here are the hints that influence a choice of access path: AND_EQUAL (table_alias index1 index2 ) CLUSTER (table_alias) FULL (table_alias) HASH (table_alias) NO_INDEX (table_alias [index_1 index2 ]) ROWID (table_alias) [index_name])
Join Order

INDEX (table_alias [index_name]) INDEX_ASC (table_alias [index_name]) INDEX_COMBINE (table_alias index1 index2 ) INDEX_DESC (table_alias [index_name]) INDEX_FFS (table_alias [index_name]) INDEX_JOIN (table_alias [index_name]) INDEX_SS (table_alias

The join order of tables in a FROM clause are affected with three hints, LEADING (table_alias), ORDERED and STAR. The STAR join method is useful only with a join of 3 or more tables, where the largest table has a concatenated index of at least 3 columns and there are no conflicting access or join method hints. The largest table is joined last in the join order, using a nested loops join to its qualifying concatenated index. The ORDERED hint will instruct the optimizer to join the tables in the order that they are specified, left to right, in the FROM clause. To be beneficial, use of this hint presumes that the query author has arranged the tables in the FROM clause according to immutable size and/or selectivity criteria, and that she understands some idiosyncrasy of the data distribution that the optimizer does not. This example query is constructed with the ORDERED hint: SELECT /*+ ORDERED */ e.last_name, e.first_name, b.branch_name FROM employee e JOIN branch b ON (branch_id) WHERE e.position = BRANCH MANAGER; The LEADING hint specifies which table is first in the join order, leaving the remainder of the ordering up to the CBO.
Join Operations

The choice of a join method is highly leveraged and can change the elapsed runtime of a query by a factor of up to several thousand times, favorably or unfavorably. Following is a synopsis of the hints that influence join operations. USE_HASH (table_1 [table_2 ]) HASH_AJ USE_MERGE (table_1 [table_2 ]) MERGE_AJ HASH_SJ MERGE_SJ

USE_NL (table_1 [table_2 ]) DRIVING_SITE (table_alias)

NL_AJ

NL_SJ

The anti-join hints HASH_AJ, MERGE_AJ and NL_AJ apply only to NOT IN or NOT EXISTS subquery blocks. The semi-join hints HASH_SJ, MERGE_SJ and NL_SJ are meaningful only when specified within EXISTS subquery blocks. Examples: SELECT /*+ DRIVING_SITE(wh1) */ wh1.* FROM my_list ml, big_warehouse_table@dwh wh1, lookup_table@dwh wh2 WHERE ml.market = wh1.market AND wh1.type = wh2.type AND wh2.status = ACTIVE; SELECT /*+ USE_HASH(E D) d.deptid, e.emplid */ FROM empl e JOIN dept d USING (e.empl_dept = d.deptid) WHERE d.dept_location = DENVER; SELECT division_name FROM division WHERE exists (SELECT /*+HASH_SJ*/ NULL FROM employee WHERE employee.division_id = division.division_id AND bonus > 25000);
Query Transformations

The query transformation category of hints has the effect of recasting the query as if it were written entirely differently for the same result. For example, observe the employment of a USE_CONCAT hint in this query: SELECT /*+ USE_CONCAT */ * FROM pub78 WHERE (state = 'CO' OR state = 'WY') AND UPPER(name) LIKE '%CATHOLIC%' The USE_CONCAT hint transforms the query (as confirmed with an EXPLAIN PLAN) as if it had been written: SELECT * FROM pub78 WHERE state = CO AND UPPER(name) LIKE '%CATHOLIC%' UNION ALL SELECT * FROM pub78 WHERE state = WY AND UPPER(name) LIKE '%CATHOLIC%' This is a list of the query transformation hints available: EXPAND_GSET_TO_UNION FACT (table_alias) MERGE (view_alias) REWRITE [(view1 [view2 ])] STAR_TRANSFORMATION
Parallel Execution

NO_EXPAND NO_FACT (table_alias) NO_MERGE (view_alias) NOREWRITE USE_CONCAT

Characteristics of query parallelism can be altered using parallel execution hints:

PARALLEL (table_alias, [degrees | DEFAULT][,{instances | DEFAULT}]) NOPARALLEL (table_alias) PQ_DISTRIBUTE (table_alias[,] outer_distribution, inner_distribution) PARALLEL_INDEX (table_alias, [degrees | DEFAULT][,{instances | DEFAULT}]) NOPARALLEL_INDEX (table_alias) This example is typical of the use of parallel query hints: INSERT /*+ APPEND PARALLEL(T_CPY,4) */ INTO t_cpy SELECT /*+ PARALLEL(T,2)*/ * FROM t;
Additional Operations

The final group of hints falls outside of the preceding categories. Here is a roster of documented additional hints: APPEND CACHE (table_alias) PUSH_PRED (view_alias) PUSH_SUBQ UNNEST NO_APPEND CURSOR_SHARING_EXACT NO_CACHE (table_alias) DYNAMIC_SAMPLING ([table_alias] level) NO_PUSH_PRED (view_alias) ORDERED_PREDICATES NO_PUSH_SUBQ NO_UNNEST

Included in this list are two hints that arguably can be incorporated into production SQL with little downside risk. The APPEND hint is used only in INSERT statements. Its inclusion instructs Oracle to bypass free space processing, and instead, to append new blocks to the table. This results in much quicker insert processing, at the possible cost of additional space usage. However, if the table is newly created, newly truncated, or if it has had few or no DELETE operations, there might not be a significant space trade-off at all. Index insertion is deferred when the APPEND hint is used. This is the direct-path insert method that can be optionally invoked with the SQL*Loader utility. Parallel INSERT implies APPEND. The DYNAMIC_SAMPLING hint is used to obtain cardinality and selectivity estimates on the fly. This can be a useful hint when performing queries on a global temporary table (GTT), for example, or where there is reason to believe that the table/index statistics might be inaccurate for use by the CBO. Notice that there is a required integer that specifies the level of sampling to be performed. Finally, there are a sizable number of undocumented hints. Ordinarily, these are used by Oracle internally, such as hints used for parallel query process coordination, and by Oracle support to identify or to work around particular CBO issues. Reliance on undocumented hints, even in development environments, is not recommended.

Why Wont Oracle Take My Hint?

Often, people complain that Oracle ignores their hints [6d]. Investigation almost always results in one of three explanations: The syntax of the hint is incorrect, so the hint is indeed disregarded. The hint is syntactically correct, but the method being hinted is not plausible to the CBO. For example, an INDEX hint might be properly formed, reference the table alias and specify a valid index for the table. However, if the index columns do not provide adequate selectivity, or all of the data required by the query, then the hinted access path will be disregarded. Event 10053, described later, can help you to understand which access paths, join orders and join methods are considered by the optimizer for a particular query. In fact, the hint is utilized in the query execution plan, but it fails to perform as expected. An easy way to determine whether or not the hint is taking effect is to generate explain plans of the query before and after inclusion of the hint.

A Few Preliminaries
Ensure that the database environment is using the cost-based optimizer (CBO). This is set with the database initialization parameter, OPTIMIZER_MODE, which ordinarily should be set to ALL_ROWS or to CHOOSE. Also, be sure that the COMPATIBLE and OPTIMIZER_FEATURES_ENABLE parameters are set to versions that support the hints you use. Confirm that the participating tables and their indexes in the query have statistics, and that the statistics are reasonably current. Be sure to confirm this for tables within views and subqueries. Check that necessary initialization parameters are set. For example, for the PARALLEL hint to work at all, you must have Oracle Enterprise Edition, and parallel query servers must be configured using Oracle database initialization parameters.

A Gallery of Ignored Hints

Incorrect: Correct: Explanation: Incorrect: Correct: Explanation: Incorrect: Correct: Explanation: Incorrect: Correct: Explanation: Incorrect: SELECT * FROM /*+ INDEX(inv,inv_pk) */ inv WHERE item_code like R%; SELECT /*+ INDEX(inv,inv_pk) */ * FROM inv WHERE item_code like R%; It is essential that the hint immediately follow the keyword; it cannot be elsewhere in the query. SELECT /*+ index(team)*/ /*+ full(player)*/ * FROM player JOIN team USING (team_id); SELECT /*+ index(team) full(player)*/ * FROM player JOIN team USING (team_id); Place all of the hints for a query block into a single comment. SELECT /*+ INDEX_FFS(books books_isbn) */ from books b where b.isbn = :ISBN; SELECT /*+ INDEX_FFS(b books_isbn) */ from books b where b.isbn = :ISBN; A common error. The table alias, whenever present, must be specified in all hints referencing a table. SELECT /* ORDER */ candidate_name FROM candidates JOIN jobs USING (job_code) SELECT /*+ ORDERED */ candidate_name FROM candidates JOIN jobs USING (job_code) Spelling the hint properly is a requirement, as is the plus sign as the first character of the comment. SELECT /*+ PUSH_JOIN_PREDICATE(v) */ name, class_code, instructor FROM student_view v LEFT OUTER JOIN classes c USING (class_code) WHERE class_code = C; Correct: SELECT /*+ PUSH_PRED(v) */ name, class_code, instructor FROM student_view v LEFT OUTER JOIN classes c USING (class_code) WHERE class_code = C; Explanation: Hints are not guaranteed to remain stable from one Oracle release to another. Beware! Incorrect: SELECT /*+ FIRST_ROWS */ zip_code from student_directory group by zip_code; Explanation: This hint is never applied to queries that perform grouping operations. Incorrect: SELECT /*+ HASH_AJ */ ssn FROM employee_table WHERE hire_code=15; Explanation: This is a valid hint used in an improper context. The HASH_AJ hint applies only to subquery blocks. Incorrect: SELECT /*+ INDEX_DESC(TM TEAM_PK) PARALLEL(PLAYER 4) */ player_name, coach, date_joined FROM player_view p JOIN team t USING (team_id) WHERE t.effdt = (SELECT MAX(effdt) FROM team TM where tm.team_id = t.team_id); Correct: SELECT /*+ PARALLEL(P.PLAYER 4) */ player_name, coach, date_joined FROM player_view p JOIN team t USING (team_id) WHERE t.effdt = (SELECT /*+INDEX_DESC(TM TEAM_PK)*/ MAX(effdt) FROM team TM where tm.team_id = t.team_id);

Explanation: Hints generally apply only within the scope of the query block where they appear. Hints for a subquery must be included in the subquery itself. Hints to be applied to table(s) in a named or in-line view can be supplied in the view query; however, it is better practice to provide a global hint in the top-level query block, as shown above.

Scope of Hints: Hinting Views

For queries involving views, possibly including views nested several levels deep, it can be worthwhile to apply a hint within the views. Generally, hints only affect the query block in which they appear. There are two ways to hint outside the current query block: (a) global hints or (b) hints imbedded in the affected view. However, imbedded hints are not recommended.

Global Hints
The execution plan of a view can be hinted from a particular query that references it, directly or indirectly, by using the global hint syntax. First, consider the following two related view definitions: CREATE VIEW manager AS SELECT * FROM employee e WHERE emp_role in (M,S,F); CREATE VIEW officer AS SELECT * FROM manager m WHERE is_officer = Y; Now, here is an example of how a global hint might be applied for use of a bitmap index on the EMPLOYEE table: SELECT /*+ INDEX(officer.m.e ix_emp_codes) */ dept_name, last_name, first_name FROM officer JOIN dept USING (dept_id); Note that the actual view name, OFFICER, was supplied in the hint because it has no alias in the query, while aliases were specified as they must be for the view MANAGER and the table EMPLOYEE. Global hints come in handy when you are working to optimize a query that references views delivered with an ERP or other application system. Global hints enable you to evaluate several plan scenarios without altering the third-party view at all.

Hints Imbedded in Views

It is possible to imbed a hint within a view. Occasionally, a well-meaning, novice query author will tune a view by trying any number of hints in the view and then seeking minimum elapsed time with a query such as this: SELECT * FROM my_view; While that might result in improved performance of the view in isolation, typically the inclusion of such a tuned view in a real-world query will result in a drastic deterioration of the performance of the query. Alternatively, a hinted view might actually benefit one query in which it is used, but could result in worsened performance for other queries that use the view. The reason for this is because the cost-based optimizer attempts to merge views and subqueries into higher-level query block(s) prior to performing optimization. The existence of certain hints in a view suppresses view merging, often resulting in uncoordinated, suboptimal query execution plans. (The NOMERGE hint specifically suppresses view merging.) Incorporating a hint into a named view tends to obscure and scatter relevant performance information, making the task of query optimization daunting, especially when several hinted views are joined. Oracle Corporation explicitly discourages the practice of incorporating hints into view definitions [3] and provides additional detail on the subject of view merging. If you still believe that you need to hint a view, first understand how the view is being merged into the overall query, using EXPLAIN PLAN, then supply your hints to the top-level affected or problem query using global hints, instead of imbedding your hints in a view definition.

Hint Selection Strategies

Typically, there are one of two motivations that a query author will have for placing a hint within a query. One motivation is recognition that the cost-based optimizer must use some generalized assumptions concerning what is most important when it is building a query plan; however, a usual assumption does not apply to the current application: Normal behavior for an INSERT is to seek and utilize free space within existing blocks of the table. However, if populating a table with many rows as quickly as possible is a more important goal than efficient use of space, then INSERT /*+APPEND*/ is a worthy hint. Space reutilization is irrelevant if the table is truncated immediately prior to the INSERT, or if there are no DELETE or UPDATE operations performed on the table at any time. If there is UPDATE or DELETE activity on the table, using the APPEND hint will continue to result in rapid loading; however, subsequent query performance will become slower and slower over time as more blocks are added to the table, while empty or nearly empty blocks remain. Normal behavior for a SELECT is to devise an execution plan that can be completed in the least amount of time. However, if the application is a scrollable browser, for example, producing the first rows of the query in a few seconds might be a more important goal than the complete query elapsed time, especially if the entirety of the result set is infrequently accessed. In this case, SELECT /*+ FIRST_ROWS */ is a worthy hint Normal behavior for a distributed query is to perform optimization on the local database. The remote tables are recast as single-table queries with WHERE clauses that might result in an indexed or other optimal access path; however, the join operation(s) occur on the local database. The cost-based optimizer presently does not have any statistics about network latency or bandwidth, so the query might result in massive amounts of data being brought across the network to be joined on the local database, resulting in very slow execution. Joining at the remote site can mitigate data volumes in these cases; this can be achieved using SELECT /*+ DRIVING_SITE(remote) */. Another common motivation for using hints is recognition that there is something evidently wrong with a particular query execution plan, because it runs much more slowly than expected. In this case, it is not nearly so clear which of more than 100 hints available is going to be effective. So, how do you choose a hint?

Brute force or guesswork

Simply throwing a quantity of hints at a problem query is a method that requires the minimal training and knowledge. There are even third-party Oracle tuning products that will automate this dubious methodology for you. Some advantages of this approach are that it can be delegated to a very junior technician and it keeps us looking busy. We can provide management with copious, documented test cases evidencing that we tried everything to tune the query, even if we were unsuccessful. Sometimes, this approach even results in a hinted query that runs faster. However, random, try this methods rarely reveal the root cause of a performance problem and are not a professional means to optimize performance.

Look at the Execution Plan

Any time a problem query is isolated, an EXPLAIN PLAN should be run on the query. While EXPLAIN PLAN output on long or complex queries can be inscrutable even to experienced Oracle professionals, there very well might be an epiphany in the plan that will allow you to create and test a hypothesis. For example, you notice that a subquery on the ZIPCODE_LOOKUP table is a full table scan. That doesnt seem optimal, and you want to test the hypothesis that the subquery is the primary cause of a performance problem and that it will perform better if it utilizes an index. So, you install an /*+ INDEX(zip z5) */ hint, run another EXPLAIN PLAN to confirm that the hint was processed, and then run a test case if the execution plan changed. Pay little heed to the differing COST values on the plan outputs; that is what the cost-based optimizer has already attempted to do! You might possibly find a faster plan that has a higher cost than a slower plan; that is symptomatic of poor information (configuration, table statistics, system statistics or data distribution) being provided to the CBO.

If the hint provides improved results, you are not done! Now the root cause must be determined. Often, it will be because statistics on the index or table are missing or stale. Gather fresh statistics on all tables (and their indexes) that participate in the query, and see if that has an impact on the execution plan. If it does not, consider analyzing a 10053 trace on the query.

Monitor V$SESSION_LONGOPS or V$SESSION_WAIT

For a query that runs several minutes or more, periodic monitoring of V$SESSION_LONGOPS or V$SESSION_WAIT might provide some insight into what is really happening in a general sense. Is the query spending a lot of time accessing data, or joining data, or grouping it (as evidenced by lengthy sort operations)? While the insight gained might be marginally useful, the approach is hit or miss and is not particularly thorough. More importantly, it is difficult to divine which hint or other optimization approach will best remedy the problem. Sometimes, the observed bottleneck can be correlated to a specific step in the EXPLAIN PLAN, and then a hypothesis can be tested.

Examine trace output from events 10053 and 10046

Trace outputs from the Oracle kernel, gathered while running the problem query, are perhaps the best and most thorough way to determine conclusively what the cost-based optimizer is thinking and doing when planning and executing the query.

Using the event 10053 trace: See What Oracle is Thinking

The 10053 event provides a comprehensive, quantitative journal of the optimizer decision paths used to choose an execution plan for a query. Sections of this trace include the query itself, parameter values used by the optimizer, base statistical information on the tables and critical columns comprising the query, access paths under consideration, general plans (join methods and orders), recosting for special features and a final result. Oracle Corporation publishes very little information about this event; however, detailed information about event 10053 trace output is available in an excellent paper by Wolfgang Breitling [1]. A recommended method for enabling this event within a session is: alter session set tracefile_identifier='RC_CBO'; alter session set events '10053 trace name context forever, level 1'; Following these two commands, run either the problem query or an explain plan for the problem query. Look for the tracefile in the user_dump_dest directory; it can be readily identified by its tracefile_identifier (in this case, RC_CBO). Browse it, attempt to understand as much of it as you can, hint your query, and start the tracing process over again. It can be very helpful to have both tracefiles (before and after hinting) and their corresponding EXPLAIN PLAN outputs to get an overview of how the hint is handled by the optimizer.

Using the event 10046 trace: See What Oracle is Doing

The 10046, or SQL trace, event can be used along with the Oracle tkprof trace formatting utility or with Trace Analyzer [6i] (downloadable from MetaLink) to understand in detail where time is being spent in the execution of a problem query. The Trace Analyzer provides a timing graph formatted identically to an EXPLAIN PLAN output, so that actual run times can be correlated to the steps in the execution plan. Consider this query: select name, city, state from pub78 where (city,state) in (select city,state from pub78 group by city,state having count(*)=1);

When traced, and then run through the Trace Analyzer, this output is presented: | Rows Row Source Operation | ------------ --------------------------------------------------| 12645 HASH JOIN (cr=8990 pr=382 pw=382 time=14.48) | 12880 .VIEW (cr=4432 pr=382 pw=382 time=7.86) | 12880 ..FILTER (cr=4432 pr=382 pw=382 time=7.79) | 34061 ...SORT GROUP BY (cr=4432 pr=382 pw=382 time=7.68) | 614578 ....TABLE ACCESS FULL OBJ# PUB78 (cr=4432 pr=0 pw=0 time=1.95) | 614578 .TABLE ACCESS FULL OBJ# PUB78 (cr=4558 pr=0 pw=0 time=1.89) Explain Plan --------------------------------------------------------------...7 SELECT STATEMENT (COST=2620 CARD=2576 BYTES=172592 IO_COST=2620 ) ...6 .HASH JOIN (COST=2620 CARD=2576 BYTES=172592 IO_COST=2620 ) ...4 ..VIEW OF 'SYS.VW_NSO_1' (COST=1794 CARD=6146 BYTES=122920 ) ...3 ...FILTER ...2 ....SORT (GROUP BY) (COST=1794 CARD=6146 BYTES=79898 IO_COST=1794 ) ...1 .....TABLE ACCESS (FULL) OF 'RANDY.PUB78' (COST=673 CARD=614578 BYTES=7989514 IO_COST=673 ) ...5 ..TABLE ACCESS (FULL) OF 'RANDY.PUB78' (COST=673 CARD=614578 BYTES=28885166 IO_COST=673 ) While the two full-table scans on PUB78 would typically arouse concern, this output indicates that over half of the query run-time is a result of the GROUP BY operation, substantially more than the total elapsed time for both fulltable scans. To get a trace of your own SQL session, the following statements should be included preceding the SQL under analysis: alter session set tracefile_identifier='RC_PROF'; alter session set sql_trace=TRUE; To also see wait events (recommended!) use this alternative invocation, instead: alter session set tracefile_identifier='RC_PROF'; alter session set events '10046 trace name context forever, level 8'; Look for files with the tracefile_identifier (in this case, RC_PROF) in the user_dump_dest directory, and apply those files to the formatter of your choice according to its documented procedures.

Whats So Bad About Hints?

Many respected Oracle experts have published their serious qualms about the use of hints, describing hints as a last resort methodology, and something to be avoided or even prohibited in critical environments. These objections have merit, and they should be considered when using hints for any purpose other than self-instruction, testing hypotheses or debugging. If you like the rule-based optimizer, youll love hints. The thrust behind this statement is that hints impair the CBO from doing what it is supposed to be doing: optimizing! You might stumble across a hint that yields really fast execution on todays data volumes, relative proportions and data skews, but then youll never know if a better, more scalable plan would have been found by the CBO as data volumes increase. The presence of hints in a production query might guarantee that the plan will remain unchanged, but that does not guarantee that performance will always be good! Upgrading is a hassle with hints. Because the CBO undergoes refinement with each Oracle release, the appropriateness of hints ensconced in views, PL/SQL, Java and stored SQL scripts must be evaluated with each

RDBMS software upgrade. Just as hints can mask downside performance issues, they can equally as well mask dramatic, upside performance opportunities that are delivered in nearly every new Oracle release. Hints inhibit portability. The really fast results you could achieve by fully hinting queries on your development system might not be reproduced successfully in every environment. Differences in hardware, OS configuration and database configuration can cause a hinted query to run terribly poorly when ported to a different environment. Hints are inscrutable. Mere presence of hints is sure to decrease readability and pertinence of a query to any developer or business analyst. The clutter of hints in a query might help the CBO work faster, but it wont help people work faster. Hints cover up sloppy practices. Pervasive use of hints on a database likely is symptomatic of deeper, unresolved issues. These include stale or missing object statistics, suboptimal database configuration, poorly written queries and even deficient application or database design. So, after writing a hint, take a hint: fix the root cause! The bottom line is that hints impose unnatural rigidity on the cost-based optimizer. Rigidity can be helpful during diagnosis of a problem, but in the long run, it can be profoundly detrimental to the scalability and adaptability of your environment.

So, no hints in production, right?

Your organizational best practices should govern, but not entirely prohibit, the deployment of hinted queries in a production environment. These are circumstances where hints might legitimately be used in production: As a tourniquet, to resolve temporarily a critical production issue. Be sure to revisit the emergency use of the hint at a later time. If possible, use an adequately scaled test or production support environment to investigate a durable solution, so that the hint becomes unnecessary for maintaining adequate performance and service levels. In a prior section captioned Hint Selection Strategies, three hints were identified as having reasonably deterministic behavior and measurable downside risk. These hints are: o APPEND o DRIVING_SITE o FIRST_ROWS As part of a carefully orchestrated and tested migration, preferably including stored outlines for plan stability, rather than migrating explicitly hinted SQL text.

What do Stored Outlines have to do with Hints?

Plenty. Oracle uses hints to record stored plans. Accordingly, the stored outline mechanism can be a satisfactory way to manage and migrate carefully screened hints into production, as part of a deliberate value decision favoring performance determinacy, or plan stability, instead of the best obtainable throughput. Stored outlines are also useful when moving carefully optimized SQL from a rule-based to a cost-based environment. Finally, stored outlines are an excellent way to provide hints to queries that would otherwise be inconvenient or inadvisable to modify, including stored SQL delivered with ERP and other third-party packages, as well as inaccessible SQL imbedded or generated in various applications, such as PeopleSoft or Crystal Reports. Bear in mind that with the use of plan stability, you will continue to have rigidity, impaired scalability and other adverse issues that come with keeping hints around, albeit in a better managed and documented framework. Stored outlines should be periodically revisited to confirm that superior query execution plans are not being suppressed by use of plan stability.

Alternatives to Hints
Think of most hints as analogous to diapers, crutches, debuggers or training wheels they are indispensable in the short run, but you certainly wouldnt choose to use them all of the time! So, once youve obtained satisfactory performance from a problem query using a hint, what is the next step? 1) Be sure that all tables participating in the problem query have current statistics. Run DBMS_STATS to gather new statistics, if necessary. Unless the tables are immense, specify computed, rather than estimated statistics. Be sure that all indexes also have current statistics. Then, evaluate the problem query again without the hint. 2) [DBA] Confirm that system statistics are periodically measured and recorded. See MetaLink Note 149560.1 [6j] 3) Attempt to adjust session CBO parameters instead of, or in combination with, using hints. Consult this list of parameters used by the optimizer, understand them and adjust them appropriately: optimizer_mode optimizer_index_caching (default value is usually deficient, see [2]) optimizer_index_cost_adj (default value is often deficient, see [2]) optimizer_dynamic_sampling optimizer_max_permutations hash_join_enabled db_file_multiblock_read_count (used differently between Oracle8i and Oracle9i) sort_area_size and hash_area_size (Oracle8i and prior) 4) [DBA] Ensure that database CBO spfile parameters, especially optimizer_mode, compatible, optimizer_features_enable, optimizer_index_caching and optimizer_index_cost_adj, are established for optimal benefit to the entire instance. 5) Carefully observe the cardinality values in the querys EXPLAIN PLAN and 10053 trace. If there is a data distribution (skew) issue, consider gathering statistics for column histograms. 6) If possible, consider expressing the query differently. The CBO is rarely confounded by the mere order of tables in a FROM list or by predicate order or transitivity. Increasingly, it has the ability to recognize and transform specific suboptimal constructions such as OR predicates. Nevertheless, expressing lengthy or complex queries in an entirely different way sometimes results in improvements to the execution plan.

Conclusion
Query hints provide a convenient, ready-to-use means to manipulate the cost-based optimizer for a specific problem query. Hints can be useful for debugging a database or application issue, for testing SQL performance scenarios, and for learning how the optimizer works. With discipline and restraint, hints can even be used for advantage in production environments. Because hints constrain the optimizers freedom to explore a full range of execution plans, they inhibit scalability and optimal performance when they remain in a SQL statement over a period of time. Therefore, hints should not be regarded as a solution to performance issues, but they should be considered as an important tool for achieving top query performance.

References
[1] Breitling, Wolfgang; A Look Under the Hood of CBO: The 10053 Event. http://www.centrexcc.com [2] Gorman, Tim; The Search for Intelligent Life in the Cost-Based Optimizer. http://www.evdbt.com [3] Green, Connie Dialeris; Oracle9i Database Performance Tuning Guide and Reference, Release 2 (9.2), Chapter 5. October 2002, Oracle Corporation Part No. A96533-02. [4] Harrison, Guy; Oracle SQL High-Performance Tuning, Second Edition. 2001 Prentice-Hall, ISBN: 0-13012381-1. [5] Holdsworth, Andrew; Oracle9i Database Performance Planning. March 2002, Oracle Corporation Part No. A96532-01. [6] Oracle Support Site (subscription required). http://metalink.oracle.com. Resources: [6a] Note: 29236.1 QREF: SQL Statement Hints [6b] Note: 50607.1 How to specify and index hint [6c] Note: 62364.1 Hints and subqueries [6d] Note: 69992.1 Why is my hint ignored? [6e] Note: 73489.1 Affect of Number of Tables on Join Order Permutations [6f] Note: 122812.1 How to Tune a Query that Cannot be Modified [6g] Note: 225598.1 How to Obtain Tracing of Optimizer Computations (EVENT 10053) [6h] Note: 35934.1 Cost Based Optimizer - Common Misconceptions and Issues [6i] Note: 224270.1 Trace Analyzer: Interpreting Raw SQL Traces Generated by Event 10046 [6j] Note: 149560.1 Collect and Display System Statistics (CPU and IO) for CBO usage

About the Author

Randy Cunningham is a consultant with SageLogix, Inc., based in Englewood, Colorado, USA. He has worked with Oracle since version 4, and has been consulting exclusively to clients using Oracle products for the past eight years. His primary professional focus is on the architecture, implementation and performance optimization of ERP and data warehousing applications, with a secondary emphasis on database replication techniques.