Module 8: Access and Query Logging

After completing this module, you will be able to: Describe how the BEGIN/END LOGGING statements capture information about user access to Teradata. Describe how to set up user access logging. Use system views to gather information about data access. Identify the reasons for using the Database Query Log. Identify the tables and views that make up the DBQL facility.

Access and Query Logging

There are two logging facilities available to the database and/or security administrator.

Access Logging Facility Used for access and security audit analysis. May be used to monitor data access requests (via access rights
checks) and log entries for requests that are granted and/or denied.

Query Logging Facility (DBQL) Used for query activity and workload analysis. Can be used to track processing behavior and/or capture detailed
information about the queries that are running on a system.

Workloads can be utilized with Teradata Analyst tools such as

Teradata Index Wizard.

New facility available with V2R5.

Access Logging
An administrator can ...

use the Access Logging facility to monitor data access requests and log
entries for requests that are granted and/or denied.

optionally capture the SQL text along with the access right check.
The following statements are used to specify objects and/or SQL requests to monitor for specific or all users.

BEGIN LOGGING statement Starts the monitoring of data access requests by Teradata.
END LOGGING statement Ends the monitoring of data access requests by Teradata.

Objects used in Access Logging

Users who are granted EXECUTE permission on the following macro can use the BEGIN LOGGING and END LOGGING statements.
Example: DD/D Macro

GRANT EXECUTE ON DBC.AccLogRule TO SecAdmin;

This allows "SecAdmin" to execute the BEGIN LOGGING and END LOGGING statements.

DBC.AccLogRule
DD/D Table

Execution of BEGIN LOGGING or END LOGGING statements causes rows (representing the rules) to be added or updated in To view the rules in this table, SELECT from this view.

DBC.AccLogRuleTbl
DD/D View

DBC.AccLogRules
DD/D Table

Based on the rules, access of specified objects or SQL statements cause entries to be placed in

DBC.AccLogTbl
(can potentially become large)

DD/D View

To view the rules in this table, SELECT from this view.

DBC.AccessLog

BEGIN LOGGING Statement

BEGIN LOGGING DENIALS WITH TEXT ON FIRST LAST FIRST AND LAST EACH A ALL , operation GRANT ; , BY username ON DATABASE dbname USER username TABLE VIEW MACRO PROCEDURE dbname. A

name

Operation BY ON

Any function for which an access right can be granted (e.g., CREATE, USER, MACRO, GRANT, etc.). username implies all users, if not specified. object-name implies all entities, if not specified. Valid object-names are: DATABASE TABLE MACRO databasename tablename macroname USER VIEW PROCEDURE username viewname procedurename

END LOGGING Statement

END LOGGING DENIALS WITH TEXT ON A

ALL , operation GRANT

; , BY username ON DATABASE dbname USER username TABLE VIEW dbname. MACRO

name

PROCEDURE

Operation BY ON

Any function for which an access right can be granted (e.g., CREATE, USER, MACRO, GRANT, etc.). username implies all users, if not specified. object-name implies all entities, if not specified. Valid object-names are: DATABASE TABLE MACRO databasename tablename macroname USER VIEW PROCEDURE username viewname procedurename

Setting Up Access Logging

STEP 1

Install Access Logging on the system: 1. Run the DIP script DIPACC to install the DBC.AccLogRule macro in system user DBC. 2. Restart the database to activate the code.

STEP 2

Create and empower a security administrator:

CREATE GRANT GRANT USER SecAdmin AS Password = secpasswd, PERM = 0, SPOOL = 200e6 ; EXECUTE ON DBC.AccLogRule TO SecAdmin ; EXECUTE ON DBC.LogonRule TO SecAdmin ;

STEP 3

Define access logging rules. Examples: 1. Log all attempts to access security macros. 2. Log all denied attempts to access DBC User. 3. Log any CREATE or DROP USER/DATABASE or GRANT commands.
BEGIN LOGGING WITH TEXT ON ON EACH ALL MACRO DBC.LogonRule, MACRO DBC.AccLogRule;

BEGIN LOGGING DENIALS WITH TEXT ON EACH ALL ON USER DBC;

BEGIN LOGGING WITH TEXT

ON EACH DATABASE, USER, GRANT;

Access Log Views

DBC.AccLogRules
Contains current logging rules generated by BEGIN and END LOGGING statements.

DBC.AccessLog

Contains log entries collected as a result of applying access log rules.

Dictionary Tables Accessed:

DBC.AccLogRuleTbl DBC.AccLogTbl

DBC.AccLogRules View
Returns information about current access logging rules.
DBC.AccLogRules
UserName AcrCreateDatabase (CDB) AcrCreateProc (CSP) AcrDropMacro (DMC) AcrDropProc (DSP) AcrGrant (GRT) AcrRestore (RST) AcrDropTrigger (DTG) DatabaseName AcrCreateMacro (CMC) AcrCreateView (CVW) AcrDropTable (DTB) AcrDump (DMP) AcrIndex (IDX) AcrSelect (SEL) CreatorName TVMName AcrCreateTable (CTB) AcrDelete (DEL) AcrDropUser (DUS) AcrExecute (EXE) AcrInsert (INS) AcrUpdate (UPD) CreateTimeStamp AcrCheckpoint (CPT) AcrCreateUser (CUS) AcrDropDatabase (DDB) AcrDropView (DVW) AcrExecProc (ESP) AcrReference (REF) AcrCreateTrigger (CTG)

ACR Columns are positional:

Position #1 = How often to log requests (F, L, B, E, blank = First, Last, Both, Each, None) Position #2 = How often to log denials (F, L, B, E, blank = First, Last, Both, Each, None) Position #3 = How often to save text (+ All entries, - Denials, = All Specified)

SELECT UserName (CHAR (6)) AS "User//Name" ,DatabaseName (CHAR (6)) AS "Dbase//Name" ,TVMName (CHAR (10)) AS "TVM//Name" ,AcrCreateDatabase, AcrCreateUser, AcrDropDatabase ,AcrDropUser, AcrGrant, AcrSelect, AcrExecute FROM DBC.AccLogRules; User Dbase TVM Name Name Name All All All All All DBC DBC DBC

CDB CUS DDB DUS GRT SEL EXE E + E- EE +

Example Results:

All E + E + E + E + E + LogonRule E + All E- E- E- E- EAccLogRule E +

BEGIN LOGGING Example

BEGIN LOGGING DENIALS BEGIN LOGGING BEGIN LOGGING BEGIN LOGGING BEGIN LOGGING DENIALS SELECT WITH TEXT WITH TEXT WITH TEXT WITH TEXT WITH TEXT ON EACH SELECT ON TABLE ON FIRST INSERT ON TABLE ON FIRST AND LAST DELETE ON TABLE ON FIRST UPDATE ON TABLE ON LAST UPDATE ON TABLE PD.Employee; PD.Employee; PD.Employee; PD.Employee; PD.Employee; 1 2 3 4

FROM WHERE

UserName (CHAR (6)) AS "User//Name" ,DatabaseName (CHAR (6)) AS "Dbase//Name" ,TVMName (CHAR (10)) AS "TVM//Name" ,AcrSelect, AcrInsert, AcrDelete, AcrUpdate DBC.AccLogRules DatabaseName = 'PD'; 1 User Dbase TVM Name Name Name All PD Employee 2 3 4

SEL INS DEL UPD E- F + B + FL=

Position #1 = How often to log requests (F, L, B, E, blank = First, Last, Both, Each, None) Position #2 = How often to log denials (F, L, B, E, blank = First, Last, Both, Each, None) Position #3 = How often to save text (+ All entries, - Denials, = All Specified)

DBC.AccessLog View
Displays entries made to DBC.AccLogTbl.
DBC.AccessLog
LogDate LogicalHostID AccountName EventCount ColumnName LogTime IFPNo OwnerName Result StatementType LogonDate SessionNo AccessType DatabaseName StatementText LogonTime UserName Frequency TVMName

Access Type

The same codes are used that indicate an access right.

Frequency

F, L, B, E = First, Last, Both or Each.

To delete entries in the DBC.AccLogTbl, use the DBC.DeleteAccessLog[X] view.

DELETE FROM DBC.DeleteAccessLog; Deletes entries older than 30 days. DELETE FROM DBC.DeleteAccessLog
WHERE LOGDATE < (CURRENT_DATE 90);

Deletes entries older than 90 days.

DBC.AccessLog View Example

Example: List all of the entries in the Access Log table for the current date. SELECT LogTime ,UserName (CHAR (10)) AS "User//Name" ,AccessType AS "Access//Type" ,Frequency AS "Log//Freq" ,Result AS "Granted//Denied" ,StatementText AS "Statement//Text" FROM DBC.AccessLog WHERE LogDate = CURRENT_DATE ORDER BY LogDate, LogTime ;

Example Results:

LogTime User Name 09:04:25 09:10:17 09:12:22 09:12:31 09:15:54 09:17:04 09:32:50 TFACT01 SYSDBA SYSDBA SYSDBA SYSDBA TFACT01 TFACT01

Access Type S I D D U U U

Log Freq E F f l F f l

Granted Denied D G G G G D D

Statement Text SELECT * FROM PD.Empl... INSERT INTO PD.Employe ... DEL FROM PD.Employee ... DEL FROM PD.Employee ... UPDATE PD.Employee SE ... UPDATE PD.Employee SE ... UPDATE PD.Employee SE ...

Note: The facing page contains the SQL that generated this report.

END LOGGING Example

Previously, these rules were created for logging on PD.Employee table.
BEGIN LOGGING DENIALS BEGIN LOGGING BEGIN LOGGING BEGIN LOGGING BEGIN LOGGING DENIALS WITH TEXT WITH TEXT WITH TEXT WITH TEXT WITH TEXT ON EACH SELECT ON FIRST INSERT ON FIRST AND LAST DELETE ON FIRST UPDATE ON LAST UPDATE ON TABLE ON TABLE ON TABLE ON TABLE ON TABLE PD.Employee; PD.Employee; PD.Employee; PD.Employee; PD.Employee;

To end the logging for PD.Employee table, the following statements can be executed:
END LOGGING DENIALS END LOGGING ON SELECT, UPDATE ON INSERT, DELETE, UPDATE ON TABLE PD.Employee; ON TABLE PD.Employee;

To verify the rules have been removed, use the DBC.AccLogRules view:
SELECT UserName (CHAR (6)) AS "User//Name" ,DatabaseName (CHAR (6)) AS "Dbase//Name" ,TVMName (CHAR (10)) AS "TVM//Name" ,AcrSelect, AcrInsert, AcrDelete, AcrUpdate DBC.AccLogRules DatabaseName = 'PD'; User Dbase TVM Name Name Name

FROM WHERE

SEL INS DEL UPD

Rules for PD.Employee have been removed.

Query Logging (DBQL) Concepts

DBQL is a V2R5 feature created to log historical query information.
Not intended for live review of queries and query steps (use Performance Monitor)

DBQL caches and eventually stores query information in multiple Teradata

Data Dictionary tables as the queries are executed.

Logging is invoked and revoked via SQL statements (BEGIN QUERY

LOGGING and END QUERY LOGGING )

Logging can be invoked for all users, a list of users, a list of account strings or a
particular user with one or more account strings.

By default, 1 row per query is logged that contains user id information and
some statistics for that query.

Options are available to expand the amount and kind of information to be logged.

Notes and limitations include

Only tracks SQL (note that TPump sends SQL to Teradata) Doesnt log logons and logoffs Utilities will be included in future

Objects used in Defining Rules for DBQL

Users who are granted EXECUTE permission on the following macro can use the BEGIN QUERY LOGGING and END QUERY LOGGING statements.
Example: GRANT EXECUTE ON DBC.DBQLAccessMacro TO Sysdba; Initially, only DBC and SystemFE users are allowed to issue BEGIN/END QUERY LOGGING statements. DD/D Tables Execution of BEGIN QUERY LOGGING or END QUERY LOGGING statements causes rows (representing the rules) to be added or updated in DBC.DBQLRuleTbl DD/D View To view the rules in this table, SELECT from this view. DBC.DBQLRules

DD/D Macro
DBC.DBQLAccessMacro

There are 6 additional tables and 6 additional views that are used to hold and view captured query data (shown on next page).

Objects used in DBQL (cont.)

The views and associated tables used to hold query data are ...
DD/D Views DBC.QryLog DBC.QryLogSteps DD/D Tables DBC.DBQLogTbl DBC.DBQLStepTbl DBQL Purpose Stores default rows (key table) One row per step One row per object referenced in query Stores full SQL text - multiple rows may be needed Queries meeting Summary or Threshold rules Not used in V2R5; plan is to store EXPLAIN of query

DBC.QryLogObjects
DBC.QryLogSQL DBC.QryLogSummary DBC.QryLogExplain

DBC.DBQLObjTbl
DBC.DBQLSqlTbl DBC.DBQLSummaryTbl DBC.DBQLExplainTbl

Details on the columns in these views are located in Appendix E.

BEGIN QUERY LOGGING Statement

BEGIN QUERY LOGGING WITH , logging_option LIMIT limit_option AND A ON ALL , , user_name , ACCOUNT = ( 'account_name' ) ; limit_option A

A BEGIN QUERY LOGGING statement without the WITH or LIMIT options causes default
rows to be placed in the DBQLogTbl. A default row contains:

User name, account string (expanded), time stamp information Unique ID for process, session, and client (host) connection First 200 characters of SQL statement Use of the WITH option(s) cause a default row to be placed in DBC.DBQLogTbl plus
additional rows in other DBQL tables.

The LIMIT option may be used to limit the amount of SQL text captured, set thresholds,
or just capture summary information.

BEGIN QUERY LOGGING WITH ... (Cont.)

BEGIN QUERY LOGGING , WITH logging_option LIMIT limit_option AND A ON ALL , , user_name ACCOUNT = ( , 'account_name' ) ; limit_option A

WITH ALL generates one row per query in DBQLogTbl which includes 200 characters of SQL statement one row per target object per query in DBQLObjTbl one row per step per query in DBQLStepTbl one or more rows per complete SQL statement in DBQLSqlTbl WITH OBJECTS one row per target object per query in DBQLObjTbl plus default row in DBQLogTbl WITH SQL logs the entire SQL for each request for each user being logged in DBQLSqlTbl
plus default row in DBQLogTbl WITH STEPINFO inserts one row per step per query in DBQLStepTbl plus default row

BEGIN QUERY LOGGING LIMIT (cont.)

BEGIN QUERY LOGGING WITH , logging_option LIMIT limit_option AND A ON ALL , , user_name ACCOUNT = ( , 'account_name' ) ; limit_option A

LIMIT SQLTEXT specify the amount of SQL text to capture in the default row of
DBQLogTbl. (Default is 200 char., 0 = off, max = 10,000 characters)

LIMIT SUMMARY (For short high volume queries example OLTP) Counts queries; count is written in DBQLSummaryTbl every 10 min (if count > 0) SUMMARY doesnt generate default rows in DBQLogTbl

LIMIT THRESHOLD (Also for short high-volume queries example OLTP) Similar to SUMMARY, but default rows are generated in DBQLogTbl Threshold, in seconds, determines whether to log a query or just count it. Query that complete <= threshold (sec.) are counted in DBQLSummaryTbl. Query that complete > threshold (sec.), DBQL logs the default row.

BEGIN QUERY LOGGING Examples

BEGIN QUERY LOGGING ON ALL;

This creates a rule for all users you will not be able to create rules for specific users. If rules exist for specific users/accounts, then you cannot create a rule for ALL.
BEGIN QUERY LOGGING ON tfact01, tfact02;

This creates 2 rules - one for each specified user. You can END QUERY LOGGING for either or both of the users.
BEGIN QUERY LOGGING ON tfact03 ACCOUNT = ('$L', '$M');

This creates 2 rules for a specific user each rule has a specific account ID.
BEGIN QUERY LOGGING ON ALL ACCOUNT = ('$L_&D&H', '$M_&D&H');

This creates 2 rules for all users each rule identifies a specific account ID. You can END QUERY LOGGING for either or both of the account IDs.
In these examples, the WITH and LIMIT options aren't used. Therefore, default rows will be created in the DBQLogTbl.

BEGIN QUERY LOGGING Examples (cont.)

BEGIN QUERY LOGGING WITH STEPINFO, SQL LIMIT SQLTEXT=0 ON ALL;

Default rows are logged as well as complete SQL text and one row per step. Since complete SQL text is captured in DBQLSqlTbl, no need to also capture SQL text
in default row. Therefore, LIMIT SQLTEXT=0 doesn't capture SQL text in default row. BEGIN QUERY LOGGING WITH ALL LIMIT SQLTEXT=0 ON ALL;

ALL options are logged for ALL users (probably generates too much information).
BEGIN QUERY LOGGING LIMIT SUMMARY = 5, 60, 600 ON ALL;

Summary option is to only count running queries based on elapsed time. Counts are
logged in DBQLSummaryTbl every 10 minutes or when cache is full. 3 values (in sec.) are required. 4 count intervals are logged (<=5, <=60, <=600, >600) Summary limit cannot be used with any other limits LIMIT THRESHOLD = 300 ON ALL;

BEGIN QUERY LOGGING

If a query runs for less than 300 seconds (5 minutes), increment the count. If a query runs longer than 5 minutes, log a default row.
In these examples, the ON option can also specify user name and/or account IDs.

END QUERY LOGGING Statement

END QUERY LOGGING ON , user_name ACCOUNT = ( , 'account_name' ) ALL , ;

If ON ALL was used in the BEGIN statement, ON ALL must be used in the END
statement.

If a list of users or a list of account strings was given in the BEGIN statement,
logging can be ended on an individual basis.

The END QUERY LOGGING statement will cause DBQL cache to be written to the
tables except for Summary cache. END QUERY LOGGING ON ALL; END QUERY LOGGING ON tfact01; END QUERY LOGGING ON tfact03 ACCOUNT=('$H'); (You can end logging for a specific user.) (You can end logging for a specific account of a user.)

DBC.DBQLRules View
Returns information about current query logging rules.
DBC.DBQLRules
UserName SqlFlag TextSizeLimit ThreshValue AccountString StepFlag SummaryVal1 ExplainFlag SummaryFlag SummaryVal2 ObjFlag ThresholdFlag SummaryVal3

Example:

SELECT

UserName (CHAR (8)) AS "User" ,AccountString (CHAR (8)) AS "Acct_ID" ,SqlFlag AS "Sql" , TextSizeLimit AS "Size" ,SummaryFlag AS "SumFl ,SummaryVal1 AS "V1" , SummaryVal2 AS "V2" , SummaryVal3 AS "V3" FROM DBC.DBQLRules ORDER BY 1; User Acct_ID Sql Size SumFl V1 V2 V3

Results:

Note: BEGIN QUERY LOGGING statements are shown on facing page.

All All tfact01 tfact02 tfact03 tfact03 tfact04 tfact05

$L_&D&H F $M_&D&H F F F $M F $L F F T

200 200 500 500 200 200 200 0

F F F F F F T F

? ? ? ? ? ? 5 ?

? ? ? ? ? ? 30 ?

? ? ? ? ? ? 60 ?

DBC.QryLog View Example

Returns information about default rows in the DBQLogTbl.
Example of the data within one default row with a QueryID of 11166.
SELECT ProcID, CollectTimeStamp, QueryID, UserID, AcctString, ExpandAcctString, LogonDateTime, StartTIme, FirstStepTime, FirstRespTime, LastRespTime, NumSteps, QueryText DBC.QryLog QueryID = 11166;

FROM WHERE

Result:
ProcID CollectTimeStamp QueryID UserID AcctString ExpandAcctString LogonDateTime StartTime FirstStepTime FirstRespTime LastRespTime NumSteps QueryText 16383 2003-06-04 10:49:02 11166 00000704 $M $M 2003-06-04 18:05:43 2003-06-04 18:10:16 2003-06-04 18:10:16 2003-06-04 18:10:30 2003-06-04 18:10:30 5 SELECT * FROM DS.Sales_History ;

Result is shown in BTEQ with SIDETITLES and FOLDLINE on.

DBC.QryLogSummary View Example

Returns information about summary rows in the DBQLSummaryTbl.
This example is based on the summary rule:
BEGIN QUERY LOGGING LIMIT SUMMARY = 5, 30, 60 ON tfact04;
SELECT * FROM DBC.QryLogSummary ORDER BY 2, 3, 6; For #2, the average time of queries can be calculated: 14 74 84 93 / 38 = / 5= / 2= / 1= .37 14.80 40.50 93.00

Result:
ProcID 16383 16383 16383 16383 16383 16383 16383 16383 16383 CollectTimeStamp 2003-06-04 18:59:08 2003-06-04 18:59:08 2003-06-04 19:09:08 2003-06-04 19:09:08 2003-06-04 19:09:08 2003-06-04 19:09:08 2003-06-04 19:09:08 2003-06-04 19:09:08 2003-06-04 19:09:08 SessionID 1005 1005 1005 1005 1005 1005 1006 1006 1006 QueryCount 17 1 38 5 2 1 4 2 1 QuerySeconds 8 17 14 74 81 93 2 33 85 LowHist 0 5 0 5 30 60 0 5 60 HighHist 5 30 5 30 60 32767 5 30 32767 1

1 In this summary collection, no queries were executed that exceeded 30 seconds. 2 In this summary collection, queries were executed in all 4 summary intervals. 3 In this summary collection, no queries were executed that ran between 30 and 60 seconds.

Access and Query Logging Summary

There are two logging facilities available to the database and/or security administrator.

Access Logging Facility

Used for access and security audit analysis

Query Logging Facility (DBQL)

Used for query activity and workload analysis
Both facilities require establishing a set of rules.

Access Logging rules are stored in DBC.AccLogRules table.

Contains current access logging rules generated by BEGIN LOGGING and

END LOGGING statements.

DBQL (Query Logging) rules are stored in DBC.DBQLRuleTbl.

Contains current query logging rules generated by BEGIN QUERY LOGGING

and END QUERY LOGGING statements.

Both facilities provide a set of views to display logged information.

Review Questions
1. In order to use the BEGIN/END LOGGING commands, what is the name of the system macro you need execute permission on? _________________________________________

2. How is this macro initially created? _________________________________________

3. What is a negative impact of the following statement? BEGIN LOGGING WITH TEXT ON EACH _________________________________________ 4. With DBQL, what is the size of the default text captured for queries? ________

5. True or False. With DBQL, the LIMIT SUMMARY option cannot be used with any other LIMIT. 6. True or False. With DBQL, the WITH SQL option only captures a maximum of 10,000 characters. 7. True or False. With DBQL, the option WITH ALL ON ALL is typically a good choice. 8. True or False. With DBQL, default rows are logged in the DBC.DBQLogTbl.

Module 8: Review Question Answers

1. In order to use the BEGIN/END LOGGING commands, what is the name of the system macro you need execute permission on? DBC.AccLogRule

2. How is this macro initially created? When the DIP script (DIPACC) is executed.
3. What is a negative impact of the following statement? BEGIN LOGGING WITH TEXT ON EACH Potentially a lot of entries are placed in the dictionary and would require a lot of PERM space. 4. With DBQL, what is the size of the default text captured for queries? 200 characters

5. True or False. With DBQL, the LIMIT SUMMARY option cannot be used with any other LIMIT. 6. True or False. With DBQL, the WITH SQL option only captures a maximum of 10,000 characters. 7. True or False. With DBQL, the option WITH ALL ON ALL is typically a good choice. 8. True or False. With DBQL, default rows are logged in the DBC.DBQLogTbl.