The Underutilized Security Bulwark: Database Logs Dr.

Anton Chuvakin WRITTEN: 2007 DISCLAIMER: Security is a rapidly changing field of human endeavor. Threats we face literally change every day; moreover, many security professionals consider the rate of change to be accelerating. On top of that, to be able to stay in touch with such ever-changing reality, one has to evolve with the space as well. Thus, even though I hope that this document will be useful for to my readers, please keep in mind that is was possibly written years ago. Also, keep in mind that some of the URL might have gone 404, please Google around. As security breaches like that of TJ Maxx grab headlines and as corporate regulations like Sarbanes-Oxley and PCI continue to emphasize preserving and securing data and assurance of IT controls, organizations are increasingly turning to log data to provide a continuous fingerprint of everything that happens within the security perimeter. Enterprises look to logs from network systems (routers, switches, firewalls), security devices (IDS/IPS, firewalls), and/or servers (Windows, Unix, Linux, and even mainframes) to gain insight into their system, protect from internal and external security threats, and satisfy auditors. However, another crucial, log-generating part of the IT infrastructure, the database, has been capturing more attention recently, even though most security issues surrounding databases have existed since commercial database systems were introduced a few decades ago. Over the past few years, insider attacks have caused more headaches than occasional malware-related incidents. According to recent surveys, a majority of respondents attribute at least some amount of data loss to insiders. Perhaps the top reason why paying attention to database security is crucial to improving enterprise information security posture is that database systems are deployed deep inside the company network. This is also why databases are less visible on a security radartheir internal deployment is seen as a shield that protects them from Internet attacks. However, such database placement makes them a prime target for insiders, who have the best opportunity to attack, compromise, and steal the data. Although, it is not just insider threats that present problems. Databases that house confidential client information (e.g. medical records or credit card numbers) that need to be available to partners and other outside parties can also be penetrated by outsiders, possibly through web application vulnerabilities. Such a breach is guaranteed to have deleterious effects. The term database security connotes controlling access to database software, structures (or metadata), the data itself, database configuration hardening, database data encryption, and database vulnerability scanning, all of which are underscored by database logging, which stands as the last but most critical line of defense against insider attacks as well as compliance risks.

Database Logging is Difficult Databases offer different arrays of logging options, but most are capable of logging user logins and logouts, database system starts, stops, restarts, various system failures and errors, user privilege changes, database structure changes, database administrator actions, and database data access. These logged events provide deep insight into the IT infrastructure and business data- insight that can help enterprises meet their security, compliance, and IT operational needs. The question then becomes if databases offer extensive logs that are crucial for accurate and effective user-activity tracking and/or preventing insider attacks, why do database logs remain the forgotten children of the log family? There are several reasons that database log management and analysis does not happen to the extent that it should. First, it is inherently complicatedthe logs themselves are unclear and often difficult to analyze; many databases log in multi-line format, where a single record might be spread across multiple lines of log data. In addition, all but the most basic database logging capabilities are typically turned off by default, however shocking that sounds in todays compliance-heavy environment. To enable proper database logging, a database administrator (DBA) must set special configuration options or sometimes restart the database software, both of which take time, manpower, and expertise. Further, unlike other areas of the IT infrastructure, where logging has a negligible impact on system performance, database logging does actually slow down the database, especially when all access to data is recorded. Highperformance databases are meant to provide thousands of data transactions per second, and logging all of those events takes power and space that most DBAs dont want to sacrifice. From the DBA perspective, their job is not to log, but to ensure the smooth functioning of the database and quick responses to database queries by customers. To top it off, few security professionals are familiar with in-depth details of database logging. But it Has to be Done Despite these challenges, database logging must be enabled and log review must happen. For example, viewing the authentication logs is the only way to verify what access control decisions are made, who views and downloads what data, who is connected to the server, who is deleting or corrupting data, and whether or not a security breach has led to unauthorized access to critical and supposedly secure information. Reviewing database logs is the only way to know the who, what, where, and how if a disgruntled employee accesses a secure customer list to steal confidential data or, worse, to modify it to cause embarrassing problems for company executives (!). It is unlikely that intrusion detection or other security technology would stop this type of problematic user-activity in time, but records of the employees search would be found, immutable, in database logs. Having such logs ready for real-time analysis and reports could mitigate the damage from that type of attack or abuse. In fact, one of the more important database logs is actually a log of DBA activity. DBAs have access to all sorts of confidential information that is stored in the databasepatient information in hospitals, financial information in financial institutions, and credit card information in retail stores. They

also have the best ability to modify or corrupt covert data. It is essential that DBA activity logs be collected, reviewed, andyes!protected from DBAs themselves as a separation of duty measure, because with near-unbridled access to the company crown jewels, they could present an internal threat to company security. Of course, this brings up another known point of resentment for DBAs towards logging! In addition to security, database logging and log analysis must be performed for IT auditors to enable regulatory compliance with the slew of government mandates for preserving and securing data. Payment Card Industry Data Security Standards (PCIDSS), designed to enhance payment account data security, mandates that companies monitor their logs; although it does not specify database logs, this information would be crucial to guaranteeing secure payment information, since this data is stored in a database. The Sarbanes-Oxley Act (SOX) mandates that companies must have an assurance of internal controls. In spirit, this regulation means that officers of the company should be sure that their financial records, stored in databases, remain intact and unmodified. Of course, it is a logical conclusion that if financial data is stored in databases and this financial data must be locked, safe and secure, the best way to guarantee data security and integrity is to review the database logs. The Health Insurance Portability and Accountability Act (HIPAA) requires that patient information (again, stored in a database) is kept secure and controlled. Imagine if a nurse were interested in peeking at celebrity health records. If she isnt caught red-handed at the computer, how would anyone know? The answer should already be obvious- look at the logs. Perhaps the user name of Nurse Smith, who has no business accessing the database, will show up as having viewed a chart detailing the latest Hollywood stars emergency-room treatment for a drug overdose, and the hospital will know that she has accessed confidential records. More generally, IT governance best practices frameworks such as the Control Objectives for Information and related Technology (COBIT) also steer IT users towards database logging. Database Logging Doesnt Have to be A Challenge Enterprises are faced with a staggering amount of logs, and databases are among the most chatty log sources. The presence of these logs and their review and analysis are essential to ensure IT security and compliance, but, unfortunately, for a variety of reasons, the cards are stacked against their easy and comprehensive collection, review, and analysis due to the reasons mentioned above. So whats to be done? As mentioned before, most commercial databases log surprisingly few events by default. For those companies that use database logging capabilities in only the most basic way, a manual log review may be suitable. However, if any kind of database security best practices are being followed and more comprehensive logging is enabled, automation of log analysis and log management via some technology is required. Simpler log analysis tools, often provided by database vendors, allow skilled DBAs to review logs on a specific database server and gain some insight into database activity, but do not provide any real-time analysis in the form of alerting or alarms.

To ease the necessary burden of log collection and review, leading security technology vendors offer more advanced log management appliances that allow automation of what could be a time-consuming, manual-labor -intensive process (or, worse, an insurmountable obstacle) including monitoring of logs, searches of logs, and instant responses in the case of a breach or suspicious database activities. Most importantly, these tools work across multiple database servers and even across database types in combination with logs from servers, security devices, and other network architecture components. Analysis of all of these logs via one device allows users to put the database data in the context of other enterprise log data and to correlate database activity with other activity elsewhere on the network. This multi-faceted log management solution paints a more accurate picture of IT infrastructure activity and is more desirable to and more efficient than using separate database, security, server, and network logging tools. Imagine a hacker connecting to a companys web server, and deploying malware to hack deeper inside the company in an attempt to access or steal database information. To investigate the situation if it comes to light (and with the proper security technology, it most definitely would come to light), a company would want to have a single solution or technology that can enable log data from multiple sources (databases, firewalls, routers, servers, applications, operating systems, and network devices) so that information about the attack and its effects (not just on the database specifically targeted but also on other parts of the IT infrastructure) is coherent and readily available and accessible. A single, one-stop security solution for log management simplifies the process greatly. Another advantage of advanced log management tools is the automation of the log management lifecycle, from the collection of logs and where they are generated to the secure transfer of the log data to the companys central server for analysis and storage. From the issuance of real-time alerts to DBAs in the event of a breach to the provision reports and analytics based on log data. From the secure storage of the logs as long as is mandated by whatever retention policy under which the company operates to the safe destruction of the logs when it is acceptable to do so. Obviously, choosing between a simple analysis tool and a more advanced log management deployment depends on many company factors-- budget, IT resources, size, and what sort of regulatory mandates the organization is subject to. Deploying an enterprise-wide solution is, of course, more complicated than simply installing a tool on an individual server, but the benefits of such broad deployment via a true log management appliance will extend far beyond solving a single tactical problem. Satisfying current operational, security, and compliance requirements requires databases to be configured with detailed logging. Unfortunately, the inevitable result of enabling this detailed logging is a cascade of log data, the likes of which no one person or team would be able to manually process. For enterprises that are serious about securing their system, using log management tools for log automation is absolutely essential. While database-vendor-specific tools are better than nothing, combining database log management with other similar projects that manage logs from other sources and using a

single platform for all of them is the desirable option to deter internal and external threats, and make shorter work of regulatory compliance. In summary, getting familiar with database logging and log analysis is a must for todays information security professionals. Security and compliance drivers will make database log management ubiquitous in the coming years, and its never too late to start getting up to speed with these complicated subjects. ABOUT THE AUTHOR: This is an updated author bio, added to the paper at the time of reposting in 2010. Dr. Anton Chuvakin (http://www.chuvakin.org) is a recognized security expert in the field of log management and PCI DSS compliance. He is an author of books "Security Warrior" and "PCI Compliance" and a contributor to "Know Your Enemy II", "Information Security Management Handbook" and others. Anton has published dozens of papers on log management, correlation, data analysis, PCI DSS, security management (see list www.info-secure.org) . His blog http://www.securitywarrior.org is one of the most popular in the industry. In addition, Anton teaches classes and presents at many security conferences across the world; he recently addressed audiences in United States, UK, Singapore, Spain, Russia and other countries. He works on emerging security standards and serves on the advisory boards of several security start-ups. Currently, Anton is developing his security consulting practice www.securitywarriorconsulting.com, focusing on logging and PCI DSS compliance for security vendors and Fortune 500 organizations. Dr. Anton Chuvakin was formerly a Director of PCI Compliance Solutions at Qualys. Previously, Anton worked at LogLogic as a Chief Logging Evangelist, tasked with educating the world about the importance of logging for security, compliance and operations. Before LogLogic, Anton was employed by a security vendor in a strategic product management role. Anton earned his Ph.D. degree from Stony Brook University.