Intrusion Detection FAQ

(http://www.sans.org/newlook/resources/IDFAQ/ID_FAQ.htm)

Terms, Theory and Research

What is ID?
ID stands for Intrusion Detection, which is the art of detecting inappropriate,
incorrect, or anomalous activity. ID systems that operate on a host to detect malicious
activity on that host are called host-based ID systems, and ID systems that operate on
network data flows are called network-based ID systems.
Sometimes, a distinction is made between misuse and intrusion detection. The term
intrusion is used to describe attacks from the outside; whereas, misuse is used to
describe an attack that originates from the internal network. However, most people
don't draw such distinctions.
The most common approaches to ID are statistical anomaly detection and patternmatching detection.

What is host-based intrusion detection?

Host-based ID involves loading a piece or pieces of software on the system to be
monitored. The loaded software uses log files and/or the system's auditing agents as
sources of data. In contrast, a network- based ID system monitors the traffic on its
network segment as a data source. Both network-based and host-based ID sensors
have pros and cons, and in the end, you'll probably want to use a combination of each.
The person responsible for monitoring the IDS needs to be an alert, competent System
Administrator, who is familiar with the host machine, network connections, users and
their habits, and all software installed on the machine. This doesn't mean that he or
she must be an expert on the software itself, but rather needs a feel for how the
machine is supposed to be running and what programs are legitimate. Many break-ins
have been contained by attentive Sys Admins who have noticed something "different"
about their machines or who have noticed a user logged on at a time atypical for that
user.
Host-based ID involves not only looking at the communications traffic in and out of a
single computer, but also checking the integrity of your system files and watching for
suspicious processes. To get complete coverage at your site with host-based ID, you
need to load the ID software on every computer. There are two primary classes of
host-based intrusion detection software: host wrappers/personal firewalls and agentbased software. Either approach is much more effective in detecting trusted-insider
attacks (so-called anomalous activity) than is network-based ID, and both are
relatively effective for detecting attacks from the outside.
Host wrappers or personal firewalls can be configured to look at all network packets,
connection attempts, or login attempts to the monitored machine. This can also
include dial-in attempts or other non-network related communication ports. The best
known
examples
of
wrapper
packages
are
TCPWrappers
(http://coast.cs.purdue.edu/pub/tools/unix)
for
Unix
and
Nuke
Nabber
(http://www.amitar.com.au/DOWNLOADS/INTERNET/PROTECTION/NukeNabber
_2_9b.html) for Windows. Personal firewalls can also detect software on the host

attempting to connect to the network, such as WRQ's AtGuard

(http://www.atguard.com).
In addition, host-based agents may be able to monitor accesses and changes to critical
system files and changes in user privilege. Well-known commercial versions include
products from AXENT (www.axent.com), CyberSafe, (www.cybersafe.com) ISS,
(www.iss.net) and Tripwire (www.tripwiresecurity.com). (There's also an Academic
Source Release of Tripwire available if your site is an academic department of a state
university.)
In addition, UNIX has a rich set of software tools to perform intrusion detection. No
one package will do everything, and the software should be tailored to the individual
computer that's being monitored. For example, if a machine has only a handful of
users, perhaps only the connections from the outside and the integrity of the system
files need to be monitored; whereas, a machine with a lot of users or network traffic
may need more stringent monitoring. Types of software that help monitor hosts
include: system and user log files (syslog); connectivity monitoring (TCPwrappers,
lastlog); process monitoring (lsof (http://vic.cc.purdue.edu/pub/tools/unix/lsof),
process accounting); disk usage monitoring (quotas); session monitoring (options to
ftpd to log all file transfers, process accounting); system auditing (audit).
UNIX host-based intrusion detection is only as good as the logging that's done.
Programs can be written to analyze log files and alert the Sys Admin via e-mail or
pager when something is amiss. System logging output can be sent to a remote site or
modified, so that the log files are put into non-standard places to prevent hackers from
covering their tracks. With the prevalence of hacking scripts, home-brew monitoring
can be set up to watch for specific instances of break-ins. Some "must-reads" for the
Sys Admin new to host-based intrusion is Practical Unix & Internet Security by
Simson Garfinkel and Gene Spafford, (2nd edition, published by O'Reilly) and
Intrusion Detection: An Introduction to Internet Surveillance,
Correlation, Trace Back, Traps, and Response, by Edward Amoroso", (published by
Intrusion.Net Books). Manual pages for network daemons give information on how to
produce logging. Any of the xxxstat programs (vmstat, netstat, nfsstat) or software
like t!'op (ftp.groupsys.com/pub/top) can help point out suspicious activity. Know
your system, and know it well.
A truly effective IDS will use a combination of network- and host-based intrusion
detection. Figuring out where to use each type and how to integrate the data is a real
and growing concern.

What is network based intrusion detection?

A network-based ID system monitors the traffic on its network segment as a data
source. This is generally accomplished by placing the network interface card in
promiscuous mode to capture all network traffic that crosses its network segment.
Network traffic on other segments, and traffic on other means of communication (like
phone lines) can't be monitored. Both network-based and host-based ID sensors have
pros and cons. In the end, you'll probably want a combination of both.
Network-based ID involves looking at the packets on the network as they pass by
some sensor. The sensor can only see the packets that happen to be carried on the
network segment its attached to. Packets are considered to be of interest if they match
a signature. Three primary types of signatures are string signatures, port signatures,
and header condition signatures.
String signatures look for a text string that indicates a possible attack. An example
string signature for UNIX might be "cat "+ +" > /.rhosts" , which if successful, might

cause a UNIX system to become extremely vulnerable to network attack. To refine the
string signature to reduce the number of false positives, it may be necessary to use a
compound string signature. A compound string signature for a common Web server
attack might be "cgi-bin" AND "aglimpse" AND "IFS".
Port signatures simply watch for connection attempts to well-known, frequently
attacked ports. Examples of these ports include telnet (TCP port 23), FTP (TCP port
21/20), SUNRPC (TCP/UDP port 111), and IMAP (TCP port 143). If any of these
ports arent used by the site, then incoming packets to these ports are suspicious.
Header signatures watch for dangerous or illogical combinations in packet headers.
The most famous example is Winnuke, where a packet is destined for a NetBIOS port
and the Urgent pointer, or Out Of Band pointer is set. This resulted in the "blue screen
of death" for Windows systems. Another well-known header signature is a TCP packet
with both the SYN and FIN flags set, signifying that the requestor wishes to start and
stop a connection at the same time.
Well-known, network-based intrusion detection systems include AXENT
(www.axent.com), Cisco (www.cisco.com), CyberSafe (www.cybersafe.com), ISS
(www.iss.net), and Shadow (www.nswc.navy.mil/ISSEC/CID).
A good ID capability will use both host- and network-based systems. Figuring out
where to use each type and how to integrate the data is a real and growing concern.

I have often heard that the best approach to computer

security is to use a layered approach. Can you describe
this approach and how an IDS fits in?
The layered approach can best be compared as an analogy of weathering out a winter
storm. Many people know the feeling of being stuck at home during a winter blizzard.
The things one does in a winter storm are to heat some soup, turn up the furnace,
snuggle up under the blankets, and start a fire in the fireplace. All of these things lead
to a warm and secure feeling while waiting for the storm to pass. It's this utilization of
separate things in the household that results in an overall approach that gives us that
warm and fuzzy feeling in a winter storm. Thus, computer security is the most
effective when multiple layers of security are used within an organization.
The most common misconception is that a firewall will secure your computer
facilities and additional steps don't need to be taken. A firewall is just one component
of an effective security model. Additional components or layers should be added to
provide an effective security model within your organization. The security model that
will protect your organization should be built upon the following layers:
Security policy of your organization
Host system security
Auditing
Router security
Firewalls
Intrusion detection systems
Incident response plan
Using multiple layers in a security model is the most effective method of deterring
unauthorized use of computer systems and network services. Every layer provides
some protection from intrusion, and the defeat of one layer may not lead to the
compromise your whole organization. Each layer has some inter-dependence on other
layers. For example, the intrusion detection systems and the incident response plan
have some interdependencies. Although they can be implemented independently, it's

best when they're implemented together. Having an intrusion detection system that
can alert you to unauthorized attempts on your system has little value unless an
incident response plan is in place to deal with problems. The most important part of
overall security organization is the security policy. You must know what you need to
protect and to what degree. All other layers of the security model follow logically
after the implementation of the organization security policy.
In summary, an intrusion detection system is just one component of an effective
security model for an organization. The overall security integrity of your organization
is dependent upon the implementation of all layers of the security model. The
implementation of the layered approach to security should be undertaken in a logical
and methodical manner for best results and to ensure the overall sanity of the security
personnel.

What is knowledge-based intrusion detection?

There are two complementary approaches to detecting intrusions, knowledge-based
approaches and behaviour-based approaches. This entry describes the first approach.
Almost all IDS tools today are knowledge-based. This is also referred to in the
literature as misuse detection.
Knowledge-based intrusion detection techniques apply the knowledge accumulated
about specific attacks and system vulnerabilities. The intrusion detection system
contains information about these vulnerabilities and looks for attempts to exploit these
vulnerabilities. When such an attempt is detected, an alarm is triggered. In other
words, any action that is not explicitly recognized as an attack is considered
acceptable. Therefore, the accuracy of knowledge-based intrusion detection systems is
considered good. However, their completeness (i.e. the fact that they detect all
possible attacks) depends on the regular update of knowledge about attacks.
Advantages of the knowledge-based approaches are that they have the potential for
very low false alarm rates, and the contextual analysis proposed by the intrusion
detection system is detailed, making it easier for the security officer using this
intrusion detection system to take preventive or corrective action.
Drawbacks include the difficulty of gathering the required information on the known
attacks and keeping it up to date with new vulnerabilities and environments.
Maintenance of the knowledge base of the intrusion detection system requires careful
analysis of each vulnerability and is therefore a time-consuming task. Knowledgebased approaches also have to face the generalization issue. Knowledge about attacks
is very focused, dependent on the operating system, version, platform, and
application. The resulting intrusion detection tool is therefore closely tied to a given
environment. Also, detection of insider attacks involving an abuse of privileges is
deemed more difficult because no vulnerability is actually exploited by the attacker.

What is behavior-based intrusion detection?

There are two complementary approaches to detecting intrusions, knowledge-based
approaches and behavior-based approaches. This entry describes the second approach.
It must be noted that very few tools today implement such an approach, even if the
founding Denning paper {D. Denning, An Intrusion Detection Model, IEEE
transactions on software engineering} recognizes this as a requirement for IDS
systems.
Behavior-based intrusion detection techniques assume that an intrusion can be
detected by observing a deviation from normal or expected behavior of the system or
the users. The model of normal or valid behavior is extracted from reference

information collected by various means. The intrusion detection system later

compares this model with the current activity. When a deviation is observed, an alarm
is generated. In other words, anything that does not correspond to a previously learned
behavior is considered intrusive. Therefore, the intrusion detection system might be
complete (i.e. all attacks should be caught), but its accuracy is a difficult issue (i.e.
you get a lot of false alarms).
Advantages of behavior-based approaches are that they can detect attempts to exploit
new and unforeseen vulnerabilities. They can even contribute to the (partially)
automatic discovery of these new attacks. They are less dependent on operating
system-specific mechanisms. They also help detect 'abuse of privileges' types of
attacks that do not actually involve exploiting any security vulnerability. In short, this
is the paranoid approach: Everything which has not been seen previously is
dangerous.
The high false alarm rate is generally cited as the main drawback of behavior-based
techniques because the entire scope of the behavior of an information system may not
be covered during the learning phase. Also, behavior can change over time,
introducing the need for periodic online retraining of the behavior profile, resulting
either in unavailability of the intrusion detection system or in additional false alarms.
The information system can undergo attacks at the same time the intrusion detection
system is learning the behavior. As a result, the behavior profile contains intrusive
behavior, which is not detected as anomalous.

The Importance of Intrusion Protection

Evolution
When we talk about Intrusion Detection Systems (IDS), management automatically
assumed it is THE solution to all network, organization and social problems. Most
people deal with this technology like it is a monolithic solution. This is not a good
way to consider any security technology, it does not work like that. The majority fails
to recognize that IDS' initial design and function is to protect the organization's vital
information from an outsider.
However, this is now slowly changing, as more organizations want to monitor their
"networks" because studies shows the majority of all losses in the commercial sector
involve insiders. They now want to use the IDS in any of the following combinations:
To track down insiders, catch them in the act, get the evidence needed for prosecution,
fire them or take them to court for indictment.
Another factor to consider is technology is still at its infancy and intrusions get missed
due to its immaturity. RAID'99 identified that in order to reach its full potential as a
forensic tool, IDS' role must evolve to include better logging and a collections of
forensic tools to use the information as evidence (http://www.raid-symposium.org/).
New attack techniques are coming out each month and the IDS technology must adapt
to these rapid changes. The list of all known attacks constantly changes rendering
codifying the statistical "signature" of a new attack a daunting task for R&D labs.
Current Network Intrusion Detection System (NIDS) products (first generation) use a
predominantly passive approach to collect data via protocol analysis by watching
traffic on the network. Most IDS have been built on signature-base and anomaly
detection, providing the capability to look for set "patterns" in packets, but they can
also be tuned to look for things you should never see. The addition of specific string
search signature (i.e. look for confidential), logging and TCP reset features has greatly
enhance the IDS capability as a detection and protection tool.

The work done by Common Vulnerabilities and Exposures (CVE) Editorial Board is a
result of a collaborative effort, which will advance and standardize attack names and
definitions across vendors. Since its implementations (1999), a large number of
organizations have declared that they are working to make their product or database
CVE-compatible. This list can be viewed at http://cve.mitre.org.

Tomorrow's IDS
Due to the inability of NIDS to see all the traffic on switched Ethernet, many
companies are now turning to Host-based IDS (second generation). These products
can use far more efficient intrusion detection techniques such as heuristic rules and
analysis. Depending on the sophistication of the sensor, it may also learn and establish
user profiles as part of its behavioral database. Charting what is normal behavior on
the network would be accomplished over a period of time.

Strength and Limits facing IDS

Today, we recognize that IDS have evolved and are still very much in research stages
to refining and moving the technology forward (RAID 2000 at http://www.raidsymposium.org/raid2000/). However, here is a list of advantages and limitations to
consider before deploying them:

Strength

A strong IDS Security Policy is the HEART of commercial IDS

Provides worthwhile information about malicious network traffic
Can be programmed to minimise damag
A useful tool for ones Network Security Armory
Help identify the source of the incoming probes or attacks
Can collect forensic evidence, which could be used to identify intruders
Similar to a security "camera" or a "burglar alarm"
Alert security personnel that someone is picking the "lock"
Alerts security personel that a Network Invasion maybe in progress
When well configured, provides a certain "peace" of mind
Part of a Total Defense Strategy infrastructure

Limits
Not a cure-all for most security ills
Produces false positive (false alarms)
Produces false negative (failed to alarm)
Large-scale attacks could overwhelm a sensor
NIDS cannot properly protect high-speed networks
All products have weaknesses
Not a replacement for:
well managed firewall
regular security audit
a strong security policy
As part of the Total Defense Strategy of an organization, they offer additional
protection and deterrence against:

Script kiddies
Hackers
Would-be hackers
Crackers
Industrial espionage
Elite Blackhat

Total Defense Strategy

IDS is just another tool part of a good security architecture and Multi-Layered
Defense Strategy. It has its strengths and weaknesses, which must be assessed and
weighed before a decision is made to deploy one on your network. The decision can
be made after you test two or three against your baseline in a lab environment. This
way, you measure as accurately as possible its effects against your network (i.e.
workload, detection accuracy, etc.). You may also want to check some IDS lab
studies. In November 1999, one was published by Network computing at
http://www.nwc.com/1023/1023f1.html
The power of IDS is that it demonstrates a positive degree of readiness, which may be
critical for long term success. If your business depends on networking, IDS is good
business and well worth the return.

Why does my intrusion-detection system generate false

alarms / no alarms?
Well, first of all it's extremely difficult to detect intrusions. We are only seeing the
first generation of commercial tools, and they are limited in scope. However, it is
clear that today's tool either generate lots of false positives (i.e. signaling attack when
there is none) or miss attacks. Even tools based on attack signatures generate a large
number of false alarms, in the most unexpected cases.
One of the most obvious reasons why false alarms occur is because tools are stateless.
To detect an intrusion, simple pattern matching of signatures is often insufficient.
However, that's what most tools do. Then, if the signature is not carefully designed,
there will be lots of matches. For example, tools detect attacks in sendmail by looking
for the words "DEBUG" or "WIZARD" as the first word of a line. If this is in the
body of the message, it's in fact innocuous, but if the tool doesn't differentiate
between the header and the body of the mail, then a false alarm is generated.
Another example deals with http requests for bad scripts. One would actually like to
know whether the request was successful or not, because it's a very different story to
react to. In the second case, it is a minor annoyance, in the first a dangerous security
breach. Since tools do not have the notion of session, creating such a signature is quite
difficult.
In the false positive realm, tools cannot cope with the amount of data to be analyzed.
Keep in mind that 99.99% of the data analyzed is for nothing, and that for each of
these pieces of data, all attack tests have to be performed! Therefore, accuracy is often
traded for speed. Also, there are many ways to detect an attack, and sometimes
attackers come quickly with new methods that bypass the detection mechanism.
Finally, there are many events happening in the course of the normal life of any
system or network that can be mistaken for attacks. A lot of sysadmin activity can be
catalogued as anomalous. Therefore, a clear correlation between attack data and
administrative data should be established to cross-check that everything happening on
a system is actually desired.

What is a bastion host?

A bastion host is a computer that is fully exposed to attack. The system is on the
public side of the demilitarized zone (DMZ), unprotected by a firewall or filtering
router. Frequently the roles of these systems are critical to the network security
system. Indeed the firewalls and routers can be considered bastion hosts. Due to their
exposure a great deal of effort must be put into designing and configuring bastion
hosts to minimize the chances of penetration. Other types of bastion hosts include
web, mail,
DNS, and FTP servers. Some network administrators will also use sacrificial lambs as
bastion hosts, these systems are deliberately exposed to potential hackers to both
delay and facilitate tracking of attempted break-ins.
Effective bastion hosts are configured very differently from typical hosts. Each
bastion host fulfills a specific role, all unnecessary services, protocols, programs, and
network ports are disabled or removed. Bastion hosts do not share authentication
services with trusted hosts within the network so that if a bastion is compromised the
intruder will still not have 'the keys to the castle.' A bastion host is hardened to limit
potential methods of attack. The specific steps to harden a particular bastion host
depend upon the intended role of that host as well as the operating system and
software that it will be running. Access Control Lists (ACLs) will be modified on the
file system and other system objects; all unnecessary TCP and UDP ports will be
disabled; all non-critical services and daemons will be removed; as many utilities and
system configuration tools as is practical will also be removed. All appropriate service
packs, hot fixes, and patches should be installed. Logging of all security related events
need to be enabled and steps need to be taken to ensure the integrity of the logs so that
a successful intruder is unable to erase evidence of their visit. Any local user account
and password databases should be encrypted if possible.
The last step to securing a bastion host may be the most difficult: securing whatever
network application the host is running. Very often the vendor of a web or streaming
media server doesn't consider security risks while developing their product. It is
usually up to the system administrator to determine through testing what ACLs they
need to modify to lock down the network application as thoroughly as possible
without disabling the very features that make is a useful tool. It is also necessary to
closely track the latest announcements from the vendor regarding security problems,
workarounds, and patches. The more popular network applications also tend to inspire
the creation of independent mailing lists, newsgroups, and websites that can be
tracked for additional insights.