The Role of Firewalls in Network Security

DEGREE PROJECT IN COMPUTER ENGINEERING,
FIRST CYCLE, 15 CREDITS

STOCKHOLM, SWEDEN 2018
The Role of Firewalls in

Network Security
A Prestudy for Firewall Threat Modeling
JANI BONNEVIER
SEBASTIAN HEIMLÉN
KTH ROYAL INSTITUTE OF TECHNOLOGY

SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE
Abstract
Firewalls help protect computer networks from intrusions and malware by en-
forcing restrictions on what network traffic is allowed to pass through the fire-
wall into the network. This thesis explores the role of firewalls in network se-
curity, with the ultimate goal of advancing attempts to create a threat model for
firewalls. Five areas are explored, namely:
• Definitions of Concepts
• Firewalls vs. Services as Targets for Direct Attack
• The Past and Future of Firewalls
• Approach to Estimating Firewall Security
• Firewall Configuration and Security Policies
These areas are explored using a questionnaire survey. Each question in the
questionnaire is either tied to a particular area, or is used to evaluate the re-
spondents’ credibility. The questionnaire has 15 questions, many of which ask
for free text answers. The group of potential respondents consists of 209 indi-
viduals, of whom about 75 % are authors of scientific articles that discuss fire-
walls, penetration testing, and other relevant topics. The rest are information
security professionals, journalists or bloggers of varying merit that were found
online.
20 responses to the questionnaire were received. Responses to qualitative ques-

tions were codified to produce some quantitative data.
The conclusions drawn based on the results include, among other things:
• Attackers tend to directly target network services rather than firewalls.

• Respondents disagreed on whether the role of firewalls is currently chang-
ing.
• A possible approach to estimating firewall security takes into account the
network services that the firewall protects.
• Firewall configurations frequently do not match the security policies of the
organizations in which the firewalls are deployed.
Keywords
firewall; firewall configuration; threat modeling; network security; information

security
Sammanfattning
Svensk titel: Brandväggars roll i nätverkssäkerhet: En förstudie för hotmodel-
lering av brandväggar
Brandväggar hjälper att skydda datornätverk från intrång och skadeprogram

genom att begränsa den trafik som tillåts passera genom brandväggen in i nätver-
ket. Denna uppsats utforskar brandväggars roll i nätverkssäkerhet med målet
att göra framsteg i försök att skapa en hotmodell för brandväggar. Fem områden
utforskas, nämligen:
• Definitioner av begrepp
• Brandväggar kontra tjänster som mål för direkta angrepp
• Brandväggens historia och framtid
• Tillvägagångssätt för att estimera brandväggssäkerhet
• Brandväggskonfiguration och säkerhetspolicyer
Dessa områden utforskas via en enkätstudie. Varje fråga i enkäten tillhör antin-
gen ett specifikt område, eller används för att evaluera respondenternas tro-
värdighet. Enkäten har 15 frågor, varav många efterfrågar fritextsvar. Gruppen
potentiella respondenter består av 209 individer, varav cirka 75 % är författare
av vetenskapliga artiklar som behandlar brandväggar, penetrationstestning och
andra relevanta ämnen. Resten är professionella säkerhetskonsulter, journalis-
ter eller bloggare med olika meriter inom informationssäkerhet eller nätverk.
20 svar på enkäten togs emot. Svar på kvalitativa frågor klassificerades för att
producera kvantitativ data.
Slutsatserna som drogs baserat på resultaten inkluderar bl.a.:
• Angripare tenderar att ha nätverkstjänster som sina direkta mål, snarare

än brandväggar.
• Respondenterna var oense om huruvida brandväggars roll just nu förän-
dras.
• Ett möjligt tillvägagångssätt för att uppskatta brandväggssäkerhet tar hän-
syn till de nätverkstjänster brandväggen skyddar.
• Brandväggskonfigurationer överrenstämmer ofta inte med säkerhetsrik-
tlinjerna i de organisationer där brandväggarna är i bruk.
Nyckelord
brandvägg; brandväggskonfiguration; hotmodellering; nätverkssäkerhet; infor-

mationssäkerhet
Glossary
attacker — A person who illegally gains access to and/or tampers with infor-
mation in a computer system.
brute force attack — A repetitive method of trial and error used to obtain in-
formation, typically a person’s username, password or cryptographic key. Soft-
ware is used to generate a large number of guesses, which are then tested until
the correct value is found.
DNS — Domain Name System. A system that maps domain names to IP ad-
dresses. In the context of this thesis, DNS refers to the network services pro-
vided by DNS servers.
firewall — A device that monitors traffic in and out of a local area network and
either allows or denies passage according to its configuration/ruleset.
FTP — File Transfer Protocol. In the context of this thesis, FTP refers to the
network services provided by FTP servers.
HTTP — Hypertext Transfer Protocol. The foundation of communication for

the World Wide Web. In the context of this thesis, HTTP refers to the network
services provided by web servers.
ICMP — Internet Control Message Protocol. Used by network devices to com-

municate outside of regular data transmission, as well as by tools such as ping
and traceroute.
penetration tester; pentester — A person hired to conduct penetration test-

ing.
penetration testing; pentesting — The authorized practice of attempting to

find vulnerabilities in a computer system or network through practical means.
SMTP — Simple Mail Transfer Protocol. A protocol for sending email. In the
context of this thesis, SMTP refers to the network services provided by outgoing
SMTP email servers.
Telnet — A protocol that enables remote terminal connections and interactions

between computers. Unencrypted and superseded by the encrypted SSH proto-
col. In the context of this thesis, Telnet is referred to as a network service rather
than a protocol.
threat model — A model containing the data required to analyze and assess
the security of an IT-system.
Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Research Strategy . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.6 Societal Benefits and Ethics . . . . . . . . . . . . . . . . . . . 4
1.7 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.8 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Theoretical Background . . . . . . . . . . . . . . . . . . . . . . . 5
2.1 Security Policies . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Firewall Fundamentals . . . . . . . . . . . . . . . . . . . . . 5
2.3 Research Methods and Methodologies . . . . . . . . . . . . 7
2.4 Practical Requirements for Scientificity . . . . . . . . . . . 10
2.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Practical Implementation of Research Method . . . . . . . 16
3.3 Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4 Weighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4 Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.1 Respondents’ Experience . . . . . . . . . . . . . . . . . . . . 21
4.2 Respondents’ Roles in Information Security . . . . . . . . . 22
4.3 Regularly Examines Firewalls . . . . . . . . . . . . . . . . . 23
4.4 Definition of a Firewall Configuration Error . . . . . . . . . 24
4.5 Definition of Firewall Breach . . . . . . . . . . . . . . . . . . 26
4.6 Firewall vs. Services as Targets for Attack . . . . . . . . . . 29
4.7 The Role of Firewalls Over the Last Five Years . . . . . . . 31
4.8 The Role of Firewalls in the Cloud . . . . . . . . . . . . . . . 32
4.9 Control Question 1: Firewall Ruleset . . . . . . . . . . . . . 33
4.10Control Question 2: Most Important Traffic to Block . . . . 35
4.11 Most Frequently Exposed Services . . . . . . . . . . . . . . . 36
4.12 Most Frequently Vulnerable Services . . . . . . . . . . . . . 40
4.13 Match Between Firewall Configurations and Security Poli-
cies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.14 Percentage of Firewalls with Configuration-Policy Mismatches 43
4.15 Correlation Between Vulnerable and Exposed Network Ser-
vices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.16 Correlation Between Q14 and Q15 . . . . . . . . . . . . . . . 46
i
5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.2 Research Methods . . . . . . . . . . . . . . . . . . . . . . . . 49
5.3 Validity and Reliability . . . . . . . . . . . . . . . . . . . . . . 50
5.4 Scientificity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5.7 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Appendix A: The Questionnaire . . . . . . . . . . . . . . . . . . . . 61
ii
1 Introduction
This section introduces the thesis, the background to the problem, the problem
statement, the purpose, the research strategy, and goals of the project.
The history of computer networking in general and the Internet in particular,

has consisted of one security disaster after the other, and that will likely continue
to be the case for the foreseeable future. Over the years, a plethora of security
features have been put in place to protect individuals and organizations from
malicious attacks.
One of those security features can be likened to a border control between the
Wild West of the Internet and private local area networks, namely, the firewall1 .
A firewall is a device or computer program created to protect networks inside the
firewall from malicious traffic by filtering the traffic into and out of networks.
Firewalls are, in some sense, not a core feature of the Internet; they have to be
bought, installed and configured correctly to do their job properly. They also
have to be continuously maintained. The same goes for applications that run
inside the network. As a computer network grows larger and more applications
are installed and used, the complexity of managing and updating these applica-
tions, as well as the firewall, increases. Thus the risk of failing to maintain the
security of the network also increases as the network grows.
1.1 Background
This study emerged from one company and their efforts to extend their product.
1.1.1 Foreseeti
Foreseeti is an IT-security company based and located in Stockholm, Sweden.
Foreseeti was founded in 2014 and strives to become a “global leader in quantita-
tive threat modeling and proactive risk management”. Foreseeti has developed
a product called SecuriCAD®, which is a threat modeling and risk management
tool that can be used to analyze IT infrastructure and model threats and weak-
nesses [1].
1.1.2 Problem Background

The SecuriCAD® software developed by Foreseeti employs a probabilistic cal-
culation engine to simulate attacks on IT infrastructure, for example corporate
networks. This calculation engine requires a lot of data to be able to perform
good attack simulations. Foreseeti reached out and wanted to investigate and
obtain more data regarding firewall security; specifically the time it takes an
1
The reader may think of a firewall as something that they have on their own computer. Those
are indeed firewalls, but not of the kind this thesis is concerned with.
1
attacker to breach a firewall in the case it contains a misconfiguration. The cor-
relation between firewall misconfigurations and the time to breach the firewall
has not been studied much. Foreseeti wanted to conduct a quantitative study on
firewalls, the results of which could be used to directly improve SecuriCAD®.
1.2 Problem Statement

A problem statement was proposed by Foreseeti and the study commenced.
However, certain complications were encountered that ultimately resulted in
a different problem statement.
1.2.1 Original Problem

Corporate firewalls supposedly play a part in restricting access to the organiza-
tion’s local networks, protecting the company from malware and attackers [2].
Configuring and managing these firewalls is, however, complex and prone to
human error [3]. Studies [4], [5] have shown that firewalls protecting corporate
networks are often poorly configured, which leads to security risks. The main
problem statement requested by Foreseeti was the following.
How likely is it that a professional penetration tester can breach a

typical enterprise firewall in a certain amount of time?
The statement essentially calls for the quantification of firewall security. This is
a question Foreseeti have had trouble answering, and this project was initially
going to attempt the same.
1.2.2 Final Problem Statement

After some work on the aforementioned problem, it became apparent during our
questionnaire prestudy (Section 3.2.2) that it was rather complex. The state-
ment seems simple enough at first glance, but only because it assumes a very
particular way of looking at firewalls and network security. There are a number
of problems, or open questions, surrounding it that have to be answered prior to
answering the exact question posed by Foreseeti. This study is thus to be seen
as a prestudy that seeks to enable answering the original problem statement, by
exploring and answering the following questions.
• Definitions of Concepts: What exactly does it mean to “breach a fire-

wall”? What constitutes a firewall configuration error?
• Firewalls vs. Services as Targets for Direct Attack: Do attackers
usually even think of firewalls as targets for direct attack, or do they target
network services?
• The Past and Future of Firewalls: Some studies of firewall security
have been done (Section 2.5), but their relevance today depends on how
2
quickly the role of the firewall changes. How, if at all, has it changed re-
cently, or will it change in the future?
• Approach to Estimating Firewall Security: Could one approach to
estimating firewall security be to study the relationship between the fire-
wall and the services it protects?
• Firewall Configuration and Security Policies: How well do firewall
configurations match the security policies of the organizations in which
they are deployed?
1.3 Purpose
The purpose of this thesis is to explore the role of firewalls as a security measure
in corporate computer networks. This is done by researching a few more specific
questions concerning firewalls, such as definitions of concepts, to what degree
they are targeted by attackers, how their role is changing, how their security can
be estimated and how well configured they tend to be. This thesis aims to be a
step along the way toward creating a threat model for firewalls.
1.4 Research Strategy

Given that the research is exploratory and mostly qualitative, the research strat-
egy is to conduct a questionnaire survey. This was suggested by the research
group’s contact at Foreseeti.
1.5 Goals
Several goals with this thesis exist, namely:
1.5.1 Academic
The academic goal of the thesis is to answer a problem statement by carrying out
a project on a scientific basis using methods and methodologies that are proven
and correct. By writing a good thesis that meets all course requirements, the
authors will finish their studies at KTH Royal Institute of Technology.
1.5.2 Industrial
The industrial goal is to provide results that would be of use to Foreseeti in the
future development of their threat model. Since the thesis tries to provide an-
swers to problems regarding firewalls used in corporate settings, other actors in
the industry may also benefit from the findings.
1.5.3 Scientific
Scientifically, the goal is to carry out a valid and reliable study that can be of use
to other researchers. As previously stated, this thesis can be seen as a prestudy
for future research.
3
1.6 Societal Benefits and Ethics
The results presented could possibly benefit corporations that wish to evaluate
the security of their systems. This, in turn, could be beneficial to society as a
whole. Increasing amounts of people’s personal information are kept online to-
day. When corporations become less vulnerable to attacks, the leakage of this
personal information might be reduced. Higher security in corporations would
likely lead to less successful attacks and breaches, which would save corpora-
tions and society a lot of money. Also, a large part of the Internet is made up of
corporate networks. By securing these networks there would be fewer hosts for
malware to spread through, which in turn could reduce the spreading of mal-
ware in general [6].
Since firewalls are an important part of corporate network security, the data
collected as part of this project is considered sensitive. All respondents are kept
completely anonymous because the information they might provide could po-
tentially be used by bad actors to identify real, vulnerable systems.
1.7 Delimitations
This study is concerned with only network firewalls and not personal firewalls,
which are applications installed on individual workstations or laptops. Network
firewalls, on the other hand, are software-, hardware- or cloud-based solutions
that protect entire networks from the dangers that lurk outside. This study is
not concerned with information security matters that do not involve firewalls,
unless brought up by respondents. One example of such a matter would be social
engineering attacks.
1.8 Outline
Section 2 provides a theoretical background that some may need in order un-
derstand the rest of the thesis. Moreover, it presents a theoretical overview of
research methods and methodologies that were considered for this study. It also
presents practical requirements for scientificity in a research method. Lastly, it
discusses previous work that is related to this thesis. Section 3 accounts for the
choice of theoretical research methods. Furthermore, each research question is
discussed. Lastly, the practical implementation of the research method is pre-
sented. In Section 4 the results of the study are presented. Section 5 discusses
the acquired results for each of the problem statements, the research methods
used in the study and the validity and reliability of the study. Furthermore, the
scientificity of the study is evaluated, conclusions are presented and possible
future work related to the study is suggested.
4
2 Theoretical Background
This section aims to give a theoretical background needed to be able to under-
stand the remainder of the thesis and also explains why our problem is a prob-
lem in the first place. This section also gives a theoretical background on various
research methods and methodologies. The specific methodologies used in this
project are discussed in Section 3.
Section 2.1 gives a short explanation of security policies. Section 2.2 explains
why firewalls exist and describe a few ways of attacking networks. Section 2.3
provides a theoretical overview of research methods and methodologies. Section
2.4 discusses what is required for a project to be scientific. Section 2.5 is an
account of related work and studies that have been made regarding firewalls
and how this study differs from them.
2.1 Security Policies

An organization of sufficient size most likely has internal policies that state how
certain things should be done within the organization. One policy that always
should be in place is the security policy, which is a document that states how
an organization plans to protect its physical and information technology assets.
The document should be monitored and updated as the organization and its
security requirements change [7]. The security policy should heavily affect the
configuration of firewalls used by the organization, as they play an important
role in the enforcement of the policy.
2.2 Firewall Fundamentals

Firewalls are a fundamental part of network security and often function as the
first line of defense, partially separating a local network from the Internet. The
task of the firewall is to prevent unauthorized network traffic from passing through
itself into the protected network [2]. To make a firewall function properly it has
to be told what traffic is to be authorized and what is not. This is done in a config-
uration file typically called a ruleset. The ruleset is an ordered list of rules, where
each rule is of the form predicate → action. The predicate typically contains a
range of source IPs, a range of destination IPs, source port, destination port, pro-
tocol and the action typically is either accept, discard, log or a combination of
these (see Table 1). When packets arrive at the firewall they are matched against
this list of rules. Overlapping and conflicting rules are normal occurrences. To
resolve the conflicts, the action of the first rule that matches the packet will be
enforced, therefore the order is crucial and changing the order could change the
behavior of the firewall drastically. The goal when configuring the firewall is to
restrict as much traffic as possible to reduce the risk of letting malicious traffic
in, while avoiding blocking legitimate traffic [8].
5
Table 1: An example firewall ruleset containing three rules.
rule Source IP Destination IP Source Port Destination Port Protocol Action

#1 any 1.2.3.4 any 25 TCP accept
#2 156.78.12.3 172.87.1.3 any 22 TCP accept
#3 any any any any any discard
2.2.1 Ways of Attacking Networks

An attacker must somehow either breach or circumvent the firewall to get fur-
ther inside the network. A variety of ways of doing so exist.
Exposed network services constitute potential attack surfaces. A few common

examples of network services are HTTP, FTP, SSH, SMTP and ICMP. They of-
ten run inside a local area network, but they can be exposed on the Internet by
the firewall or be placed outside of the firewall entirely. In some cases, such as
HTTP, this may be desirable. If a network service is exposed on the Internet,
anyone (who possesses an elementary understanding of networks) can easily
find it with a port scan and send traffic to the service. If the exposed service is
vulnerable, it is an open door through which an attacker can steal information,
carry out sabotage, or with some luck, reach further into the network and do the
same on an even larger portion of the network. In that case, the attacker has
effectively circumvented or breached the firewall.
Phishing is a social engineering attack in which an attacker typically sends emails

from a forged sender address to individuals working at the targeted organiza-
tion, with the intent of tricking the individuals into either downloading mal-
ware or sharing sensitive information with the attacker. These emails are made
to look very authentic and often claim to be very urgent. For example, the email
could claim to be sent from a systems administrator asserting that the receiver
urgently has to install a new security update that is attached to the email. The
attached file is of course not a security update, but a malicious program that
the attacker can use to harvest credentials, monitor the receiver and potentially
connect the receiver’s computer to the attacker’s, thereby creating a connection
through the firewall [9].
Phishing is a widely used technique that takes advantage of the fact that a hu-
man is much easier to manipulate and trick than a security system. According
to Dhamija [10], computer users, in general, lack the knowledge regarding op-
erating systems and security needed to distinguish phishing from legit emails
and websites. Users are also easily deceived by the often very well made faked
images and text in phishing attempts. By utilizing phishing, an attacker can
essentially circumvent all security measures in place. It does not matter how
6
secure the perimeter of the infrastructure is, if the attacker can trick the correct
person into giving up the correct information or downloading malware.
Denial of firewalling is an attack that stems from the more general denial of
service-attacks. During a DoF attack, carefully crafted traffic is used to overload
a firewall. The overloading of the firewall has two possible outcomes. One is
that all traffic is denied, resulting in the network not being reachable; a form of
sabotage. The other, generally more preferred outcome (for the attacker), is that
the firewall becomes so busy that it cannot authenticate the traffic but instead
lets all traffic through. A well-crafted DoF attack can thus disable the firewall
entirely, leaving the network completely vulnerable for further attacks [11].
Configuration errors are a matter of definition studied in this study. What fol-
lows is therefore an introductory explanation of approximately what one might
mean by the word.
Configuration errors are either errors in the firewall ruleset or errors such as
using the default password for the administration interface, exposing the ad-
ministration interface publicly on the Internet or allowing unencrypted remote
access to the administration interface via Telnet [4].
What specifically constitutes a firewall ruleset error depends on the wider se-
curity policy of the organization in question. Any firewall ruleset that does not
comply with the organization’s stated security policy should be treated as a fire-
wall ruleset error. Generally, if a firewall allows an unauthorized agent to access
internal systems or information, it should most likely be considered a firewall
ruleset error.
Other configuration errors may compromise the security of the firewall inde-
pendently of the actual firewall ruleset. It does not matter how well configured
the firewalls rulesets are if the actual configuration of the firewall itself is left
insecure. For example, exposing the administration interface publicly on the
Internet is a major security issue even if the interface is password protected. An
attacker could perform a brute force attack to figure out the password and get
access to the configuration of the firewall. Another example is using the default
password for the administration interface. This renders the firewall useless if
an attacker gets access to the interface.
2.3 Research Methods and Methodologies

Håkansson has written a rather comprehensive overview of research method-
ologies [12]. What follows is a summary of some of them.
7
2.3.1 Categorization of Research Methods
Research methods in [12] are divided into two main categories; quantitative and
qualitative. A quantitative study is concerned with collecting and/or analyzing
large sets of concretely measurable data. A qualitative study, on the other hand,
is more exploratory in nature and aims to reach tentative hypotheses and the-
ories. Finally, triangulation is the practice of utilizing both quantitative and
qualitative methods in order for them to complement each other.
2.3.2 Philosophical Assumptions

The philosophical assumption is the starting point that the rest of the research
is based upon.
Positivism and Realism can be seen as quantitative assumptions, while Inter-

pretivism and Criticalism are more distinctly qualitative.
Positivism assumes that “the reality is objectively given and independent of the
observer and instruments.” Therefore Positivism is useful in projects of experi-
mental and testing character.
Realism assumes that there exists an objective reality independent of any ob-
server or interpretation, but views scientific knowledge as a mere approxima-
tion of the truth. The realist collects data by observing a phenomenon and then
works with understanding the collected data and developing knowledge from it
[12].
Interpretivism assumes that “access to reality (given or socially constructed)

is only through social constructions such as language, consciousness, shared
meanings, and instruments” [13]. They understand phenomena by interpret-
ing the meaning that people assign to them.
Criticalism assumes that “the reality is socially, historically, and culturally con-
stituted, produced and reproduced by people” Criticalism can be used to learn
about social, historical and cultural aspects of people and things produced by
people [12].
2.3.3 Research Methods

Research methods describe and provide the procedures used to accomplish the
research task.
The Experimental research method studies reasons for, and effects of certain
phenomena. It can be used to compare effects of different scenarios against each
other, given different causes. An example usage is system performance testing.
8
The Descriptive research method studies and describes characteristics of the dif-
ferent phenomenon in different scenarios, but does not study causes and effects
of the phenomenon. The Descriptive research method often uses surveys, case
studies or observations to produce and describe representations of situations.
The Non-Experimental research method draws conclusions based on existing

scenarios. The method is used to describe behavior or opinions and can for ex-
ample be used to study users’ behavior or opinions.
The Conceptual research method is used to interpret existing concepts or cre-

ating new ones. The Conceptual research method can for example be used to
examine content in a system.
The Fundamental research method is “curiosity-driven” and aims to generate

“new ideas, principles and theories” [12]. It is used in all kinds of research and
useful for finding new questions to study more closely with other methods.
2.3.4 Research Approaches

The research approach determines how conclusions are drawn and from what
data. The main two approaches are the inductive and deductive ones. The in-
ductive research approach aims to discover things to study and create hypothe-
ses. The data are often collected using qualitative methods. The deductive re-
search approach studies things that are already known to exist. Theories are
tested by deducing and testing a hypothesis by using, almost exclusively, quan-
titative methods with large data sets. Finally, the abductive approach is a com-
bination of the two, that “derives likely conclusions from an incomplete set of
observations” [12].
2.3.5 Research Strategy / Design

The research strategy informs how the research is to be planned and carried out
in practice. Examples of strategies listed in [12] include Experimental, Survey,
Ex post facto, Case study and Exploratory.
In Experimental research, the aim is to conduct experiments where as many

variables as possible are controlled, prove or falsify hypotheses and study rela-
tionships between variables.
Surveys exist in two main varieties; cross-sectional and longitudinal. Cross-

sectional surveys examine the relationships of a wide range of variables at a sin-
gle point in time. Longitudinal surveys do the same over some period of time.
The strategy works well for both quantitative and qualitative studies.
9
Ex post facto research is carried out after the data have already been collected.
It attempts to find causal factors by studying the past.
Case study uses multiple sources of evidence in a real-life context. It is par-

ticularly useful when the distinction between the studied phenomenon and its
context is unclear.
Exploratory research aims to find relationships between as many variables as

possible, often using qualitative surveys. It identifies issues, rather than pro-
vides definite answers to them.
2.3.6 Data Collection

The data collection method determines to a large degree what results can be pre-
sented and what conclusions can be drawn. Common data collection methods
for quantitative research are Observations, Experiments, Questionnaires and
Case studies. For qualitative research; Questionnaires, Interviews, Observa-
tions and Case studies. The quantitative methods are suited for collecting large
data sets, while the qualitative methods are more suited for collecting smaller
but more in-depth, data sets.
2.4 Practical Requirements for Scientificity

Andersson and Ekholm have written a report in which they discuss scientific
work methods and the practical requirements for scientificity in projects [14].
On page 17 in the report, they give an example of a scientific work method based
on their interpretation of a method originating from Bunge [15], in the form of
the following series of steps.
1. Identify a problem within an area of research.

2. Describe the problem carefully.
3. Study existing knowledge within the problem area. Find information, meth-
ods or tools that are relevant to the problem statement.
4. Explain and solve the problem statement based on knowledge from step
3. If existing knowledge on the problem statement is not enough to solve
the problem statement, move to step 5. Otherwise, move to step 6.
5. Propose new ideas, theories or hypothesis and collect new empirical data
to solve the problem.
6. Present either exact or approximate solutions to the problem.
7. Derive the consequences of the proposed solution.
8. Test the proposed solution.
9. Correct the proposed solution according to the test results.
10. Examine the proposed solution with respect to existing knowledge (step
3) and identify new problem statements.
10
Based on this practical sequence, Andersson and Ekholm also present a method
to evaluate the scientific quality of research projects that utilize the method
above. The evaluation involves identifying the existence of certain aspects in
the written work that describes the project. These aspects are described by An-
dersson and Ekholm as characteristic of a project with a scientific approach.
1. Background (Problem owner/stakeholders)

2. Hypothesis/testable consequences
3. Purpose/goals
4. Delimitations
5. Theory describing problem
6. Research method
7. Execution
8. Expected result
9. Achieved result
Based on whether these aspects are accounted for, one can get a hint of whether
a given project used a scientific approach or not.
2.5 Related Work

Wool performed two quantitative studies [4], [5] of firewall ruleset configura-
tion errors in real, deployed firewalls. Both of these studies suggest that fire-
walls, in general, are poorly configured and that the number of configuration er-
rors is positively correlated with the number of rules in the ruleset. Wool’s stud-
ies and our study are tangentially related, but while Wool studied the phenom-
ena of configuration errors in great detail, our study discusses it with broader
strokes as part of a bigger picture. Furthermore, Wool defines a firewall config-
uration error as being any configuration of a set of configurations that he con-
siders to be insecure in some way. Even though the research is probably the best
of its kind, the studies, as well as Wool’s set of configuration errors, are rather
old. This study aims to address both the definition of configuration errors and
the relevance of previous research.
Kamara et al. [16] propose a methodology for analyzing vulnerabilities in fire-

walls. Twenty known firewall vulnerabilities are analyzed and categorized ac-
cording to the suggested methodology. Our work does not analyze or categorize
any vulnerabilities, but again, discusses them in more general terms.
Alsaleh et al. [17] present a set of quantitative metrics for measuring the security
level of an enterprise firewall based on its ruleset configuration. The metric can
be used to compare the security of different firewalls against each other. While
the metric is useful for many purposes and certainly would be interesting to ap-
ply to data such as Wool’s, we cannot make use of it given our choice of research
11
methods. This study instead explores an alternative approach to estimating fire-
wall security.
12
3 Method
What follows is a description of the study in terms of the methods and method-
ologies discussed in Section 2.3.
The study uses triangulation, although it is somewhat more qualitative than

quantitative. It assumes the philosophical viewpoint of realism and uses the
fundamental research method. The study is primarily an inductive one. A cross-
sectional survey, with a questionnaire directed at information security researchers
and professionals constitutes the main research strategy and data collection el-
ements of the study. The quantitative data is analyzed statistically. The qual-
itative data is, apart from being considered as it is, codified in order to extract
quantitative data from them.
3.1 Research Questions

In this section, each of the research questions defined in the problem statement
are discussed. Each questionnaire question is related to one of the research
questions (Section 1.2.2).
3.1.1 Definitions of Concepts

What exactly does it mean to “breach a firewall”? What constitutes a firewall
configuration error?
The project group hopes to collect definitions by experts for these concepts, not
least to see whether an established consensus already exists. The proposed def-
initions are collected with the following questionnaire questions.
Q4: How do you define a firewall configuration error? (Free text

answer)
Q5: How would you define the act of “breaching a firewall”? (Free
text answer)
3.1.2 Firewalls vs. Services as Targets for Direct Attack

Do attackers usually even think of firewalls as targets for direct attack, or do
they target network services?
The answers to this question can provide insights into how firewall security
could be modeled. In the case that firewalls are a target for attackers, it is im-
portant to consider the degree to which firewalls themselves are resistant to at-
tacks. In the other case, firewall configuration is of greater importance. Par-
ticularly relevant are answers from penetration testers who regularly breach or
13
circumvent firewalls. Answers to this question are collected using the following
questionnaire question.
Q6: Do attackers tend to directly target firewalls or do they tend

to target exposed network services? (Free text answer)
3.1.3 The Past and Future of Firewalls

How, if at all, has the role of firewalls changed recently, or will it change in the
future?
The question tries to determine the degree to which previous studies on firewall
security are still relevant, or for how long they might remain relevant. Answers
are collected using the following questionnaire questions.
Q8: Has the role of the firewalls changed during the last 5 years?
How and why? (Free text answer)
Q9: Will the role of firewalls change as more companies move to

the cloud / SDN-networks? How and why? (Free text answer)
3.1.4 Approach to Estimating Firewall Security

Could one approach to estimating firewall security be to study the relationship
between the firewall and the services it protects?
If it is the case that network services, rather than firewalls, are targets for direct
attack, one should explore ways to model firewall security with services in mind.
To study this relationship, two sets of answers are wanted; one that determines
the most frequently exposed network services and another that determines the
most frequently vulnerable ones. The correlation between the two could then be
studied. Answers are collected using the following questionnaire questions.
Q12: In your experience, which network services in corporate net-

works are most frequently exposed on the Internet? (Free text an-
swer; list services in descending order of frequency)

works do most frequently contain vulnerabilities? (Free text an-
3.1.5 Firewall Configuration and Security Policies

How well do firewall configurations match the security policies of the organi-
zations in which they are deployed?
14
One possible way of defining a firewall configuration error (which was one of the
questions in Section 3.1.1) is as a mismatch between the firewall configuration
and the organization’s security policy. Given this definition, the project group
wants to study how well configured firewalls tend to be. Answers are collected
using the following questionnaire questions. They are essentially the same ques-
tion, formulated differently.
Q14: How well does the configuration of the typical perimeter fire-
wall you have encountered match the organization’s security pol-
icy? (Answer given as a number ranging from 1-5, where 1 represents
“Several Mismatches” and 5 represents “Perfect Match”)
Q15: Approximately what percentage of perimeter firewalls that

you have encountered have mismatches between their rulesets and
the organization’s security policy?
3.1.6 Control Questions

These are questions to which the answers are known, that are used to assess the
respondents’ credibility.
Q10: Given this firewall ruleset, which of the following statements

are true? (Options; select all that apply. The ruleset and options are
shown in Section 4.9)
Q11: Which of these types of inbound traffic is the most important

to block with a firewall? (Options; select one. The ruleset and op-
tions are shown in Section 4.10)
3.1.7 Information About the Respondent

A couple of questions are asked about the respondent’s role and experience in
the field of information security. They are mainly used to assess the respon-
dents’ credibility.
Q1: For how many years have you been working with, or research-
ing information security?
Q2: What is your primary role within the field of information se-
curity? (Options or free text answer)
Q3: Do you regularly examine/analyze/test different firewalls? The

question refers to individual firewalls and their configurations; not
brands, models or types. (Yes / No)
15
3.2 Practical Implementation of Research Method
The practical work of this project aimed to utilize the theoretical research method
(described in the beginning of Section 3) while ensuring some level of scien-
tificity (discussed in Section 2.4) and keeping in concordance with the project
triangle, as proposed by Ekholm [18].
This research project was therefore divided into three phases. The phases are
listed in chronological order and depend on each other. Each phase is an essen-
tial part of the project and contributes to the end result. One of the phases was
conducted as an iterative process. What follows is a description of each phase
of the project.
3.2.1 Literature Study Phase

The first phase of the project was the literature study, which commenced as soon
as the problem statement was proposed by Foreseeti. The aim of the literature
study was for the authors to get a better understanding of the problem at hand.
Three main areas were researched, namely:
• Firewalls
• Network security
• Network architecture
The literature study was mainly conducted by reading published literature that
was found through research libraries such as IEEE Xplore2 and ACM Digital
Library3 . The research yielded relevant information in the form of previous
studies and references that could be used in this study, as well as perspectives
that were used to formulate the research questions.
After researching these problem-specific areas, literature regarding scientific re-

search methods and project methods was read, such as [14], [15] and [12]. This
research was used to decide what methods and methodologies were appropri-
ate to be used in the project and ultimately resulted in the project method now
described.
3.2.2 Questionnaire Design and Publication Phase

The literature study reinforced the choice of using a questionnaire as the method
for data collection. The design and creation of said questionnaire became the
second phase, which itself can be divided into three parts, carried out in the
order given below.
2
https://ieeexplore.ieee.org
3
https://dl.acm.org/
16
• Questionnaire design. The initial design of the questionnaire and the
questions at hand were formed.
• Questionnaire prestudy. A prestudy was conducted, where the ques-
tionnaire was tested and evaluated.
• Redesign of the questionnaire. After the prestudy, the questionnaire
was redesigned according to the feedback from the prestudy.
The design of the questionnaire was done according to some tips given by Har-
rison [19]. The questionnaire was kept as short as possible, with as few and
as straightforward questions as possible. 15 questions in total made for a good
length. This should be enough questions to collect the desired data, while still al-
lowing the respondents to properly fill in the questionnaire in a short amount of
time. The questionnaire had to be general enough so that it could be filled out by
people working with information security, but not necessarily strictly firewalls.
As recommended by Harrison [19], a prestudy was conducted where the ques-

tionnaire was answered and discussed (separately) by two information security
professionals; one penetration tester from the industry and one academic pro-
fessor. This was done to verify that the questionnaire was well designed, under-
standable and possible to answer properly. These two discussions gave some
great insight that was used to further improve the questionnaire prior to it be-
ing sent out.
During the entirety of this project phase, another process was also carried out,
namely that of finding potential respondents. In order to conduct as good of
a study as possible, it was important to find competent respondents who pos-
sessed knowledge of information security in general and firewalls in particular.
To help ensure that a sufficient number of responses were received, from various
perspectives, the questionnaire was not solely distributed to firewall specialists.
People from various professions in the field of information security were sought,
although the bulk (about 75 %) of potential respondents ended up being authors
of scientific articles that discuss firewalls, penetration testing and other relevant
topics. The rest were information security or network professionals, journalists
or bloggers of varying merit that were found online.
Subsequently, the questionnaire was deemed ready for publishing. The ques-
tionnaire was created and distributed via Google Forms, which allowed for an
easy and quick distribution of the survey to a large number of respondents around
the world. It also allowed for anonymity (with some caveats, see Section 5.5.1)
which was important to offer to all respondents. The questionnaire remained
open for responses for one week. This was determined to be enough time to al-
low most people who wanted to respond to do so. About half of the respondents
submitted their response within the first 24 hours of the questionnaire being
17
opened.
Questionnaire design
Find potential
respondents Questionnaire prestudy
Questionnaire redesign
Publish questionnaire
Figure 1: Questionnaire design and publication process.
3.2.3 Result Analysis Phase

The third phase commenced when the questionnaire was closed. Qualitative re-
sponses were codified in order to extract some quantitative data from them (ex-
plained further in Section 3.3). Interesting responses were quoted in the thesis.
For some questionnaire questions, the results were weighted (explained further
in Section 3.4). Additionally, correlations were studied and charts were created.
To utilize the advantages of iterative processes, the analysis of the data was done
in iterations. One iteration was carried out for each of the research questions
listed in Section 3.1. In each iteration, the data from the questionnaire regard-
ing that research question were analyzed and discussed. Thanks to this iterative
process, the project ran no risk of running out of time prior to deriving any con-
clusions at all.
3.3 Coding
Qualitative free text answers are codified in an inductive manner, meaning that
the set of possible coding categories for each question is not known in advance,
18
but derived from the responses. The percentage PC of responses codified under
category C is calculated as
RC
PC =
R
where RC is the number of responses codified under C, and R is the total number
of responses (very elementary math).
For some questionnaire questions, a single response can be codified under more
than one category. The sum of the percentages of responses codified under each
of the categories respectively may in such cases exceed 100 %. For example, if
the categories are “Yes” and “No”, a nuanced response (e.g. “Yes, because ___,
but on the other hand, ___”) may be codified as both. A more decisively positive
response would be codified as simply “Yes”. In this example, PYes = RRYes = 22 =
100% and PNo = RRNo = 21 = 50%, and the sum would be 100% + 50% = 150% >
100%.
3.4 Weighting
The results of two of the questionnaire questions (Q12 and Q13) are weighted us-
ing a custom method. The exact formula is presented in Section 4.11.1 (because
it is quite closely tied to the specific questions), but its essence is that the value
of each response is multiplied by the sum of the respondent’s stated experience
(Q1) and score on control questions (Q10 and Q11).
Experience ranges from 0 to the maximum length of a person’s career (on the
order of tens of years) and the control question score ranges from 0 to 6. The
balance between these two aspects affects the weighting significantly and was
considered carefully. Ultimately, however, the current balance, where the max-
imum control question score is worth as much as 6 years of experience, is ad-
mittedly somewhat arbitrary. The group concluded that experience in the field,
even if not directly related to firewalls, is more important for credibility than the
control question score. On the other hand, the control question score is arguably
more reliable, because it is not subject to fabrication in the same way that the
stated amount of experience is.
The weighting is applied to a score that is based on the ranking of options in

responses (also explained in Section 4.11.1). Because the weighting method is
experimental and without basis in literature, the unweighted scores are also pre-
sented. Furthermore, just in case the scoring system itself turns out to be flawed,
results based on the number of mentions of options, regardless of ranking, are
also presented.
19
20
4 Result
The section begins with a presentation of the results for each individual ques-
tionnaire question (Sections 4.1–4.14). Then, correlations between the results
of some of the questions are investigated (Sections 4.15–4.16). This all builds
up to the discussion and conclusions regarding the research questions in the
remainder of the thesis.
The questionnaire was sent to 246 potential respondents. At least 37 of our

requests could not be delivered, bringing the number down to 209. 20 responses
were received, of which 15 were from academic researchers. All questions were
optional, so not all respondents answered all questions.
4.1 Respondents’ Experience

Q1: For how many years have you been working with, or research-
ing information security?
The average respondent claimed to have about 11 years of experience in the field
of information security.
Table 2: Statistics for Q1.

Responses 16
Response rate 80 %
Average 10.88
Median 10
Standard deviation 6.73
21
Q1 - Distribution of responses
6
5
Number of responses
0
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34
Years of experience
Figure 2: Chart for Q1.
4.2 Respondents’ Roles in Information Security

Q2: What is your primary role within the field of information se-
curity? (Options or free text answer)
75 % of respondents described themselves as academic researchers. The pro-

portion is the same as that of authors of scientific articles among potential re-
spondents (Section 3.2.2).

Responses 20
Response rate 100 %
22
Table 4: Results for Q2.
Number Percentage
Academic researcher 15 75.00 %
Penetration tester 2 10.00 %
Administrator 1 5.00 %
Architect 1 5.00 %
Journalist 1 5.00 %
Q2 - Role in the field of information security

16
14
12
Number of responses
10
0
Academic researcher Penetration tester Administrator Architect Journalist
4.3 Regularly Examines Firewalls

Q3: Do you regularly examine/analyze/test different firewalls? The
question refers to individual firewalls and their configurations; not
brands, models or types. (Yes / No)
35 % of respondents answered “Yes” and the rest answered “No”.
23
Responses 20
Responses by academic researchers 15
Responses by others 5
Response rate 100 %
Table 6: Results for Q3. Percentages of respondents who answered a certain

way. The percentages are with regard to the column’s specified group.
All who responded Academic researchers Others
Yes 35.00 % 33.33 % 40.00 %
No 65.00 % 66.67 % 60.00 %
Q3 - Respondent regularly examines firewalls

100
All who responded
90 Academic researchers
Others
80
70
Percentage of responses
60
50
40
30
20
10
0
Yes No
4.4 Definition of a Firewall Configuration Error

Q4: How do you define a firewall configuration error? (Free text
answer)
69 % of respondents mentioned mismatches between firewall rulesets and se-

curity policies as either their definition or a part of their definition of a firewall
24
configuration error. Other concepts were mentioned in 44 % of responses.
A single response could be codified into multiple categories. Thus the presented
percentages do not necessarily add up to 100 % (Section 3.3).

Responses 16
Response rate 80 %
Table 8: Results for Q4. Percentages of respondents who mentioned a certain

concept in their definition. The percentages are with regard to the col-
umn’s specified group.
Coding category All who responded Academic researchers Others

Ruleset-policy mismatch 68.75 % 75.00 % 50.00 %
Ruleset too permissive 25.00 % 16.67 % 50.00 %
Contradictory rules 18.75 % 16.67 % 25.00 %
Q4 - Definition of firewall configuration error

100
All who responded
Others
80
70
60
50
40
30
20
10
0
Ruleset-policy mismatch Ruleset too permissive Contradictory rules
25
Examples of responses that were codified as “Ruleset-policy mismatch” (possi-
bly among other things):
“At the highest level it is when the firewall does not implement a
defined business’ security policy. Unfortunately, some businesses do
not create such a definition leading to a problem in understanding
errors.”
“It’s either a configuration that don’t respect rules and guidelines, or

configuration that represents contradictions with other rulsets.”
“Any configuration that does not align with the corporation’s secu-
rity policy.”
Some responses were somewhat ambiguous, but if they discussed intent or ex-
pectations, they were also codified as “Ruleset-policy mismatch”.
“A bug that doesn’t separate allowed from not-allowed in the in-

tended way.”
“When a firewall configuration fails to perform as expected.”
Others did not mention policies at all.
“Letting unwanted packets that threaten security enter organizations’

network.”
“A man-made mistake resulting in over-permissive ruleset.”
“Any configuration that enables an unauthorized party to gain con-

trol.”
4.5 Definition of Firewall Breach

Q5: How would you define the act of “breaching a firewall”? (Free
text answer)
54 % of respondents mentioned illegitimate traffic passing through the firewall

as their definition or part of their definition of what it means to breach a firewall.
Other concepts were mentioned in 60 % of responses.
A single response could be codified into multiple categories. Thus the presented
percentages do not necessarily add up to 100 % (Section 3.3).
26
Responses 13
Response rate 65 %
Table 10: Results for Q5. Percentages of respondents who mentioned a certain
concept in their definition. The percentages are with regard to the
column’s specified group.

Illegitimate traffic through firewall 53.85 % 60.00 % 33.33 %
Exploitation of firewall vulnerability 30.77 % 30.00 % 33.33 %
Unauthorized access to systems 15.38 % 10.00 % 33.33 %
Circumvention 15.38 % 20.00 % 0.00 %
27
Q5 - Definition of breaching a firewall
100
All who responded
Others
80
70
60
50
40
30
20
10
n
l
ty
s
al
em
io
ili
w
nt
ab
st
e
fir
ve
sy
er
um
gh
ln
to
vu
ou
rc
ss
Ci
ce
hr
l
al
ac
ct
we
fi
ed
fir
af
riz
tr
of
ho
e
n
at
tio
ut
m
na
ita
iti
U
eg
lo
p
Ill
Ex
Examples of responses that were codified as “Illegitimate traffic through fire-

wall” (possibly among other things):
“Illegitimate network traffic passing through the firewall ”
“A hacker manages to initiate a connection through the FW for a ma-

licious purpose”
“It could mean one of two things (i) using misconfigurations to get
through it, or (ii) using a hack or exploit to corrupt or otherwise cir-
cumvent a well defined set of policies, Generally the outcome is pack-
ets getting to where they should not.”
Exploitations of firewall vulnerabilities were the second most mentioned con-

cept (as also seen in the last of above responses).
28
“Exploiting a vulnerability in a firewall to gain access to the device
or changing the behavior of the firewall.”
Some considered circumvention to be a “breach”. Granted, there is some am-

biguity here regarding what these respondents really meant, because their an-
swers were so concise.
“Devising a round-about way to bypass the firewall”.
“Bypassing firewall logic that filters unwanted packets”
4.6 Firewall vs. Services as Targets for Attack

Q6: Do attackers tend to directly target firewalls or do they tend
to target exposed network services? (Free text answer)
The free text answers were codified to produce the presented quantitative data.
57 % of respondents said attackers tend to target services, while 36 % said at-
tackers target both services and firewalls. Thus, about 93 % of respondents
mentioned network services and 36 % mentioned firewalls. No one said that
attackers only target firewalls.

Responses 14
Response rate 70 %
Table 12: Results for Q6. Percentages of respondents who answered a certain
way. The percentages are with regard to the column’s specified group.
Services 57.14 % 70.00 % 25.00 %
Both 35.71 % 30.00 % 50.00 %
Firewall 0.00 % 0.00 % 0.00 %
Neither 7.14 % 0.00 % 25.00 %
29
Q6 - Firewalls vs. services as targets for direct attack
100
All who responded
Others
80
70
60
50
40
30
20
10
0
Services Both Firewall Neither
The single respondent whose answer was codified as “Neither” suggested that
phishing attacks are a far more likely and easier way of penetrating a firewall
than using malware.
One respondent whose answer was codified as “Services”, noted however that
attacks on firewalls certainly do exist, such as denial of service and other hacks.
Some simply stated “Both” as their sole answer, while others expanded upon
what roles the alternatives play.
“Both, if the firewall can be compromised intrusion and attacks be-

come relatively straightforward. Otherwise targeting insecure net-
work services is a good alternative.”
“it depends on the attacker’s intention, if his target is behind a fire-

wall surely he will attack the firewall to reach it, but exposed network
services are always a piece of cake.”
Others were more decidedly on one side.
“Hackers are not interested in attacking firewall. They are interested
30
in the services it protects.”
4.7 The Role of Firewalls Over the Last Five Years

Q8: Has the role of the firewalls changed during the last 5 years?
How and why? (Free text answer)
The free text answers were codified and resulted in the quantified data presented
below. 47 % claimed that the role of firewalls have changed during the last five
years, while 52 % claimed that it has not. The definition of “role” was purposely
not specified in this question, because what professionals in the field regard as
the role of the firewall is in itself a relevant question. In other words, omitting
the definition allowed for more diverse qualitative responses.

Responses 17
Response rate 85 %
Table 14: Codified results for Q8. Percentage of respondents who answered a
certain way. The percentages are with regard to the column’s specified
group.
Yes 47.06 % 50.00 % 40.00 %
No 52.94 % 50.00 % 60.00 %
Nearly half of the respondents thought that the role had changed and the other
half thought that it had not. Many elaborative responses arguing for both sides
were received.
One respondent claimed that the time span was too short.
“No. To short of a timespan. Increase to 10 years and firewalls has

been going from traditional port-based to application aware firewalls.”
Another respondent claimed that the fundamentals of the firewall had not changed,
but it had become better at what it does.
“Fundamentally there were limited changes - it’s still a checkpoint

system. However there are quite some advances in terms of the speed
31
of firewall checking, limited deep packet analysis, and firewall man-
agement.”
One respondent claimed that firewalls now have to inspect more protocols as
applications become more advanced and hide their traffic behind generic pro-
tocols.
“The technical scope in terms of the protocols inspected has become

wider. There is a tug of war between App designers who use generic
protocols such as http and the FWs that try to control meaningful
entities that try to hide behind the generic protocol.”
Others claimed that changes to network architectures have lead to consequences

regarding firewalls.
“Many services migrating to the cloud makes old firewalls less use-
full.”
“Of course, because of SDN, NAT and programmable switches“
4.8 The Role of Firewalls in the Cloud

Q9: Will the role of firewalls change as more companies move to
the cloud / SDN-networks? How and why? (Free text answer)
The free text answers were codified and resulted in the quantified data presented
below. 67 % claimed that the role of firewalls will change as we move from
traditional network architectures to cloud-based network architectures. 25 %
claimed that the role would not change and 8 % of the responses could not be
codified.

Responses 12
Response rate 60 %
32
Table 16: Codified results for Q9. Percentage of respondents who answered a
certain way. The percentages are with regard to the column’s specified
group.
Yes 66.67 % 75.00 % 50.00 %
No 25.00 % 12.50 % 50.00 %
Not codified 8.33 % 12.50 % 0.00 %
Fewer responses were received than for Q8, but there were still some elaborative
answers given.
One motivation for the role of firewalls not changing with cloud/SDN network
architectures was the following.
“I think firewalls will maintain their legacy role, as a ground-level

layer of protection, and one of many systems that can help inform
the emerging generation of platform security solutions.”
Another respondent reasoned differently.
“Yes, with crucial services being moved to the cloud, hackers will tar-
get the cloud more than the enterprise network.”
One respondent argued that the role of firewalls will not change, as they are not
enough.
“i don’t think so, firewalls aren’t enough, otherwise they are indis-
pensable in any good security policy. and new security solutions
used in SDN nets and Cloud are based on firewalls.”
One respondent answered yes on the question, but did not specify in what way
the cloud protection will differ from “traditional” firewalls.
“Yes, Companies will have to rely on the cloud provider protection.”
4.9 Control Question 1: Firewall Ruleset

Q10: Given this firewall ruleset, which of the following statements
are true? (Options; select all that apply)
33
Figure 8: The ruleset referred to in Q10.
The options were the following.
• The local network most likely uses NAT. Correct.

• The ruleset most likely allows hosts on the local network to browse the
WWW. Correct.
• The ruleset contains masked/overlapping rules. Correct.
• The ruleset is a realistic and complete example of how a typical corporate
firewall might be configured.
• The ruleset contains a serious security flaw.
One point was awarded for each statement that was correctly checked or unchecked,
so the highest possible score was five. There was no way of knowing whether a
respondent abstained from answering or thought that all statements were incor-
rect. Thus it was assumed that all respondents answered and leaving all check-
boxes empty resulted in two points.
Table 17: Results for Q10. Number of points scored by respondents.

Average 3.3
Median 3
34
Q10 - Distribution of scores
7
5
Number of responses
0
0 1 2 3 4 5 6
Score
4.10 Control Question 2: Most Important Traffic to Block

Q11: Which of these types of inbound traffic is the most important
to block with a firewall? (Options; select one)
The options were the following.
• Telnet. Correct.
• ICMP.
• Whois.
• FTP.

Responses 16
Response rate 80 %
35
Table 19: Results for Q11. Percentages of respondents who answered with each
of the possible answers. The percentages are with regard to the col-
umn’s specified group.
All who responded Academic researchers Others
Telnet 75.00 % 84.62 % 33.33 %
FTP 0.00 % 0.00 % 0.00 %
ICMP 18.75 % 15.38 % 33.33 %
Whois 6.25 % 0.00 % 33.33 %
Q11 - Most important service to block

100
All who responded
Others
80
70
60
50
40
30
20
10
0
Telnet FTP ICMP Whois
4.11 Most Frequently Exposed Services

works are most frequently exposed on the Internet? (Free text an-
4.11.1 Analysis
Results are presented for all services that were mentioned in either Q12 or Q13.
Three ways of measuring the results were used. The rationale behind these
methods is discussed in Section 3.4.
36
• Mentions. The percentage of respondents that mentioned a given ser-
vice.
• Score. For each response, the listed services were given points based on
their position in the list. The first service was given five points, the next
was given four points, the next three and so on. The score S for a given
service was simply the sum total of points across all responses, according
to the following formula.
∑
5
S= pnp
p=1
where p is the number of points “awarded” based on position and np is

the number of times across all responses that the service occurred in the
position worth p points.
• Weighted score. The weighted score takes into account the respon-
dents’ stated experience and number of correct answers to control ques-
tions. It is scaled such that one weighted point is worth the same as one
unweighted point. The weighted score SW for a given service was calcu-
lated according to the following formula.
( np )
∑5 ∑
SU = p (Eip + Cip )
p=1 i=1
SU Ssum
SW =
SUsum
where
– SU is the unscaled, weighted score.
– p is the number of points “awarded” based on position.
– np is the number of responses in which the service occurred in the
– Eip is the experience in years of respondent number i that mentioned
the service in the position worth p points (Q1, Section 4.1).
– Cip is the number of correct answers (at most 6) to control questions
(Q10, Q11) by respondent number i that mentioned the service in the
– Ssum is the sum of the unweighted scores of all services.
– SUsum is the sum of the unscaled, weighted scores of all services.
4.11.2 Results
Both HTTP and SMTP were mentioned by at least half of the respondents. HTTP
received a score of 47, SMTP about half of that and then FTP less than half of
SMTP’s score.
37
Responses 14
Response rate 70 %

Service Mentioned by Score Weighted score
HTTP 71.43 % 47 49.70
STMP 50.00 % 25 27.50
FTP 21.43 % 10 8.03
Telnet 14.29 % 8 6.41
WLAN 7.14 % 5 1.28
VPN 7.14 % 5 8.54
DNS 7.14 % 4 4.44
Proxy 7.14 % 4 2.39
SMB 7.14 % 2 1.71
NTP 0.00 % 0 0.00
NBT 0.00 % 0 0.00
IRC 0.00 % 0 0.00
SNMP 0.00 % 0 0.00
Sum 110 110.00
38
Q12 - Most exposed network services - Mentioned by percentage of responses
100
90
80
70
60
50
40
30
20
10
0
HTTP SMTP FTP Telnet WLAN VPN DNS Proxy SMB NTP NBT IRC SNMP
Figure 11: Frequencies of services in Q12.
Q12 - Most exposed network services - Score

55
Score
50 Weighted score
45
40
35
30
Score
25
20
15
10
0
Figure 12: Scores of services in Q12.
39
4.12 Most Frequently Vulnerable Services
works do most frequently contain vulnerabilities? (Free text an-
4.12.1 Analysis
Same as Q12, see Section 4.11.1.
4.12.2 Results
Almost half of the respondents mentioned HTTP. The other services were men-
tioned in at most 22 % of responses. In terms of score, HTTP, DNS, SMTP and
FTP were found to be the most frequently vulnerable, in that order.

Responses 9
Response rate 45 %

Service Mentioned by Score Weighted score
HTTP 44.44 % 20 24.33
STMP 22.22 % 9 8.56
FTP 22.22 % 9 7.46
Telnet 11.11 % 3 2.59
WLAN 0.00 % 0 0.00
VPN 0.00 % 0 0.00
DNS 22.22 % 10 9.42
Proxy 0.00 % 0 0.00
SMB 11.11 % 5 3.92
NTP 11.11 % 3 2.35
NBT 11.11 % 2 1.57
IRC 11.11 % 4 5.65
SNMP 11.11 % 4 3.14
Sum 69 69.00
40
Q13 - Most vulnerable network services - Mentioned by percentage of responses
100
90
80
70
60
50
40
30
20
10
0
Figure 13: Frequencies of services in Q13.
Q13 - Most vulnerable network services - Score

26
Score
24
Weighted score
22
20
18
16
14
Score
12
10
8
6
4
2
0
Figure 14: Scores of services in Q13.
41
4.13 Match Between Firewall Configurations and Security Policies
Q14: How well does the configuration of the typical perimeter fire-
wall you have encountered match the organization’s security pol-
icy? (Answer given as a number ranging from 1-5, where 1 represents
“Several Mismatches” and 5 represents “Perfect Match”)
As can be seen below in table 24, the average and median rating was 2.50. Not a
single respondent answered 5, which suggests that respondents usually find at
least one mismatch.

Responses 14
Response rate 70.00 %
Average 2.50
Median 2.50

4
3
Number of responses
0
1 2 3 4 5 6
Several Mismatches Perfect Match
Match between typical firewall and organization security policy
Figure 15: Results for Q14.
42
4.14 Percentage of Firewalls with Configuration-Policy Mismatches
Q15: Approximately what percentage of perimeter firewalls that
you have encountered have mismatches between their rulesets and
the organization’s security policy?
On average, respondents stated that about 59 % of firewall rulesets contain mis-

matches with applicable security policies.

Responses 11
Response rate 55.00 %
Average 58.64
Median 50.00

2
Number of responses
0
0 10 20 30 40 50 60 70 80 90 100 110
Percentage of firewalls with mismatches
Figure 16: Results for Q15.
4.15 Correlation Between Vulnerable and Exposed Network Services

Here, the results from Q12 (Section 4.11) and Q13 (Section 4.12) are compared.
Somewhat of a positive correlation appears to exist between levels of exposure
and vulnerability. Four services clearly stood out as the most insecure; HTTP,
43
SMTP, FTP and DNS. As showcased in Figures 17, 18 and 19, these four services
had the highest scores for both exposure and vulnerability. The red lines in the
plots are a linear function fitted to the data by Gnuplot “using the nonlinear
least-squares Marquardt-Levenberg algorithm” [20].
Percentage of responses that mentioned service as frequently vulnerable
Q12 vs Q13 - Correlation between most exposed and most vulnerable services - Mentions
100
90
80
70
60
50
HTTP
40
30
DNS FTP SMTP
20
10 SNMP
NBT
NTP
IRC SMB Telnet
0 WLAN
Proxy
VPN
0 10 20 30 40 50 60 70 80 90 100
Percentage of responses that mentioned service as frequently exposed
Figure 17: Correlation between frequency of mentions of services in Q12 and

Q13.
44
Q12 vs Q13 - Correlation between most exposed and most vulnerable services - Unweighted score
22
20 HTTP
18
16
Vulnerability score
14
12
10 DNS
FTP SMTP
8
6
SMB
4 SNMP
IRC
NTP Telnet
2 NBT
0 Proxy
WLAN
VPN
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54
Exposure score
Figure 18: Correlation between unweighted scores of services in Q12 and Q13.
Q12 vs Q13 - Correlation between most exposed and most vulnerable services - Weighted score
28
26
24 HTTP
22
20
Vulnerability score
18
16
14
12
10 DNS
8 SMTP
FTP
6 IRC
4 SMB
SNMP Telnet
NTP
2 NBT
0 WLAN
Proxy VPN
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56
Exposure score
Figure 19: Correlation between weighted scores of services in Q12 and Q13.
45
4.16 Correlation Between Q14 and Q15
Here, there results for Q14 (Section 4.13) and Q15 (Section 4.14) are compared.
The questions provide two ways of comparing firewall configurations and se-
curity policies. In a perfect world, the typical firewall configuration would per-
fectly match the security policy, and thus the rating on Q14 would be 5. In Q15
the respondents were to estimate the percentage of firewalls they had encoun-
tered that contained mismatches with security policies. Not a single respondent
answered 0 %, while two respondents answered 100 %.
The responses to Q14 and Q15 were expected to be negatively correlated (typical
firewall matches policy well ⇐⇒ few firewalls have mismatches). Figure 20
displays the correlation between the responses to these two questions, which it
was not as clear as expected. All in all this result suggests that firewall config-
urations generally match their applicable security policies poorly. The red line
in the plot is a linear function fitted to the data by Gnuplot “using the nonlinear
least-squares Marquardt-Levenberg algorithm” [20].
Q14 vs Q15 - Correlation between responses
100
Percentage of FWs with ruleset-policy mismatches
90
80
70
60
50
40
30
20
10
0
1 2 3 4 5
Several Mismatches Perfect Match
Ruleset-policy match for typical firewall
Figure 20: Correlation between responses to Q14 and Q15.
46
5 Discussion
This section discusses the research methods applied (Section 5.2), the acquired
results (Section 5.1) and the validity and reliability of the study (Section 5.3).
The conclusion of the study is presented in Section 5.6 and possible future work
that is related to this study is presented in Section 5.7.
5.1 Results
In this section, the results tied to each research question are discussed.
5.1.1 Definitions of Concepts

The definitions of two concepts were researched using the questionnaire ques-
tions Q4 and Q5 (results presented in Sections 4.4 and 4.5)
Based on the results of Q4, there appears to be a consensus that a firewall con-
figuration error could, at the very least, be defined as a mismatch between the
firewall ruleset and the organization’s security policy. The validity of this result
is reinforced by the fact that no such idea had been mentioned at that point in
the questionnaire; many respondents independently arrived at something ap-
proximating this definition.
It just so happens that toward the end of the questionnaire, there were two ques-
tions regarding how well firewall configurations match security policies (Q14,
Q15). Given the definition we have now arrived at, these two questions can be
thought of as “how well configured do firewalls tend to be?”.
Based on the results of Q5, a firewall breach could be defined as illegitimate

traffic passing through the firewall. Unfortunately, this definition does not state
how a breach is executed. It does, however, appear to exclude things such as
circumvention, social engineering, etc.
5.1.2 Firewalls vs. Services as Targets for Direct Attack

The sole questionnaire question for this problem was Q6 (results presented in
Section 4.6). 93 % of respondents mentioned network services as targets for
direct attack, whereas only 36 % mentioned firewalls.
From this one might conclude that services are a far more critical part of network
security, but that firewalls themselves are not insignificant either.
It is possible that the phrasing of the question suggested to some that it was
an either-or question, even though it was possible to answer “Both”. Had the
question explicitly included “Both” as an alternative, perhaps more respondents
47
would have answered with that. On the other hand, as it is now we can be more
certain that those who did answer “Both” really meant it, rather than simply
opting for it as a safe bet.
5.1.3 The Past and Future of Firewalls

Q8 and Q9 of the questionnaire were used to study this problem (results pre-
sented in Sections 4.7 and 4.8) The same number of respondents answered “Yes”
on Q8 as Q9, and a majority of those who answered “Yes” on Q8 also answered
“Yes” on Q9. Fewer responses were received for Q9 than for Q8. It may be that
respondents, in general, found it more difficult to reason about the future than
to do so about the past, which would make sense.
While the result was quite even on Q8, in the case of Q9 67 % of respondents
thought that the role of the firewall would in fact change as the network archi-
tectures shift toward the cloud. This suggests that much of the research that
exists in the field may become less relevant as this shift progresses.
These results also give rise to some other questions. Will the ways of attacking
networks change as networks become cloud-based and if so, how? Will existing
security measures become obsolete given the new network architecture? Given
that corporations that move to the cloud will lose some control over security
measures, how can they assure their customers of the security of their data?
5.1.4 Approach to Estimating Firewall Security

Two questionnaire questions were used to research this problem, namely Q12
and Q13 (results presented in Sections 4.11, 4.12 and 4.15).
The weighting of scores did not significantly affect the outcome, as the same
services stood out as the most insecure. In fact, the weighted and unweighted
results may be identical within the margins of error that this study has anyway.
The score-based and mention-based results were also very similar.
Somewhat of a positive correlation appeared to exist between levels of exposure

and vulnerability. A possible causality could be that some services have more
vulnerabilities discovered in them because they are more often exposed (we do
not think that the opposite, exposure as a consequence of vulnerability, seems
likely). Arguably the services with high scores play the biggest roles for network
security (and by extension firewall configuration), but one can not necessarily
conclude that these services in and of themselves are the most insecure ones.
If one wishes to evaluate the security of a firewall configuration, we propose that

the threat model would take into account a measure of the risks associated with
48
particular network services, similar to the one used here.
5.1.5 Firewall Configuration and Security Policies

The questionnaire questions Q14 and Q15 were used to study this topic (results
presented in Sections 4.13, 4.14 and 4.16).
The results suggested that firewall configurations generally match their appli-
cable security policies poorly. Studying the cause of this was outside the scope
of this study, but can be speculated upon. It may be difficult to translate the
security policy into rules in a firewall ruleset, or perhaps it is as one respondent
said in their optional comment.
“Organizational policies for firewalls are often non-existent.”
Note that just because a firewall complies with the security policy, does not mean
that it is secure. A perfectly compliant firewall configuration can still be inse-
cure and allow attackers to breach the network. It is then not only the firewall
configuration, but also the organization’s security policy that needs to be re-
considered. Regardless, it is still a problem if firewalls do not match security
policies, because then corporations may not have the security they believe they
have.
5.2 Research Methods

This section discusses the research methods applied in the study.
5.2.1 Research Category - Quantitative vs. Qualitative

This research was categorized as a triangulation study, or in other words, it was
both qualitative and quantitative. That said, it did lean more in the qualitative
direction. Strictly quantitative data would simply not have been helpful in an-
swering many of the problem statements. The research questions were rather
investigatory and qualitative research allowed for the respondents to provide
more thorough and motivated answers, with less influence from the researchers’
preconceptions.
5.2.2 Research Approach - Inductive

As already stated (in Section 1.2.1), the project started with a concrete problem
statement.

However, difficulties arose when attempting to study the problem using an ab-
49
ductive approach. Thus a new set of problems statements were formulated and
the project was forced to employ a more inductive approach instead, as many
of the new problem statements were not really about measuring anything, but
rather exploring what there is to measure.
5.2.3 Data Collection - Questionnaire

Because the research was inductive and the research questions required a more
qualitative approach, a questionnaire seemed more appropriate than for exam-
ple an experimental approach. The issue with experiments is that they often
utilize test benches or other experimental tools to produce results that very of-
ten are quantitative [12]. Creating an experimental environment to answer the
problem statements of this particular study would be difficult and subject to the
researchers’ preconceptions. A questionnaire, on the other hand, can relatively
easily and inexpensively be designed to collect relevant answers to the problem
statements.
5.3 Validity and Reliability

This section discusses the validity and reliability of the results acquired from
this study. Shenton refers to four specific criteria that should be discussed in
order to ensure a trustworthy study; credibility, transferability, dependability
and confirmability [21].
5.3.1 Credibility
Credibility is concerned with the question “How congruent are the findings with
reality?” [21] and is one of the most important criteria for ensuring a trustwor-
thy study. Shenton provides a few examples of provisions that can be used to
promote confidence in that the results of the study are trustworthy, such as the
following.
• Adoption of established research methods.

• Developing familiarity early with participating organizations/individuals
before the first data collection takes place.
• Establishing tactics to help ensure honesty amongst participants, for ex-
ample by allowing anonymity, or by accepting rejection and not harass the
participant with participation requests [21].
The project method used in this study adopted several of these provisions. Well
established research methods have been adopted and motivated. Data collection
was done via a questionnaire with a controlled group of potential respondents.
The potential respondents were chosen based on their presumed skill set and
ability to answer the questions. The questionnaire was completely anonymous,
with the ambition of allowing the participants to be honest (although anonymity
50
may not be entirely good in our case, see Section 5.5). The main reason for
the control questions was that the competence of respondents could to some
degree be determined. The questionnaire was distributed by email, with a single,
personalized email sent to each recipient (except in one case, where two emails
were sent by mistake). There was no nagging or begging for participation.
5.3.2 Transferability
Transferability “is concerned with the extent to which the findings of one study
can be applied to other situations” [21]. Shenton suggests that the question re-
garding transferability is highly impacted by the boundaries of the study, and
gives examples of a few things that need to be conveyed to the reader prior to
discussing transferability, such as the following.
• The number of organizations/individuals that participate in the study.

• Any restrictions on the type of people who contributed data.
• The data collection methods employed.
• The number and length of data collecting sessions.
• The time period over which the data were collected.
These things have been clearly addressed in this thesis.
5.3.3 Dependability
Dependability is concerned with whether a repetition of the study, with the same
method under the same circumstances, gives the same result. Shenton [21] spec-
ifies three sections that should exist in the scientific report to allow readers to
reason about the dependability of the study, namely:
• Descriptions of methods and plans.

• Detailed descriptions regarding the data gathering.
• Evaluation of the process and it’s results.
This thesis includes all of these, and therefore it should be possible for anyone to
repeat this study quite closely. Even if our methods themselves are dependable,
our results may not be, due to the small number of respondents. Thus the results
of a repeated study may not be the same.
5.3.4 Confirmability
Confirmability is concerned with the objectivity of the study. Real objectivity
is very hard to ensure in studies utilizing questionnaires or tests since they are
designed by humans, and therefore the intrusion of the researchers’ biases is
inevitable. A few examples of things that increase the objectivity of a study are
the following.
51
• Decisions made and methods adopted should be acknowledged and moti-
vated within the report.
• Preliminary theories that were not born out of data should be discussed.
• Descriptions on how research questions gave rise to work should be in-
cluded in the report [21].
The chosen methods, questionnaire questions and strategies for answering the
research questions have been motivated and described, mostly in Sections 1
and 3. Many of the questions in the questionnaire allowed for free text an-
swers, which allowed for objectivity, since the respondent is not forced to choose
among answers or definitions predetermined by the researchers. Some of the
questions, for example Q9 (Section 4.8), contained certain assumptions, which
goes against objectivity as the question itself may be biased.
5.4 Scientificity
Our work quite closely matches the scientific work method described in Section
2.4, particularly steps 1–6. Steps 8–10 (testing, correction and examination of
the solution) have not been done at all; we simply present our results and con-
clusions and that is all.
In table 26 the scientificity of the study has been evaluated according to the eval-
uation method described in Section 2.4. The study seems to fall short with re-
gard to the existence of a hypothesis; there simply is none (although the neces-
sity of a hypothesis in an inductive study is not entirely evident to us).
Based on the criteria laid out by Andersson and Ekholm [14], we may conclude
that our study is somewhat scientific, but not perfectly so.
Table 26: Evaluation of the scientificity of the study.

Aspect Missing Mentioned Examined
1 Background X
2 Problem statement X
3 Hypothesis X
4 Purpose/goals X
5 Delimitations X
6 Problem theory X
7 Research method X
8 Execution X
9 Expected result X
10 Achieved result X
52
5.5 Limitations
Given the size of the field of information security, the number of respondents
in the study was very low with only 20; a fact that significantly diminishes the
reliability of the study. Possible reasons for the low response rate are discussed
in 5.5.1.
Anonymity and the use of an open online questionnaire impact the credibility of
the result negatively because it cannot be guaranteed that all actual respondents
were among the potential respondents that had been chosen by us. That said,
we have no reason to suspect that the questionnaire at any point was subject to
uncontrolled distribution.
A prestudy was conducted to verify the first version of the questionnaire. The
second version of the questionnaire was not properly verified due to time con-
straints, but another verification would have been valuable because the problem
statement itself was reformulated as a result of the prestudy and the changes to
the questionnaire were quite radical.
5.5.1 Possible Reasons for Few Responses

The participation rate of the questionnaire was 10 %, and there are many possi-
ble reasons for why this might be the case.
Some people may refrain from providing information regarding their compe-
tence and profession in a service (Google Forms) where they know that all of the
entered data can be read by Google. The respondents are anonymous to us, but
they may not be to Google.
Email addresses to researchers were extracted from published articles they had
authored, but it is possible that many of the researchers have since moved on
and no longer use the addresses found in the articles.
Some of the potential respondents may reside in China. Considering that the
country employs the so-called “Great Firewall of China” (a phrase used to de-
scribe Internet censorship in the country [22]), there is no way of knowing whether
our emails were delivered to these individuals.
Apart from these possibilities, the low participation rate probably just stems
from completely “normal” causes, such as missing the email in the inbox, lacking
interest, not having time, or forgetting to respond.
53
5.6 Conclusion
The purpose of this study was to explore the role of the firewall in network se-
curity. This was done by researching five more specific problems. Two of them
were concerned with the relationship between firewalls and network services,
and it is in this area we believe this study makes its foremost contribution. With
regard to the question about firewall configurations, our results are in line with
findings from other studies, not least those by Wool [4], [5]. Realistically, we do
not consider our results to be that revolutionary nor reliable. What follows is a
short summary of our findings.
• Definitions of Concepts: A firewall configuration error is generally a

mismatch between the firewall ruleset and the organization’s security pol-
icy or otherwise intended behavior.
• Firewalls vs. Services as Targets for Direct Attack: Attackers tend
to directly target network services rather than firewalls, although firewalls
themselves are not an insignificant target either.
• The Past and Future of Firewalls: Respondents disagreed on whether
the role of firewalls is currently changing, but seemed to argue from dif-
ferent perspectives.
• Approach to Estimating Firewall Security: A possible approach to
estimating firewall security takes into account the network services that
the firewall protects.
• Firewall Configuration and Security Policies: Firewall configura-
tions frequently do not match the security policies of the organizations in
which the firewalls are deployed.
5.7 Future Work

This study is thought of as a prestudy for one that can answer the original prob-
lem statement (Section 1.2.1), which was:

Naturally, the findings of this study should be of use in a future study that re-
sumes the work on this problem. Our proposition for such a future study is that
it be an experimental one, for example:
1. Determine an appropriate definition of a firewall breach (based on Section

5.1.1).
2. Set up an experimental environment with a mock network consisting of
a firewall, the most crucial services (based on Section 5.1.4) with some
known vulnerabilities and perhaps some workstations. Several network
54
and firewall configurations could be created, between which the exposure
of services and/or vulnerabilities in the services vary.
3. Let penetration testers attack the network and measure the time it takes
them to breach the firewall according to the determined definition. The
services present in the network and/or exposed by the firewall would con-
stitute variables to be accounted for.
Additional questions exist that we believe could be studied more closely.
• This thesis discusses the role of traditional firewalls in traditional network

infrastructures and also how the role of these firewalls will change as we
move toward cloud- and SDN-networks. Some respondents argued that
the firewall will keep its main purpose to filter and authorize traffic, but
that the actual implementation and configuration of firewalls will have
to change in response to the requirements of new cloud architectures. A
study that examines in what way these implementation and configuration
changes would best be incorporated could therefore be in place.
• Another potential study could be to compare firewall vulnerabilities in tra-
ditional networks with those in the cloud, to see if the new architecture
leads to different vulnerabilities or if the same security issues are present
in that environment.
55
56
References
[1] Foreseeti. (2018) About foreseeti. Resource accessed: 2018-03-27.
[Online]. Available: https://www.foreseeti.com/about
[2] J. E. Canavan, Fundamentals of Network Security. Norwood, MA, USA:

Artech House, Inc., 2001, p. 153-155.
[3] T. Wong, “On the usability of firewall configuration,” 05 2012. [Online].

Available: https://cups.cs.cmu.edu/soups/2008/USM/wong.pdf
[4] A. Wool, “A quantitative study of firewall configuration errors,” Computer,

vol. 37, pp. 62–67, 06 2004. [Online]. Available: http://www.eng.tau.ac.
il/~yash/computer2004.pdf
[5] ——, “Trends in firewall configuration errors: Measuring the holes in swiss
cheese,” Internet Computing, IEEE, vol. 14, pp. 58 – 65, 09 2010.
[6] A. Zaharia, “Ransomware distribution: How one in-

fection can go network-wide,” 2016, visited 2018-
04-15. [Online]. Available: https://heimdalsecurity.com/blog/
ransomware-distribution-one-infection-network-wide/
[7] M. Rouse. (2007) security policy. Resource accessed: 2018-06-08.

[Online]. Available: https://searchsecurity.techtarget.com/definition/
security-policy
[8] A. X. Liu, “Firewall policy change-impact analysis,” ACM Trans. Internet

Technol., vol. 11, no. 4, Mar. 2008. [Online]. Available: http://doi.acm.
org./10.1145/2109211.2109212
[9] J. Hong, “The state of phishing attacks,” Commun. ACM, vol. 55, no. 1,
pp. 74–81, Jan. 2012. [Online]. Available: http://doi.acm.org.focus.lib.
kth.se/10.1145/2063176.2063197
[10] R. Dhamija, J. D. Tygar, and M. Hearst, “Why phishing works,” pp. 581–
590, 2006. [Online]. Available: http://doi.acm.org/10.1145/1124772.
1124861
[11] A. X. Liu, A. R. Khakpour, J. W. Hulst, Z. Ge, D. Pei, and J. Wang,

“Firewall fingerprinting and denial of firewalling attacks,” Trans. Info.
For. Sec., vol. 12, no. 7, pp. 1699–1712, Jul. 2017. [Online]. Available:
https://doi.org/10.1109/TIFS.2017.2668602
57
[12] A. Håkansson, Portal of Research Methods and Methodologies for
Research Projects and Degree Projects. CSREA Press U.S.A, 2013,
p. 67–73, [ed] Hamid R. Arabnia, Azita Bahrami, Victor A. Clincy,
Leonidas Deligiannidis, George Jandieri. [Online]. Available: http:
//urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-136960
[13] M. D. Myers, Qualitative Research in Business & Management, il-

lustrated edition ed. Sage Publications Ltd, Nov. 2008, p. 38.
[Online]. Available: http://www.amazon.com/exec/obidos/redirect?tag=
citeulike07-20&path=ASIN/141292166X
[14] N. Andersson and A. Ekholm, “Vetenskaplighet - utvärdering av tre imple-

menteringsprojekt inom it bygg & fastighet 2002,” Institutionen för Byg-
gande och Arkitektur, Lunds Universitet, Tech. Rep., 2002, p 16-17.
[15] M. Bunge, Epistemology and Methodology I: Exploring the World. Rei-

del, 1983, vol. 5.
[16] S. Kamara, S. Fahmy, E. Schultz, F. Kerschbaum, and M. e. Frantzen,

“Analysis of vulnerabilities in internet firewalls,” vol. 22, 04 2003.
[Online]. Available: https://www.cs.purdue.edu/homes/fahmy/papers/
firewall-analysis.pdf
[17] M. N. Alsaleh, S. Al-Haj, and E. Al-Shaer, “Objective metrics for firewall

security: A holistic view,” pp. 470–477, Oct 2013. [Online]. Available:
https://ieeexplore.ieee.org/document/6682762/
[18] S. Eklund, Arbeta i projekt. Studentlitteratur, 2011, vol. 4, pp. 128–129.
[19] C. Harrison, “Tip sheet on question wording,” 2007, visited 2018-

04-19. [Online]. Available: https://psr.iq.harvard.edu/files/psr/files/
PSRQuestionnaireTipSheet_0.pdf
[20] T. Williams and C. Kelley, “gnuplot 5.2: An interactive plotting

program,” 05 2018, resource accessed: 2018-05-29. [Online]. Available:
http://gnuplot.info/docs_5.2/Gnuplot_5.2.pdf
[21] A. K. Shenton, “Strategies for ensuring trustworthiness in qualitative

research projects.” Education for Information, vol. 22, no. 2, pp. 63–75,
2004. [Online]. Available: http://dblp.uni-trier.de/db/journals/efi/efi22.
html#Shenton04
[22] F. Shen, “Great firewall of china,” pp. 599–602, 01 2014. [On-
58
line]. Available: https://www.researchgate.net/publication/281030754_
Great_Firewall_of_China
59
60
4/26/2018 Corporate Firewall Security Survey
Corporate Firewall Security Survey

We are interested in the role of firewalls in network security. Feel free to skip any question if you feel
that you do not have a good answer. All answers are completely anonymous.
Definitions:
"firewall" = Firewall protecting an corporate local area network. NOT a personal firewall.
This survey is conducted as part of a Bachelor's thesis by Jani Bonnevier <jkarki@kth.se> and
Sebastian Heimlén <heimlen@kth.se> at KTH Royal Institute of Technology.
1. For how many years have you been working

with, or researching information security?
2. What is your primary role within the field of information security?

Mark only one oval.
Academic researcher
Penetration tester
Network/firewall administrator
Journalist / blogger
None (I do not work within the field)
Other:
3. Do you regularly examine/analyze/test different firewalls?

The question refers to individual firewalls and their configurations; not brands, models or types.
Mark only one oval.
Yes
No
Your Opinions
Definitions:
4. How do you define a firewall configuration error?
https://docs.google.com/forms/d/16z1FpCL0perwfQzQZ0IjotQXaafmt7LniLitU4Y5Jug/edit 1/4
5. How would you define the act of "breaching a firewall"?
6. Do attackers tend to directly target firewalls or do they tend to target exposed network
services?
7. Do firewalls instill a false sense of security? How?
8. Has the role of the firewalls changed during the last 5 years? How and why?
9. Will the role of firewalls change as more companies move to the cloud / SDN-networks?
How and why?
10. Given this firewall ruleset, which of the following statements are true?
Check all that apply.
The local network most likely uses NAT.

The ruleset most likely allows hosts on the local network to browse the WWW.
The ruleset contains masked/overlapping rules.
The ruleset is a realistic and complete example of how a typical corporate firewall might be
configured.
The ruleset contains a serious security flaw.
11. Which of these types of inbound traffic is the most important to block with a firewall?
Mark only one oval.
ICMP
Telnet
Whois
FTP
Your Estimations
Base your answers to the following questions on your own experience and expert knowledge.
Definitions:
12. In your experience, which network services in corporate networks are most frequently
exposed on the Internet?
Specify in order, with most frequent first.
13. In your experience, which network services in corporate networks do most frequently
contain vulnerabilities?
Specify in order, with most frequent first.
14. How well does the configuration of the typical perimeter firewall you have encountered
match the organization's security policy?
A "perimeter firewall" is a firewall deployed between a local area network and the Internet.
Mark only one oval.
1 2 3 4 5
Several mismatches Perfect match
15. Approximately what percentage of perimeter

firewalls that you have encountered have
mismatches between their rulesets and the
organization's security policy?
Percent, 0-100. A "perimeter firewall" is a
firewall deployed between a local area network
and the Internet.
16. Optional comment

Thank you for answering the survey! If you have any comments, feel free to post them below.
Powered by
TRITA TRITA-EECS-EX-2018:134
www.kth.se

The Role of Firewalls in Network Security

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

The Role of Firewalls in Network Security

Transféré par

Droits d'auteur :

Formats disponibles

DEGREE PROJECT IN COMPUTER ENGINEERING,

FIRST CYCLE, 15 CREDITS

The Role of Firewalls in

KTH ROYAL INSTITUTE OF TECHNOLOGY

20 responses to the questionnaire were received. Responses to qualitative ques-

• Attackers tend to directly target network services rather than firewalls.

firewall; firewall configuration; threat modeling; network security; information

Brandväggar hjälper att skydda datornätverk från intrång och skadeprogram

Slutsatserna som drogs baserat på resultaten inkluderar bl.a.:

• Angripare tenderar att ha nätverkstjänster som sina direkta mål, snarare

brandvägg; brandväggskonfiguration; hotmodellering; nätverkssäkerhet; infor-

HTTP — Hypertext Transfer Protocol. The foundation of communication for

ICMP — Internet Control Message Protocol. Used by network devices to com-

penetration tester; pentester — A person hired to conduct penetration test-

penetration testing; pentesting — The authorized practice of attempting to

Telnet — A protocol that enables remote terminal connections and interactions

The history of computer networking in general and the Internet in particular,

1.1.2 Problem Background

1.2 Problem Statement

1.2.1 Original Problem

How likely is it that a professional penetration tester can breach a

1.2.2 Final Problem Statement

• Definitions of Concepts: What exactly does it mean to “breach a fire-

1.4 Research Strategy

2.1 Security Policies

2.2 Firewall Fundamentals

rule Source IP Destination IP Source Port Destination Port Protocol Action

2.2.1 Ways of Attacking Networks

Exposed network services constitute potential attack surfaces. A few common

Phishing is a social engineering attack in which an attacker typically sends emails

2.3 Research Methods and Methodologies

2.3.2 Philosophical Assumptions

Positivism and Realism can be seen as quantitative assumptions, while Inter-

Interpretivism assumes that “access to reality (given or socially constructed)

2.3.3 Research Methods

The Non-Experimental research method draws conclusions based on existing

The Conceptual research method is used to interpret existing concepts or cre-

The Fundamental research method is “curiosity-driven” and aims to generate

2.3.4 Research Approaches

2.3.5 Research Strategy / Design

In Experimental research, the aim is to conduct experiments where as many

Surveys exist in two main varieties; cross-sectional and longitudinal. Cross-

Case study uses multiple sources of evidence in a real-life context. It is par-

Exploratory research aims to find relationships between as many variables as

2.3.6 Data Collection

2.4 Practical Requirements for Scientificity

1. Identify a problem within an area of research.

1. Background (Problem owner/stakeholders)

2.5 Related Work

Kamara et al. [16] propose a methodology for analyzing vulnerabilities in fire-

The study uses triangulation, although it is somewhat more qualitative than

3.1 Research Questions

3.1.1 Definitions of Concepts

Q4: How do you define a firewall configuration error? (Free text

3.1.2 Firewalls vs. Services as Targets for Direct Attack

Q6: Do attackers tend to directly target firewalls or do they tend

3.1.3 The Past and Future of Firewalls

Q9: Will the role of firewalls change as more companies move to

3.1.4 Approach to Estimating Firewall Security

Q12: In your experience, which network services in corporate net-

Q13: In your experience, which network services in corporate net-

3.1.5 Firewall Configuration and Security Policies

Q15: Approximately what percentage of perimeter firewalls that

3.1.6 Control Questions