Linux based Unauthorized Process Control

Si-Jung Kim

Choul-Woong Son

Cheon-Woo Lee

Research Institute
AT&T Co.,Ltd.
Taejeon, Korea
Sjkim6183@hanmir.com

Dept. of Computer Engineering

Hannam University.
Taejeon, Korea
Choulang@hannam.ac.kr

Dept. of Computer Information

Woosong College.
Taejeon, Korea
cwlee@it.wsi.ac.kr

AbstractThere is a problem of Linux security arising as

Linux system has come out as a core of electric automated
system and played a role of important operating system of
mobile communications. This paper describes security of Linux
and requirements, analyzes well-known weakness of Linux,
suggests solutions through existing security solution, and
introduces an improved Linux kernel module. It also suggests a
security module of Linux kernel improved by controlling
invaders to access and detecting unauthorized processors by
modifying kernel module and provides integrity of system
information by updating invasion information of operating
system in real time. It is expected to be used as a solution in
various server security solutions like firewall and IDS in the
future.
Keywords Linux secutiry , Kernel module, Rrootkit, Access
control

I.

INTRODUCTION

It checks process control based on unauthorized Linux

through improved Linux kernel-leveled security solution and
takes a look at improved Linux security solutions [1][2].
II.

BACKGROUND

Linux kernel, high capacity software, performs functions

like process management, file system, memory management
and network management. Linux kernel source can used
freely by anyone and it follows GNU Public License which
lets source be modified freely and redistributed. Weakness of
Linux is announced in the sites such as Security Focus,
CVE(Common Vulnerability and Exposure) and Security
Tracker.
Requirements of Linux security can be defined as
follows[3][4].

Management of user authentication and account

Linux empathized on open source has been developed a

lot after first version leased. As network environment got
common recently, security policies for that are desperate.
Network security policies based on Linux are firewall,
Intrusion Detection System, and so on. They show the limit
as security policies for the inner server which contains
important information.

Access control on file and directory

Management of process

Access control on Network

Hacking prevention function

Self-protection function

So, the instances of information outflow and invasion

happen a lot even though there is a network based security
system. The security of Linux OS has built-in functions such
as authentication with user identification, reuse prevention of
Discretionary Access Control object, and audit trail .

Performance and installation

Due to weakness of Discretionary Access Control built in

Linux it is not safe from hackings like Trojan horse and so
on. Therefore, improved policies and related goods to
security are required to use Linux as a sager server.
Linux is applied to a various fields because it is open
source, but it has a weakness of security because of it as
well, so it should be solved first. This paper takes a look at
kernel-based security solutions so as to solve the weakness of
Linux security and defines necessary items of Linux security
through this.

Recently the number of weakness cases of kernel, a core

security mole of Linux, has been increased to about 4 times.
It shows that the interest in kernel has been increased by
security related specialists or hackers. It takes a lot of time to
analyze the weaknesses of kernel and check.
It takes long time to confront by using patches. It is
because of some cases: errors from design of Linux kernel,
programming errors, and unknown causes. The weaknesses
of Linux security are as follows.

Weakness patch technology of kernel

Access control technology

Optional access control

Invasion prevention technology

978-1-4244-9224-4/11/$26.00 2011 IEEE

Encryption technology

Audit record technology

Attackers on Linux misuse the weaknesses of security in

order to get a system access authority, access to information
illegally, change the use of other computer to distribute spam
information, and participate in attack on high effectiveness
system. Access control of Linux uses an access control
method to decide whether the user enables to access to
specific resource.
Linux based systems use Discretionary Access Control. It
controls object access based on user group. The reason why
the access control technique causes a problem is that the
program intruded by security invasion inherits the access
control authority from the user. So, it is not good. Mandatory
Access Control and Role-Based Access Control that use the
minimum authority principle are safe.

SELinux (secutity m o ud ule)

Security m o d ule
ind ep end ent

Figure 1 shows the process and composition of

hierarchical Linux security decision.

K ernel w ith em b ed d ed secu rity _< call>

Secu rity_< call> (./lin u x/in clu d e/lin u x/secu rity.h )

./linu x/secu rity/selin u x/h o o ks.c

A ccess vecto r cach e (avc)

LKM is a kernel program that can be loaded in Linux

kernel dynamically or unloaded. Generally, it is used to load
a device driver and it is to get rid of discomfort of
recompiling the kernel and rebooting whenever a new device
is connected to the system. An attacker can load the kernel
module that she/he made in the system and control the
system freely by misusing these handy functions. This
convenience is a very negative aspect for security.
Most LKM rootkits use the method which system call
function that attacker on normal System Call Interception or
hooking made lets be operated. Kernel operates the function
that the user wants by referring the address of system call
function defined in "sys_call_table", a global variable. LKM
rootkits originally makes attackers system call routine
operate by changing the location of normal system call
function to the address of the system call made by the
attacker. LKM Next continues to explain with examples
how to snatch system call with some of LKM rootkit module
and hide attackers process, file, and specific strings.
A. Hiding files
An attacker has to hide himself/herself(the existence)in
the system at any time. Especially, the files or directories an
attacker use must not be shown to the system manager. Jobs
as follows can be done by using LKM rootkit.
"getdents" is the system call called when operating ls
instruction, it uses a kernel mode like below and snatches
getdents, a system call. That is how the attacker operates
hacked_getdentsmade by the attacker. And hacked_getdents
routine does files made by the attacker not to output.
int hacked_getdents( )

Secu rity server (./lin u x/security/selin ux/ss)

int init_module()

B inary p o licy file

orig_getdents = sys_call_table[__NR_getdents];
sys_call_table[_NR_getdents] = hacked_getdents;
Figure 1. Layerd LinuxSecurity Process

void cleanup_module()
The callback hooks initialized into security_ops are
defined dynamically as a loadable kernel module but
otherwise contain dummy stub functions in the event that no
security module is loaded.
These stub functions implement the standard Linux DAC
policy. The callback hooks exist at all points where object
mediation must be provided for security.
These include task management program loading, file
system management IPC, module hooks, and network hooks
[3][4]

III.

LOADABLE KERNEL MODULE ROOTKIT

sys_call_table[_NR_getdents] = orig_getdents;

B.

Hiding Strings

Traditional Rootkit uses the method that makes

attackers process, IP address, or ID not to appear by
modifying program like ps, netstat, and who. However, it has
a problem that it can be discovered easily by the system
manager.
When outputting something from the system, it uses
write() system call. So it operates write system call in a
version of Trojan in order not to output specific strings, i.e.

attackers id, ip address, etc. as below by snatching write

system call.
int hacked_write( )
int init_module( )

void cleanup_module()
sys_call_table[__NR_write] = orig_write;
The existing Rootkit can hide attackers process,
directory, file, and even the fact of connection. However, it
provides functions that user wants by changing program
codes of user layer like ps, df, netstat, top, and Isof. Hence
this Rootkit can be detected easily by checking the size of
file, trail of system call used, and integrity of file.
so as to analyze a system Rootkit is installed in, no more
system instructions can be used to detect, analysis programs
like kstat and carbonite.
kstat and carbonite are tools based on Linux, so they can
be used only in Unix-series OS.
Security solutions of kernel module are as follows[5][6].

File protection function: kstat p provides detailed

information about process. When operating kstat p,
process id has to be given as an input value[8][9].
IV.

orig_write = sys_call_table[__NR_write];
sys_call_table[__NR_write] = hacked_write;

LIDS : Linux intrusion Detection System is a host-based

intrusion detection system and composed of security
management tool and kernel patch to improve kernel
security. It provides with the security function of access
to file, process, kernel, and network by using ACL.[7]
PaX : It is the way of intercepting at the kernel buffer
overflow attack using weaknesses of software which
happens most recently. Security mechanism is
composed of NOEXEC and ASLR techniques mainly.

Figure 2 below shows the structure of LSM .

U ser level Process

O pen System C all

Look up Inod e
Error C hecks
LSM M odule Policy Engine
D A C C heck
LSM H ook
C om plete request

Prim ary Securiry

M odule
Secondary Securiry
M odule

access
Inode

Figure 2. Structure of LSM Hook

SYSTEM ACESS CONTROL

According to development of network environment,

openness among systems helps normalize information
sharing and provide users with convenience. Whereas it is
easier to access to important confidential information of
individuals or organizations and system intrusion like
hacking has been increasing rapidly as well. The way of
encrypting users data and storing it has gathered a big
interest. The method to encrypt data and store it is safer than
physical security of the system and it can prevent leak of
important data from theft of disk itself[10][11].
Linux based running program is composed of machine codes
stored as types of operable in the disk and the set of the
codes.
When the process related to action is generated, kernel
manages information of each process. In order to do
scheduling suitable for specific process, kernel will manage
Process Control Block, group of information about process
control flow and the space for the scheduling unit.
The information that kernel manages for each process
includes that a process uses several resources if being
operated and it gets back File Descriptor as a result.
Also instructions and data signals being operated in the
process which is belonged to a tree related to process have to
be managed independently for each process[12][13].
Kernel allocates a lot of data structures to manage
information of process when process is allocated. The
structures in core position among data structures for process
are task_struct and file_struct.

M o d ified Linux
kernel

Pro cess list

Task Linked list

C o m pare

C o m p are w ith Intrusion

Pattern
C hecked Po licy

Figure 4. Imformation of Process(1)

Search U nautho rized Pro cess

A ccess co ntro l
Lo g C o llectio n
M essag e send

Figure 3. Flow of Process.

In order to store processor list in kernel module, the

information for the process is generated in array.
Initial array is applied after initializing. After achieving a
process list, it sets an array through process ID and checks
the elements of the list.
If there are any differences between lists of process, it
regards there is a process and decides it as an abnormal
access.

Figure 5. Imformation of Process(2)

Log file used in system operation in Figure 6 below is

necessary information of system operation to analyze
intrusions.

It takes a stage that follows the information of the process

that kernel progresses.
Inode object manages the data block of process being existed
in the actual disk.
Proposed module operates check on process and forms log
files, and then sends log file and the message of it.
Figure 4 shows the information of process achieved form
instructions and Figure 5 is a checking result of information
of the process by circulating linked list of tack-struct.
Figure 6. Backup Logfile

When LKM detection module finds LKM rootkit, the

module has to sent the current log file and system
information to backup system before an attacker erases the
traces of intrusion. That is how to provide an integrity.
Log files are the record that records users' behaviors accessed
to the system. Therefore, they have semantic information of
security like what the intruder from outside did in the system
and what instructions users used and system operating
information like what the system managed and errors.

The intruder that achieved the root authority enables to delete

log files or change.
Once the job has been done well, anything for the
improvement of the system security cannot be done because
there will not be any information about intrusion path or
intrusion method because the related data to trail or analysis
has been deleted or modulated even if the manager gets to
know the fact of intrusion later.
Hence the gain of logs is a basis of managemental security.
In the first step among 3 steps of collecting information of
process, the LKM Rootkit detection module proposed earlier
transfers the log information of the system being operated
currently when detecting LKM Rootkit.
These logs are the important intrusion information to be used
for the intruder to erase her/his traces has to be backup in the
state of integrity.

V.

CONCLUSION

This paper looked at the weaknesses of Linux kernel

announced and investigated patch technology, access control
technology, and intrusion prevention technology that are the
security element technologies of Linux kernel level to solve
those problems.
Before or after an approacher from outside to Linux system
deletes log files of her/his own access behavior or modulates
them, it transfers messages about information of system
processor to the manager in real time, thus it enables to
watch abnormal approaches and abnormal processes and
intercepts re-accesses.
It analyzed log files with integrity and modulated log files
and showed an effect that gets rid of loss of operating
information of system or minimizes.
The proposed module is expected to be grafted in improved
intrusion detection technology.
A further study on active managing technology that
preserves system operating information of intrusion with
integrity is required in the future.

REFERENCES
[1]
[2]
[3]

[4]
[5]
[6]
[7]

[8]

http://www.ibm.com/developerworks/linux/library/l-selinux/
http://sourceforge.net/projects/stjude/
Verwoerd T, Hunt R, Policy and Implement of an Adaptive
Firewall, Proceedings of the 10th IEEE International Conference on
Networks, Nation University of Singgapore, 27-30 August 2002.
http://www.ibm.com/developerworks/linux/library/l-selinux/
http://sourceforge.net/projects/stjude/
Daniel P. Bovet, Marco Cesati, Understanding of Uinux kernel,
Hanbitmedia, 2001. 12.
Jung-min Kang, In-Suk Jang, Tak-Jun Nam, Trend of Security in
Linux Kernel,
Korea Institute of Imffformation Seccurity ans
Cryptology vol. 15 -2 , 2005.4
http://www.ibm.com/developerworks/linux/library/l-selinux/

[9] http://sourceforge.net/projects/stjude/
[10] Daniel P. Bovet, Marco Cesati, Linux kernel , Hanbitmedia,
2001. 12.
[11] Yong-Chang Ryu, Linux Device Drive, Hanbitmedia, 2009. 09.
[12] Gae_Chan Lee, Heon_Woo Lee, Serch of LKM RootKit Version 2.0,
2002
[13] Peter Jay Salzman, "Programming Guid of Linux Kernel Module,
2003.