I was able to successfully undelete the rootkit itself, most of its unpacked components, and most of the original (untrojaned) binaries. I was also able to perform an analysis of the damage to the compromised machine.
/dev/ida.
The evidence thus far seemed to indicate that the rootkit was installed at
19:45 (system time), so I looked for
filesystem events right around then. The
ctimes on ps, netstat, and ifconfig
suggest that they are trojaned versions installed by the attacker.
/etc/services and
/etc/inetd.conf look to be more
or less normal, and do not offer much additional information.
/etc/rc.d/rc.sysinit also
looks to be pretty normal, although there is a strange looking
lsattr command at the end. lsattr in this
context would not serve a purpose, and the arguments do not look right, so
this is probably what the attacker is using to start up his services at
boot time.
debugfs(8) and then sifting through the newly undeleted files.
eve /var/tmp/honeynet schmolli
20:57:34 (0) $ echo "lsdel" | debugfs honeypot.hda8.dd 2>/dev/null | \
> tail +3 | awk '{print $1}' | grep '[0-9][0-9]*' | \
> sed 's:^.*$:dump <\0> /var/tmp/dumps/\0.dump:' | \
> debugfs honeypot.hda8.dd >/dev/null 2>&1
eve /var/tmp/honeynet schmolli
21:04:32 (0) $ echo "lsdel" | debugfs honeypot.hda8.dd 2>/dev/null | \
> tail +3 | awk '{print $1}' | grep '[0-9][0-9]*' | \
> sed 's:^.*$:stat <\0>:' | \
> debugfs honeypot.hda8.dd > /var/tmp/dumps/inode.stats 2>/dev/null
eve /var/tmp/honeynet schmolli
21:07:06 (0) $ cd /var/tmp/dumps/
eve /var/tmp/dumps schmolli
21:07:21 (0) $ file *
16110.dump: ASCII test
2038.dump: empty
2039.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), not stripped
2040.dump: ASCII test
2041.dump: Bourne shell script text
2042.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), stripped
2043.dump: Bourne-Again shell script text
2044.dump: English text
2045.dump: Bourne shell script text
2046.dump: English text
2047.dump: perl commands text
2048.dump: English text
2049.dump: data
2050.dump: ASCII test
2051.dump: data
2052.dump: ASCII test
2053.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), not stripped
2054.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), stripped
2058.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), stripped
2059.dump: ASCII test
2060.dump: ASCII test
2061.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), not stripped
23.dump: gzip compressed data, deflated, last modified: Fri Mar 2 21:09:06 2001, os: Unix
30188.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), stripped
30191.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), stripped
48284.dump: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), stripped
56231.dump: ASCII test
8097.dump: empty
8100.dump: English text
There is only one file (23.dump)
detected as compressed data, which makes it the leading candidate for being
the rootkit.
eve /var/tmp/dumps schmolli 21:22:56 (0) $ gunzip -c 23.dump | tar tf - last/ last/ssh last/pidfile last/install last/linsniffer last/cleaner last/inetd.conf last/lsattr last/services last/sense last/ssh_config last/ssh_host_key last/ssh_host_key.pub last/ssh_random_seed last/sshd_config last/sl2 last/last.cgi last/ps last/netstat last/ifconfig last/top last/logclear last/s last/mkxfsThis result pretty much confirmed
23.dump as the rootkit.
At this point, I suspected that the reason for the modification time on the
root directory of the compromised machine was that this was where the
attacker downloaded the rootkit to, but it could also have just been a core
file. To find out, I used debugfs to dump the blocks allocated
to the root directory and then pulled out all the strings. If the system
has been quiescent, then this should be roughly equivalent to a list of what
has recently been in the root directory.
eve /var/tmp/honeynet schmolli 21:32:56 (0) $ echo "dump <2> /var/tmp/2.dump" | \ > debugfs honeypot.hda8.dd >/dev/null eve /var/tmp/honeynet schmolli 21:34:34 (0) $ strings -n 3 /var/tmp/2.dump lost+found boota home usr var procIV tmp dev etc bin lib mnt opt rootA sbinb floppy lk.tgz lastThis luckily produced results immediately, saving me the trouble of doing the same for any of the larger directories. "
last" is the
name of the directory in the compressed tarfile, so I am confident that the
identity of 23.dump prior to deletion was "lk.tgz".
Examination of the rootkit reveals that it is made up of a mix of
precompiled binaries, shell scripts, and perl scripts. Of particular
interest is the install script,
which is the probable mechanism used to install the rootkit. I was unable
to locate the "computer" file which is produced as a side
effect of the install script, which suggests that the
disk space was reclaimed.
I performed a simple analysis of the unpacked rootkit to determine the purpose of each component. The results are summarized in the following table.
cleaner |
cleaner is a shell script for performing syslog
cleaning. |
ifconfig, last.cgi, netstat, ps, top |
These appear to be trojaned/unsafe binaries. This theory is
supported by /dev/rpm and /dev/last,
which look like standard configuration files for trojaned versions
of netstat, ps, top. |
inetd.conf, services |
These two are basic /etc/ configuration files. I believe it likely that they are installed in order to establish a basic network daemon environment that does not clash with services the attacker wants to run. |
install |
install is a shell script for installing the rootkit
and then sending out mail containing information about the system.
install also modifies /etc/rc.d/rc.sysinit
to make a call to a modified /usr/bin/lsattr, which in
turn starts an ssh daemon and a packet sniffer. |
linsniffer, logclear |
linsniffer is a packet sniffer.
logclear is a simple script for restarting linsniffer
and clearing its log |
lsattr |
lsattr is a shell script that starts an ssh daemon
(mkxfs) and packet sniffer (linsniffer).
The install script adds a call to lsattr
to /etc/rc.d/rc.sysinit. |
mkxfs, s, ssh_host_key, ssh_host_key.pub, ssh_random_seed,
sshd_config |
mkxfs is a version 1.2.27 ssh daemon. s
is an associated config file that instructs the daemon to start on
port 5. The install script configured the system such
that mkxfs is started by the lsattr shell
script at boot time. sshd_config does not appear to
be used. |
sense |
sense is a perl script that parses the output from
linsniffer. |
ssh, ssh_config |
These appear to be an ssh client and config file. They do not serve an obvious function, and may just be included for purely mundane reasons. |
sl2 |
Based upon the output from strings(1) and limited
experimentation, I have concluded that sl2 is a
packet flooder. |
The last part of my investigation was to confirm that the versions of
/bin/ps, /bin/netstat, and
/sbin/ifconfig originally present on the system were among
the deleted files. I did this by generating a list of all files in the
compromised root filesystem and thier md5sums, then searching for the
md5sums of undeleted files in that list.
eve /honey root
19:15:08 (0) # find /honey -type f -exec md5sum {} \; | \
> sort -k2 > /var/tmp/honey.md5sumx
eve /honey root
19:20:54 (0) # cd /var/tmp/dumps
eve /honey root
19:21:02 (0) # md5sum *.dump | while read md5 file ; do
> if ! grep -q $md5 ../honey.md5sums ; then ls -l $file ; fi
> done
-rwxr-xr-x 1 schmolli schmolli 239 May 24 17:57 16110.dump*
-rwxr-xr-x 1 schmolli schmolli 611931 May 24 17:57 2039.dump*
-rwxr-xr-x 1 schmolli schmolli 3713 May 24 17:57 2041.dump*
-rwxr-xr-x 1 schmolli schmolli 796 May 24 17:57 2042.dump*
-rwxr-xr-x 1 schmolli schmolli 1345 May 24 17:57 2043.dump*
-rwxr-xr-x 1 schmolli schmolli 79 May 24 17:57 2045.dump*
-rwxr-xr-x 1 schmolli schmolli 880 May 24 17:57 2048.dump*
-rwxr-xr-x 1 schmolli schmolli 344 May 24 17:57 2050.dump*
-rwxr-xr-x 1 schmolli schmolli 688 May 24 17:57 2052.dump*
-rwxr-xr-x 1 schmolli schmolli 4620 May 24 17:57 2054.dump*
-rwxr-xr-x 1 schmolli schmolli 53588 May 24 17:57 2058.dump*
-rwxr-xr-x 1 schmolli schmolli 520333 May 24 17:57 23.dump*
-rwxr-xr-x 1 schmolli schmolli 66736 May 24 17:57 30188.dump*
-rwxr-xr-x 1 schmolli schmolli 60080 May 24 17:57 30191.dump*
-rwxr-xr-x 1 schmolli schmolli 42736 May 24 17:57 48284.dump*
-rwxr-xr-x 1 schmolli schmolli 33135 May 24 17:57 56231.dump*
-rwxr-xr-x 1 schmolli schmolli 16329 May 24 17:57 8100.dump*
Based upon file sizes and experimentation, I was able to quickly identify
the following deleted binaries.
2058.dump |
/last/top |
trojan |
30188.dump |
/bin/netstat |
original |
30191.dump |
/bin/ps |
original |
48284.dump |
/sbin/ifconfig |
original |