Scan 15 (May 2001), by whizzard hn8963@pelennor.net

1. Show step by step how you identify and recover the deleted rootkit from the / partition.

Using debugfs, the ext2 file system debugger, deleted inodes can be found and identified, as shown below.

aragorn: debugfs honeypot.hda8.dd                                    ~/honeynet 
debugfs 1.18, 11-Nov-1999 for EXT2 FS 0.5b, 95/08/09
debugfs:  lsdel
29 deleted inodes found.
 Inode  Owner  Mode    Size    Blocks    Time deleted
 56231      0 100644  33135   13/  13 Thu Mar 15 05:17:36 2001
 16110      0 100644    239    1/   1 Thu Mar 15 05:20:25 2001
  2058      0 100755  53588   54/  54 Thu Mar 15 19:45:02 2001
 30188      0 100755  66736   67/  67 Thu Mar 15 19:45:02 2001
 30191      0 100555  60080   60/  60 Thu Mar 15 19:45:02 2001
 48284      0 100755  42736   43/  43 Thu Mar 15 19:45:02 2001
  2047      0 100755   4060    4/   4 Thu Mar 15 19:45:03 2001
  2049      0 100600    540    1/   1 Thu Mar 15 19:45:03 2001
  2051      0 100600    512    1/   1 Thu Mar 15 19:45:03 2001
  2053      0 100700   8268    9/   9 Thu Mar 15 19:45:03 2001
  2059      0 100700     75    1/   1 Thu Mar 15 19:45:03 2001
  2060      0 100644    708    1/   1 Thu Mar 15 19:45:03 2001
  2061      0 100755 632066  622/ 622 Thu Mar 15 19:45:03 2001
    23      0 100644 520333  512/ 512 Thu Mar 15 19:45:05 2001
  2039      0 100755 611931  602/ 602 Thu Mar 15 19:45:05 2001
  2040      0 100644      1    1/   1 Thu Mar 15 19:45:05 2001
  2041      0 100700   3713    4/   4 Thu Mar 15 19:45:05 2001
  2042      0 100644    796    1/   1 Thu Mar 15 19:45:05 2001
  2043      0 100755   1345    2/   2 Thu Mar 15 19:45:05 2001
  2044      0 100644   3278    4/   4 Thu Mar 15 19:45:05 2001
  2045      0 100755     79    1/   1 Thu Mar 15 19:45:05 2001
  2046      0 100644  11407   12/  12 Thu Mar 15 19:45:05 2001
  2048      0 100644    880    1/   1 Thu Mar 15 19:45:05 2001
  2050      0 100644    344    1/   1 Thu Mar 15 19:45:05 2001
  2052      0 100644    688    1/   1 Thu Mar 15 19:45:05 2001
  2054      0 100755   4620    5/   5 Thu Mar 15 19:45:05 2001
  2038   1031  40755      0    1/   1 Thu Mar 15 19:46:09 2001
  8097      0  40700      0    1/   1 Fri Mar 16 04:03:12 2001
  8100      0 100644  16329  177/ 177 Fri Mar 16 04:03:12 2001
debugfs:

In this case, there are relatively few deleted inodes, but if necessary, interesting inodes could be picked out of this list using the owner, size, and deletion time shown. Each of the files referenced in the list of deleted inodes was recovered, using the following method:

debugfs:  dump <56231> recovered.56321                                          

An analysis of the resulting recovered files using the file command yields the following:

aragorn: file recovered.*                                          ~/honeynet/1
recovered.00023: gzip compressed data, deflated, last modified: Fri Mar  2 21:09
:06 2001, os: Unix
recovered.02038: empty
recovered.02039: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), not stripped
recovered.02040: PCX image data, version 2.5
recovered.02041: Bourne shell script text
recovered.02042: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), stripped
recovered.02043: Bourne-Again shell script text
recovered.02044: English text
recovered.02045: Bourne shell script text
recovered.02046: English text
recovered.02047: perl script text
recovered.02048: English text
recovered.02049: data
recovered.02050: ASCII text
recovered.02051: data
recovered.02052: ASCII text
recovered.02053: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), not stripped
recovered.02054: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), stripped
recovered.02058: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), stripped
recovered.02059: ASCII text
recovered.02060: ASCII text
recovered.02061: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), not stripped
recovered.08097: empty
recovered.08100: English text
recovered.16110: ASCII text
recovered.30188: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), stripped
recovered.30191: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), stripped
recovered.48284: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically 
linked (uses shared libs), stripped
recovered.56231: ASCII text

Further inspection of the contents of each of these files reveals that the gzip compressed data (inode 23) is a compressed archive containing the rootkit. Many of the other deleted files appear to be the contents of this rootkit.

2. What files make up the deleted rootkit?

An analysis of the files in the rootkit archive follows.

Filename Description
cleaner a shell script which removes all lines matching a pattern from logfiles in /var/log
ifconfig replacement utility; presumably backdoored
inetd.conf replacement inetd configuration file; enables only a telnet and pop3 daemon
install a shell script used to install the rootkit; it replaces system utilities with backdoored versions, installs an ssh server and client in a hidden directory, starts linsniffer, and sends mail containing information about the infected system to two email addresses
last.cgi a cgi executable which allows remote command execution via a web server
linsniffer a network sniffer designed to capture usernames and passwords for common network services, such as ftp, telnet, pop3, and rlogin
logclear shell commands which kills linsniffer, deletes its output file, and restarts it
lsattr a shell script which starts mkxfs (see below) and linsniffer
mkxfs an sshd daemon; presumably backdoored
netstat replacement utility; backdoored to hide network connections based on a configuration in /dev/caca
ps, top backdoored versions of these system utilities which hide processes based on a configuration in /dev/dsx
sense a perl script used to process linsniffer output
services replacement services definition file; does not appear to differ materially from a default services file
sl2 a packet generator which appears to be useful in creating syn floods
ssh an ssh client; possibly backdoored
s, sshd_config, ssh_*, pidfile config files for ssh and sshd (mkxfs)

It is interesting to note that while the rootkit's install script attempts to create configuration files for its backdoored copies of netstat, ps, and top, it appears to name these files incorrectly.

Bonus Question: Was the rootkit ever actually installed on the system? How do you know?

I believe the rootkit was installed on the system. This conclusion was made by checking the system for changes made by the install script. These changes include modified versions of the system files ifconfig, netstat, ps, inetd.conf, and services, as well as the creation of files and directories under /dev.