Honeynet - Scan 15
Michael Bishop - mbishop@bytealliance.com

Question 1:
Steps to recover the files:

After downloading the image file, verify the contents, unzip and extract it.

$ cat honeynet.tar.gz | openssl md5
0dff8fb9fe022ea80d8f1a4e4ae33e21

$ gunzip honeynet.tar.gz
$ tar -xvf honeynet.tar
honeynet/
honeynet/honeypot.hda8.dd
honeynet/README

$ cd honeynet
$ cat honeypot.hda8.dd | openssl md5
5a8ebf5725b15e563c825be85f2f852e

Ok, so we've got the file and all of the checksums are good.
Next, mount the file system and verify everything is good. We mount it read-only so we don't disturb anything.

$ mkdir image
$ mount -o ro,loop honeypot.hda8.dd image
$ cd image
$ ls
bin   dev  floppy  lib         mnt  proc  sbin  usr
boot  etc  home    lost+found  opt  root  tmp   var
Ok, so we now proceed to recover the files which were deleted. To do this, we need to unmount the file system and we're going to be looking at the inodes directly using debugfs.

$ cd ..
$ umount image

Next, we want to recover the deleted files. Since we know the time of the attack, this is easily done using debugfs.
Note: Original thoughts on how to recover deleted files from the mini-howto.
Unless it's been updated, I believe it is terribly outdated now.

Ok, so now we generate a list of all the inodes which have been deleted. Using debugfs with the lsdel command. Since it's about 30 files, we go ahead and pipe this to a file called deleted.txt so we can print it.

$ echo lsdel | debugfs honeypot.hda8.dd > deleted.txt

The output is something similar to:

debugfs:  29 deleted inodes found.
 Inode  Owner  Mode    Size    Blocks    Time deleted
 56231      0 100644  33135   13/  13 Thu Mar 15 05:17:36 2001
 16110      0 100644    239    1/   1 Thu Mar 15 05:20:25 2001
  2058      0 100755  53588   54/  54 Thu Mar 15 19:45:02 2001
 30188      0 100755  66736   67/  67 Thu Mar 15 19:45:02 2001
 30191      0 100555  60080   60/  60 Thu Mar 15 19:45:02 2001
 48284      0 100755  42736   43/  43 Thu Mar 15 19:45:02 2001
  2047      0 100755   4060    4/   4 Thu Mar 15 19:45:03 2001
  2049      0 100600    540    1/   1 Thu Mar 15 19:45:03 2001
  2051      0 100600    512    1/   1 Thu Mar 15 19:45:03 2001
  2053      0 100700   8268    9/   9 Thu Mar 15 19:45:03 2001
  2059      0 100700     75    1/   1 Thu Mar 15 19:45:03 2001
  2060      0 100644    708    1/   1 Thu Mar 15 19:45:03 2001
  2061      0 100755 632066  622/ 622 Thu Mar 15 19:45:03 2001
    23      0 100644 520333  512/ 512 Thu Mar 15 19:45:05 2001
  2039      0 100755 611931  602/ 602 Thu Mar 15 19:45:05 2001
  2040      0 100644      1    1/   1 Thu Mar 15 19:45:05 2001
  2041      0 100700   3713    4/   4 Thu Mar 15 19:45:05 2001
  2042      0 100644    796    1/   1 Thu Mar 15 19:45:05 2001
  2043      0 100755   1345    2/   2 Thu Mar 15 19:45:05 2001
  2044      0 100644   3278    4/   4 Thu Mar 15 19:45:05 2001
  2045      0 100755     79    1/   1 Thu Mar 15 19:45:05 2001
  2046      0 100644  11407   12/  12 Thu Mar 15 19:45:05 2001
  2048      0 100644    880    1/   1 Thu Mar 15 19:45:05 2001
  2050      0 100644    344    1/   1 Thu Mar 15 19:45:05 2001
  2052      0 100644    688    1/   1 Thu Mar 15 19:45:05 2001
  2054      0 100755   4620    5/   5 Thu Mar 15 19:45:05 2001
  2038   1031  40755      0    1/   1 Thu Mar 15 19:46:09 2001
  8097      0  40700      0    1/   1 Fri Mar 16 04:03:12 2001
  8100      0 100644  16329  177/ 177 Fri Mar 16 04:03:12 2001
Now, we run debugfs again with the -w parameter so we can make some changes. We are going to recover the files by modifying the inodes and setting their deleted time field to 0 and their link count to 1.
Luckily, this is not from an ancient kernel which cleared the indirect inode lists when a file is deleted.
To modify the inode, we use the mi command within debugfs.
$ debugfs -w honeypot.hda8.dd
debugfs 1.18, 11-Nov-1999 for EXT2 FS 0.5b, 95/08/09
debugfs:  mi <56231>
                          Mode    [0100644]
                       User ID    [0]
                      Group ID    [0]
                          Size    [33135]
                 Creation time    [984655056]
             Modification time    [984655056]
                   Access time    [984655056]
                 Deletion time    [984655056] 0      <--+- note these two lines
                    Link count    [0] 1              <--|
                   Block count    [68]
                    File flags    [0x0]
                      File acl    [0]
           High 32bits of size    [0]
              Fragment address    [0]
               Fragment number    [0]
                 Fragment size    [0]
               Direct Block #0    [229685]
               Direct Block #1    [229686]
               Direct Block #2    [229687]
               Direct Block #3    [229688]
               Direct Block #4    [229689]
               Direct Block #5    [229690]
               Direct Block #6    [229691]
               Direct Block #7    [229692]
               Direct Block #8    [229693]
               Direct Block #9    [229694]
              Direct Block #10    [229695]
              Direct Block #11    [229696]
                Indirect Block    [229697]
         Double Indirect Block    [0]
         Triple Indirect Block    [0]
We issue the mi command for each inode in our list of deletions.

After we've recovered all of the files, we run e2fsck to clean up the filesystem. Our recovered files won't be assigned to a directory, so this will place all of the files into the lost+found directory. e2fsck will ask a lot of questions, so simply answer 'y' to them all.

$ e2fsck -f honeypot.hda8.dd
Now, mount the file system again and look in the lost+found directory.
This time, we don't mount it read-only because we need to make some changes.
$ mount -o loop honeypot.hda8.dd image
$ cd image/lost+found
$ ls
#16110  #2040  #2043  #2046  #2049  #2052  #2058  #2061   #30191  #8097
#2038   #2041  #2044  #2047  #2050  #2053  #2059  #23     #48284
#2039   #2042  #2045  #2048  #2051  #2054  #2060  #30188  #56231
As a hunch, we search for shell scripts. An easy way to do this is:
$ head -1 * | strings
==> #2041 <==
#!/bin/sh
==> #2043 <==
#!/bin/bash
==> #2045 <==
#!/bin/sh
==> #2047 <==
#!/usr/bin/perl
This command generated a lot of extra output, but the remaining lines were clipped because these are the ones we are looking for. We start browsing around these files, and when we get to #2041 and it's clear that this is an installation script.

Ok, now we search for the archive file. The "file" command comes in useful:

$ file *
#23:      gzip compressed data, deflated, last modified: Fri Mar  2 21:09:06 2001, os: Unix
Again, I clipped the output of the remaining files because this is the one we were looking for. We open it up so we can examine what it contains:
$ tar -xvzf "#23"
last/
tar: Archive contains future timestamp 2002-02-08 07:08:13
last/ssh
last/pidfile
last/install
last/linsniffer
last/cleaner
last/inetd.conf
last/lsattr
last/services
last/sense
last/ssh_config
last/ssh_host_key
last/ssh_host_key.pub
last/ssh_random_seed
last/sshd_config
last/sl2
last/last.cgi
last/ps
last/netstat
last/ifconfig
last/top
last/logclear
last/s
last/mkxfs

$ cd last
$ ls
cleaner     last.cgi    mkxfs    s         ssh               ssh_random_seed
ifconfig    linsniffer  netstat  sense     ssh_config        sshd_config
inetd.conf  logclear    pidfile  services  ssh_host_key      top
install     lsattr      ps       sl2       ssh_host_key.pub
>From here we are able to see all the files that make up the kit, plus we have the install script which shows what files went where.
Question 2:
What files make up the deleted rootkit?

This question was either answered in the end of the last question, or you want to know what files on the system were affected. If you want to know what files were replaced, then the list (pulled from "install") is:


Bonus Question:
Was the rootkit installed? How can you tell?

Yes, it was installed.
The easiest way to tell after looking at the instal script is the presence of the "/dev/ida/.drag-on" and "/dev/ida/.. " directories.
Additionally, if we run diff on some of the files in the rootkit and the files installed on the system they return no differences.

Time spent:

The time was split into 2 sessions, but total time was approximately 4 hours.