Project Honeynet, Scan of the Month Scan 15, May 2001 Jeremiah Shirk Contents: Tools used for analysis Question 1 Question 2 Bonus Question ============================================================================== Tools used for analysis ============================================================================== First of all, thanks to Dan Farmer and Wietse Venema for The Coroner's Toolkit, . These tools are extremely useful for this sort of analysis. Tools used: Linux utilities losetup (and loopback device) mount file tar strings diff cmp The Coroner's Toolkit utilities grave-robber unrm lazarus ils ils2mac mactime icat ============================================================================== Question 1. Show step by step how you identify and recover the deleted rootkit from the / partition. ============================================================================== The first step is to mount the image at /hn using the loopback device. # losetup /dev/loop0 honeypot.hda8.dd # mkdir /hn # mount -r /dev/loop0 /hn Next, run grave-robber to collect some information we'll use later. # grave-robber -c /hn -o LINUX2 -F -i -M -m -t -V But, since we know that we are looking for a deleted rootkit, what we really want is to recover information from currently unallocated blocks in the partition. # unrm /dev/loop0 > unrm.out Once the unallocated blocks are copied to unrm.out, we use lazarus to extract and analyze them. The -h flag generates a nice set of html pages for easy browsing. # lazarus -h -w laz_html -D laz_blocks unrm.out Running 'file' against the files extracted to the directory laz_blocks gives us some additional information. In particular, one thing that really stands out is a block of compressed data, 9.z.txt. # file laz_blocks/9.z.txt laz_blocks/9.z.txt: gzip compressed data, deflated, last modified: Fri Mar 2 21:09:06 2001, os: Unix Further investigation reveals that this is the beginning of a gzipped tar file. # tar tzf laz_blocks/9.z.txt last/ last/ssh tar: Skipping to next header gzip: stdin: unexpected end of file tar: 4 garbage bytes ignored at end of archive tar: Child returned status 1 tar: Error exit delayed from previous errors This definitely looks promising, and suggests that our rootkit may still exist in unallocated blocks on the partition, as a .tar.gz archive. So, we switch tactics a little bit and go straight for finding and resurrecting the archive. List the inodes of deleted files in the partition image and format them for mactime. # ils honeypot.hda8.dd | ils2mac > honeypot.hda8.dd.ilsbody Generate the list of mactimes, owners, etc. # mactime -p /hn/etc/passwd -g /hn/etc/group -b \ honeypot.hda8.dd.ilsbody 3/1/2001 > mactime.txt Examine mactime.txt, looking for sufficiently large files that were accessed on Mar. 15. This one looks promising: Mar 15 01 19:44:50 611931 .a. -rwxr-xr-x root root Extract and check inode 2039. # icat honeypot.hda8.dd 2039 > 2039; file 2039 2039: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), not stripped Hmm, that's not it. How about this one: Mar 15 01 19:36:48 520333 m.. -rw-r--r-- root root # icat honeypot.hda8.dd 23 > 23; file 23 23: gzip compressed data, deflated, last modified: Fri Mar 2 21:09:06 2001, os: Unix That could definitely be it. Using tar to list the contents of the archive shows us we've found a rootkit. # tar tzf 23 | more last/ last/ssh last/pidfile last/install last/linsniffer last/cleaner ... ============================================================================== Question 2. What files make up the deleted rootkit? ============================================================================== The following files are included in the rootkit: last/ssh - ssh client last/pidfile - used by mkxfs (sshd) last/install - installs the rootkit and emails the (presumed) author(s) last/linsniffer - captures the first part of net connections, i.e. to grab login and password last/cleaner - log file cleaner last/inetd.conf - install copies this to /etc/inetd.conf last/lsattr - despite the name, actually executes ssh daemon and linsniffer last/services - install copies this to /etc/services last/sense - processes the output of linsniffer last/ssh_config - ssh client config file last/ssh_host_key - ssh private key last/ssh_host_key.pub - ssh public key last/ssh_random_seed - random seed file for ssh daemon last/sshd_config - ssh daemon config file last/sl2 - looks like the syn-flooder synk4, based on the output of strings last/last.cgi' - CGI for remote command execution, possibly copied to host by install last/ps - appears to be ps from LRK 5, based on '/dev/dsx' appearing in the binary. Oddly, /dev/dsx is never created by the rootkit installer. last/netstat - strings reveals a reference to /dev/caca, suggesting a trojaned netstat. Appears to be very similar to the netstat packaged with LRK 5. last/ifconfig - md5sum matches that in the precompiled ifconfig packaged with LRK 5. This ifconfig is modified to hide the PROMISC flag when sniffing. last/top - appears to be ps from LRK 4/5, based on '/dev/dsx' appearing in the binary. Oddly, /dev/dsx is never created by the rootkit installer. last/logclear - clear linsniffer logfile, and restart linsniffer last/s - ssh daemon config file last/mkxfs - despite the name, this is actually an ssh daemon. ============================================================================== Bonus Question: Was the rootkit ever actually installed on the system? How do you know? ============================================================================== The rootkit was clearly installed on this system, based on the number of files on the system which have been created, replaced, or modified by the rootkit installer. Presumably, we would see others in other partitions, such as /usr. For example: Replaced /etc/inetd.conf /etc/services /sbin/ifconfig /bin/netstat /bin/ps Created /dev/rpm /dev/last /dev/ida/.drag-on/* Modified /etc/rc.d/rc.sysinit