1. Identify and explain the purpose of the binary.

The binary has two primary purposes. The first is to provide backdoor access to the intruder in the future. The second purpose is to launch denial of service attacks against hosts on the internet from the compromised host. I was unable to identify the binary as any publically known and available tool.

2. Identify and explain the different features of the binary. What are it's capabilities.

The binary is a backdoor server that receives messages in packets with an IP protocol field value of 11. There are 12 types of messages that can be sent to the server with each message triggering a different function on the server.

The functionality of the agent can be divided into two groups. Commands for administration:

  1. Retrieve status information about which command is running on the server.
  2. Send information to the server about the IP address of the client.
  3. Run a shell command and ignore any output.
  4. Run a shell command and send output back to the client.
  5. Run a password protected shell on a port.
Commands for attacking hosts:
  1. Flood a host with either UDP or ICMP packet fragments.
  2. Flood a host with TCP SYN packets.
  3. Flood a host with DNS requests.
  4. Flood a host by spoofing DNS requests to various DNS servers on the internet. The packets are spoofed so that the target address appears as the source of the requests.

3. The binary uses a network data encoding process. Identify the encoding process and develop a decoder for it.

The information transferred between the client and the server is encoded using a simple algorithm. The first byte of encoded data is added, modulo 256, to the constant value 23. Each of the remaining bytes is encoded by adding their value to the encoded value of the previous byte and the constant 23. Once again the addition is performed modulo 256. 

  byte_0 = (byte_0 + 23) mod 256
  byte_N = (byte_(N-1) + 23) mod 256
Here are routines in C that could be used to encode and decode data. 

encode(char *data, int len)
{
  int i;

  data[0] += 23; 
  for(i = 0; i != len; i++)
    data[i] += (data[i - 1] + 23)
}

decode(char *data, int len)
{
  int i;

  /* It's a bit easier but not necessary to decode in reverse */
  for(i = len - 1; i; i--)
    data[i] -= (data[i - 1] + 23);   

  data[0] -= 23;
}

4. Identify one method of detecting this network traffic using a method that is not just specific to this situation, but other ones as well.

Since messages to the binary are transmitted over an unusual IP protocol value, the messages between the client and the server can be detected by watching for traffic on this IP protocol (value 11). As far as I know this protocol (network voice protocol) isn't widely used for anything else.

5. Identify and explain any techniques in the binary that protect it from being analyzed or reverse engineered.

There were many obstacles that made analysis of the binary more difficult. These included the fact that the binary was statically linked and stripped and compiled with optimizations. There's no way to know if the binary was built this way to deliberately thwart analysis, or if it was just a lucky side effect. The binary did however contain at least two intentional features to prevent easily reverse engineering the program. The first is that, as already discussed, the binary encoded messages between the client and the server. Another technique is the lack of character strings in the binary. It seems as though the author attempted to write the program using as few string constants as possible in order to prevent easily being able to discover the purpose of the binary with programs such as 'strings'.

6. Identify two tools in the past that have demonstrated similar functionality.

This binary resembles many different DDoS agents that have been discovered and analyzed. Two of the earliest agents of this type that were discovered were Trin00 and the Tribe Flood Network(TFN). A more comprehensive list of agents of this type can be found here.