`T
Honeynet Project, Scan 16 (June 2001)

<'Scan 16 (June 2001)'http://project.honeynet.org/scans/scan16/>, 
by Matthew Rench &lt;<'hn7421@pelennor.net'mailto:hn7421@pelennor.net>&gt;

`P
1. Identify the encryption algorithm used to encrypt the file.

A simple algorithm was used to "encrypt" this file in which each byte was
bitwise-inverted. This is equivalent to taking the one's complement of each
byte, or exclusive-or'ing each byte by 0xFF. The same procedure is used both
to encrypt and decrypt.

`P
2. How did you determine the encryption method?

I first observed the size of the encrypted file. At only 532 bytes, it
seemed unlikely that it could be a binary executable. My initial guess was
that this file was likely to be an ASCII text file: either a shell script,
a configuration file, or some other text data file.

The file also contained
no magic number recognizable by the unix "file" command on my system. This
ruled out at least pgp and gnupg encryption.

I proceeded to create a histogram of the character use in the unknown file. To
do this, I used the following C code to generate a count of each character:

`C
int i, count[256];                                                              

for (i=0; i<256; i++)
  count[i]=0;

while ((i=fgetc(stdin)) != EOF)
  count[i]++;

for (i=0; i<256; i++)
  printf("%03d %d\n", i, count[i]);
`c

and plotted the resulting data file using gnuplot:

`C
aragorn: gnuplot << END                                                         
set data style impulse
set terminal postscript
set output 'histo.ps'
plot [0:255] 'somefile.histo'
END
`c

resulting in this plot:

<P><IMG SRC="histo.jpg">

This histogram has several interesting features. First, I noticed that only
a small portion of the available character space is used in this file. This
rules out any cipher such as rc4 or DES which attempts to make its output look
random (as can be seen by <A HREF="rc4histo.jpg">comparing</A> with a histogram
of this file encrypted with the rc4 cipher).

Next, I noticed the strong bimodal characteristic of the histogram. That is,
most of the character usage was concentrated in two distinct groups. If the
file was indeed ASCII text, this could be the separation of lower case and
upper case alphabet characters.

Finally, I noticed that all characters present in this file were in the upper
half of the character range (128-255). Assuming that the original file was made
up of ASCII text in the lower half of the character range (0-127), this would
indicate an encryption algorithm which included setting or toggling the high
bit of each character.

`P
3. Decrypt the file, be sure to explain how you decrypted the file.

To test out this assumption, I first tried inverting the high bit of each
character in the encrypted file using the following C code:

`C
int c;                                                                          

while ((c=fgetc(stdin)) != EOF) {
  c=c^0x80;
  fputc(c, stdout);
}
`c

As this produced no immediately useful result, I tried again, inverting the
entire byte by substituting 0xFF for 0x80 in the code above.

And voila. This produced the following output:

`C
[file]
find=/dev/pts/01/bin/find
du=/dev/pts/01/bin/du
ls=/dev/pts/01/bin/ls
file_filters=01,lblibps.so,sn.l,prom,cleaner,dos,uconf.inv,psbnc,lpacct,USER

[ps]
ps=/dev/pts/01/bin/psr
ps_filters=lpq,lpsched,sh1t,psr,sshd2,lpset,lpacct,bnclp,lpsys
lsof_filters=lp,uconf.inv,psniff,psr,:13000,:25000,:6668,:6667,/dev/pts/01,sn.l,
prom,lsof,psbnc

[netstat]
netstat=/dev/pts/01/bin/netstat
net_filters=47018,6668

[login]
su_loc=/dev/pts/01/bin/su
ping=/dev/pts/01/bin/ping
passwd=/dev/pts/01/bin/passwd
shell=/bin/sh

su_pass=l33th4x0r
`c

`P
4. Once decrypted, explain the purpose/function of the file and why it was 
encrypted.

This file appears to be a configuration file for one or several backdoored
programs, possibly installed as part of a rootkit. Several of the parameters
specify the locations of system utilities (possibly an archive of the original
utilities). Others contain patterns which specify what processes, files, and
network connections should be hidden from the user by the backdoored utilities.
The last line seems to set the password by which the intruder might gain access
to the system again in the future.

This file is most likely encrypted to prevent its purpose from being discovered
by users or administrators of the infected system.

`P
5. What lesson did you learn from this challenge?

This challenge gave me an opportunity to analyze an unknown, encrypted file
to discover its encryption method and contents.

A couple lessons regarding encrypted files come to mind. First, it is
trivially easy to obscure the contents of a file using a method such as the
one used here. This means that it is not always good enough to use simple
visual inspection when doing forensics work. Second, it is also relatively
simple to identify and break an encryption method such as the one used here,
even with little or no background information. We are reminded that security
through obscurity, although it may look "good enough", offers very little
protection.

`P
6. How long did this challenge take you?

I have spent approximately one and a half hours on this project, which
includes 30 minutes spent identifying and removing the file encryption, and
one hour writing this report.

`P
Bonus Question: This encryption method and file are part of a security toolkit.
Can you identify this toolkit?

No. (Technically, this is a correct answer. :)