Analyzis
of an encrypted file found on a compromised system
Preliminary report - Wednesday, June the 20th
2001
Nicolas Dubée (ndubee@secway.com)
Grégoire Sirou (gsirou@secway.com)
Summary
Remarks about this document
Modifications
between this version and the previous one are highlighted in yellow: modification.
This
document is © Secway June 2001. Authors: Nicolas Dubée (ndubee@secway.com),
Grégoire Sirou (gsirou@secway.com). We would like to thank linouss for his help
and talent.
Any
question or remark about this document should be addressed to any of the
authors given above.
Introduction
This document, labeled Preliminary
Report, summarizes the findings of Nicolas Dubée and Grégoire Sirou of Secway
on the analysis of an encrypted file found on a Solaris system compromised in
March 2001. Technical details were given by the customer to Secway in early
June 2001. The actual analysis and preliminary report were done on Wednesday,
June the 20th, and delivered to the customer on Friday, June the
22nd.
Objectives
Objectives
of this analysis are:
-
The
identification of the encryption algorithm used to encrypt the file.
-
The
decryption of the file.
-
An
analysis of the decrypted content.
Each step
is to be documented with descriptions of our methodologies and tools.
Results
As
previously stated, the file to analyze was found on a compromised Solaris system.
This file[1]
is supposed to have been introduced on the system by the person who performed
the intrusion. A quick glance at the file’s content shows that it is apparently
encrypted. No other indication was given about the compromised system.
Facts about
the file:
-
File
name: somefile
-
No
indication on the actual location (full path) of the file
-
Size:
532 bytes
-
MD5
checksum: 643be63febcba6f2a9d24a370a861e00
-
The
timestamp of the file (2001-06-04) does not appear to be valid[2],
and was probably broken during the customer’s attempt to decrypt the file
-
The
owner and group attributes of the file (honeynet/webacct) were also broken
A screen
output of the file shows that it consists mostly of non-printable characters.
Given its hexadecimal view[3],
odds are it has to be considered as a pure binary character set, with no
special character encoding. Odds are also that the plaintext corresponding has
to be considered as an ascii content.
Inspection
reveals many repeating characters and patterns, such as 0x99,0x96. We can also
prove this by the fact that the file compresses well: for example, gzip’s LZ77
compression cuts down the file to half of its original size.
Finally,
Linux’s file utility giving us the output “data”; no file signature or header seems
to be present.
We can
deduce from these facts that the file has been encrypted by a weak encryption
algorithm (maybe only an obfuscation one), and that it hasn’t probably been
generated by a file encryption utility but by an embedded function in a bigger
tool.
We perform,
using our test program given in Annex 1, a frequency analysis of the file. The
goal of this analysis is to see any peak in the characters (one character being
here one octet) distribution, by counting occurrences of each character. Given
those numbers of occurrences, we can then draw a histogram representing each
character’s frequency in the file. From this histogram, strong facts on the
algorithm used can be deduced.
Our test
program gives the following characters occurrences:
tab[0x0] = 0
tab[0x1] = 0 tab[0x2] = 0 tab[0x3] = 0 tab[0x4]
= 0
tab[0x5] = 0 tab[0x6]
= 0 tab[0x7] = 0 tab[0x8] = 0
tab[0x9] = 0 tab[0xa] = 0 tab[0xb] = 0 tab[0xc]
= 0
tab[0xd] = 0 tab[0xe]
= 0 tab[0xf] = 0 tab[0x10] = 0
tab[0x11] = 0 tab[0x12] = 0 tab[0x13] = 0 tab[0x14]
= 0
tab[0x15] = 0 tab[0x16] = 0 tab[0x17] = 0 tab[0x18]
= 0
tab[0x19] = 0 tab[0x1a] = 0 tab[0x1b] = 0 tab[0x1c]
= 0
tab[0x1d] = 0 tab[0x1e] = 0 tab[0x1f] = 0 tab['
'] = 0
tab['!'] = 0 tab['"'] = 0 tab['#'] = 0 tab['$'] = 0
tab['%'] = 0 tab['&'] = 0 tab['''] = 0 tab['(']
= 0
tab[')'] = 0 tab['*'] = 0 tab['+'] = 0 tab[',']
= 0
tab['-'] = 0 tab['.'] = 0 tab['/'] = 0 tab['0']
= 0
tab['1'] = 0 tab['2'] = 0 tab['3'] = 0 tab['4']
= 0
tab['5'] = 0 tab['6'] = 0 tab['7'] = 0 tab['8']
= 0
tab['9'] = 0 tab[':'] = 0 tab[';'] = 0 tab['<']
= 0
tab['='] = 0 tab['>'] = 0 tab['?'] = 0 tab['@']
= 0
tab['A'] = 0 tab['B'] = 0 tab['C'] = 0 tab['D']
= 0
tab['E'] = 0 tab['F'] = 0 tab['G'] = 0 tab['H']
= 0
tab['I'] = 0 tab['J'] = 0 tab['K'] = 0 tab['L']
= 0
tab['M'] = 0 tab['N'] = 0 tab['O'] = 0 tab['P']
= 0
tab['Q'] = 0 tab['R'] = 0 tab['S'] = 0 tab['T']
= 0
tab['U'] = 0 tab['V'] = 0 tab['W'] = 0 tab['X']
= 0
tab['Y'] = 0 tab['Z'] = 0 tab['['] = 0 tab['\']
= 0
tab[']'] = 0 tab['^'] = 0 tab['_'] = 0 tab['`']
= 0
tab['a'] = 0 tab['b'] = 0 tab['c'] = 0 tab['d']
= 0
tab['e'] = 0 tab['f'] = 0 tab['g'] = 0 tab['h']
= 0
tab['i'] = 0 tab['j'] = 0 tab['k'] = 0 tab['l']
= 0
tab['m'] = 0 tab['n'] = 0 tab['o'] = 0 tab['p']
= 0
tab['q'] = 0 tab['r'] = 0 tab['s'] = 0 tab['t']
= 0
tab['u'] = 0 tab['v'] = 0 tab['w'] = 0 tab['x']
= 0
tab['y'] = 0 tab['z'] = 0 tab['{'] = 0 tab['|']
= 0
tab['}'] = 0 tab['~'] = 0 tab[0x7f] = 0 tab[0x80] = 0
tab[0x81] = 0 tab[0x82] = 0 tab[0x83] = 0 tab[0x84]
= 0
tab[0x85] = 0 tab[0x86] = 1 tab[0x87] = 1 tab[0x88]
= 2
tab[0x89] = 11 tab[0x8a] = 7 tab[0x8b]
= 28 tab[0x8c]
= 52
tab[0x8d] = 11 tab[0x8e]
= 1 tab[0x8f]
= 34 tab[0x90]
= 10
tab[0x91] = 29 tab[0x92]
= 2 tab[0x93]
= 28 tab[0x94]
= 0
tab[0x95] = 0 tab[0x96]
= 24 tab[0x97]
= 6 tab[0x98]
= 3
tab[0x99] = 14 tab[0x9a]
= 24 tab[0x9b]
= 18 tab[0x9c]
= 12
tab[0x9d] = 14 tab[0x9e]
= 9 tab[0x9f]
= 0 tab[0xa0]
= 6
tab[0xa1] = 0 tab[0xa2] = 4 tab[0xa3]
= 0 tab[0xa4]
= 4
tab[0xa5] = 0 tab[0xa6] = 0 tab[0xa7]
= 0 tab[0xa8]
= 0
tab[0xa9] = 0 tab[0xaa] = 1 tab[0xab]
= 0 tab[0xac]
= 1
tab[0xad] = 1 tab[0xae] = 0 tab[0xaf]
= 0 tab[0xb0] = 0
tab[0xb1] = 0 tab[0xb2]
= 0 tab[0xb3]
= 0 tab[0xb4]
= 0
tab[0xb5] = 0 tab[0xb6] = 0 tab[0xb7]
= 0 tab[0xb8]
= 0
tab[0xb9] = 0 tab[0xba] = 1 tab[0xbb] = 0 tab[0xbc] = 0
tab[0xbd] = 0 tab[0xbe]
= 0 tab[0xbf]
= 0 tab[0xc0]
= 0
tab[0xc1] = 0 tab[0xc2]
= 14 tab[0xc3]
= 0 tab[0xc4]
= 0
tab[0xc5] = 4 tab[0xc6]
= 0 tab[0xc7]
= 3 tab[0xc8]
= 2
tab[0xc9] = 9 tab[0xca]
= 1 tab[0xcb]
= 2 tab[0xcc]
= 3
tab[0xcd] = 2 tab[0xce]
= 13 tab[0xcf]
= 18 tab[0xd0]
= 45
tab[0xd1] = 5 tab[0xd2]
= 0 tab[0xd3]
= 30 tab[0xd4]
= 0
tab[0xd5] = 0 tab[0xd6]
= 0 tab[0xd7]
= 0 tab[0xd8]
= 0
tab[0xd9] = 0 tab[0xda] = 0 tab[0xdb] = 0 tab[0xdc]
= 0
tab[0xdd] = 0 tab[0xde]
= 0 tab[0xdf]
= 0 tab[0xe0]
= 0
tab[0xe1] = 0 tab[0xe2] = 0 tab[0xe3]
= 0 tab[0xe4]
= 0
tab[0xe5] = 0 tab[0xe6] = 0 tab[0xe7]
= 0 tab[0xe8]
= 0
tab[0xe9] = 0 tab[0xea] = 0 tab[0xeb]
= 0 tab[0xec]
= 0
tab[0xed] = 0 tab[0xee] = 0 tab[0xef]
= 0 tab[0xf0] = 0
tab[0xf1] = 0 tab[0xf2] = 0 tab[0xf3]
= 0 tab[0xf4]
= 0
tab[0xf5] = 22 tab[0xf6] = 0 tab[0xf7] = 0 tab[0xf8] = 0
tab[0xf9] = 0 tab[0xfa] = 0 tab[0xfb]
= 0 tab[0xfc]
= 0
tab[0xfd] = 0 tab[0xfe]
= 0 tab[0xff]
= 0
We draw the
following frequencies histogram from the data above. The x-axis represents the
characters (if non-printable, their ASCII value), the y-axis the number of
occurrences of the corresponding character in the file.
Figure 1: Frequencies histogram
We quickly
see that the characters are only signed bytes. If we consider the plaintext as
a pure ASCII text (i.e.: alphanumeric and punctuation characters), we can
suppose that this cipher is actually a very simple one, most probably a XOR
cipher with a fixed and short key (1 byte). The fact that all bytes in the file
are above the sign limit (128) makes us believe that all bytes in the key have
high bits set.
We write a
C program[4] that performs a XOR operation on
all bytes of the file with a fixed, 1 byte key, and prints out the result.
We first
try this program with the (char )0xff key, which is actually the equivalent of
a NEG operation on all bytes. We try this key since the frequencies analysis
above showed us that all bytes in the file were signed, probably because of a
sign inversion.
This
program gives us the following output :
[file]
find=/dev/pts/01/bin/find
du=/dev/pts/01/bin/du
ls=/dev/pts/01/bin/ls
file_filters=01,lblibps.so,sn.l,prom,cleaner,dos,uconf.inv,psbnc,lpacct,USER
[ps]
ps=/dev/pts/01/bin/psr
ps_filters=lpq,lpsched,sh1t,psr,sshd2,lpset,lpacct,bnclp,lpsys
lsof_filters=lp,uconf.inv,psniff,psr,:13000,:25000,:6668,:6667,/dev/pts/01,sn.l,prom,lsof,psbnc
[netstat]
netstat=/dev/pts/01/bin/netstat
net_filters=47018,6668
[login]
su_loc=/dev/pts/01/bin/su
ping=/dev/pts/01/bin/ping
passwd=/dev/pts/01/bin/passwd
shell=/bin/sh
su_pass=l33th4x0r
Considering
the nature of the given file (found on a compromised system), this output looks
like the proper one.
We deduce
that the cipher used was a NEG inversion (equivalent to XOR 255).
This file
has a typical configuration file structure. In brackets ([ and ]) are section
names, in sections are variables-values pairs.
The user
(i.e. the intruder) customizes the package by changing the values fields.
Given the
self-explanatory sections and variables names, we can easily guess that this
file is actually a rootkit configuration file. Each section gives the
configuration options of a part of the rootkit. For example, the [netstat]
section contains configuration options of the rootkit’s own netstat
replacement. In this case, the net_filters variable looks like a list of open
or connected ports that will not be printed by the netstat replacement.
We find
that the urk rootkit, written by K2, has the same encryption feature, and has
the same configuration file structure. Tested version was 0.9.8, as found on www.ktwo.ca.
The sample
configuration file found in the URK distribution[5]
is:
[file]
find=/usr/man/man1/xxxxxxbin/find
du=/usr/man/man1/xxxxxxbin/du
ls=/usr/local/bin/ls.gnu
file_filters=xxxxxx,yyyyyy,aaaaaa,mmmmmmmmm
[ps]
ps=/usr/man/man1/xxxxxxbin/ps
ps_filters=nedit,bash
[netstat]
netstat=/usr/man/man1/xxxxxxbin/netstat
net_filters=innu.org
[login]
su_pass=h4x0r
su_loc=/usr/man/man1/xxxxxxbin/su
ping=/usr/man/man1/xxxxxxbin/ping
passwd=/usr/man/man1/xxxxxxbin/passwd
shell=/usr/man/man1/xxxxxxbin/bash
Also,
quoting urk-dist-0.9.8/inv.c, we see:
#include <stdio.h>
#include <stdlib.h>
int main (int argc, char **argv)
{
int c;
FILE
*file1,*file2;
/* simple
error checking */
if(argc
<= 1) {
printf("Inverses the bit's in a file to make it
unreadable.\n");
printf("inv [file1] [file2]\n");
return
-1;
}
/* read and
write a file in binary mode, if error then exit */
if (( file1
= fopen(argv[1],"rb")) == NULL ) {
fprintf(stderr, "Cannot open input file: \"%s\".\n",
argv[1]);
return
-2;
}
file2=fopen(argv[2],"wb");
/* while
there is still input */
while((c = getc(file1))
!= EOF) {
c=~c;
fputc(c,file2);
}
printf("File processed...\n");
/* close */
fclose(file1);
fclose(file2);
return 0;
}
This
utility, part of the urk rootkit, inverses all bytes in a file given in
argument.
Quoting
url-dist-0.9.8/file.c:
#ifdef HIDE
tmpnam(atmpfile_name);
atmpfile=fopen(atmpfile_name,"wb");
if(atmpfile
&& somefile) {
while((c
= getc(somefile)) != EOF) {
c=~c;
fputc(c,atmpfile);
}
}
fclose(somefile);
fclose(atmpfile);
somefile =
fopen(atmpfile_name,"r+");
#endif
If the
preprocessor’s HIDE variable is defined, the code snippet above is compiled.
This code is part of a routine that gives the value of a variable in a section
and file given in arguments to the function.
We can thus
conclude that this file was probably part of URK or a derivate rootkit.
However, inspecting features of the URK
rootkit, we don’t see any lsof Trojan. Moreover, the decrypted configuration
file shows us that the rootkit puts files in /dev/pts/01, which is the very
usual place for the ‘01’ rootkit.
This ‘01’
rootkit may actually be the SunOS Rootkit v2.5,
by Tragedy/Dor, and derived from URK by K2. This ‘01’ rootkit has been widely
used in conjunction with the snmpXdmid exploit. CERT reported use of automated
tools exploiting snmpXdmid vulnerable machines and installing this ‘01’ rootkit[6].
Conclusion
Analysis of
a 532-bytes file has made possible a quite precise and probable description of
the attack. This “Scan of the month” showed us that it is possible to get many
hints from a single point of start.
Time spent
on the project was about: 45 minutes for the analysis of the file (2 persons,
including coding) and 45 minutes to write this report (1 person, including
translation).
Appendix
#include
<stdio.h>
#include
<unistd.h>
void doit(char *file)
{
FILE *in;
unsigned int c, tab[256];
if((in=fopen(file, "r")) == NULL)
{
perror("fopen");
exit(2);
}
memset(tab, 0, sizeof(tab));
while((c = fgetc(in)) != EOF)
tab[c] ++;
#ifndef NORMAL
for(c = 0; c < 256; c++)
if(isprint(c))
printf("tab['%c'] = %d %c", c,
tab[c], c%4 ? '\t':'\n');
else
printf("tab[0x%x] = %d %c", c,
tab[c], c%4 ? '\t':'\n');
#else
for(c = 0; c < 256; c++)
if(isprint(c))
printf("%c\t%d\n",
c, tab[c]);
else
printf("%d\t%d\n", c, tab[c]);
#endif
printf("\n");
fclose(in);
}
int main(int argc,
char **argv)
{
if(argc != 2) {
fprintf(stderr,"Usage: %s
<file>\n", argv[0]);
fprintf(stderr," where <file> is the file to
parse.\n\n");
exit(1);
}
doit(argv[1]);
return 0;
}
#include
<stdio.h>
#include
<unistd.h>
void doit(char *file)
{
FILE *in;
unsigned int c;
if((in=fopen(file, "r")) == NULL)
{
perror("fopen");
exit(2);
}
while((c = fgetc(in)) != EOF)
printf("%c",~(char)c);
fclose(in);
}
int main(int argc,
char **argv)
{
if(argc != 2) {
fprintf(stderr,"Usage: %s
<file>\n", argv[0]);
fprintf(stderr," where <file> is the file to
parse.\n\n");
exit(1);
}
doit(argv[1]);
return 0;
}