Scan 16: 1. Identify the encryption algorithm used and decrypt The first task in any crypto analysis is a byte count. If the file is crackable the algorithm is usually very simple. Often each byte is XORed with a constant value and a byte count will show up any patterns which could give the algorithm away. The following Python code was used: import sys bytes = {} data = sys.stdin.read () for x in range (0, 256): bytes[x] = 0 for x in data: bytes[ord(x)] += 1 for x in range (0, 256): print str(x) + " " + str(bytes[x]) The script was run as: % cat somefile | python2 count.py > data % gnuplot gnuplot> plot 'data' with lines The ASCII-art version of the graph is: 60 ++----------+-----------+-----------+-----------+-----------+----------++ + + + + + 'data' ****** + | | 50 ++ * ++ | * | | * * | | * * | 40 ++ * * ++ | ** * | | ** * | 30 ++ *** ** ++ | *** ** | | ***** ** | 20 ++ ***** ** * ++ | ***** ** * | | ****** * ** * | | ****** * *** * | 10 ++ ****** ***** * ++ | **** * ****** * | + + + **** ** ****** *+ + 0 *********************************-*-*-***********-************---------++ 0 50 100 150 200 250 300 The pattern given shows a number of things. Firstly - it is a weak cipher. If the cipher were strong (e.g. DES, TwoFish and the like) the distribution of the bytes would be uniform. (A uniform distribution does not make a good cipher - but all good ciphers have them). Secondly, it's almost certainly not a Caeser cipher (which adds a constant value to the bytes) because there are definite areas in the output which do not look like the distribution of text. Thus it is likely to be a constant XOR based system. There are well known methods for breaking XOR ciphers (the interested reader is referred to [1]) but they were not needed in this case. Since the cipher looks so primitive, it was worth a shot at it being an XOR with a 1-byte key as they are so simple to code. The following Python code was used to decrypt the file with every key assuming a 1-byte constant XOR: import sys data = sys.stdin.read () for y in range (0, 256): print "**********************************************" print str(y) for x in data: sys.stdout.write (chr(ord(x) ^ y)) This generates a lot of output, but the human brain is very good at picking out text. However it turns out that the actual text comes out at the very bottom with a key of 255. This means that the cipher is even more primitive than was thought at first. XORing with 255 is the same as NOTing each byte. So the actual cipher is simply the NOT of each byte. The decrypted file is thus: [file] find=/dev/pts/01/bin/find du=/dev/pts/01/bin/du ls=/dev/pts/01/bin/ls file_filters=01,lblibps.so,sn.l,prom,cleaner,dos,uconf.inv,psbnc,lpacct,USER [ps] ps=/dev/pts/01/bin/psr ps_filters=lpq,lpsched,sh1t,psr,sshd2,lpset,lpacct,bnclp,lpsys lsof_filters=lp,uconf.inv,psniff,psr,:13000,:25000,:6668,:6667,/dev/pts/01,sn.l,prom,lsof,psbnc [netstat] netstat=/dev/pts/01/bin/netstat net_filters=47018,6668 [login] su_loc=/dev/pts/01/bin/su ping=/dev/pts/01/bin/ping passwd=/dev/pts/01/bin/passwd shell=/bin/sh su_pass=l33th4x0r 2. The purpose of the file The file is plainly a root kit file. A google search for some of the strings in the file turned up [2]. In [2] it becomes clear that this file is used to configure wrappers for a root kit The file is in sections, one for each wrapper and each section contains 1) The location(s) of the real binary(s) 2) Some filter information Taking, for example, one the filter lines: lsof_filters=lp,uconf.inv,psniff,psr,:13000,:25000,:6668,:6667,/dev/pts/01,sn.l,prom,lsof,psbnc This is configuring the wrapper for lsof not to show any open fds for a number of utilities and ports. One would suspect that the cracker would install back doors on some of these ports and use the lsof filter to hide them. Interestingly - the filters for netstat do not hide the same ports. One port is common to both (6668) and is a common IRC port. This suggests that the attacker wanted to hide a connection to an IRC server. Many DDoS systems use IRC to coordinate the actions of the zombies (as detailed in [3]) so a DDoS zombie could be expected on the system. 3. Lessons and Why It Was Encrypted Firstly - although the cipher is too poor to even be considered a cipher (technically it is a code, not a cipher, because it has no key) it is enough to hide the configuration from utilities like strings which is how the administrator in [2] found it. This makes it perfectly clear that strings is no replacement for md5sum and should never be relied on for finding hostile binarys. Hiding from strings is now common place and code like TFN uses strings too short ( < 3 letters) for `strings` to pickup by default, for the same effect. Playing cat and mouse and making strings try simple ciphers is not a solution as using much more powerful ciphers is very easy. 4. Time Taken About 10 minutes (not including writing it up) Bonus Question: > This encryption method and file are part of a security toolkit. Can you identify this toolkit? [2] suggests that it is part of "SunOS Rootkit v2.5 (C) 1997-2001 Tragedy/Dor". A search for this turns up 78795fed5abb0aaed98b41a62cafb393 rootkitSunOS.tgz Which does not seem to have anything to do with it. However the code seems to be related to the rootkit described in detail in [4]. A little more searching didn't turn up anything conclusive. Adam Langley agl@linuxpower.org Freenet Project References: [1] Applied Cryptography, Bruce Schneier, Section 1.5 [2] http://archives.neohapsis.com/archives/sf/sun/2001-q2/0088.html [3] Gibson, The strange tale of the DOS attacks against GRC.COM http://grc.com/dos/grcdos.htm [4] http://www.sans.org/y2k/the_compromise.htm