Results of the Reverse Challenge submissions
Last Modified: 7 July, 2002, 23:45 CDT
Wow! We received a total of 35 submission for the Reverse Challenge. These submissions were outstanding, most contestants put an incredible amount of time and effort into the Challenge, we were extremelly impressed! This made juding very tough (and time consuming :). Unfortunately, not everyone that submitted had time to complete all of the documentation. Of the 35 entries, we identified 7 as incomplete and were not evaluated (we simply did not have the time). We then judged the remaining 28 entries, using the process defined in the Challenge. The entries that did the best used both passive and active measures to analyze the binary. Also, the highest ranked entries had the best documentation. Many of the entries were technically similar, but it was the writeups that identified the winners. That does not mean they had the longest documentation. Instead their documentation was concise, simple to read and understand, and yet had all the details involved.
We are posting the Top 20 submissions from the challenge. Each one of these submissions will receive a signed copy of our book Know Your Enemy. The Top 3 winners (Dion Mendel, CoPS Lab at the University of North Texas, and Chris Eagle) get to choose as an additional award a copy of IDA Pro Advance, IDA Pro Standard, or a free pass for Black Hat Briefings. Finally, the folks from DataRescue awarded a $200 gift certificate to the student with the best Advisory and Summary documents. They feel that Gijs Hollestelle, as a student, had the most concise yet detailed submission, as such they are awarding him the $200 Amazon gift certificate. Any of these individuals can trade amongst themselves the awards they received.
Place | Submission | Points |
---|---|---|
1st Place | Dion Mendel | 43.4 |
2nd Place | (CoPS) Lab at the University of North Texas | 42.2 |
3rd Place | Chris Eagle | 41.5 |
4th Place | Solar Eclipse | 37 |
5th Place | Marcin Gozdalik | 35.9 |
6th Place | Mat | 35.9 |
7th Place | Felix von Leitner | 35.4 |
8th Place | Eric Landuyt | 33.8 |
9th Place | Gijs Hollestelle | 33.1 |
10th Place | CERIAS Computer Forensics Research Group | 32.7 |
11th Place | HP Spain | 32.6 |
12th Place | Sean Burford | 32.5 |
13th Place | sniph | 32.3 |
14th Place | Be-Secure, Telecom Italia Labs | 32.1 |
15th Place | Secure Software, Inc. | 31.7 |
16th Place | xmux | 31.7 |
17th Place | Chris Ren | 31.2 |
18th Place | Hong-Siang Teo | 31.1 |
19th Place | John Keener | 30.5 |
20th Place | Christophe Grenier | 29.2 |
Of the 28 submissions that were evaluated, the average time spent analyzing and documenting the binary was 70 hours (one entry had spent 280 hours). This is more then twice as much time spent on the Forensic Challenge last year, where people conducted a forensic analysis of a hacked system. Why does it take twice as long to analyze a single binary, as opposed to an entire system? We are not sure, to be honest we were a little surprised by the time results. However, we have some guesses.
Hours | Cost/Hr. | Total | +15% | -15% | |
---|---|---|---|---|---|
Investigation | 70 | $33.65 | $2355.50 | $2708.83 | $2002.18 |
Benefits @ 28% | $659.54 | $758.47 | $560.61 | ||
Total Labor Cost | $3015.04 | $3467.30 | $2562.79 | ||
Median Cost +/- 15% | $3015.04 | +/- $353.34 |
So, a company's cost would be $3015.04 for the analysis and documentation of a single binary. However, the cost for a company would most likely be much greater. Reverse Engineering requires very advance skills, can your company afford to lose the time of one of your most advance engineers for almost two weeks? For most organizations, you will most likely have to contract for this type of expertise, where the costs will be much higher. The cost to contract out this analysis would most likely run as much as $350 a hour. At that rate, the average cost for analyzing this binary would have been $28,000.
The Reverse Challenge is now over, but this Challenge-project is going to live on in several ways.
(Note that there will be no prosecutions of anyone involved in this intrusion. This is not about catching the person who did this intrusion, but rather about what can be learned from it. Whoever did this is veeerrrrry lucky its working out this way. This time. ;)
- The Honeynet Project