|
Summary
Wow! We received a total of 35 submission for the Reverse Challenge. These
submissions were outstanding, most contestants put an incredible amount of
time and effort into the Challenge, we were extremelly impressed! This made juding
very tough (and time consuming :). Unfortunately,
not everyone that submitted had time to complete all of the documentation. Of the
35 entries, we identified 7 as incomplete and were not evaluated (we simply did not
have the time). We then judged the remaining 28 entries, using the process defined in the
Challenge. The entries that did the best used both
passive and active measures to analyze the binary. Also, the highest ranked entries had the
best documentation. Many of the entries were technically similar, but it was the writeups that
identified the winners. That does not mean they had the longest documentation.
Instead their documentation was concise, simple to read and understand, and yet
had all the details involved.
Top 20 Submissions
We are posting the Top 20 submissions from the challenge. Each one of these
submissions will receive a signed copy of our book
Know Your Enemy. The Top 3 winners (Dion Mendel, CoPS Lab at the University
of North Texas, and Chris Eagle) get to choose as an additional award a copy
of IDA Pro Advance, IDA Pro Standard,
or a free pass for Black Hat
Briefings. Finally, the folks from DataRescue
awarded a $200 gift certificate to the student with the best Advisory and Summary documents. They feel that
Gijs Hollestelle, as a student, had the most concise yet detailed submission,
as such they are awarding him the $200 Amazon gift certificate. Any of these
individuals can trade amongst themselves the awards they received.
Time/Cost Analysis
Of the 28 submissions that were evaluated, the average time spent
analyzing and documenting the binary was 70 hours (one entry
had spent 280 hours). This is more then twice as much time spent
on the Forensic Challenge last year,
where people conducted a forensic analysis of a hacked system. Why
does it take twice as long to analyze a single binary, as opposed
to an entire system? We are not sure, to be honest we were a little
surprised by the time results. However, we have some guesses.
- Reverse Engineering is extremely scientific and detailed. There
is little guessing involved. Properly done, you should be able to
determine exactly how things work. This level of detail takes more
time.
- There are few OpenSource reverse engineering tools available. As
a result, for many of the submissions, people developed their own tools.
- Overall, the level and detail of documentation has improved, requiring
more time and effort.
So, based on an average of 70 hours per analysis and documentation, what is
the cost? Honeynet member David Dittrich has
outlined a process to determine this, based on an annual salary of 70,000.
|
Hours |
Cost/Hr. |
Total |
+15% |
-15% |
| Investigation |
70 |
$33.65 |
$2355.50 |
$2708.83 |
$2002.18 |
| Benefits @ 28% |
|
|
$659.54 |
$758.47 |
$560.61 |
| Total Labor Cost |
|
|
$3015.04 |
$3467.30 |
$2562.79 |
| Median Cost +/- 15% |
|
|
|
$3015.04 |
+/- $353.34 |
So, a company's cost would be $3015.04 for the analysis and documentation
of a single binary. However, the cost for a company would most likely
be much greater. Reverse Engineering requires very advance skills, can
your company afford to lose the time of one of your most advance engineers
for almost two weeks? For most organizations, you will most likely have to contract for this
type of expertise, where the costs will be much higher. The cost to
contract out this analysis would most likely run as much as $350 a hour. At that
rate, the average cost for analyzing this binary would have been $28,000.
The End?
The Reverse Challenge is now over, but this Challenge-project is
going to live on in several ways.
If you have any suggestions, questions, or comments on the
Reverse Challenge, feel free to contact us at
challenge@honeynet.org
Shouts and greetz
The Project would like to thank all six judges that volunteered their time
and effort in the evaluation of the submissions. Without their time and
effort, this Challenge would never have been possible. These six judges
were:
- Gera
- Halvar
- Niels
- Job de Haas
- K2
- David Dittrich
We would also like to thank the folks who supplied the awards, including
DataRescue,
Black Hat, and our publisher Addison-Wesley.
And finally, thanks to the security community. Without your contributions, this
Challenge would have never been possible.
- The Honeynet Project
|
