Reverse Engineering Tips

by Sanjay Mishra

We are often called in to reverse engineer electronics and reverse engineer software. Before give any reverse engineering quotes we make sure it is a legal request. We check with the provisions of the DMCA act and make sure that we are not doing anything illegal. Most of our requests come from owners of large systems that are non functional in some way because critical documentation or source code is not available. As an ethical reverse engineering firm we do not crack software.

Well the first thing to know is that reverse engineering is not a science but is an art form. To be able to reverse engineer a system you have to know the domain that you are performing the reverse engineering in. While reversing software or reversing electronics you will have to guess at the way the original designers implemented the system and experience with similar systems is invaluable.

Before you begin reverse engineering you must identify the goals of the reverse engineering process clearly. Reverse engineering takes a lot of time and you want to do just as much reverse engineering as is necessary.

Once you have identified the goals, you must identify the system communication interfaces where you can tap into easily. For example,

• if the system communicates via RS-485 or RS-232 you can hook into these serial communication lines easily. You can create fake serial ports using an utility (look at our serial port tips page) or feed the system data to investigate its behavior.
• If it communicates via Ethernet, you can sniff the packets and try figuring out what is happening
• If it communicates via RF you will need to do a bit more work. For narrow band RF communications, you can hook up a cheap Radio Shack scanner and feed its output to your PC's soundcard to demodulate the signal. Frequency Shift Keying can be demodulated even by simple visual inspection of the captured waveforms in a program like Audacity.

The other easy thing to do for software that runs on a generic OS (Windows, Linux, BSD, Solaris) is to trap all system calls. Utilities like strace on Unix/Linux like systems are invaluable. For windows, there is a bunch of utilities available at

http://technet.microsoft.com/en-us/sysinternals/default.aspx

(It used to be sysinternals.com before Microsoft acquired the site)

For embedded systems you can pull out the programmable memory or the programmable microcontroller and read it with a reader tool. You then may use a tool like IDA Pro to try and make sense out of it. Having the address and data buses of the system accessible are a great help. You can then hook up the system to a logic analyzer and see what is happening.

If you have I2C bus or similar bus on the embedded system you can hookup a logic analyzer tool like Intronix Logic Port to your system and decode what the bus is doing. I have used both big bulky HP Logic analyzers and the USB based Logic Port and found Logic Port to be more useful besides being cheaper for what I do.

Another option for embedded systems is to hook into the On Chip Debug facility like JTAG and see the code executing. Of course you will only see the code executing in assembly (unless you have source code) but you can quickly identify critical points and narrow your search down to the area that you are interested in.

Reverse Engineering Software

If you are reverse engineering x86 based software and if your software can run under a vmware player you must do that. We are assuming that you are reverse engineering software the rights to which you own but for some reason whose source code is lost. Vmware player provides great help in being able to decode the software, set up custom OS builds.

A software tool like SoftICE is of great help in reverse engineering system software like drivers. Development on SoftIce has been stopped. Syser offers a commercial alternative that works for multi core CPUs also.

Microsoft offers two kernel-mode debuggers - WinDbg and KD. However these require interlinked computers. The Rasta Ring 0 debugger is an open source alternative which works on x86 systems and also allows for debugging Linux, BSD etc

I have found Ollydbg to be of great help in decoding.

Reverse Engineering Tutorial

I am working on a reverse engineering tutorial which will be easy to follow. I do not want to do a cracking tutorial but something which illustrates the issues a person would run into while doing ethical reverse engineering. I want the tutorial to be illustrative and all its components to be easily available. If you have any suggestions for the tutorial let us know in the form below.

---