Geekly Articles each Day
📜 ⬆️ ⬇️

IDA Pro Upgrade. Debugger plugin. Part I. Theory


Hello. I decided to try to start a series of articles on the modernization of our beloved IDA Pro .
In each of the tutorials I will try to solve a rather complicated and poorly studied topic: writing various modules:

And, if processor modules , plug-ins and scripts are a topic that is more or less revealed, then the rest is almost total darkness ( at the end of the article I will give a list of references and projects where there is at least something ).

So, the first article from the cycle will be devoted to writing a plug-in debugger , or rather a preliminary theory. In the standard delivery of the IDA SDK , there are already sources of the main debuggers ( Windows , Linux , Mac ). But what about, for example, Amiga , M68000 ?

This article is more likely to be devoted to boring and not very theory. It is strongly recommended to get acquainted with it, because Many moments in the future may be incomprehensible precisely because of the impossibility of presenting a general concept.

To begin with, we will define what plug-ins debuggers happen.
')

Types of debugger plugins


The debugging plugins for IDA are of two main types:

There are still some others, but they will not be discussed in these articles (or will they?). I will describe here only a local debugger, since the remote one requires twice the code (client and server), and I have not studied it well enough.

Components of the debager plugin



So, about all the components in order.

IDA - visual interface


Actually, IDA itself is an adapter between the user interface and the debager , sending commands to it , and waiting for any reaction ( messages ). Commands may be:

Debugger plugin


The task of the plug-in is to respond to the commands from IDA ( see above ), and send it the events of the following types triggered in the emulator :

You need to send messages so that IDA knows what is happening now and displays the current state of debugging. When a breakpoint or step event occurs, for example, a debugger window should appear, in order to wait for further actions from the user.

Emulator - all head


What is the purpose of the emulator ? Of course, follow the instructions. He also knows how to keep context. Sometimes, it holds information about breakpoints (and sometimes it’s not - then you have to implement them at the debager level). It also happens that the emulator itself does not have the function of suspending / continuing execution, and we have to implement it somehow. One thing pleases - usually emulator source codes for many platforms already exist, and, often, they need only a little bit of modernization for our task.

So, the emulator reports to the debager about the occurrence of events in itself:

Developments


Let's look at the Step event :
  1. I press the F7 ( Step Into ) key in IDA ;
  2. In debugger gets the command to complete the last coming before this event . Usually, the debager in this case will have to tell the emulator to continue execution;
  3. Next, the debugger receives the command to perform the step in the stream , responds to this by executing one instruction of the emulator ;
  4. The emulator reports back to the debager that the STEP event has arrived, placing it in the event queue ( see the event queue below );
  5. Debager gets the dispatched event from the event queue, and reports it to IDA ;
  6. IDA responds to an upcoming event. In this case, realizes that the step has been completed, and shows the current EIP address, which we got up to after the step;
  7. After we have seen enough of the IDA window with the state of registers and other things, and we want to do something further (for example, another step by pressing F7 ), the cycle again goes to the first item.

Event Queue :
Such a thing that constantly revolves in a separate debuger thread, in anticipation of the onset of new events of the emulator . When receiving a new event, you must pause the emulation (because the step implies a process frieze), and then pass the event to IDA . When the IDA reports that the event has been processed, a command to complete the processing of the current event will come to the debager .

As you can see, the topic is very muddy, and not very simple. But, it seems so at first glance. In the next article, we will begin the actual writing of a debugger plugin , and at the same time we will see what and how.

- References and projects:


Source: https://habr.com/ru/post/262433/

More articles:


All Articles