Adding a new processor family to IDA pro
It was necessary not so long ago to gut the firmware from the M16C (Mitsubishi / Renesas). I was surprised to find out that IDA v6.1.xxx does not “hold” this family of controllers, alas. However, the SDK is available, so it's not scary - we will correct the situation. As practice has shown, there is nothing beyond the complexity in writing your module (no rocket science, tea).
I am not an expert on IDA pro and writing modules for it. Since the task was to analyze the firmware, the module was written in a hurry (on the knee), part of the code was pulled from the SDK, without understanding how it works (and whether this is necessary at all). We managed to figure out the rest of the code and it was “creatively” rethought.
I did not have tests and time to write them. The correctness of the work was verified by disassembling the firmware listing in the IDE from Renesas. Thus, it is possible (and certainly is, although I did not come across in the process of work) errors. If someone has a desire to write tests or modify the module, I will be glad.
Anyway, the module turned out quite working and allowed to perform the task. In this article I will present my thoughts on all of this. So use this work at your own risk.
')
The sources are here .
Since I do not particularly respect “Studio” (MSVC ++), I used MinGW to build it and pile up my toolchain. You can take it here . This toolchain is self-sufficient - contains a compiler and tools for building.
Preparation for assembly is as follows. Unpack the IDA_Plugins.7z somewhere, clone the repository from GitHub and copy the m16c_xx directory to the root of the IDA_Plugins directory, then run build.cmd.
Each processor module that is actually a regular dll (with a slight difference - the DOS header is slightly modified) must export a structure named LPH, with a processor_t type. It stores pointers to key functions and module structures.
Of all the variety of fields of this structure, we are primarily interested in the following function pointers:
With their help, basically, all the work is done. The ana () function is called each time when analyzing a new instruction (if there is something to analyze), its signature:
The task of this function, by sampling the byte at the current instruction pointer, try to decode the instruction, if not, continue to select subsequent bytes until all instruction and its operands are decoded. Then fill in the global cmd fields and return the length of the instruction in bytes.
The emu () function is intended to emulate an instruction, its signature:
The purpose of this function is to:
The out () function creates and displays a textual representation of the assembler instruction, its signature:
The outop () function creates and displays a textual representation of the operands of an assembler instruction, its signature:
The source data is analyzed by the ana () function, which we have to implement. Its task, reading successively the bytes of the firmware, is to determine the instructions, their operands and the lengths of the instructions.
After recognizing the instruction and its parameters, fill in the fields of the global variable cmd, which has the type insn_t :
As can be seen from the description, instructions can have up to 6 operands (which is “behind our eyes” for us - in our case, the operations contain a maximum of 3 operands).
So far, I have written a module only for the RISC controller - everything is quite simple there. We put a mask on the KOP field (Operation Code) instructions, check additional conditions in the branches of the switch statement and, in fact, everything (this is how the code for the Microchip PIC is analyzed, an example can be found in the SDK). The advantages of RISC here are obvious - an abbreviated set of commands and their identical length. Alas, for the M16C, I counted more than 300 unique teams (it all depends on the point of view - I took into account unique COPs), and yes even variable-length commands (CISC). Thus, the switch statement does not suit us because of its cumbersome and fraughtness with subtle errors.
Here it is necessary to make a small lyrical digression. Since the machine code (as well as the assembler itself) is in fact a regular language ( grammar ), it must, without any problems, be understood using the state machine (FSM), which, in essence, is the processor.
There was no desire to manually automate a manual, but there was a positive experience with Ragel - a compiler of finite automata with a special description language FSM into code in C, C ++, C #, Objective-C, D, Java, OCaml, Go or Ruby. It does not create any external dependencies, only a self-contained source in the selected programming language.
Among other things, Ragel is interesting because it allows you to parse the input data on the fly. Those. there is no need to form and transmit a large buffer to the parser for analysis, which is guaranteed to contain a command, but it can be limited to small amounts of data, up to one byte, with state preserved between calls. It is perfect for us!
The result was a DSL for parsing processor commands.
The advantage of this DSL over switch , primarily in its linearity. No need to jump on the branches of the operator to understand how it works or behavior modification. All processing of CPCs and operands for a specific team is concentrated in one place. Example:
Or, as an option, a command with operands:
It seems to me, quite clearly and conveniently. One of the enum opcodes enumeration values ( ins.hpp file) is placed in cmd.itype , which will later indicate the textual representation of the instruction, the number of operands, and describe the interaction of the instruction with operands. And also the fields of operands are filled.
The instructions themselves, the number of operands and the effect on operands is described in the instruc_t instructions [] ( ins.cpp ) array . In principle, the format of records is simple and intuitive:
It is seen that the instruction " ADC.B " has 2 operands, the first one is simply used, and the second one is changed during the instruction execution. Which is logical: ADC is addition with transfer ( ADdition with Carry ) and the operation looks like this:
Further, the execution of the instruction itself in the emu () function is emulated .
As you can see, if there is an argument, we will convert it to a digestible view in the TouchArg () function. This function looks like this:
Depending on the type of the operand, we appropriately fill in the fields of the op_t structure (“decode” the operand).
The out () function is responsible for this action. It looks like this:
Print the textual representation of the instruction and, if there are, operands with minimal formatting.
Here the code is more interesting, but everything is also quite simple:
Depending on the type of operand, we make its output to the screen accordingly.
At the moment there is one significant neponyatka. When you try to create a string (ASCII-Z String) in Undefined data, the string is created only up to an address multiple of four bytes, even if the zero byte has not yet been encountered. If the zero byte occurs earlier, then the string ends with it. With an array, the same problem.
The most annoying thing is that I don’t even know where to dig in this situation. Can someone tell me?
Thus, writing plugins for IDA pro is not very difficult. It's simple, except that with a large number of commands target assembler, quite tedious. The introduction of DSL to parse the CPCs and operands greatly simplifies and speeds up development.
Denial of responsibility
I am not an expert on IDA pro and writing modules for it. Since the task was to analyze the firmware, the module was written in a hurry (on the knee), part of the code was pulled from the SDK, without understanding how it works (and whether this is necessary at all). We managed to figure out the rest of the code and it was “creatively” rethought.
I did not have tests and time to write them. The correctness of the work was verified by disassembling the firmware listing in the IDE from Renesas. Thus, it is possible (and certainly is, although I did not come across in the process of work) errors. If someone has a desire to write tests or modify the module, I will be glad.
Anyway, the module turned out quite working and allowed to perform the task. In this article I will present my thoughts on all of this. So use this work at your own risk.
')
Sources and plugin build
The sources are here .
Since I do not particularly respect “Studio” (MSVC ++), I used MinGW to build it and pile up my toolchain. You can take it here . This toolchain is self-sufficient - contains a compiler and tools for building.
Preparation for assembly is as follows. Unpack the IDA_Plugins.7z somewhere, clone the repository from GitHub and copy the m16c_xx directory to the root of the IDA_Plugins directory, then run build.cmd.
Introduction
Each processor module that is actually a regular dll (with a slight difference - the DOS header is slightly modified) must export a structure named LPH, with a processor_t type. It stores pointers to key functions and module structures.
Of all the variety of fields of this structure, we are primarily interested in the following function pointers:
processor_t LPH = { … ana, // analyze an instruction and fill the 'cmd' structure emu, // emulate an instruction out, // generate a text representation of an instruction outop, // generate a text representation of an operand … } With their help, basically, all the work is done. The ana () function is called each time when analyzing a new instruction (if there is something to analyze), its signature:
int idaapi ana(void); //analyze one instruction and return the // instruction length The task of this function, by sampling the byte at the current instruction pointer, try to decode the instruction, if not, continue to select subsequent bytes until all instruction and its operands are decoded. Then fill in the global cmd fields and return the length of the instruction in bytes.
The emu () function is intended to emulate an instruction, its signature:
int idaapi emu(void); //emulate one instruction The purpose of this function is to:
- creating cross-references from (to) this instruction, both for data and for code;
- creating stack variables (unfortunately, I haven't figured out how it works yet)
- something else there.
The out () function creates and displays a textual representation of the assembler instruction, its signature:
void idaapi out(void); //output a single disassembled instruction The outop () function creates and displays a textual representation of the operands of an assembler instruction, its signature:
bool idaapi outop(op_t &x); //output an operand of disassembled // instruction Instruction analysis
The source data is analyzed by the ana () function, which we have to implement. Its task, reading successively the bytes of the firmware, is to determine the instructions, their operands and the lengths of the instructions.
After recognizing the instruction and its parameters, fill in the fields of the global variable cmd, which has the type insn_t :
class insn_t { public: ea_t cs; // Current segment base paragraph. Set by kernel ea_t ip; // Virtual address of instruction (within segment). // Set by kernel ea_t ea; // Linear address of the instruction. Set by kernel uint16 itype; // instruction enum value (not opcode!). // Proc sets this in ana uint16 size; // Size of instruction in bytes. Proc sets this in ana union { // processor dependent field. Proc may set this uint16 auxpref; struct { uchar low; uchar high; } auxpref_chars; }; char segpref; // processor dependent field. Proc may set this char insnpref; // processor dependent field. Proc may set this op_t Operands[6]; // instruction operand info. Proc sets this in ana char flags; // instruction flags. Proc may set this }; As can be seen from the description, instructions can have up to 6 operands (which is “behind our eyes” for us - in our case, the operations contain a maximum of 3 operands).
So far, I have written a module only for the RISC controller - everything is quite simple there. We put a mask on the KOP field (Operation Code) instructions, check additional conditions in the branches of the switch statement and, in fact, everything (this is how the code for the Microchip PIC is analyzed, an example can be found in the SDK). The advantages of RISC here are obvious - an abbreviated set of commands and their identical length. Alas, for the M16C, I counted more than 300 unique teams (it all depends on the point of view - I took into account unique COPs), and yes even variable-length commands (CISC). Thus, the switch statement does not suit us because of its cumbersome and fraughtness with subtle errors.
Here it is necessary to make a small lyrical digression. Since the machine code (as well as the assembler itself) is in fact a regular language ( grammar ), it must, without any problems, be understood using the state machine (FSM), which, in essence, is the processor.
There was no desire to manually automate a manual, but there was a positive experience with Ragel - a compiler of finite automata with a special description language FSM into code in C, C ++, C #, Objective-C, D, Java, OCaml, Go or Ruby. It does not create any external dependencies, only a self-contained source in the selected programming language.
Among other things, Ragel is interesting because it allows you to parse the input data on the fly. Those. there is no need to form and transmit a large buffer to the parser for analysis, which is guaranteed to contain a command, but it can be limited to small amounts of data, up to one byte, with state preserved between calls. It is perfect for us!
The result was a DSL for parsing processor commands.
DSL to parse commands
The advantage of this DSL over switch , primarily in its linearity. No need to jump on the branches of the operator to understand how it works or behavior modification. All processing of CPCs and operands for a specific team is concentrated in one place. Example:
#// 0x00 0000 0000 BRK M16C_BRK = 0x00 @ { cmd.itype = M16C_xx_BRK; cmd.Op1.type = o_void; cmd.Op1.dtyp = dt_void; }; Or, as an option, a command with operands:
#// 0x01..0x03 0000 00DS MOV.B:S R0L, DEST M16C_MOV_B_S_R0L_DEST = (0x01..0x03) @ { cmd.itype = M16C_xx_MOV_B_S_R0L_DEST; MakeSrcDest8(SRC_DEST_R0L, cmd.Op1); switch(*p & 0x03) { case 0x01: MakeSrcDest8(SRC_DEST_DSP_8_SB_, cmd.Op2); break; case 0x02: MakeSrcDest8(SRC_DEST_DSP_8_FB_, cmd.Op2); break; default: MakeSrcDest8(SRC_DEST_ABS16, cmd.Op2); break; } }; It seems to me, quite clearly and conveniently. One of the enum opcodes enumeration values ( ins.hpp file) is placed in cmd.itype , which will later indicate the textual representation of the instruction, the number of operands, and describe the interaction of the instruction with operands. And also the fields of operands are filled.
Emulation of instruction execution
The instructions themselves, the number of operands and the effect on operands is described in the instruc_t instructions [] ( ins.cpp ) array . In principle, the format of records is simple and intuitive:
instruc_t instructions[ ] = { ... { "ADC.B", CF_USE1|CF_CHG2 }, { "ADC.W", CF_USE1|CF_CHG2 }, { "ADC.B", CF_USE1|CF_CHG2 }, { "ADC.W", CF_USE1|CF_CHG2 }, { "ADCF.B", CF_CHG1 }, { "ADCF.W", CF_CHG1 }, { "ADD.B:G", CF_USE1|CF_CHG2 }, { "ADD.W:G", CF_USE1|CF_CHG2 }, { "ADD.B:Q", CF_USE1|CF_CHG2 }, ... }; It is seen that the instruction " ADC.B " has 2 operands, the first one is simply used, and the second one is changed during the instruction execution. Which is logical: ADC is addition with transfer ( ADdition with Carry ) and the operation looks like this:
[ Syntax ] ADC.size src,dest ^--- B, W [ Operation ] dest <- src + dest + C Further, the execution of the instruction itself in the emu () function is emulated .
int emu( ) { unsigned long feature = cmd.get_canon_feature( ); if( feature & CF_USE1 ) TouchArg( cmd.Op1, 1 ); if( feature & CF_USE2 ) TouchArg( cmd.Op2, 1 ); if( feature & CF_CHG1 ) TouchArg( cmd.Op1, 0 ); if( feature & CF_CHG2 ) TouchArg( cmd.Op2, 0 ); if( !( feature & CF_STOP ) ) ua_add_cref( 0, cmd.ea + cmd.size, fl_F); return 1; } As you can see, if there is an argument, we will convert it to a digestible view in the TouchArg () function. This function looks like this:
void TouchArg( op_t &x, int isload ) { switch ( x.type ) { case o_near: { cref_t ftype = fl_JN; ea_t ea = toEA(cmd.cs, x.addr); if ( InstrIsSet(cmd.itype, CF_CALL) ) { if ( !func_does_return(ea) ) flow = false; ftype = fl_CN; } ua_add_cref(x.offb, ea, ftype); } break; case o_imm: if ( !isload ) break; op_num(cmd.ea, xn); if ( isOff(uFlag, xn) ) ua_add_off_drefs2(x, dr_O, OOF_SIGNED); break; case o_displ: if(x.dtyp == dt_byte) op_dec(cmd.ea, xn); break; case o_mem: { ea_t ea = toEA( dataSeg( ),x.addr ); ua_dodata2( x.offb, ea, x.dtyp ); if ( !isload ) doVar( ea ); ua_add_dref( x.offb, ea, isload ? dr_R : dr_W ); } break; default: break; } } Depending on the type of the operand, we appropriately fill in the fields of the op_t structure (“decode” the operand).
The output text of the instructions
The out () function is responsible for this action. It looks like this:
void out() { char str[MAXSTR]; //MAXSTR is an IDA define from pro.h init_output_buffer(str, sizeof(str)); OutMnem(12); //first we output the mnemonic if( cmd.Op1.type != o_void ) //then there is an argument to print out_one_operand( 0 ); if( cmd.Op2.type != o_void ) { //then there is an argument to print out_symbol(','); out_symbol(' '); out_one_operand( 1 ); } if( cmd.Op3.type != o_void ) { //then there is an argument to print out_symbol(','); out_symbol(' '); out_one_operand( 2 ); } term_output_buffer(); gl_comm = 1; //we want comments! MakeLine(str); //output the line with default indentation } Print the textual representation of the instruction and, if there are, operands with minimal formatting.
Textual representation of operands
Here the code is more interesting, but everything is also quite simple:
bool idaapi outop(op_t &x) { ea_t ea; switch (x.type) { case o_void: return 0; case o_imm: OutValue(x, OOF_NUMBER | OOF_SIGNED | OOFW_IMM); break; case o_displ: { //then there is an argument to print OutValue(x, OOF_NUMBER | OOF_SIGNED | OOFW_IMM); switch (x.dtyp) { case dt_byte: break; case dt_word: out_symbol(':'); out_symbol('8'); break; case dt_dword: out_symbol(':'); out_symbol('1'); out_symbol('6'); break; default: ea = toEA(cmd.cs, x.addr); if (!out_name_expr(x, ea, x.addr)) out_bad_address(x.addr); break; } out_symbol('['); out_register(M16C_xx_RegNames[x.reg]); out_symbol(']'); } break; case o_phrase: out_symbol('['); out_register(M16C_xx_RegNames[x.reg]); out_symbol(']'); break; case o_near: ea = toEA(cmd.cs, x.addr); if (!out_name_expr(x, ea, x.addr)) out_bad_address(x.addr); break; case o_mem: ea = toEA(dataSeg(), x.addr); if (!out_name_expr(x, ea, x.addr)) out_bad_address(x.addr); break; case o_reg: out_register(M16C_xx_RegNames[x.reg]); break; default: warning("out: %a: bad optype %d", cmd.ea, x.type); break; } return true; } Depending on the type of operand, we make its output to the screen accordingly.
Deficiencies and problems identified
At the moment there is one significant neponyatka. When you try to create a string (ASCII-Z String) in Undefined data, the string is created only up to an address multiple of four bytes, even if the zero byte has not yet been encountered. If the zero byte occurs earlier, then the string ends with it. With an array, the same problem.
The most annoying thing is that I don’t even know where to dig in this situation. Can someone tell me?
Conclusion
Thus, writing plugins for IDA pro is not very difficult. It's simple, except that with a large number of commands target assembler, quite tedious. The introduction of DSL to parse the CPCs and operands greatly simplifies and speeds up development.
Source: https://habr.com/ru/post/268339/
All Articles