Is your disassembler working correctly?

Today I want to talk about one interesting complexity of decoding / disassembling IA-32 instructions.

Before reading this article, I recommend contacting the article “Prefixes in the IA-32 Command System” , which describes the general structure of the IA-32 command and the existing prefixes. In this article I will discuss in more detail about mandatory prefixes (the mandatory prefixes) and some nuances associated with them.

It all started with reading an article about a language designed to create decoders - GDSL . This article provides some examples already known to me, as well as new features that I have not heard about before. I will tell you about them now.

Some instructions, such as MULSS , MULSD and MULPD (vector multiplication instructions), have the same opcode 0x0f 0x59 , but different mandatory prefixes ( 0xf2 , 0xf3 and 0x66 respectively). The question arises, what should happen if there are several such prefixes in the instruction code at the same time? It would probably be more logical to determine what the instruction is, by the last prefix. But it's not always the case! If the last prefix is 0xf2 or 0xf3 , then it is considered mandatory, but 0x66 is mandatory only if the prefixes 0xf2 and 0xf3 missing in the encoding of this instruction. Examples of correct disassembler output for these instructions can be found in the table:
')

Instruction code	Instruction	Mandatory prefix
66 f3 f2 0f 59 ff	MULSD xmm7, xmm7	f2
66 f2 f3 0f 59 ff	MULSS xmm7, xmm7	f3
66 0f 59 ff	MULPD xmm7, xmm7	66
f2 66 0f 59 ff	MULSD xmm7, xmm7	f2
0f 59 ff	MULPS xmm7, xmm7	-

At some point I questioned the correctness of these statements, but, unfortunately, the documentation gives an ambiguous idea about this issue. After that, it was decided to conduct tests on a real processor, and it turned out that it works that way. Verification of this was carried out using assembler inserts. An example of one of the tests is shown below:

 #include <stdio.h> int main() { double a[2] = {2, 2}, b[2] = {0, 0}; __asm__ __volatile__ ( // Copy data from a to xmm7 register "movupd %1, %%xmm7\n" //"mulsd %%xmm7, %%xmm7\n" ".byte 0xf2, 0x66, 0x0f, 0x59, 0xff\n" // Copy data from xmm7 register to b "movupd %%xmm7, %0\n" :"=m"(*b) :"m"(*a) : ); printf("%lf %lf\n", b[0], b[1]); return 0; } 

Compiling and running it, you will see the following:

 $ gcc -O0 -Wall mulsd.c $ ./a.out 4.000000 2.000000 

That is, only the first element of the vector multiplied, while the second remained unchanged, which corresponds to the instruction MULSD , and not MULPD .

In this example, several disassemblers included in the known packages were tested. Few of them coped with their task, which was immediately reported to the developers of these products. A summary of the results is given below:

Product version	Result	The error I reported
Wind River Simics, 4.8	success	-
Xed	success	-
objdump, 2.23	mistake	16083
GNU GDB, 7.5	mistake	16089
nasm, 2.09	mistake	3392269
ODA, 0.2.0	mistake	Email Correspondence
objconv, 2.31	mistake	Email Correspondence
IDA, 6.4 (Evaluation Version)	mistake	Email Correspondence
llvm-objdump, 3.2	mistake	17697

It should be noted that gdb and objdump are part of binutils and use the same library for disassembling. One of the developers of ODA, Anthony DeRosa, in response to my error message, said that they use the libopcodes library that comes with binutils. That is, a correction in one place should entail an adjustment of at least three products at once, but, unfortunately, none of the binutils have yet answered me.

Does the disassembler that you use work correctly?