memset - side of darkness

After reading the article The most dangerous function in the world of C / C ++, I found it useful to go deeper into the evil in the dark cellar of memset , and write a supplement in order to broaden the essence of the problem.

In C, memset () is commonly used, concealing many pitfalls. Excerpt from C ++ Reference:

void * memset (void * ptr, int value, size_t num);
Fill block of memory
It can be interpreted as interpreted by the text.
Parameters
ptr - pointer to memory.
value - Value to be set. The value of the fills is what the block of memory uses.
num - Number to by value. size_t is an unsigned integral type.
Return value
ptr is returned.

As already noted several times, there are many rakes, which are attacked even by experienced developers. From the Andrey2008 article described in the article , a brief summary of common errors :

№1. Trying to calculate the size of an array or structure, do not use sizeof () for pointers to an array / structure, it will return you a pointer size of 4 or 8 bytes, instead of the size of the array / structure.
')
№2. The third argument, memset (), takes as input the number of bytes, not the number of elements, not taking into account the data type. I will add more, for example, the type int can occupy both 4 and 8 bytes, depending on the architecture. In this case, use sizeof (int).

Number 3. Do not confuse the arguments. The correct sequence is a pointer, a value, a length in bytes.

№4. Do not use memset when working with class objects.

But this is only the tip of the iceberg.

Alternative to memset

memset is a low-level function that binds the developer to take into account all the features of the computer architecture and its use must be justified. Let's first consider the alternative = {0} , instead of memset , they say it allows you to initialize an array or string at the compilation stage, which should improve program performance, unlike memset (also ZeroMemory), which initialize data at runtime. I decided to check it out.

void doInitialize() { char p0[25] = {0} ; //   25   0 char p1[25] = "" ; //   25   0 wchar_t p2[25] = {0} ; //  25   0 wchar_t p3[25] = L"" ; //   25   0 short p4[62] = {0} //  62   0 int p5[37] = {-1} ; //      -1 unsigned int p6[10] = {89} ; //     89 } 

C99 [$ 6.7.8 / 21]

If you’re trying to find out what to do, then be initialized implicitly storage duration.

At the same time, such initialization removes problems No. 1, No. 2, No. 3 with a confusion of parameters and buffer sizes. That is, we will not confuse the second and third arguments in some places, it is not necessary to transfer the size. Let's see how compilers transform such code. I can not check all the compilers right away, the gcc included in android-ndk-r10c, as well as gcc in ubuntu 14.04, were on hand.


Let's see how the compiler behaves on such a piece of code:

 void empty_string(){ int i; char p1[25] = {0}; printf("\np1: "); for (i = 0; i < 25; i++) printf("%x,",p1[i]); } 

So, without optimization (-O0), array initialization is compiled into such an assembler code (we scan the binaries with the help of objdump):



As expected, without optimization, we get run-time code that will eat O (n) processor time (where n is the buffer length). What the compiler will do with optimization (-O3) can be seen below.

gcc -O3, 32-bit, ARM

 000083ac <empty_string>: 83ac: e59f002c ldr r0, [pc, #44] ; 83e0 <empty_string+0x34> 83b0: e92d4038 push {r3, r4, r5, lr} 83b4: e08f0000 add r0, pc, r0 83b8: ebffffb2 bl 8288 <printf@plt> 83bc: e59f5020 ldr r5, [pc, #32] ; 83e4 <empty_string+0x38> 83c0: e3a04019 mov r4, #25 83c4: e08f5005 add r5, pc, r5 83c8: e1a00005 mov r0, r5 83cc: e3a01000 mov r1, #0 83d0: ebffffac bl 8288 <printf@plt> 83d4: e2544001 subs r4, r4, #1 83d8: 1afffffa bne 83c8 <empty_string+0x1c> 83dc: e8bd8038 pop {r3, r4, r5, pc} 

We see that the piece of code with zeroing in the run-time just disappeared, we got the promised performance O (1), let's see where does printf take its values? We are interested in this piece:

 83bc: ldr r5, [pc, #32] 83c0: mov r4, #25 ;//  r4    for,     83c4: add r5, pc, r5 ;//  r5   "%x,"  ,      002c7825 83c8: mov r0, r5 ;// r5    r0    ,    printf() 83cc: mov r1, #0 ;//   0 (  p1[i])    printf() 83d0: bl 8288 <printf@plt> 83d4: subs r4, r4, #1 ;//      83d8: bne 83c8 <empty_string+0x1c> ;//     0,      83c8 

That is, the compiler simply threw out the array, and instead of its values ​​it uses 0, as a constant pledged at the compile stage. Good, but what happens if we use memset ? Let's look at a few pieces of objdump, for example, under ARM:

Without optimization -O0 :

  83d8: e24b3024 sub r3, fp, #36 ; 0x24 83dc: e1a00003 mov r0, r3 83e0: e3a01000 mov r1, #0 83e4: e3a02019 mov r2, #25 83e8: ebffffa3 bl 827c <memset@plt> 

Optimized with -O3 :

  83c0: e58d3004 str r3, [sp, #4] 83c4: e58d3008 str r3, [sp, #8] 83c8: e58d300c str r3, [sp, #12] 83cc: e58d3010 str r3, [sp, #16] 83d0: e58d3014 str r3, [sp, #20] 83d4: e58d3018 str r3, [sp, #24] 83d8: e5cd301c strb r3, [sp, #28] 

That is, the optimization simply removes the memset call, inserting it inline. In such cases, memset will always work for O (n) time, but initialization with = {0} when optimizing works for a constant, in our case, without at all taking away processor cycles, brazenly throwing out the fact of the existence of an array and replacing all its elements zeros. But let's see if this is always the case and what happens if we write a non-zero value after initialization? The test function will look like this:

 void empty_string(){ int i; char p1[25] = {0}; p1[0] = 65; printf("\np1: "); for (i = 0; i < 25; i++) printf("%x,",p1[i]); } 

After compilation we get the already familiar block of code:

  8404: e3a02041 mov r2, #65 ; 0x41 8408: e08f0000 add r0, pc, r0 840c: e58d3004 str r3, [sp, #4] 8410: e58d3008 str r3, [sp, #8] 8414: e58d300c str r3, [sp, #12] 8418: e58d3010 str r3, [sp, #16] 841c: e58d3014 str r3, [sp, #20] 8420: e58d3018 str r3, [sp, #24] 8424: e5cd301c strb r3, [sp, #28] 8428: e5cd2004 strb r2, [sp, #4] 

And it looks as if the compiler has inserted into us an optimized memset version. And let's see what happens if the size of the array grows significantly? Say, not 25 bytes, but 25 kilobytes!

  83fc: e24ddc61 sub sp, sp, #24832 ; 0x6100 8400: e24dd0a8 sub sp, sp, #168 ; 0xa8 8404: e3a01000 mov r1, #0 8408: e59f2054 ldr r2, [pc, #84] ; 8464 <empty_string+0x6c> 840c: e1a0000d mov r0, sp 8410: ebffff99 bl 827c <memset@plt> 

Wow!

Line = {0} goes to the side of darkness, memset rejoices!

However, let's not forget, yet we decided to confuse the problem with the parameters; now it’s impossible to mix up the arguments.

String initialization

Also, it would not be superfluous to consider the option of initializing the array = "" . In the C language, null-terminated strings are used, that is, the first character with a byte value of 0x00 indicates the end of the string. Therefore, to initialize a string, it makes no sense to reset all elements; you just need to zero the first one. Here are some ways to initialize an empty string:

 void doInitializeCString() { char p0[25] = {0} ; //     0 char p1[25] = "" ; //     0 char p2[25] ; p2[0] = 0 ; //     0 char p3[25] ; memset(p3, 0, sizeof(p3)) ; //  25   0 char p4[25] ; strcpy(p4, "") ; //     0 char *p5 = (char *) calloc(25, sizeof(char)) ; //     0 } 

The surest way how initialization will work with = "" will parse the objdump again. Without optimization, we will not see anything special, everything is similar there = {0} , we will immediately consider with the -O3 option. So compile under ARM:

And, suddenly, we get zeroing of all elements of the array.

  83c0: e58d3004 str r3, [sp, #4] 83c4: e58d3008 str r3, [sp, #8] 83c8: e58d300c str r3, [sp, #12] 83cc: e58d3010 str r3, [sp, #16] 83d0: e58d3014 str r3, [sp, #20] 83d4: e58d3018 str r3, [sp, #24] 83d8: e5cd301c strb r3, [sp, #28] 

Oh well! Why in the null-terminated string reset all unused characters? It is enough to zero one single byte. Hmm, and if there will be 25 thousand bytes, what will it do? Here's what:

  8474: e24ddc61 sub sp, sp, #24832 ; 0x6100 8478: e24dd0a8 sub sp, sp, #168 ; 0xa8 847c: e3a0c000 mov ip, #0 8480: e28d3f6a add r3, sp, #424 ; 0x1a8 8484: e1a0100c mov r1, ip 8488: e59f204c ldr r2, [pc, #76] ; 84dc <empty_string+0x6c> 848c: e28d0004 add r0, sp, #4 8490: e503c1a8 str ip, [r3, #-424] ; 0x1a8 8494: ebffff78 bl 827c <memset@plt> 

Looks like dark memset is haunting us. If you still want to fight against darkness, then it is worth mentioning what other traps are waiting for you.

memset may initialize numbers with incorrect values.

If you want to fill an array of integers with nonzero values, read byte-filled data.

 void doInitializeToMistakenValues() { char pChar[25] ; unsigned char pUChar[25] ; short pShort[25] ; unsigned short pUShort[25] ; int pInt[25] ; unsigned int pUInt[25] ; //  2-  4-      memset(pChar, 1, sizeof(pChar)) ; // 1 memset(pUChar, 1, sizeof(pUChar)) ; // 1 memset(pShort, 1, sizeof(pShort)) ; // 257 memset(pUShort, 1, sizeof(pUShort)) ; // 257 memset(pInt, 1, sizeof(pInt)) ; // 16843009 memset(pUInt, 1, sizeof(pUInt)) ; // 16843009 //  unsigned    0xFF memset(pChar, -1, sizeof(pChar)) ; // -1 memset(pUChar, -1, sizeof(pUChar)) ; // 255 memset(pShort, -1, sizeof(pShort)) ; // -1 memset(pUShort, -1, sizeof(pUShort)) ; // 65535 memset(pInt, -1, sizeof(pInt)) ; // -1 memset(pUInt, -1, sizeof(pUInt)) ; // 4294967295 } 

Consider how it turns out. Here we have, let's say, an array of int, passing the unit to the second parameter, what happens?

Here's what:

0x01010101 - in hexadecimal notation, each byte will be filled with one, and the correct value
0x00000001 will not be available with the memset function. But in fact, this is not a bug, this is a feature.

Here only ignorance of these features leads to unpredictable errors.

memset can set an invalid value

If the double elements are set to bytes -1, we get the value Not-A-Number (NaN), and after subsequent calculations, each operation with the value NaN will return NaN, thus disrupting the whole chain of calculations.

In the same way, setting -1 to the bool type is incorrect and it formally will not be either true or false. Although in most cases it will behave like true. In most cases…

And the last, memset is intended only for work with simple data structures . Never use memset with managed data structures, this function is intended only for low-level operations.


The article used materials memset is evil .

Also read about the printf vulnerabilities .