64-bit code in 2015: what's new in diagnosing possible problems?

64-bit errors are hard enough to detect, since they are akin to a time bomb: they can be felt far from immediately. The PVS-Studio static analyzer facilitates the task of finding and correcting such errors. However, several more steps were taken in this direction: 64-bit diagnostics were recently reviewed more closely, as a result of which their distribution by importance levels changed. This article focuses on these changes, as well as how it affected the work with the tool and the search for errors. Examples of 64-bit errors from real applications are attached.

What is the article about?

For a start, I would like to make specifics in content. The article covers the following topics:

1. Changes in the PVS-Studio analyzer affecting the search for 64-bit errors;
2. Overview of 64-bit first-level errors found by the PVS-Studio analyzer and brief comments on them;
3. Comparison of efficiency in finding the most important errors by means of PVS-Studio and Microsoft Visual Studio 2013.

The first paragraph speaks for itself: it will cover the main changes in PVS-Studio regarding the analysis of 64-bit errors, as well as how they will affect the operation of the tool.

The second main section is devoted to found 64-bit errors in real projects. In addition to the code snippets from the projects, comments on them will also be given, so you may be able to learn something new for yourself.
')
The third section compares the effectiveness of finding these errors with the PVS-Studio static analyzer and the Microsoft Visual Studio 2013 environment tools. Moreover, in the case of Visual Studio, both the compiler and the static analyzer were used to search for errors.

Do not forget that here are written only some errors. In a real project, they will most likely be much larger and they will be more diverse. At the end of the article, links are suggested that more fully acquaint you with the world of 64-bit errors.

Changes in PVS-Studio due to 64-bit errors

Not so long ago, we carefully looked at 64-bit diagnostics and more accurately distributed them by severity levels.

Now the distribution of 64-bit errors looks like this:

Level 1. Critical errors that cause harm in any application. An example would be storing a pointer in a 32-bit variable of type int. If you are developing a 64-bit application, you should definitely study and correct the first level warnings.

Level 2. Errors, which usually manifest themselves only in applications that process large data arrays. An example is the use of an 'int' variable for indexing a huge array.

Level 3. Everything else. As a rule, these warnings are not relevant. However, for some applications, this or that diagnosis can be extremely useful.

Thus, by setting the filtering on 64-bit first-level errors, you will receive a list of messages pointing to sections of code that are more likely to be erroneous. Do not underestimate these warnings, since the consequences of 64-bit errors can be very different, but clearly unpleasant and often unexpected. It is about them that will be discussed.

I think you will understand how hard it would be to find such errors without a tool like PVS-Studio as you read the article.

Analysis of 64-bit errors

Care should be taken to ensure that data types are used correctly. With this, perhaps, let's start.

LRESULT CSaveDlg::OnGraphNotify(WPARAM wParam, LPARAM lParam) { LONG evCode, evParam1, evParam2; while (pME && SUCCEEDED(pME->GetEvent(&evCode, (LONG_PTR*)&evParam1, (LONG_PTR*)&evParam2, 0))) { .... } return 0; } 

Analyzer Warnings:

• V114 Dangerous explicit type pointer conversion: (LONG_PTR *) & evParam1 test.cpp 8
• V114 Dangerous explicit type pointer conversion: (LONG_PTR *) & evParam2 test.cpp 8

In order to understand the nature of the error, you need to look at the variable types 'evParam1', 'evParam2', as well as the declaration of the method 'GetEvent':

 virtual HRESULT STDMETHODCALLTYPE GetEvent( /* [out] */ __RPC__out long *lEventCode, /* [out] */ __RPC__out LONG_PTR *lParam1, /* [out] */ __RPC__out LONG_PTR *lParam2, /* [in] */ long msTimeout) = 0; 

As can be seen from the analyzer message, a dangerous explicit type conversion is performed. The point is that the type 'LONG_PTR' is a ' memsize-type ', having a size of 32 bits per Win32 ( ILP32 data model ) and 64 bits on a Win64 architecture ( LLP64 data model ). At the same time, the 'LONG' type has a size of 32 bits on both architectures. Since the 64-bit architecture of the above types have different sizes, it is possible to work incorrectly with the objects referenced by these pointers.

Let's continue the topic of dangerous type conversions. Take a look at the following code:

 BOOL WINAPI TrackPopupMenu( _In_ HMENU hMenu, _In_ UINT uFlags, _In_ int x, _In_ int y, _In_ int nReserved, _In_ HWND hWnd, _In_opt_ const RECT *prcRect ); struct JABBER_LIST_ITEM { .... }; INT_PTR CJabberDlgGcJoin::DlgProc(....) { .... int res = TrackPopupMenu( hMenu, TPM_RETURNCMD, rc.left, rc.bottom, 0, m_hwnd, NULL); .... if (res) { JABBER_LIST_ITEM *item = (JABBER_LIST_ITEM *)res; .... } .... } 

Analyzer warning: V204 Explicit conversion from 32-bit integer type to pointer type: (JABBER_LIST_ITEM *) res test.cpp 57

To begin with, it would be nice to take a look at the 'TrackPopupMenu' function used in this code. It returns the ID of the user-selected menu item, or a null value in case of an error or if there was no choice. The type 'BOOL' for these purposes is clearly chosen unsuccessfully, but what to do.

The result of executing this function, as can be seen from the code, is entered into the variable 'res'. In case if any element was selected by the user (res! = 0), then this variable is reduced to the type of the pointer to the structure. An interesting approach, but since in the article we are talking about 64-bit errors, let's think about how this code will be executed on 32 and 64-bit architectures, and what could be the problem?

The catch is that on a 32-bit architecture, such conversions are valid and feasible, since the types of 'pointer' and 'BOOL' are the same size. But the rake will make itself felt on the 64-bit architecture. In Win64 applications, the above types have different sizes (64 and 32 bits, respectively). The potential error is that the high bit values ​​in the pointer may be lost.

We continue the review. Code snippet:

 static int hash_void_ptr(void *ptr) { int hash; int i; hash = 0; for (i = 0; i < (int)sizeof(ptr) * 8 / TABLE_BITS; i++) { hash ^= (unsigned long)ptr >> i * 8; hash += i * 17; hash &= TABLE_MASK; } return hash; } 

Analyzer warning: V205 Explicit conversion of pointer type to 32-bit integer type: (unsigned long) ptr test.cpp 76

Let us see what is the problem of casting a variable of type 'void *' to type 'unsigned long' in this function. As already mentioned, these types have a different size in the LLP64 data model, where the type 'void *' is 64 bits, and 'unsigned long' is 32 bits. As a result, the high-order bits contained in the 'ptr' variable will be truncated (lost). The value of the variable 'i' increases as the iterations increase, and as a result, a bitwise shift to the right as the iterations pass will affect more and more bits. Since the size of the 'ptr' variable was truncated, with some iteration all the bits contained in it will be filled with 0. As a result of the above, on a Win64 application the 'hash' will be incorrectly compiled. Due to filling in 'hash' with zeros, collisions can occur, that is, getting the same hashes for different input data (in this case, pointers). As a result, this can lead to inefficient program operation. If the conversion to 'memsize-type' were performed, truncation would not occur, and then the shift (and therefore the compilation of the hash) would be carried out correctly.

Look at the following code:

 class CValueList : public CListCtrl { .... public: BOOL SortItems(_In_ PFNLVCOMPARE pfnCompare, _In_ DWORD_PTR dwData); .... }; void CLastValuesView::OnListViewColumnClick(....) { .... m_wndListCtrl.SortItems(CompareItems, (DWORD)this); .... } 

Analyzer warning: V220 Suspicious sequence of types of castings: memsize -> 32-bit integer -> memsize. The value being cast: 'this'. test.cpp 87

Diagnostics V220 signals a double dangerous data conversion. At the beginning, the variable 'memsize-type' turns into a 32-bit value, and then immediately expands back to 'memsize-type'. In fact, this means that the values ​​of the higher bits will be “cut off”. It is almost always a mistake.

Continue to reveal the topic of dangerous transformations:

 #define YAHOO_LOGINID "yahoo_id" DWORD_PTR __cdecl CYahooProto::GetCaps(int type, HANDLE /*hContact*/) { int ret = 0; switch (type) { .... case PFLAG_UNIQUEIDSETTING: ret = (DWORD_PTR)YAHOO_LOGINID; break; .... } return ret; } 

Analyzer warning: V221 Suspicious sequence of types of castings: pointer -> memsize -> 32-bit integer. The value being cast: '"yahoo_id"'. test.cpp 99

Noticed the trend that with each example of transformation becomes more and more. There are 3 of them. And 2 of them are dangerous, for the same reasons as all described above. Since 'YAHOO_LOGINID' is a string literal, its type is 'const char *', having the same size in the 64-bit architecture as the type of 'DWORD_PTR', so the explicit conversion is correct. But then the bad things begin. The type 'DWORD_PTR' is implicitly converted to an integer 32-bit. But that's not all. Since the result returned by the function is of the type 'DWORD_PTR', another implicit conversion will be performed, this time back to the 'memsize-type'. Obviously, in this case, the use of the returned value is at your own risk.

I want to note that the Visual Studio 2013 compiler issued the following warning:

warning C4244: '=': conversion from 'DWORD_PTR' to 'int', possible loss of data

Here the actual question will be possible: why is the warning issued by Visual Studio 2013 given only in this example? The question is fair, but have patience, this will be written below.

In the meantime, we will continue to consider errors Consider the following code containing a class hierarchy:

 class CWnd : public CCmdTarget { .... virtual void WinHelp(DWORD_PTR dwData, UINT nCmd = HELP_CONTEXT); .... }; class CFrameWnd : public CWnd { .... }; class CFrameWndEx : public CFrameWnd { .... virtual void WinHelp(DWORD dwData, UINT nCmd = HELP_CONTEXT); .... }; 

Analyzer warning: V301 Unexpected function overloading behavior. WinHelpA 'in derived class' CFrameWndEx' and base class' CWnd '. test.cpp 122

The example is interesting because it was taken from the report when checking the Visual C ++ 2012 libraries. As you can see, even the Visual C ++ developers make 64-bit errors.

Enough detail about this error is written in the corresponding article . Here I wanted to explain the essence briefly. On the 32-bit architecture, this code will be correctly processed, since the types 'DWORD' and 'DWORD_PTR' have the same size, in the heir class this function will be redefined, and the code will be executed correctly. But the pitfall has not gone away and will let you know about yourself on the 64-bit architecture. Since in this case the types 'DWORD' and 'DWORD_PTR' will have different sizes, polymorphism will be destroyed. We will have on hand 2 different functions, which goes against what was meant.

And the last example:

 void CSymEngine::GetMemInfo(CMemInfo& rMemInfo) { MEMORYSTATUS ms; GlobalMemoryStatus(&ms); _ultot_s(ms.dwMemoryLoad, rMemInfo.m_szMemoryLoad, countof(rMemInfo.m_szMemoryLoad), 10); .... } 

Analyzer Warning: V303 The function 'GlobalMemoryStatus' is deprecated in the Win64 system. It is a safer to use the 'GlobalMemoryStatusEx' function. test.cpp 130

In principle, no special explanation is required, everything is clear from the message of the analyzer. You must use the 'GlobalMemoryStatusEx' function, since the 'GlobalMemoryStatus' function may not work correctly on a 64-bit architecture. More information about this is written on the MSDN portal in the description of the corresponding function .

Note.

Please note that all the above errors can occur in the most ordinary application software. In order for them to occur, the program does not necessarily have to work with large amounts of memory. And that is why diagnostics that reveal these errors belong to the first level.

What will Visual Studio 2013 tell us?

Compiler warnings

Before telling about the results of testing the static analyzer of Visual Studio 2013, I would like to dwell on the compiler warnings. Attentive readers probably noticed that only 1 such warning was given in the text. What is the matter, you ask? But the fact is that there were simply no other warnings related to 64-bit errors. And this is at the 3rd level of issuing warnings.

But it is worth compiling this example with all included warnings (EnableAllWarnings), as we get ...



And, quite unexpectedly, warnings lead to header files (for example, winnt.h). If you do not be lazy and find in this heap of warnings those that relate to the project, you can still get something interesting, for example:

warning C4312: 'type cast': conversion from 'int' to 'JABBER_LIST_ITEM *' of greater size

warning C4311: 'type cast': pointer truncation from 'void *' to 'unsigned long'

warning C4311: 'type cast': pointer truncation from 'CLastValuesView * const' to 'DWORD'

warning C4263: 'void CFrameWndEx :: WinHelpA (DWORD, UINT)': member function

In general, the compiler issued 10 warnings in the file with these examples. Only 3 warnings from this list clearly indicate 64-bit errors (warnings of the C4311 compiler and C4312). Among these warnings are those that indicate a narrowing type conversion (C4244) or that the virtual function will not be redefined (C4263). These warnings also indirectly indicate 64-bit errors.

As a result, having eliminated one way or another repeating each other warnings, we will receive 5 warnings concerning the 64-bit errors we are considering.

As you can see, the Visual Studio compiler could not detect all 64-bit errors. I remind you that the PVS-Studio analyzer found 9 first-level errors in the same file.

“But what about the static analyzer built into Visual Studio 2013?” You ask. Maybe he coped better and found more errors? Let's get a look.

The static analyzer included in Visual Studio 2013

The result of testing these examples with a static analyzer embedded in Visual Studio 2013 is 3 warnings:

• C6255 Unprotected use of alloca
  _alloca indicates a stack overflow exception. Consider using _malloca instead.
  64BitsErrors - test.cpp (Line 58);
• C6384 Pointer size division
  Dividing sizeof a pointer by another value.
  64BitsErrors - test.cpp (Line 72);
• C28159 Consider using another function instead
  Consider using 'GlobalMemoryStatusEx' instead of 'GlobalMemoryStatus'. Reason: Deprecated. See MSDN for details
  64BitsErrors - test.cpp (Line 128);

But we are looking at 64-bit errors, right? How many errors from this list are related to 64-bit ones? Only the latter (using a function that may return incorrect results).

It turns out that the static analyzer Visual Studio 2013 found 1 64-bit error against 9 found by the PVS-Studio analyzer. Impressive, isn't it? Imagine the difference in large projects.

And now once again I want to remind you that in terms of error detection, static code analyzers embedded in the Visual Studio 2013 and Visual Studio 2015 environments are the same (for more information, see the corresponding note in more detail).

What is the result?

The most striking is to reflect the results of the verification of code examples in the form of a table.



As can be seen from the table, 9 64-bit errors were detected with the help of PVS-Studio, and 6. With the common means of Microsoft Visual Studio 2013 - 6. You may say that it is not such a big difference. I do not agree. Let's estimate why:

• We talked only about the most critical 64-bit errors. Even 3 missed mistakes, this is already a lot. And if we take the rarer errors for which the PVS-Studio analyzer generates warnings of the 2nd and 3rd levels, then it can find much more than Visual Studio. This article may give some insight. The article is a bit outdated, now the gap will be even greater.
• Calling for help a compiler with the 4th warning level turned on is not always possible. But both at the fourth and at the third level of warnings we will receive only 2 warnings (by means of the analyzer and the compiler) related to 64-bit errors. Not too impressive result.
• If you set the flag "/ Wall", we get a bunch of warnings that are not related to the project. In practice, using "/ Wall" will be difficult. You can separately include some warnings, but there will still be a lot of excess noise.

As can be seen from the above, in order to detect 64-bit errors found by means of Microsoft Visual Studio 2013, you need to do a certain amount of work. Now imagine how much it will increase, be it a real, really big project.

What is with PVS-Studio? We run diagnostics, set up filtering by 64-bit errors and necessary warnings with a few mouse clicks, we get the result.

Conclusion

I hope that I managed to show that transferring applications to the 64-bit architecture is associated with a number of difficulties. Errors similar to those described in this article are easy enough to make, but it is extremely difficult to find. Add to this the fact that not all such errors are detected by Microsoft Visual Studio 2013, and to find those you need to do a certain amount of work. At the same time, the PVS-Studio static analyzer coped with the task, showing a decent result. At the same time, the process of searching and filtering errors is easier and more convenient. Agree that in really large projects without such a tool would have been difficult, so in such cases a good static analyzer is simply necessary.

Developing a 64-bit application? Download the PVS-Studio trial, check your project and see how many 64-bit messages of the first level you will have. If a few still show up - please correct them, and make this world a little better.

Additional materials

As promised, I provide a list of additional materials on the topic of 64-bit errors:

• Terminology. 64-bit error ;
• Andrey Karpov. A collection of examples of 64-bit errors in real programs .
• Andrey Karpov. C ++ 11 and 64-bit errors .
• Andrey Karpov, Evgeny Ryzhkov. Lessons learned from developing 64-bit C / C ++ applications .

This article is in English.

If you want to share this article with an English-speaking audience, then please use the link to the translation: Sergey Vasiliev. 64-Bit Code in 2015: New in the Diagnostics of Possible Issues .