Application Verifier for a Programmer: Testing Windows Applications

Perhaps in your project and do not write try { /* code */ } catch (...) { } in order to avoid exceptions when working with memory, they know how to close handles and know about the virtualization of Windows Vista, and programs never fall on incomprehensible and rarely repeated reasons.

Then you are lucky, you can proceed to the next topic.

But sometimes it seems that strange things happen. The program "falls" out of the blue, the memory is leaking somewhere, and once again they called you with a complaint about the strange behavior of the program running on the 24/7 server, but you of course "wrapped" their problem, making sure that it was hardware dependent and alright Still, the development of programs for Windows is often a tricky business, and no one is immune from errors due to carelessness or because of ignorance of architecture. I will not learn how to prevent these errors - I do not know. But here is one tool for effective debugging I can advise.

It's about Microsoft Application Verifier. But this is not a debugger. On the contrary, without a debugger, by itself, the thing is relatively useless. But in conjunction with it allows you to detect a number of important platform-dependent problems. In addition, it will not be possible to obtain a “Compatible with windows 7” certificate without testing using AppVerifier (for Vista Certified itself, it is not the same, but apparently this is not accepted). And this certificate is for the user some guarantee that the program that received it may not do better, but at least it will not harm . Okay, the "water" is over, let's get down to business.
')

Mode of application

Download and install AppVerifier for Hubman, I'm sure not difficult. Let's run (from under the real-administrator, under Vista + it won't work differently) its graphical shell:



On the left is a list of applications for testing; on the right there is a list of sections to check for the selected application. The MSDN claims that AppVerifier is designed for testing programs in C ++, but generally applicable to any native code.

The graphical shell does not produce any tests, only allows you to select the desired items. The checks themselves are implemented thanks to the so-called "layers", dynamically connected libraries vfbasics, vfcompat, vfLuaPriv, vfprint (you can admire them in the system32 ). When you run the application under test, they connect to it and intercept the call to system functions such as HeapAlloc, GetTickCount, CloseHandle and many others. The interceptor performs a number of additional checks, then calls the original function, and therefore, with the exception of a few cases considered further , this will not affect the operation of the application under test. Is that some loss of performance will be noticeable. Subjectively, in the worst case, the program will “slow down” five times, and whether you need any specific numbers or not - I will leave it to your discretion.

There is an important feature here : in spite of the fact that when we add, we select the file of the application under test, the checks are tied only to its name without a path . On the one hand, you do not have to worry in which configuration (and in which folder) to build the project (usually the folders for Debug and Release are different), but on the other, you can forget about the installed checks, and launching the program from the desktop, wonder if it is “ does not work".

We will talk about the meaning of the test points a little later, but now we will add, for example, notepad.exe and install all the jackdaws. Run the notebook, add a couple of lines, try to save. Oh, bad luck:



Not the only outcome of the situation, perhaps you will receive another warning window, or even do without it. What happened? Turn again to the graphical add-in AppVerifier. This time we will select the Logs item from the main menu, we will see a list of log files associated with the applications under test. According to the launch log.



Physically, these log files are located in the AppVerifierLogs folder in the root of the user profile. It will be difficult to read them with bare hands (binary format), so poking the “View” button for the corresponding log. It will dump it in xml and open the default viewer for xml:



For those who closely watched: the error shown in this screenshot does not correspond to the error message (which is the normal behavior of the program) from the previous screenshot, but happens a little later.

Here and a brief description of the problem, and trace. And from me a hint how to look for errors, not warnings. By the way, if errors are present, the program does not receive certification for compatibility with Vista / Win7. Wait, but this is a notebook ?! Well, yes, just shhh.

Patient treatment

Now run the debugger. Let it be a debugger built into the studio, or free WinDbg from the Debugging Tools for Windows (it’s certainly more sophisticated, but now it doesn’t matter).

And here is our patient:

int _tmain( int argc, _TCHAR* argv[])
{
int *p = new int ();
delete p;
*p = 0; // p = 0 will be OK, but *p = 0 is error!
}

The potential danger of this fragment is easy to assess if the lines with delete and overwriting memory were stretched over time. But neither in the debug, nor in the release assembly such a problem is not detected (Visual Studio, the default configuration).

Now add a program to test the Basics group in the Application Verifier. And run it from under the debugger (from the F5 studio, for example). AppVerifier spoke to us in the voice of the studio:



And in Debug Output the corresponding structural exception is shown:

=======================================
VERIFIER STOP 00000013: pid 0xB54: First chance access violation for current stack trace.

02B59FF8 : Invalid address causing the exception.
0082142F : Code address executing the invalid access.
0013F670 : Exception record.
0013F68C : Context record.

=======================================


It says that with the exception of (00000013), with which memory address (02B59FF8) and at what address of the code (0082142F) occurred. To the lucky ones who downloaded the Windows Debug Symbols, they will show the place in the source code where the problem occurred and Stack Trace, which led to the exception.

Well, we found this problem, which means we fixed it. For other classes of errors, the algorithm is preserved, but the correction procedure may not be so trivial.

Detectable problems

Let's now figure out what problems will allow us to reveal AppVerifier. All testing options are divided into groups. Excluding the group “Low Resource Simulation” and the tests “TimeRollOver” and “HighVersionLie” checks do not change the behavior of the application (if no errors are detected).

1. Distorting checks

1.1. Low Resource Simulation

Here it is the reason for the fall of the notebook. Tests of this group allow you to simulate the behavior of the system when there is a shortage of resources. An application can easily be refused (by a random number generator) in allocating memory, creating a file, Event, windows, or writing to the registry. Usually there is some “quiet” time around 2-5 seconds, when the application is allowed to use the resources at full strength; This is done so that the application can start at all (it was invented not so long ago, it was sadder before). The normal behavior of the program is stability; showing warning dialogs, but not “crashes”. So in the code you need to provide for these situations.

1.2. TimeRollOver in the Misc group

Consider the following sample code that executes an action several times, but no more than one second:

DWORD time_end = GetTickCount() + 1000; // 1s timelimit
do { action(); } while (GetTickCount() < time_end);

The trick is visible to the naked eye; if time_end very close to DWORD_MAX , but less than DWORD_MAX-1000 , and action() sometimes takes more than a second, then the cycle will work a little longer than we would like. Namely, 50 days (DWORD_MAX / (1000 * 60 * 24)).

And this is not the only case that you say about the next fragment?

char buf[8];
sprintf(buf, "%i" , GetTickCount());

To diagnose such problems, checking TimeRollOver “runs” the value of the GetTickCount () function faster. A full cycle before resetting the value takes 5 minutes.

1.3. HighVersionLie in the Compatibility group

If suddenly you use the GetVersionEx function, then this test will help you to find the code branches with incorrect verification of a valid OS version.

OSVERSIONINFO osvi;
ZeroMemory(&osvi, sizeof (OSVERSIONINFO));
osvi.dwOSVersionInfoSize = sizeof (OSVERSIONINFO);
GetVersionEx(&osvi);

BOOL bIsWindowsXP_or_Later = (osvi.dwMajorVersion >= 5) && (osvi.dwMinorVersion >= 1);
if (!bIsWindowsXP_or_Later)
printf( "Windows XP or later required.\n" );



In this fragment, a clear error is made; in order to cut off Windows 2000 (5.0), an additional check is introduced on the minor version of XP (5.1), but the code also discards Windows Vista (6.0). On Windows 7 (6.1) will work. Is this really the reason for the poor compatibility with Windows Vista? Microsoft claims that 70% of incompatible programs with Vista do not work, including because of this problem .

But diagnosing such a situation on a developer's computer is difficult - he has one, fixed OS version. You can use a virtual machine with a different OS version, or you can just poke the HighVersionLie checkbox. Then the value of GetVersionEx will be modified (usually by the rule dwMajorVersion += 3; dwMinorVersion = 0 ).

2. Non-modifying checks

2.1. Memory in the Basics group

Validation of calls to HeapAlloc, GlobalAlloc and other Windows Heap Manager APIs. It does not monitor memory leaks, but it can be solved in other ways .

2.2. TLS in the Basics group

Monitors the correctness of calls to the Local Storage API.

2.3. Exceptions in the Basics group

Monitors the appropriateness of interception of exceptions, in particular, attempts to “silence” exceptions Access Violation, “unmask” exceptions in plugs like try { } catch (...) { } .

2.4. Handles in the Basics group

Monitors the admissibility of operations on handles, the correctness of handles and their lifetime. A little more in English .

2.5. Locks in the Basics group

Checks the correctness of the use of critical sections, prevents the critical section from being dumped from another stream relative to the installation of the critical section.

2.6. DirtyStacks in the Misc group

Periodically fills the unused part of the stack with the 0xCD pattern, which allows detecting uninitialized variables or function parameters.

2.7. DangerousAPIs in the Misc group

Notifies of the use and undesirable potentially dangerous API functions like TerminateThread .

2.8. Luapriv

Limited-user-account privileges test. Checks if a program needs administrative privileges, does not perform a program of actions that are allowed only for a real administrator.

It consists of two parts: predictive (lists all the actions of the program that only the administrator can perform) and diagnostic (refuses the program in administrative actions with the error ACCESS_DENIED ). Thus, it is not necessary for the programmer to test the program by logging in separately to the guest. It also checks a number of features related to virtualization under Windows Vista and later.

Conclusion

AppVerifier is an interesting tool that allows you to identify and solve a number of "floating" and "hidden" (and sometimes specially hidden) problems. To use it as a whole is not difficult, with certain skills it is convenient. And if you want to get a certificate "Windows compatible", then acquaintance with him can not be avoided. I personally have already helped me on two projects, I hope it will be useful for you too.

* All source code was highlighted with Source Code Highlighter .