Testing installers in Windows when you need to quickly and cheaply

Happy web testers, take Selenium and you will not go wrong. Happy java-testers, for them there are test frameworks, in especially difficult cases, siculi. Console applications were brought to the tests - python, perl are nice here. What about the desktop? Testing of blackbox applications in windows, in particular, installers (for example, msi packages) led me to the autoit camp, inasmuch as my automation every time runs on the same rake, which I highlighted in the following

What you need from the installer autotest:

1. Click, instead of me, where necessary - in the Russian and English versions of the interface and so that without editing the source code. There are many different ways to do this, but I liked the Autoit way more - it’s enough to know the name of the object instance and you can send a push message to it, the only minus is you need to know the name of the window where our button is located, both in Russian and English.
2. Do not float with a bunch of the same code, for example, if I need to go through windows 1, 2 and 3 to press button 2 or button 3, I would like to combine pressing buttons in windows 1, 2, 3 into some kind of abstraction “path to buttons 2 and 3 "and call it as needed.
3. Reports about the debris, how far I had time to report, in general, what I did. Maybe even make screenshots before pressing the buttons?
   Hidden text

   Somehow then implement
4. Report on the result, if everything is done, Smok test that all the actions actually performed, and not that the buttons were pressed. For the installer - whether the files have appeared, whether the services are active, whether the interface is running.
5. I do not want to recompile and scatter new exe. I want to edit a crashed autotest right on the machine where it fell, the writing environment of the autotest should not change the environment - JAVA, .net, etc. Hundreds of cases - my tests went off with a bang, the client does not have - they forgot to attach any .net. Basically, for this reason, the options for using siculi- have fallen off, it is very expensive to roll back java, to check that everything did not work without it either.
6. A tester in an ordinary world full of pain and despair is not what is shown in the marketing smooth movie about tfs if it’s very lucky - you and the girl who carefully follow the test plan (I’m not lucky and the girl is absent). She doesn't need python, thousands of thousands of features from cpan. It is necessary - to remove the routine. Even as a bonus in perishable reality, testers will be half-betting students, cool, but as soon as a student masters perl, python, he will run away to coders and be lucky if he has time to complete the testing task that he has been given. So I came to the idea of ​​a minimum of opportunities for programming during testing.
7. The execution path is not the same on all operating systems, somewhere uac window appears, somewhere not, I do not want to bother with it every time. There must be some branching method when performing a test path.

Putting everything in the list, I came to the next idea - I need my clicker windows with settings in ini-files and reports .
Ok, we take into account the experience of generations, we will use the unix-way: we divide the task into two parts - the clicker and the report generator.
A clicker clicks on the objects specified in the simplest, preferably self-documented, text file - buttons, simulates keystrokes.

The report builder is a separate prog, written for each specific case, it can check services, create files in the right place, compare the output for the dir command, for example, with the standard and so on, if necessary. In most cases, it is not even necessary, because experience shows that it is constantly required to check the new functionality and have to watch it yourself.
')
An example of settings for a clicker: the ways.ini file, which contains unique test execution paths:

Actually now, having a description of what can be done with the installer, we will assemble a test path, describing it in another ini-file, testway1.ini.

If we need to check the button, we make another ini file, testway2.ini

countOps - the number of readable settings from the ways.ini - now there are 7 of them: LogMessage, WindowsNameEnglish, WindowsTextEnglish, Element, WindowsNameRussian, WindowsTextRussian, SendKeyText. sLogPath is the file name for messages, LogMessage. The rest is probably clear from the names. The Element property will have to be defined through the Au3Info.exe program.

Sample code that can work with such ini-files, clickAndType.au3:

The launch run requires a parameter — the name of the file to get the settings (testway2.ini, for example, which indicates what to execute from the ways.ini file).

How can all this economy achieve branching from point 7? It's very simple, when starting, the test hangs in the tray and waits for the specified window to appear, you can run several autotest programs in parallel, one of which, for example, will respond to the window requiring you to enter a password, enter it and the initial test will continue its execution as if nothing happened.
About the second prog - report generator, I probably will not write, it’s all very individual there.

Ok, it can click for me, what's next, Brain? It is necessary to get rid of the vicious practice of launching an autotest with your own hands, a good idea is to prepare special virtuals, make a BeforeInstallXXX snapshot that will contain a batch file of the form, runme.bat, in autorun:

It remains only to place the installer under test in the file sphere and reboot the virtual machines. If you are lucky enough, you can also place the autotest on the file balloon, then all changes will be centralized and you will save time on copying testway.ini, ways.ini configuration files to each virtual machine.

As your requests grow, most likely you will come to the conclusion that the runme.bat functionality needs to be complicated, as far as I can tell, soon there will be a third program to run, which will replace runme.bat, and, apparently, will know how Some conditions choose the option to start clickAndType.exe

9 out of 10 I put on the fact that everything described above is a bicycle that was invented 1000 times already 1000 different people and in fact everything rested on the fact that they all had a fatal flaw — they were not written by me, the bonus was written by programmers who, in spurts maximum versatility often spawned monsters, and I, being the main customer, and the user was interested in the simplicity of the tool and implemented it. At the same time I will say that I was not lucky to find any information on testing installers in windows with simple googling. Now she will be here.