Dynamic (non-linear) GUI testing

What?

Perform actions on GUI elements in random order.

What is it for?

The person performing the testing is Homo sapiens, i.e. he has some intelligence. This very intellect prevents (very rarely, but prevents) applications to find “absurdities of behavior” associated with unforeseen situations. He simply cannot imagine such an illogical situation.
The user is much higher than QA in quantity and may be significantly inferior to him in IQ. Hence, the probability of unexpected user behavior is by no means extremely small.
So, what to us, possessing free resources and desire, prevents to take measures on prevention of similar situations? - Nothing.
Now we will formulate specific tasks in which “meaningless clicking” on buttons can be useful:

• Supplement existing application stability testing by introducing a non-linear user behavior model into the GUI.
• Investigate the consumption of resources for all possible variants of the application (initiated from the GUI).

First, the test variant described in this article should be used, indeed, only as an addition to the existing GUI testing. To rely only on the chaotic "clatter" on the buttons is at least silly. There is no check what exactly is happening and whether it is happening at all. Therefore, the first option can be considered as additional negative stability testing.
The second is the most effective on an infinite length of time, which is often impossible. Therefore, choosing the measurement period, one should proceed from the complexity of the application, its type and purpose. For example, it probably makes no sense to drive a “non-server” application for 24 hours, consisting of two buttons and one check box, which multiplies something by two, and then the result is halved.

How do we do?

The following description is intended for testing applications on the Windows platform.
I suggest to use the python + pywinauto bundle. Although pywinauto has some limitations in terms of access to window elements, for most cases this should be enough.
Frankly, I do not see an alternative. All the GUI testing automation tools I know do not have the dynamism shown below - already during the execution of the test, we receive a list of controls, determine their type and perform a valid action.
Also, do not underestimate the capabilities of the Python itself and its modules. Here you can remove the video, measure the CPU and send the message to the right place, in which case send it

What do we need?

• Python 2.7 http://python.org/getit/
• pywinauto http://sourceforge.net/projects/pywinauto/
• virtual machine (to taste)

I also recommend using the SWAPY utility, with its help it is convenient to look at the properties of the controls, it also generates code for pywinauto. It can also be used to check whether pywinauto controls for your application or not.
')

Test specification

1. Launch the application window.
2. Click on the available control (close the window).
3. Check the fail criteria.
4. Repeat steps 1 - 3 for the specified time.
5. At the end of this time, consider the test passed.

Fail criteria - the condition under which the test is considered failed. For example, if the Crash report window is running, the Internet is not pinged, etc. The test is also considered to be failed in any event (unforeseen situation).

Underwater rocks

• Sometimes the window needs to be closed.
  Example: a child window does not have a button that closes it and returns focus to the parent window.
• The window can not process Close - do not fail the test.
  Example: a dialog box with OK and Cancel buttons only.
• Pywinauto may not see the controls on the window.
  Here we will not do anything - we are looking for another bundle. Check through SWAPY or through

  pywinauto.application.WindowSpecification.PrintControlIdentifiers 

Code

According to the specification:

1. Run the binary, wait for the main window to appear:
   

    pywinauto.application.Application().start_(binary_path) pywinauto.timings.WaitUntil(WAIT_TIMEOUT, CHECK_INTERVAL, _check_window) 

   
   
2. Using enabled_and_visible() we get a list of available controls. Randomly select which element to click or close the window:
   

    if ready_contr_list and random.randint(0,len(ready_contr_list)): control = random.choice(ready_contr_list) print('Click on - "%s"' % control.Texts()[0].encode('unicode-escape', 'replace')) highlight_control(control) control.Click() else: try: window.Close() except: pass else: print('Close window') 

   
   
3. Fail criteria. Nothing came to mind. Put the stub:
   

    if 1==0: print('') result = TEST_FAILED break 

   
   

Full text below. I would like to note a few points:

1. make_action is only able to send a single left click signal to a control (or close the window). If the topic is interesting, it will be possible to complicate the logic.
2. highlight_control highlights the active control. Just beautiful.
3. To run the script you will need to specify:
   
   • The path to the executable file. You can with the parameters:
     BINARY_PATH = r'"C:\path\app.exe" –params 1 2 3'
   • Regular for the header of the main window:
     TITLE_RE = 'My app - .*'
   • The class of the main window. We look in SWAPY :
     CLASS_NAME = '#32770'
   
   

Result

Tested a native Windows RDP client.
Literally in an hour, I managed to catch the crash of the RDP client. Success? - Maybe. Manually failed to repeat.
However, no one has yet canceled the dumps, so an autopsy will show ...

The full text of the script:

 import pywinauto import random import thread import time import sys ''' GUI dynamic testing ''' TEST_FAILED = 1 TEST_PASSED = 0 TEST_EXEC_TIME = 60 * 60 WAIT_TIMEOUT = 30 CHECK_INTERVAL = 0.2 BINARY_PATH = r'"C:\WINDOWS\system32\mstsc.exe"' TITLE_RE = 'Remote Desktop Connection' CLASS_NAME = '#32770' def _check_window(): ''' Check window is opened ''' try: pywinauto.findwindows.find_windows(title_re=TITLE_RE, class_name=CLASS_NAME)[0] except: return False else: return True def start_binary(binary_path): ''' Start a binary, wait for window opens ''' if not _check_window(): pywinauto.application.Application().start_(binary_path) pywinauto.timings.WaitUntil(WAIT_TIMEOUT, CHECK_INTERVAL, _check_window) return 0 def get_top_window(title_re, class_name): ''' Return the top window of the binary ''' if not _check_window(): start_binary(BINARY_PATH) app = pywinauto.application.Application() try: app.Connect_(title_re=TITLE_RE, class_name=CLASS_NAME) except pywinauto.findwindows.WindowAmbiguousError: app.Connect_(title_re=TITLE_RE, class_name=CLASS_NAME, active_only=True) return app.top_window_() def enabled_and_visible(all_conrt_list): ''' Return list of ready for action controls ''' ready_contr_list = [] for contr in all_conrt_list: if contr.IsEnabled() and contr.IsVisible(): ready_contr_list.append(contr) return ready_contr_list def highlight_control(control): ''' Highlight control ''' def _highlight_control(control, repeat = 1): while repeat > 0: repeat -= 1 control.DrawOutline(thickness=1) time.sleep(0.7) control.DrawOutline(colour=0xffffff, thickness=1) time.sleep(0.4) thread.start_new_thread(_highlight_control,(control,3)) return 0 def make_action(window): ''' Make action on a control or close a window ''' all_conrt_list = window.Children() ready_contr_list = enabled_and_visible(all_conrt_list) if ready_contr_list and random.randint(0,len(ready_contr_list)): control = random.choice(ready_contr_list) print('Click on - "%s"' % control.Texts()[0].encode('unicode-escape', 'replace')) highlight_control(control) control.Click() else: try: window.Close() except: pass else: print('Close window') def main(): ''' main section ''' start_time = time.time() result = -1 try: #start testig build start_binary(BINARY_PATH) #testing cycle while (time.time() - start_time) < TEST_EXEC_TIME: #get top window window = get_top_window(TITLE_RE, CLASS_NAME) #make an action make_action(window) #check fail criteria if 1==0: print('') result = TEST_FAILED break else: result = TEST_PASSED print('Test passed') except Exception, e: result = TEST_FAILED print('Test failed.\n Exception %s' % e) sys.exit(result) if __name__ == '__main__': main()