The effectiveness of automatic application testing

Attack of the clones.
Episode: Poker.

At one command, without delay, several thousand clones rushed through narrow network corridors, not knowing any doubts and fear of defeat! In order to come together in a duel and win! The clones, most of them not masters, but programmed to win, just clearly follow the instructions that, if lucky, lead them to the goal. The rules for all are alone, every man for himself, but there is not a single chance to stand up and defeat the person who finds himself in this mess ...

This is not an episode of the legendary Star Wars saga and not a preview of the fantastic story. This is a description of the load testing server (built on Java technologies), conducted during the development of the gaming application "Poker" for social networks.
')
Load testing was preceded by the implementation of the main server functionality: command set, network interface (Apache MINA), game engine and database (Hibernate, MySQL). The development process used continuous integration (Hudson continuous integration server), unit testing (JUnit), integration testing of the database, load testing of the network interface and application logic. In conditions when the flash client of the game was in the early stages of development, the task was to carry out the integration and load testing of the game server. An instruction algorithm for poker bots was described, where the clone's behavior depended on the current state of the game situation, a set of cards. Given that there was no goal to create an automatic player with high skill, the algorithm was simple to implement, but effective. This allowed testing the game server by means of automatic players along with real players. And clones often took victory from people in numbers, not in skill.

As noted above, during the development, unit testing was used to verify the functionality of individual classes. Not without it and in the development of the “brain” of the engine - the calculation of the poker “hand” (determining the winning combination of cards during the hands of cards). For each combination tests were written, allowing them to check. Poker rules determine the following winning combinations:

• High card;
• A pair (two cards of the same rank);
• Two pairs (two pairs of the same rank and two more cards of the same rank);
• Set (three cards of the same rank);
• Straight (a sequence of five cards of different suits);
• Flash (any five cards of the same suit);
• Full house (three cards of the same rank and two cards of the same rank);
• Care (four cards of the same denomination);
• Straight Flush (a sequence of five cards of the same suit);
• Flash piano (five high cards of any one suit).

Thus, there are 10 winning card combinations and in the simple case, a solution can be found for a maximum of 10 calculation cycles. But even a novice poker player will notice that some of the game combinations have common properties, for example, a straight-flush has the property of a straight (a sequence of five cards ...) and a flush (... of the same suit). Thus, the number of calculation cycles was reduced to 4. And we had a set of unit tests for each combination ... but there was no confidence in the calculation. This problem was solved by writing a test that used statistics on poker combinations (probability of a poker combination):

================================================= ==============
| Winning combinations test
================================================= ==============
| HIGH_CARD | 17.497% | 34994 |
| ONE_PAIR | 43.847% | 87694 |
| TWO_PAIR | 23.4185% | 46837 |
| SET | 4.8045% | 9609 |
| STRAIGHT | 4.653% | 9306 |
| FLUSH | 2.989% | 5978 |
| FULL_HOUSE | 2.581% | 5162 |
| QUADS | 0.175% | 350 |
| STRAIGHT_FLUSH | 0.033% | 66 |
| ROYAL_FLUSH | 0.0020% | 4 |
================================================= ==============
handsNumber: 200,000

In this test, 200,000 random combinations are calculated, the number of their occurrences is presented in percentage and in numerical (for example, the set fell in test 9609 in ~ 4.81% of cases). The discrepancy between the results of this test and the statistical data indicated an error resulting from the fact that they did not initially take into account a number of special rules in combinations that experienced players are well known.

Thus, the effectiveness of automatic testing (modular, integration, load) of the developed applications was once again confirmed.