Web services play poker

Hello, Habr.

I am very attracted to sports programming, and I also love poker. So I decided to kill two birds with one stone by running the poker bots tournament.

Unlike other similar events, the participant's program can be implemented as a web service with which the croupier will communicate via HTTP.

The original idea of ​​the project belongs to my friend (he is not from Habr). The first time it sounded like this:

I believe that there is nothing wrong with bots playing in poker rooms. Although, maybe, it would be more honest to organize a special poker club where robots would play only with each other. So I would love to write such a bot. And you?

I do not remember what I said. It was more interesting for me to make such a service. Challenge accepted;)
')

Engine

The amusing picture just above indicates the main components of the system and their connections with each other. A separate comment is required for the part related to the poker engine. I could not find a suitable ready-made engine, so I wrote a small (~ 1000 lines) python-package that implements the logic of the poker croupier and communication with bots. A simple unix-style interface is added to the package for generating poker hands and conducting them directly.

The engine "does not know anything" about the logic of the tournament, the user database, etc. He acts exclusively in the role of "croupier" and returns the result of his work in the form of XML containing the final state of the distribution.

This implementation makes it possible to use the same engine both inside the poker room and on the user's computer during the development / testing of the bot (what I call the SDK). This ensures that the bot will be in the same conditions on the user's test lab and during a real tournament game, as well as reducing code duplication.

The source code of the engine is published on GitHab (MIT license). For use in the form of an SDK for Windows, an exe-file with python-interpreter enabled is built (using py2exe ). Under Linux and Mac, the SDK comes in source and uses the interpreter installed (by default) in the operating system.

In the current implementation, the engine supports only one version of poker - “Texas hold'em” with fixed bets. Judging by Wikipedia , this is one of the most popular poker games in the world.

I did not do the SDK as a web service so that the user could run the system completely locally. Thus, you can write hundreds of regression tests on the bot and run them very quickly, even after each change. Moreover, to implement such a scenario, you do not need to patch the sources or write your own interface to the engine - everything works out of the "by design" box.

The tests themselves are written easily and pleasantly thanks to the stateless API.

API

If I have a choice, I almost always make a stateless API. In this case, this choice was not so obvious, because It is necessary to transfer the bot's entire distribution history to the bot (this is the minimum necessary information for making a decision).

Currently, this information (I call it the “state” of distribution) is transmitted to the bot in XML format using an HTTP POST request. In the worst case, this XML can weigh up to 4K. It’s a lot or a little to judge you, but this is the price I’m willing to pay to simplify the bot development process and lower the entry threshold for new members.

Development is really quite simple - the participant simply needs to write a program that parses some XML, analyzes the situation described in it and makes a certain decision (pass, support, increase, go all-in, etc.).

The XML document schema had to be developed independently.

Example

<?xml version="1.0"?> <game> <table button="2"> <player sit="0" name="vbo" stack="80" in_stack="100"/> <player sit="2" name="lenny" stack="80" in_stack="100"/> <player sit="5" name="psycho" stack="90" in_stack="100"/> </table> <posts> <post amount="10" player="psycho" type="small_blind"/> <post amount="20" player="vbo" type="big_blind"/> </posts> <betting> <round name="preflop"> <action amount="20" player="lenny" type="call"/> <action amount="10" player="psycho" type="fold"/> <action amount="20" player="vbo" type="check"/> </round> <round name="flop"> <action amount="0" player="vbo" type="check"/> <action amount="0" player="lenny" type="check"/> </round> <round name="turn"/> <round name="river"/> </betting> <community> <card rank="Q" suit="H"/> <card rank="A" suit="H"/> <card rank="7" suit="S"/> <card rank="T" suit="S"/> </community> </game> 

This scheme seems to me to be quite simple for a person to understand (including due to easy redundancy, for example, // player / @ stack) and a convenient robot for analysis. In addition, I plan in the near future to make a JSON version of the scheme - as you like.

For the internal presentation of the distribution engine uses an extended version of the scheme, where there is a deck, for each bot its transport is specified (URL, etc.), pocket cards, etc. The final state of distribution also includes the showdown node, which makes it clear which player won and which cards he had (if he had to open up). In the current implementation, only the bot author sees the final distribution state when he views the game statistics on the poker room site.

All other information is transmitted (in the case of HTTP transport) in separate fields of the POST request. This is done so that you can write a simple bot without parsing XML at all. Moreover, such an implementation makes writing tests more convenient - a test bot may not parse XML at all and act, for example, randomly or following a simple algorithm (in the spirit of "check / call any" or "bet / raise good cards").

For this, the “croupier” also gives the bot a list of actions that it is possible to perform in this situation. If a bot performs an action that is not in this list, its response is considered as fold, and the response text (usually, this is a trace trace of exceptions) is recorded in the logs on the side of the poker room. At the moment, I myself am reading these logs and writing to the participant an email if the bot breaks. Of course, in the very near future this process will be automated;)

Tournament

In sports programming contests, the following always upsets me:

• They do not start on time.
  For example, last weekend before launching a large project at work.
• They end at the most interesting moment.
  For example, when I came up with a brilliant idea that would make my program well, quite amazing.

Therefore, from the very beginning I decided to make the main contest “endless”. Imagine a real online poker club for people. You can come there at any time, throw some money, sit at the table and have a great time. You can improve your skills and sit down at tables with higher stakes, or you can just have fun playing “candy wrappers” in your free time.

Of course, I wouldn’t do something like that right away, only for robots, I wouldn’t - it would be very disappointing, for example, after six months of development, to realize that the service has a very high entry threshold and I play there in splendid isolation.

Therefore, for starters, I launched something like a sandbox - a poker bots tournament that I can join at any time and get an idea of ​​the level of my program compared to other participants.

At the beginning of each day, the poker room makes a rating of the participants of the tournament, based on the amount of "money" that they managed to win (or had to lose) for the previous day. After that, the balance of each user is set to zero and the competition starts from the beginning. As part of the competition, the poker room tries to ensure that all the bots are constantly playing, and, if possible, with various rivals. In the current implementation, the following rules are used for this:

• If the bot is free, it immediately sits down at the first available table. At the same time, priority is given to tables at which the bot has long (or never) sat at the tables with the least number of players.
• The bot gets up from the table as soon as it loses or plays more than N hands with it. The last rule applies to equal rivals not playing with each other ad infinitum.
• The poker room supports the "optimal" number of open tables - so that everyone has enough and there are still places where you can change after the fakap. This number is linearly dependent on the number of active players.

Algorithms that implement these rules (in the diagram this is the “Tournament worker”) work in parallel, asynchronously and completely independently. This ensures uniformity of the load on the machine and the constancy of the number of hands per unit of time. Also, this architecture makes the system more scalable.

Workers, like the site, are implemented in PHP (I have more experience here than in python) and work with the PostgreSQL database. To ensure parallel processing of the queue of tables, pg_try_advisory_lock is used when selecting candidates. I can not vouch that this method is the most correct, t.ch. I will be glad to discuss this issue in the comments.

Plans

I still do not see the point of putting the source of the poker room in open access - the code turned out not too complicated and too tied to the project. Instead, I will focus on improving the “engine” (for example, adding support for no-limit poker), implementing the API for “learning” bots (i.e., to get the results of past games), as well as developing a new type of game in which participants separate API will be able to manage seating at the tables and leave the table as needed. In general, be able to behave almost like people.

Also, I plan to reduce the threshold for entry for beginners (or simply far from the web) programmers - to give the opportunity to develop a bot directly in a browser in a language like CoffeeScript and test it through a special web interface. It would be great to make some visual editor of the algorithm. Of course, he does not have to give absolute opportunities, but this will allow even non-programmers to use the service, which is very useful. At the moment I have no idea how to make such an editor, t.ch. If you have them, please post in the comments.

The project is done "just for fun", t.ch. The deadlines for the implementation of all these “cows” to publish is useless. Judging by the git log, I started doing the current implementation in early February. From this we can draw conclusions about my speed.

If the reader is interested in playing, the beta version of the poker room works here . And on youtube, you can watch a short video on how to make a simple PHP bot, test it and register it in a tournament.

UPD [05.31.2013] The project was frozen a few months after the post was published. The reason is that in spite of the positive feedback from the community, only a few dozen real users have appeared at the service, and they very quickly stopped updating their solution. If you would be interested to resume the project, write to me in the LAN, I would be happy to give it in good hands.