The history of one poker bot
Now the possibility of writing a poker bot is no longer in doubt (there is even a series of articles in the Hacker magazine), but in 2008 everything was not so obvious and all that could be found was a single article in English that described only general ideas never got accustomed to our project. But even now, few people will share successful stories.
In this article I will describe our experience in developing a bot that was actually tested in real poker rooms and even learned how to play and not lose. Those who are going to write their bot will find interesting things in the article and ways to make their lives easier. Who cares - welcome under cat.
The idea of writing a bot visited my friend who shared it with me and with my neighbor in the hostel, and the friend himself was also with the hostel.
The roles were divided by the following plan:
• Logic Designer (idea author)
• Parser Developer (Neighbor)
• Developer of eyes and hands (i)
It was decided to write a bot for limit hold'em (the author of the idea at that time was earning some money playing this type of game)
In such a composition and with such roles, we moved into battle. As a result, we got 3 iterations of development with breaks between them in 3-4 months, which I will try not to forget anything to tell in more detail.
What a time it was. Three green students began to think. We needed tools, initially the choice was this:
• C ++ - game logic
• VB.Net - Everything Else
But after a few hours of development, we decided to teach C # and began using it instead of BASIC.
Unfortunately, I can’t tell anything about the brains that were in the DLL in C ++, and no one can (just no one remembers), I only remember that there was a hell in it, in which even the author was confused and sometimes surprised that something accidentally works / does not work.
The poker room we chose for testing had excellent chat rooms from which it was decided to take information to analyze what was happening. My neighbor took up this analysis.
I needed to learn how to give him a chat on time, it remained to understand when it comes on time.
So, the first questions that needed to be resolved were:
• How to find among all windows of system necessary?
• How to understand that it's time to go?
• How to walk?
• How to get a chat?
More detail about solving problems below (there is a feeling that it was in that order that I then solved them).
I then heard something about WinApi, but what kind of animal it is and how to teach it - I was not in the know. Very soon, I learned that by a mysterious handle you can learn a lot about the window and do a lot of things, I also learned that you can watch the window well. Armed with Spy ++, I plunged into the world of the then simple recognition of the handle by pointing a cross on the window)))
Smoking allowed me to find out what you can ask the color of the dot on the desktop and what the system can say that the mouse clicked the button.
The solution of the problem became obvious:
1. look for points by the color of which you can find out what buttons appeared (i.e. It's time to walk)
2. get the coordinates of the window
3. add to the coordinates of the point to determine
4. constantly make the window active and check the color of the point
5. If the color has become necessary - it's time to go
')
Already knowing how to use Spy ++, it was easy to find the handle. Having smoked again, I understood how to send Ctrl + A, Ctrl + C to the window. Further by means of C # received the contents of the buffer. Chat is.
While it was determined how to understand that it was time to walk - it became clear how to walk. Calculate the coordinates of the button, put the cursor on the dot, Send the left mouse click to the system. It was like.
This task turned out to be the most difficult - I couldn’t solve it for a long time until I turned to my teacher (thanks to Chernosvitov AV) for C # / C ++ and he remembered the EnumWindowsProc function or something like that. Using Spy ++, I looked for the window's features, but in the end this feature turned out to be a header, which by the way can also be obtained from the window handle, after which using EnumChildWindowsProc (enumeration of all child windows) was a chat.
After the first iteration, we had a bot who played limit hold'em, knew how to understand that it was time to walk, knew how to walk. He had some logic. He could only play alone and completely blocked the computer (due to constant switching to the window before each action) on which he was tested.
A few months later we decided to write again, to speed up the process, we decided to take the computers into one room and work together there.
During this iteration, it was carried out:
• Moving to one language - we moved completely to C #.
• Transition to anlim tournaments
• Many tables tied
• Tried using system hooks to determine if it's time to go.
• Improved chat reception
• Improved progress mechanism
The logic was rewritten in C # and the starting tables for playing tournaments from pokerstrategy.com were used, which made it possible to make reasonably clean and clear logic. There were also attempts to make a logic configurator (we wanted to sell the bot to the side so that everyone would write their own logic), but we decided that this was utopia and stopped the development of the idea.
Along with the new transparent logic, the possibility of playing on several tables was added. Realization of such a possibility was a curve - each table in turn to the fore, checking the color for the need for a move - if it's time to go, then we go. However, this scheme worked, and it was a victory. But now the bot blocked the computer completely and tightly.
By this time, I was already strong enough to start digging with Windows system hooks. Since C # does not hang system hooks on the arrival of messages to the window - a project with CodeProject (http://www.codeproject.com/KB/system/WilsonSystemGlobalHooks.aspx) was used, which consists of a DLL and C # logic that runs this DLL and receiving information from her. Hooks allowed not to do a bunch of switching between windows to determine the progress, but to catch the exact order of messages, after which it was necessary to go, did not work, plus the hooks slowed down the system and eventually refused to hooks.
During another sleepless night, I watched in Spy ++ the commands that come to the chat, and I saw that certain commands corresponded to the selection and copying of everything. The code of these commands was recorded, the chat copy was rewritten to use the new commands, then the copying was done completely using WinApi commands.
It turned out that you can send a message to the window, that you clicked on it with the mouse - this allowed us to rewrite the walk to send messages directly to the window about clicks. Now the cursor could no longer twitch across the screen. The computer is gradually returning from the bot to the owner =)
After this iteration, a bot played on many tables in tournament hold'em with transparent logic (by the way, the table proposed at PokerStrategy.com is a loser if you follow it completely and not think more). The bot could already leave a small chance to somehow work the host computer.
By the third iteration, two of us reached, my neighbor dropped out of the project and his part of the code was rewritten from scratch (it was difficult to maintain someone else's code, especially since it was necessary to significantly develop it). During the third iteration we reached:
• Using a new mechanism for determining the time course
• Using the new chat mechanism
• Using new logic
I still wonder why I didn’t guess earlier that you can determine the color of a point not only on the desktop, but also in the window specifically. Having understood this, I rewrote the definitions of the move in a new way. Now it was not necessary to constantly push the windows to the fore in order to understand that it was time to walk. Such a mechanism works stably and fails only in one case - when the game table is collapsed into the tray.
The poker room chat had a class InetnetExplolerClass or something like that. To this iteration, I found a way to extract text from such objects and code on VB.Net that worked. The code was rewritten in C # and now getting the chat was without any copying to the buffer and was more stable.
It became clear that for tournaments you need an equator and ICM model calculation, and we started trying to get it. The equator written by my partner could calculate 5000 games per second, while the Equiliator from PokerStrategy.com could have 1500000 on the same configuration - it was a failure and something had to be done about it.
One day, sitting at night, I went to the folder with the files of software PokerStrategy.com (Equiliator, ICM Trainer) and found that they are written on .Net. I was overjoyed about this and I spent the next 2 days to run this software under my debugging, but in the end we could fully see how to use the libraries of this company and successfully learned how to use them.
As a result, our bot considered its chances of winning before the move, and at the extreme stage of the tournaments used pure ICM, which gave it certain advantages.
As a result, we got a bot that played zero or a very easy plus on microlimites in tournaments on many tables (he played 20 easily) using a lot of mathematics.
The next step will be screwing to the bot statistics on players will lead to an improvement in the game.
Unfortunately, at the moment the development is worth and is worth a long time. We still have a lot of small jambs, which can occur once in many hours and it is very difficult to debug such things.
If you have a desire to write your bot - I hope that the solutions we found will allow you to write it faster.
In this article I will describe our experience in developing a bot that was actually tested in real poker rooms and even learned how to play and not lose. Those who are going to write their bot will find interesting things in the article and ways to make their lives easier. Who cares - welcome under cat.
How it all began
The idea of writing a bot visited my friend who shared it with me and with my neighbor in the hostel, and the friend himself was also with the hostel.
The roles were divided by the following plan:
• Logic Designer (idea author)
• Parser Developer (Neighbor)
• Developer of eyes and hands (i)
It was decided to write a bot for limit hold'em (the author of the idea at that time was earning some money playing this type of game)
In such a composition and with such roles, we moved into battle. As a result, we got 3 iterations of development with breaks between them in 3-4 months, which I will try not to forget anything to tell in more detail.
Iteration 1
What a time it was. Three green students began to think. We needed tools, initially the choice was this:
• C ++ - game logic
• VB.Net - Everything Else
But after a few hours of development, we decided to teach C # and began using it instead of BASIC.
Unfortunately, I can’t tell anything about the brains that were in the DLL in C ++, and no one can (just no one remembers), I only remember that there was a hell in it, in which even the author was confused and sometimes surprised that something accidentally works / does not work.
The poker room we chose for testing had excellent chat rooms from which it was decided to take information to analyze what was happening. My neighbor took up this analysis.
I needed to learn how to give him a chat on time, it remained to understand when it comes on time.
So, the first questions that needed to be resolved were:
• How to find among all windows of system necessary?
• How to understand that it's time to go?
• How to walk?
• How to get a chat?
More detail about solving problems below (there is a feeling that it was in that order that I then solved them).
How to understand that it's time to go?
I then heard something about WinApi, but what kind of animal it is and how to teach it - I was not in the know. Very soon, I learned that by a mysterious handle you can learn a lot about the window and do a lot of things, I also learned that you can watch the window well. Armed with Spy ++, I plunged into the world of the then simple recognition of the handle by pointing a cross on the window)))
Smoking allowed me to find out what you can ask the color of the dot on the desktop and what the system can say that the mouse clicked the button.
The solution of the problem became obvious:
1. look for points by the color of which you can find out what buttons appeared (i.e. It's time to walk)
2. get the coordinates of the window
3. add to the coordinates of the point to determine
4. constantly make the window active and check the color of the point
5. If the color has become necessary - it's time to go
')
How to get a chat?
Already knowing how to use Spy ++, it was easy to find the handle. Having smoked again, I understood how to send Ctrl + A, Ctrl + C to the window. Further by means of C # received the contents of the buffer. Chat is.
How to walk?
While it was determined how to understand that it was time to walk - it became clear how to walk. Calculate the coordinates of the button, put the cursor on the dot, Send the left mouse click to the system. It was like.
How to find among all windows of system necessary?
This task turned out to be the most difficult - I couldn’t solve it for a long time until I turned to my teacher (thanks to Chernosvitov AV) for C # / C ++ and he remembered the EnumWindowsProc function or something like that. Using Spy ++, I looked for the window's features, but in the end this feature turned out to be a header, which by the way can also be obtained from the window handle, after which using EnumChildWindowsProc (enumeration of all child windows) was a chat.
Summary of Iteration 1
After the first iteration, we had a bot who played limit hold'em, knew how to understand that it was time to walk, knew how to walk. He had some logic. He could only play alone and completely blocked the computer (due to constant switching to the window before each action) on which he was tested.
Iteration 2
A few months later we decided to write again, to speed up the process, we decided to take the computers into one room and work together there.
During this iteration, it was carried out:
• Moving to one language - we moved completely to C #.
• Transition to anlim tournaments
• Many tables tied
• Tried using system hooks to determine if it's time to go.
• Improved chat reception
• Improved progress mechanism
Transition to anlim tournaments
The logic was rewritten in C # and the starting tables for playing tournaments from pokerstrategy.com were used, which made it possible to make reasonably clean and clear logic. There were also attempts to make a logic configurator (we wanted to sell the bot to the side so that everyone would write their own logic), but we decided that this was utopia and stopped the development of the idea.
Many tables screwed
Along with the new transparent logic, the possibility of playing on several tables was added. Realization of such a possibility was a curve - each table in turn to the fore, checking the color for the need for a move - if it's time to go, then we go. However, this scheme worked, and it was a victory. But now the bot blocked the computer completely and tightly.
Tried using system hooks to determine if it’s time to go.
By this time, I was already strong enough to start digging with Windows system hooks. Since C # does not hang system hooks on the arrival of messages to the window - a project with CodeProject (http://www.codeproject.com/KB/system/WilsonSystemGlobalHooks.aspx) was used, which consists of a DLL and C # logic that runs this DLL and receiving information from her. Hooks allowed not to do a bunch of switching between windows to determine the progress, but to catch the exact order of messages, after which it was necessary to go, did not work, plus the hooks slowed down the system and eventually refused to hooks.
Improved chat reception
During another sleepless night, I watched in Spy ++ the commands that come to the chat, and I saw that certain commands corresponded to the selection and copying of everything. The code of these commands was recorded, the chat copy was rewritten to use the new commands, then the copying was done completely using WinApi commands.
Improved progress mechanism
It turned out that you can send a message to the window, that you clicked on it with the mouse - this allowed us to rewrite the walk to send messages directly to the window about clicks. Now the cursor could no longer twitch across the screen. The computer is gradually returning from the bot to the owner =)
Results of iteration 2
After this iteration, a bot played on many tables in tournament hold'em with transparent logic (by the way, the table proposed at PokerStrategy.com is a loser if you follow it completely and not think more). The bot could already leave a small chance to somehow work the host computer.
Iteration 3
By the third iteration, two of us reached, my neighbor dropped out of the project and his part of the code was rewritten from scratch (it was difficult to maintain someone else's code, especially since it was necessary to significantly develop it). During the third iteration we reached:
• Using a new mechanism for determining the time course
• Using the new chat mechanism
• Using new logic
Using the new mechanism for determining the time course
I still wonder why I didn’t guess earlier that you can determine the color of a point not only on the desktop, but also in the window specifically. Having understood this, I rewrote the definitions of the move in a new way. Now it was not necessary to constantly push the windows to the fore in order to understand that it was time to walk. Such a mechanism works stably and fails only in one case - when the game table is collapsed into the tray.
Using the new chat mechanism
The poker room chat had a class InetnetExplolerClass or something like that. To this iteration, I found a way to extract text from such objects and code on VB.Net that worked. The code was rewritten in C # and now getting the chat was without any copying to the buffer and was more stable.
Using new logic
It became clear that for tournaments you need an equator and ICM model calculation, and we started trying to get it. The equator written by my partner could calculate 5000 games per second, while the Equiliator from PokerStrategy.com could have 1500000 on the same configuration - it was a failure and something had to be done about it.
One day, sitting at night, I went to the folder with the files of software PokerStrategy.com (Equiliator, ICM Trainer) and found that they are written on .Net. I was overjoyed about this and I spent the next 2 days to run this software under my debugging, but in the end we could fully see how to use the libraries of this company and successfully learned how to use them.
As a result, our bot considered its chances of winning before the move, and at the extreme stage of the tournaments used pure ICM, which gave it certain advantages.
Results of iteration 3
As a result, we got a bot that played zero or a very easy plus on microlimites in tournaments on many tables (he played 20 easily) using a lot of mathematics.
Results
The next step will be screwing to the bot statistics on players will lead to an improvement in the game.
Unfortunately, at the moment the development is worth and is worth a long time. We still have a lot of small jambs, which can occur once in many hours and it is very difficult to debug such things.
If you have a desire to write your bot - I hope that the solutions we found will allow you to write it faster.
Source: https://habr.com/ru/post/127378/
All Articles