How to win the game with unknown rules?
Hi, Habr! In short, we invite everyone to take part in the unusual machine learning contest Black Box Challenge , which is held with the support of DCA.
The conditions are simple: you need to write a bot that can play the game with unknown rules. The winners will receive valuable prizes: from Xbox One to 300 thousand rubles for the first place and the opportunity to get into the steep companies to the position of a machine learning specialist.
Below we describe the competition in the question-answer format.
So, you need to teach the bot to play the game with unknown rules. At each step of the game, the bot can find out the state vector of the game environment, think what it wants to do in this state, and do one of four possible actions.
')
A bot receives a reward for committed actions, sometimes immediately, sometimes only some time after the action. That is, it is not always possible to immediately understand whether the action was good or not.
In addition, the game is not deterministic, in the rewards there is an element of chance. But chance is already frozen in files with levels. This is necessary so that the result at the level does not depend on the launch. You can imagine it this way: we threw a coin 100 times and saved all the outcomes. After that, we use these outcomes to calculate rewards.
More information about the conditions and interface of the game can be found on the competition website . There you can also get acquainted with test examples, specifications and useful links to study the issue (although there is not enough time).
Kaggle is the most famous international machine learning competition platform. All competitions on Kaggle are arranged approximately as follows:
Our competition is different:
In other words, the solution to the problem on Kaggle is a static file with answers. The solution in our competition is a trained agent interacting with an unknown gaming environment.
In short, the fact that the rules are unknown. Each year, our partner Mail.ru holds an open competition in artificial programming - Russian AI Cup. For example, last year it was necessary to create a program to manage one or a group of code vehicles.
If you look closely at the last year's winners of the Russian AI Cup, it turns out that all their decisions used a priori human knowledge about the game device and did not use machine learning at all.
We thought it was more interesting when the system does not use human expertise, but learns itself to understand the environment, in particular, to play games.
To contribute to this, we created a game with unknown rules.
Yes, this area of ​​machine learning is called Reinforcement Learning . Not so long ago, the Google company Deep Mind (the one that created AlphaGo ) released an excellent article “ Playing Atari with Deep Reinforcement Learning ”, in which she taught the program to play dozens of Atari 2600 games on a human level. Some participants are trying to use their ideas for our game, but judging by the chat in Telegram, it is not very successful so far :)
On the site we have laid out an example of a simple bot who can play our game a little. As we came up with it, it will still be our secret - we do not want to limit your imagination to your idea.
The result of this bot (2090 points on the validation sample), we consider the base level for the participants. At the moment, out of more than 850 registered participants, 42 were able to overcome this level, it turned out to be not as easy as it seemed to us.
Many of the participants who overcame the basic level achieved this simply by slightly improving the odds of our simple bot. We hope that you will be able to surpass them with new approaches. :)
It is enough to have Python with the numpy module installed and be able to program on it.
If for some reason you do not have python, then we recommend installing the latest version of Anaconda . Anaconda is a Python with many useful libraries for machine learning and data analysis.
We only support Python. More recently, you could only use Python 2.7, but we decided that there was no point in resisting the future and made support for Python 3.5.
If something doesn’t work for you, feel free to contact us immediately. We try to help everyone.
The prize fund is more than 600,000 rubles.
These funds will be distributed as follows:
1st place: 300,000 rubles
2nd place: 170,000 rubles
3rd place: 125,000 rubles
In addition, the next 5 best participants will get Microsoft Xbox One.
Competition partners - Data-Centric Alliance, Mail.ru, Skyeng are constantly looking for advanced specialists in machine learning and data analysis.
Participants who performed well in our competition and offered non-trivial approaches will be invited for interviews. This means that looming career ahead with even more interesting tasks in the field of machine learning.
May 30, 2016, 23:59 Moscow time: the deadline for downloading solutions.
June 10, 2016: the definition of winners.
We have a forum where we post news about the competition, answer questions, and where participants share ideas. We also have two channels in Telegram for discussions: English-speaking and Russian-speaking .
Ask your questions in the comments, we will be happy to answer!
The conditions are simple: you need to write a bot that can play the game with unknown rules. The winners will receive valuable prizes: from Xbox One to 300 thousand rubles for the first place and the opportunity to get into the steep companies to the position of a machine learning specialist.
Below we describe the competition in the question-answer format.
What kind of competition?
So, you need to teach the bot to play the game with unknown rules. At each step of the game, the bot can find out the state vector of the game environment, think what it wants to do in this state, and do one of four possible actions.
')
A bot receives a reward for committed actions, sometimes immediately, sometimes only some time after the action. That is, it is not always possible to immediately understand whether the action was good or not.
In addition, the game is not deterministic, in the rewards there is an element of chance. But chance is already frozen in files with levels. This is necessary so that the result at the level does not depend on the launch. You can imagine it this way: we threw a coin 100 times and saved all the outcomes. After that, we use these outcomes to calculate rewards.
More information about the conditions and interface of the game can be found on the competition website . There you can also get acquainted with test examples, specifications and useful links to study the issue (although there is not enough time).
What is the difference from Kaggle?
Kaggle is the most famous international machine learning competition platform. All competitions on Kaggle are arranged approximately as follows:
- Member downloads data
- Teaches any model (for example, a classifier) ​​using this data
- Predicts unknown data with his model and creates a static file with answers.
- Fills the file with the answers to the server Kaggle, where it is checked the quality of its predictions. The winner is the one whose answer file is more accurate.
Our competition is different:
- The participant downloads the game simulator and training levels.
- Teaches the bot to play at training levels
- Fills the source of his bot on our server, where it is checked how many points it is gaining at an unknown level for it. The one whose bot is gaining more points wins.
In other words, the solution to the problem on Kaggle is a static file with answers. The solution in our competition is a trained agent interacting with an unknown gaming environment.
Then what is the difference from the Russian AI Cup?
In short, the fact that the rules are unknown. Each year, our partner Mail.ru holds an open competition in artificial programming - Russian AI Cup. For example, last year it was necessary to create a program to manage one or a group of code vehicles.
If you look closely at the last year's winners of the Russian AI Cup, it turns out that all their decisions used a priori human knowledge about the game device and did not use machine learning at all.
We thought it was more interesting when the system does not use human expertise, but learns itself to understand the environment, in particular, to play games.
To contribute to this, we created a game with unknown rules.
Someone already knows how to learn cars to play games?
Yes, this area of ​​machine learning is called Reinforcement Learning . Not so long ago, the Google company Deep Mind (the one that created AlphaGo ) released an excellent article “ Playing Atari with Deep Reinforcement Learning ”, in which she taught the program to play dozens of Atari 2600 games on a human level. Some participants are trying to use their ideas for our game, but judging by the chat in Telegram, it is not very successful so far :)
How are things now with the participants?
On the site we have laid out an example of a simple bot who can play our game a little. As we came up with it, it will still be our secret - we do not want to limit your imagination to your idea.
The result of this bot (2090 points on the validation sample), we consider the base level for the participants. At the moment, out of more than 850 registered participants, 42 were able to overcome this level, it turned out to be not as easy as it seemed to us.
Many of the participants who overcame the basic level achieved this simply by slightly improving the odds of our simple bot. We hope that you will be able to surpass them with new approaches. :)
What you need to start to participate?
It is enough to have Python with the numpy module installed and be able to program on it.
If for some reason you do not have python, then we recommend installing the latest version of Anaconda . Anaconda is a Python with many useful libraries for machine learning and data analysis.
What languages ​​can I write?
We only support Python. More recently, you could only use Python 2.7, but we decided that there was no point in resisting the future and made support for Python 3.5.
If something doesn’t work for you, feel free to contact us immediately. We try to help everyone.
What are the prizes?
The prize fund is more than 600,000 rubles.
These funds will be distributed as follows:
1st place: 300,000 rubles
2nd place: 170,000 rubles
3rd place: 125,000 rubles
In addition, the next 5 best participants will get Microsoft Xbox One.
And what about vacancies?
Competition partners - Data-Centric Alliance, Mail.ru, Skyeng are constantly looking for advanced specialists in machine learning and data analysis.
Participants who performed well in our competition and offered non-trivial approaches will be invited for interviews. This means that looming career ahead with even more interesting tasks in the field of machine learning.
What is the timeline?
May 30, 2016, 23:59 Moscow time: the deadline for downloading solutions.
June 10, 2016: the definition of winners.
How to follow the news?
We have a forum where we post news about the competition, answer questions, and where participants share ideas. We also have two channels in Telegram for discussions: English-speaking and Russian-speaking .
Ask your questions in the comments, we will be happy to answer!
Source: https://habr.com/ru/post/281707/
All Articles