How we made a highload ++ game with voxel graphics and VR
In fact, this is a lightweight technical longrid, we hope that after reading you will have an additional interest in making some kind of game, or at least you will learn how it works.
About Speaker: Alexander Khayorov ( @allexx ) leads the development department at Ingram Micro Cloud . The guys in the team of Alexander consider themselves not just excellent engineers, but call themselves a great team of voxel Jedi, optimization masters, 3D gurus and masters of big data! [note: similar to job titles on LinkedIn and Medium] ')
This cool team, preparing for a speech at Highload ++ 2017, decided to entertain the audience and make something new and interesting for the stand. Therefore, they gash the game, the creation of which will be discussed further.
Hostess note: from the organizers, we very much welcome the efforts to prepare for participation in the conference. They pay for themselves many times, attracting participants, and, as it turns out, benefit the team.
So let's go!
Often, when sorting out mail, I look through the headlines of informational tapes, where a variety of news flashes. Once I saw the headline "Who are indie developers." For some reason he hooked me, and I decided to read this article. I opened it - there were a lot of numbers, letters and statistics.
Reference: Indie developers are people who create games without a special budget and without the financial support of computer game publishers.
As a rule, indie developers are not burdened with the framework of the script and templates, so they get quite interesting games, movies, and so on.
In this article, I learned a few funny facts:
97% of mythical indie developers are men;
60% of them are not, not singles, but they make games alone;
On average, it takes 1 to 3 months to make an indie game.
It so happened that we also started to make the game.
Why we started making the game
I always liked the games from the Reddit service. I think that you have spent more than one month on this resource.
A social experiment by Reddit
Every year Reddit conducts a social experiment, as they call it. Although in reality these are different games for the community. In 2017, a social experiment took place, as always, at the beginning of April (on fool's day).
The essence of the game was promising. The developers of Reddit created a picture of 1,000 x 1,000 pixels. Each registered user on Reddit, and there are several million of them, was asked to paint over 1 pixel of this canvas with one color from a fairly wide palette.
The event lasted 72 hours, and each participant every 5 minutes could draw only one point on the screen. People created different pictures, fought among themselves, repainting pixels. Some countries and communities united and collaborated.
In the end, it turned out an interesting panel, from which someone then made a puzzle, someone tied socks with such an ornament, etc.
I really liked the idea that you can engage people in massive online games , even with a simple gameplay. This was probably the main inspiration for us to create an online game. In general, I believe that the games are full of fun and I am sure that you sometimes play different games and this brings you pleasure.
But for the techie, the most interesting thing was inside, since the game attracted a huge number of people - more than a million registered unique users participated in it - up to 90 thousand people at the same time painted over pixels every second!
Professionally, I develop web services and various M2M services when a server communicates with servers. This is actually very important and responsible, but sometimes a little boring. Therefore, the new developer experience is always interesting . I'll share it with you right now.
Looking ahead, I will say that we really learned a lot of new and interesting about the game, without having any experience from the beginning.
The game
Features
We got together with the guys and sketched out a skeleton of ideas about what to make the game. At first we decided to go with trumps and write cool features.
Definitely wanted to make a multiplayer .
We decided that the game would be networked , and even massive, and not in split-screen format.
We chose 3D graphics.
Although it would probably be easier to make 2D without experience, the team wanted volume.
We decided to take the HYIP technology - let it be Virtual Reality .
We understood that without overvalued technology, it was impossible to make a successful game.
This is how we formed the Features set, but without a certain gameplay.
Concessions
But, after thinking about a week, we decided that we needed to somehow limit our Wishlist and made small compromises.
We really did not want to write a separate client, and we decided that the game would be browser-based , especially since we heard that modern browsers have become very powerful and allow you to make 3D graphics.
The second important factor was that we, as engineers, of course, strive for beauty , but we have a mediocre attitude to graphics. Therefore, the idea of using small voxels or blocks in order to create the whole world and construction seemed to us to be a kind of panacea.
Finally, we decided that the world would not be infinite (although I am sure that the universe is infinite) and limited ourselves to 1000 × 1000 × 200 voxels, thereby simplifying the technical features of the game a little.
Reference: Voxel is almost a pixel, but in a 3-dimensional world. For example, in Minecraft, the world is represented by voxels.
On this we parted for some time and later decided to talk only on the matter, in particular, to discuss the gameplay and rules.
Gameplay and rules
It seems that the gameplay is a very simple thing! We know how to play StarCraft, Doom or Quack. But when you create your game, you have a lot of ideas. These thoughts are scattered in different directions and it is very difficult (especially in a team) to agree on how the game will look and what will happen in it.
This process was not linear, we could not immediately determine everything. The final version of our rules looks quite simple:
We need to find a spot - a kind of building - which is why the game is called Urban (City).
On the spot there is a flag to be captured, points are awarded for this.
The participant with the most points gets a prize.
Players can freely move around the common world and retake flags (a variation of "capture the flag").
On this we decided on all the managerial functions and decided to take up architecture.
Architecture
Since we are creating enterprise products, we decided to start with the enterprise architecture of our game. We thought that it should definitely be:
The players .
Special stand for playing in a virtual reality helmet.
Browser , as we decided it would be a browser game. People will go there and receive data from there.
Some kind of " storage " - because we have a multiplayer game on the network, we need to store data about different players somewhere.
It was the original, rather naive architecture diagram.
The problem of replication in distributed systems
How should the "store" communicate with the browser where the game works? Here the interesting moments begin, and I would like to take you to a more familiar area.
In general, the problem of replication in distributed systems has been well studied. Replications are used daily in our industry. But any game, especially online, is an atypical situation of replications when it is necessary to provide replication only for the game to happen. If you do not have replication and coherence in the distributed system, consistency between players, then the game can be terminated because it is impossible to play it.
There are two types of replications:
Active replication , when data is taken directly from one client and transmitted to all other players, for example, information is sent that a player has captured the flag. Other clients receive exactly the same animation, the same events, etc.
Passive replication , in this case, the player sends information to a server, and already this server provides data transfer to all other players.
Let's try to figure out what the salt is, what is better, what is worse, and if there is any difference between active and passive replication.
Pros and cons of active replication:
+ In active replication, everything is simple and intuitive : we take the data, send it to all other players, receive information from all other players.
+ The second important point - this system is quite effective . Indeed, you do not need any additional device in the form of a server that will receive, process and transmit data to other players.
- But there is a huge minus - systems built on active replication are very fragile . The appearance of network problems, for example, losses, delays or difficulties with clients, is enough; a completely inconsistent system is obtained. Synchronization breaks, and our task is to have the same common world.
In fact, getting out of a situation with a broken sync is quite difficult. There are various protocols for this, for example, Paxos, but all of them imperceptibly complicate the simple scheme of active replication, they require time and computational resources.
For example, I can say that the classic StarCraft game is built on active replication. This is one of the clear examples of using a simple but fragile model. That is why, when certain problems appear in synchronization, the game usually has to be completed and restarted.
Pros and cons of passive replication:
+ Unlike active, passive replication is very resistant to desynchronization . If something goes wrong, there is a special device - a server that can bring the system back to normal.
The second point, and often very undervalued and important, is the safety of games. Recently there was a game VKpixel Battle - a direct analogue of the game Place from Reddit, in which it was also possible to paint a board. This game was hacked in a few hours ( source ). In the passive model, the security of the game is noticeably easier to provide for the reason that, again, there is a server where you can control a lot.
- But there are no pros without cons. With the advent of the server - a device that receives and sends data, a classic point of failure occurs. If you do not use special tools, such as sharding, division into nodes, you can lose the whole game by crashing the server.
We did not think long, and, like the developers of all modern games, chose a passive replication method. In fact, now the vast majority of games (about 99%) use passive replication. Therefore, our powerful enterprise architecture has another component - the game-backend, which takes on the task of synchronization.
Data structure
Let's talk a little about the storage system. The fact is that the choice of storage we treated, so to speak, without proper attention, as is customary in some circles.
In order to understand where to store data, you need to know how to present them. This is probably the most important question that determines certain criteria. Consider the data structure that is possible in our game.
It has a lot of voxels and a limited world. You can take all the information about the cubes (voxels) and store them in a regular linear large array. This is a very simple and straightforward way. Its main advantage is constant writing and reading in a cell. We can safely get information about the voxel, and this will require some constant time.
In fact, a lot of modern games, including Minecraft, started with such a model, and it proved to be quite successful.
The downside is the high memory consumption . The fact is that our world is growing in three dimensions, so the amount of memory required to store this data increases nonlinearly.
This problem can be solved, for example, with the help of octree trees . We will not go into what it is, I suggest for understanding just to look at the picture.
If the 3-dimensional world is represented in the form of a large cube, then it can be divided into 8 parts - octas. Each of them can also be divided into 8 sections. This mechanism saves a lot of storage structure due to areas in which data is missing.
There is a clear advantage here - this is a more profitable structure in terms of storage . The minus is initially not so obvious, but quickly gets out in practice. The point is that you need to use different caches. As a rule, everyone who uses octree complains that it works much slower than regular arrays.
Finally, there is a third option, I would call it a compromise - you can take an ordinary linear array and divide it into some uniform areas (chunks). This gives certain advantages. For example, unlike a regular array, you can load individual blocks. We do not need to download the whole world of one user. We can load only blocks coming from it.
But there is a certain minus - it certainly complicates the whole model of a regular array , because you need to store references to chunks, operate chunks, add logic to the front-end of the game, etc.
We decided that the last option was optimal for our game and we chose it. After that, the choice of storage for us went unnoticed: we decided - let it be MongoDB . Now I would not want to breed holivar on this topic - I am sure that this can also be implemented on many other excellent databases. However, we had little experience with this base and therefore decided to dwell on it for experiments.
Front End / Backend Interaction Protocol
We have a certain structure, there are clients, probably, some browser will appear in which all this will be displayed. But how to transfer data between storage and game? It's time to think about the protocol.
As we already know, our world is represented as a parallelepiped with a base of 1000 × 1000 and a height of 200. You can ignore the height and always use the maximum height in the unloading. This greatly simplifies the creation of the game.
In turn, each square, according to our standard, we broke into 32 × 32 voxels inside the chunks.
As I said, we decided to use MongoDB. The voxels are stored in it quite simply, for this we use ordinary documents. Inside the document looks like, as we slide below:
service id field
coordinates,
the name of the player's color,
some service name,
the date when this block appeared.
Now we have at least two tasks. For example, we direct the player. He is born in the world at some point.
We determine the nearest chunks that a player can see, and form a radius. Actually it is not absolutely correct name. We form an understanding of the distance and see how many chunks it affects. Next, we make a get request to the database, in which we speak the current coordinate of the player and this radius.
Create a set - block or cube on the map. Here you simply send the coordinates of the cube that you want to install.
These are direct requests to our repository.
In the opposite direction, you can observe requests for updating other cubes.
In this case, the point is that, being in the world and having received some of its original state, we want to observe new cubes that other users create. To do this, we subscribe to events and get updated messages about all the dice that appear in the game.
Finally, we wanted to add a bit of interactivity to the game, and therefore added a special message about user activity - when someone entered the game, or went out, or intercepted a flag that allows you to get points.
So we have formed an interaction protocol.
We had a small choice of how we can communicate between the backend and the front end: HTTP, WebSocket, HTTP2. We decided to stop at WebSocket for obvious reasons - in order to intuitively reduce the potential delays that were possible.
We also thought that it would be nice to get the results of the game - for example, to see who entered the game, what were his points. To do this, we made a separate “view”, which we screwed to the frontend and decided to use HTTP for it, so as not to authorize it and not complicate this process.
So we have formed a certain stack in architecture. The key point of interaction in it was WebSocket.
Game backend
Backend, at a minimum, can be called the heart of our system.
Since we have some experience in web development, we decided not to use new technologies, unknown for us. We settled on Python 3.6 and aiohttp . That is, we have a fully asynchronous server.
We added a modified loop to it - this is uvloop , which allows you to work even faster.
Since we are using MongoDB, we needed some kind of asynchronous client. For this we used motor , which perfectly allows us to communicate with MongoDB in asynchronous mode.
Finally, in order to expose our server to the outside, we used gunicorn , which develops in Python.
So we have a completely traditional stack that allows exchange data over our designated protocol.
Game-frontend
This is the highlight of our game - the way the game begins to live and exist.
First of all, we have a question - how can we write a game engine, how to make games and how it all works. After a brief study of the material, we realized that there is a library WebGL , which allows a lot. It works with the OpenGL library, which already in turn works with hardware, with drivers, with a video card, etc.
Therefore, the first thought was to use the usual native JavaScript, see which API provides WebGL and start making the game. We quickly dismissed this idea, because from our experience in web development we understood that developing a web server itself is a very strange and long undertaking . Moreover, we did not have so much time.
After a brief search, we found a JS library called Voxel.js . In fact, it became the Grail for us, because it represents a lot of tools.
The library has existed for more than 3 years, and, as the author claims, in fact, is a tool-kit for creating modern browser games, and voxel games. It contains everything that is needed.
The concept of a world in which there are axes and coordinates;
Work with blocks (voxels) that understand the concept of chunks;
Video game creation tools:
bounding polyhedra in order to understand collisions, being near, working with objects, etc .;
ray casting is a method that allows you to create a 2D view on a monitor from 3D objects;
textures.
Most importantly, there is the concept of the player and his management . In fact, we got almost finished game!
All necessary events - the cycle of creation of the world, the cycle of installation of the block, etc. All of them can subscribe and produce their actions.
What does the whole stack look like
We write our code using the Voxel.js library - this is our special logic.
In turn, Voxel.js communicates with the library called three.js. I will dwell on it a little bit, because it is the cornerstone and important stone in this whole scheme.
Next, three.js already communicates with WebGL, which is actually built into the browser (uses its API).
WebGL already works with this equipment - with the OpenGL driver, which draws a picture on the monitor.
In fact, it is not so easy to create a scene using three.js, and in particular with Voxel.js, which it uses. Although it seems there is nothing complicated at all.
Let's look at the code, but we will not be afraid in advance.
To create a cool 3D image in the browser, you really need only 3-4 things:
First of all, we need a stage , as in the ordinary world.
In this scene there must be a camera - we create a camera object, in which we designate the dimensions and the maximum / minimum distance that we see.
After that, we need a device that will render - to create for the camera on stage what we want to see. To do this, we create a render . In this case, just used WebGL technology, and our video card will work to create the world.
In principle, we have everything to make the game work. The only thing missing some objects.
4. In order to create an object, we need:
Geometry (geometry) is how an object will look in space. For example, if it is a cube, then it must have vertices and faces.
Material - in order to color the object. The easiest option is to use color shading.
Combining the geometry and material, we get the so-called mesh (or some kind of mesh object).
Next, the simple animate function is called . Like setInterval in the browser, each time it draws a new frame and it turns out animation. In fact, this mechanism is used in ordinary 3D games and is available in the browser.
The virtual reality
I would like to talk a little bit about VR. As I said before, we wanted to use HYIP technology to get attention. We decided to use VR. It was a random, spontaneous choice, but after studying the topic, we realized that VR is not a very new and not very high technology. This picture is dated the beginning of the XX century - this is VR, but the XX century!
You just need to use this helmet to view real 3D objects - the only thing that is static.
Oculus rift
We decided not to use a static image, but still create graphics with video. Therefore, our choice fell on the Oculus Rift .
Again, the choice was quite intuitive and spontaneous. There is nothing special about it, the Oculus Rift has at least 3 basic devices:
helmet;
a special device in order to understand the tracking, that is, the movement of the head, which is auxiliary (left);
the controller is a source of commands that can be represented by different devices.
None of us has dealt with VR technology and Oculus helmet before.
Pros:
+ This is really a VR-picture - it is quite real and allows us to deceive our brain and imagine that we are in a 3D world.
+ A good screen resolution and a very decent picture update speed. I personally did not observe any noticeable deterioration of my state of health while viewing the picture, unlike the first old virtual reality helmets.
+ This is really a very convenient device, well-made, which is perfectly in the hand and fits perfectly on the head. We can say that this is really a consumer device.
Minuses:
- I imagined VR so that you can just put on a helmet, devices and it will be cool! In fact, this is a huge number of wires. In fact, each device requires a cable, or even two.
- The second important moment for me and slight disappointment - I expected that I would take my usual laptop and be able to program while sitting on the sofa. It turned out that there are serious limitations: you can only use Windows, Oculus and the SDK is not available for macOS / Linux. Perhaps this is a company's strategy or a feature of the operating system - it's hard for me to say.
- I could not even start a simple video or a game in VR without connecting a decent enough video card. I am not an ardent gamer of modern 3D games, so I had to look for it in order to start developing.
I think that if you are not hampered by these drawbacks, then you can safely try modern virtual reality helmets.
How to make games for VR
For us, this topic was completely closed and incomprehensible. We thought we needed special magic, perhaps devices or a degree. Not really.
A-Frame
Now there are not so many tools that allow to make VR games also in the browser.
Probably the first thing that can be found on the Internet is the A-Frame framework. .