Perfect storm. Postmortem unannounced project.
Today we went with Vovcheg, our guru game designer, to take him a loan to buy an apartment and discussed our current project.
We agreed that it is surprising that both of us have not been fired yet. We managed to make a huge number of very significant mistakes: I am in management, Vovcheg in design. The project is not finished yet, but it is already possible to draw certain conclusions.
')
The project lasts for the 8th month, will last with all the numerous platforms for another 2-3 months. At the same time, we now clearly see what kind of mistakes cost the company 2-3 months of development and tens of thousands of dollars (I hope that the bill for hundreds of thousands will not work).
TEAM
The main mistake in managing mine is the team selection error:
- A bet on a young team that was never able to, in fact, “turn around” on this very difficult project in all aspects.
- A bunch of young PM + experienced PMA, which failed immediately. It seemed to me that I would be able to realize what, then, seemed to work in Nival.
- We fatally made a mistake with the assessment of the complexity of the task of game design, which led to the fact that a bunch of designer-programmer was removed, which is almost a disaster for the project of our complexity.
It took me as much as 2 months to realize how bad the situation is, it only justifies me quite a bit that out of these 2 months 2 weeks I was on vacation in a warm place. What I saw after the holidays was a disaster. I understood, __ categorically and on a large scale, I screwed up. It seems that never before have I been wrong SO cool.
The correction was a complete change of the entire core team and its concentration in one room of the Minsk office. Only after that the project actually began with the speed with which it was supposed to start 2 months ago.
Surprisingly, I was 1.5 months with the project almost every day, I controlled key points, while “turning on the brain” and seeing the obvious I could only after the holidays.
I was once again convinced that in a difficult situation I am not able to correct the situation with PM directing. The only way I can solve such problems on projects is to solve them by myself. To close the holes by yourself. Unfortunately, I was not better than all other people in this aspect.
Vovcheg's design error was that it was impossible to throw a not well-developed idea into the remote team. We saw very well what is happening with the project, when the team does, it does NOT understand and does NOT accept. None of the team even then did not understand closely what exactly we want to do. Really, I didn’t even understand it! At the same time, over the course of weeks, seeing, on the one hand, what kind of non-playable garbage is in the build, on the other, a break with the Vision that we and Vovcheg want to realize.
3 very important conclusions on this part, which, of course, look obvious.
- The most difficult things I will do myself, only myself. When we have another cool PM, this will change.
- Core team of complex projects will be always in the same room of the same office. And I'll be damned if I ever try to break it again.
- Never will I never again “throw people into the water with the hope that they will emerge,” I will “learn to swim neatly and consistently,” this concerns building teams and developing PM.
- You can not do something, if you do not have at least 1 person who supports and understands the idea, then we did not have it.
HOW A PERFECT STORM IS BORN
We screwed up with Vovcheg very much in entities where we were considered to be experienced guys - Pre-production and Vertical Slice (reference playable is also called).
We went through the concept phase very well. We had a good and clear vision. We have been thinking about the concept for a long time and many people. Not to say that we came up with something incredible, but the concept phase was passed really well. A good start was made.
Clear and correct vision was the only ray of hope. The realization that there is a goal, that it is achievable, but we still do not know a clear way how to get there - this is what saved the project. If this were not, there would be a typical Death March Project, or, as I call it, the Disaster Project. For 8 years the company has had many such projects. I have not yet learned how to avoid them. A clear concept phase allowed us to keep the delay within 2-2.5 months.
Pre-production was completely failed. Despite the fact that some documents were formally made, some (non-playable) Vertical Slice developed, some plans were written, it was reported that some risks were removed, - in fact, it was all fake. The main thing was not done - the certainty of what we are doing. Without this, the plans and the GDD could not have a real filling.
All this led to disaster.
We started production.
A lot of people came to the project. Reserves have already been eaten. Instead of concentrating on solving the main problems of the project - to get a real Vertical Slice, to introduce Definiteness to key features - we had to be strongly dispersed on providing a large team with work. We pressed on the programmer of the engine to make changes even faster. When a large production team was producing results, integrating this whole thing into a build only made the problem worse, we saw that nothing worked at all. It was necessary to redo much, people howled from this, the spiral of problems was twisted.
The changes we made to the engine, have become very expensive. This has already become apparent to external people; absolutely simple changes cannot cost days. So something is wrong with the system inside. The new lead programmer of the project Dimusik, unfortunately, did not realize how much then we were in deep ass.
I found myself in a situation where I had fundamental problems everywhere. In design - we do not know what we are doing. We have nothing but vision-a and a bad Vertical Slice build. We already have some kind of production trait and a lot of people who are embittered by the fact that you have to make a lot of changes that you need to give tasks, but there are no normally thought-out tasks. We have drawn a lot of art, but the quality of many elements was far from the “Wow” we were counting on.
In order to somehow solve this, we had to change the Vertical Slice. I understood that there would be no chance yet in time. Catching time on time means that you’ll go out simultaneously with a big release on American TV. It was a real deadline that could not be moved.
At this moment the programmer comes to me and says that you need to stop any work and throw out 80% of the entire code base. What is the only solution. We both understand that this is true. At best, if we rewrite the current complex elements in the simplest form, for example, full random, instead of the current complex AI, we will do it all in 2 weeks. We will return to the working build in 2 weeks. We understood that we would return to the previous point soon after a month. And it only allows us to make at least some changes to the project.
Remember, I recently talked about the fact that the project is like a play , where the manager's task is to “cram in unshaken” and make everyone a winner.
My project became a TRAGEDY. My personal tragedy. The tragedy of the team and our entire company.
I clearly realized that we were late for minimum by 2 months and that we were actually at the beginning of pre-production.
Transferring the deadline for 2 months, for a big brand, where we have big guarantees for revenue, was a CATASTROPHE.
LIFE AFTER A CATASTROPHE
It was only when realistic terms returned to the project, when we clearly realized that we were at the beginning of pre-production, when a very experienced and robust team sitting in the same room took up the project, and the project began to move at a very fast pace. Ideas were generated not by one designer, but by the whole team, that is, PM, designer, artists, programmers, and later - qa. Changes were made very quickly. The project was changing very quickly.
Literate lead programmer Dimusik then read the Perfect McConnell Code and was able to effectively implement the ideas outlined there, a very clear and very simple architecture of 3 state (instead of complicated and confusing 10-20 state) was formulated. These 3 states became the core of the engine. Everyone, starting with me, ending with QA, knew this state.
For the first time, I saw for myself that by correctly asking questions to the customer (designer) you can seriously help the designer to better understand what he wants, you can help find a solution that is much simpler and better.
We have implemented a scripting system on many levels of projects, a real data driven approach. What took 2 weeks and 10 iterations of the work of the designer-programmer, was now done in 3 days by the programmer, right away and almost without bugs, and then the designer himself very easily and quickly set up the feature in a couple of days. This was a real shock to me.
Later we learned that Nival does the same for scripting magic in Allods Online. Hardcode 200 spells in heroes 5 - it was the old inefficient approach, from which we also left.
It was 10x. We all know that the best people are 10x medium. Now I could measure it very accurately, and indeed it was 10x. It was a shock to me again.
DARK SIDE AGILE
I knew before well what evil Agile can be. On the project, I saw with concrete examples, how big an Evil can be Agile in the wrong hands. Some features we did in the agile style, that is, to be honest, not thinking much what we want and what will change (there will be iterations for changes later).
When at a certain moment we had to make a big next changelog, the lead programmer Dimusik behaved quite differently. We both realized that there must be a better way.
Dimusik for 2 days did not write a single line of code. He talked with the designer a lot and for a long time, they discussed and thought through various options, Dimusik suggested another way to implement the feature, which was radically simpler, gave rise to noticeably fewer problems, but, in essence, features did the same. Dima drew the shemki, drew tables of different interaction options, worked through all the details in great detail. Fortunately, I had the sense to understand that this is not “2 days of doing nothing”, but our future success.
After these 2 days, Dima implemented this already simple feature in 1 day. And when she earned from the very first iteration, completely without bugs (except with only 2 very small blots, which were rather completions) the whole team was in shock. It was fixed for 3 days, of which only 1 day the code was written, it was 1 iteration, instead of 2-3-4 weeks and 3-5-10 iterations.
It became obvious to everyone how much better the good old forgotten (?) All (?) Waterfall approach could be. Only after that I truly, on my own blood, realized how important it is to always adhere to the barely perceptible balance between Agile and Waterfall approaches. I talked about this before, at the last CWI , for example, but only now I saw such a glaring example.
“Keep it simple” is Dimusika’s slogan, tough and very simple architecture, the abandonment of Agile in many elements of the project, scripts and data driven approach in almost all elements of the project - this was the foundation of the revival of the project.
WHERE ARE WE
I already spent 8 months and 50+ person-months on a portable project for many platforms (multiply by the cost of a person-month and get the cost). The project employs the best people of the company, the maximum of people I could “effectively occupy” - these are 4 programmers, 4 artists, 2 dedicated teams.
Now 23:29, Thursday February 12th. Yesterday we received 6 devastating alpha play tests. Never before in all 5 years and 20+ projects play-tests were not so bad. All players are united in the fact that “nothing is clear about what is happening on the screen, the goal is not clear, it is boring to perform the same actions”.
It is ridiculous to say, but we actually did not pass the alpha, we have already done a lot on the beta, I have 3.5 weeks or 21 working days left. We are in a hurry to complete submission of materials on all key Tier1 American channels, otherwise we will simply miss “our window” to launch.
I know we'll be late. It is impossible to hand over a project in 3.5 weeks when the alpha stage has not yet been completed. We will be late for 1 to 3 weeks. We will fight for every day, do everything to bring the release at least for a day, for each of the days.
TODAY FULFILLED DREAMS
Four hours ago, Vovcheg did something very important for him - he bought a good apartment in Minsk, and now, when I write these lines, Vovcheg writes a new pack of changes at home, which we conditionally call “Death Changes” (before that there was “Mad changes pack "), Which will be cheap and that will greatly improve the game. Despite the devastating alpha review, we believe that there are changes that will be good and cheap.
Our faith is our little ray of hope, this is our vision, we have come a long way in search of a goal, and we feel that we are already very close. We are almost here. Our intuition tells us that we are already very close. It seems we are almost happy.
Is there life after the deadline? I dont know.
But I know for sure that I will be proud of our Game, proud of the team that has gone this way, I know that when Vovcheg and I return from San Francisco after listening to Kajim and his "Solid Game Design: Making the 'Impossible' Possible" , we will make our 'Impossible' Possible - we will make our new project much better than the current one.
Denis Voikhansky,
Executive Producer, where, at Reaxion, we create games we want to play ourselves.
PS Perhaps after we announce our game, you will have a chance to evaluate what we have been creating all these months.
We agreed that it is surprising that both of us have not been fired yet. We managed to make a huge number of very significant mistakes: I am in management, Vovcheg in design. The project is not finished yet, but it is already possible to draw certain conclusions.
')
The project lasts for the 8th month, will last with all the numerous platforms for another 2-3 months. At the same time, we now clearly see what kind of mistakes cost the company 2-3 months of development and tens of thousands of dollars (I hope that the bill for hundreds of thousands will not work).
TEAM
The main mistake in managing mine is the team selection error:
- A bet on a young team that was never able to, in fact, “turn around” on this very difficult project in all aspects.
- A bunch of young PM + experienced PMA, which failed immediately. It seemed to me that I would be able to realize what, then, seemed to work in Nival.
- We fatally made a mistake with the assessment of the complexity of the task of game design, which led to the fact that a bunch of designer-programmer was removed, which is almost a disaster for the project of our complexity.
It took me as much as 2 months to realize how bad the situation is, it only justifies me quite a bit that out of these 2 months 2 weeks I was on vacation in a warm place. What I saw after the holidays was a disaster. I understood, __ categorically and on a large scale, I screwed up. It seems that never before have I been wrong SO cool.
The correction was a complete change of the entire core team and its concentration in one room of the Minsk office. Only after that the project actually began with the speed with which it was supposed to start 2 months ago.
Surprisingly, I was 1.5 months with the project almost every day, I controlled key points, while “turning on the brain” and seeing the obvious I could only after the holidays.
I was once again convinced that in a difficult situation I am not able to correct the situation with PM directing. The only way I can solve such problems on projects is to solve them by myself. To close the holes by yourself. Unfortunately, I was not better than all other people in this aspect.
Vovcheg's design error was that it was impossible to throw a not well-developed idea into the remote team. We saw very well what is happening with the project, when the team does, it does NOT understand and does NOT accept. None of the team even then did not understand closely what exactly we want to do. Really, I didn’t even understand it! At the same time, over the course of weeks, seeing, on the one hand, what kind of non-playable garbage is in the build, on the other, a break with the Vision that we and Vovcheg want to realize.
3 very important conclusions on this part, which, of course, look obvious.
- The most difficult things I will do myself, only myself. When we have another cool PM, this will change.
- Core team of complex projects will be always in the same room of the same office. And I'll be damned if I ever try to break it again.
- Never will I never again “throw people into the water with the hope that they will emerge,” I will “learn to swim neatly and consistently,” this concerns building teams and developing PM.
- You can not do something, if you do not have at least 1 person who supports and understands the idea, then we did not have it.
HOW A PERFECT STORM IS BORN
We screwed up with Vovcheg very much in entities where we were considered to be experienced guys - Pre-production and Vertical Slice (reference playable is also called).
We went through the concept phase very well. We had a good and clear vision. We have been thinking about the concept for a long time and many people. Not to say that we came up with something incredible, but the concept phase was passed really well. A good start was made.
Clear and correct vision was the only ray of hope. The realization that there is a goal, that it is achievable, but we still do not know a clear way how to get there - this is what saved the project. If this were not, there would be a typical Death March Project, or, as I call it, the Disaster Project. For 8 years the company has had many such projects. I have not yet learned how to avoid them. A clear concept phase allowed us to keep the delay within 2-2.5 months.
Pre-production was completely failed. Despite the fact that some documents were formally made, some (non-playable) Vertical Slice developed, some plans were written, it was reported that some risks were removed, - in fact, it was all fake. The main thing was not done - the certainty of what we are doing. Without this, the plans and the GDD could not have a real filling.
All this led to disaster.
We started production.
A lot of people came to the project. Reserves have already been eaten. Instead of concentrating on solving the main problems of the project - to get a real Vertical Slice, to introduce Definiteness to key features - we had to be strongly dispersed on providing a large team with work. We pressed on the programmer of the engine to make changes even faster. When a large production team was producing results, integrating this whole thing into a build only made the problem worse, we saw that nothing worked at all. It was necessary to redo much, people howled from this, the spiral of problems was twisted.
The changes we made to the engine, have become very expensive. This has already become apparent to external people; absolutely simple changes cannot cost days. So something is wrong with the system inside. The new lead programmer of the project Dimusik, unfortunately, did not realize how much then we were in deep ass.
I found myself in a situation where I had fundamental problems everywhere. In design - we do not know what we are doing. We have nothing but vision-a and a bad Vertical Slice build. We already have some kind of production trait and a lot of people who are embittered by the fact that you have to make a lot of changes that you need to give tasks, but there are no normally thought-out tasks. We have drawn a lot of art, but the quality of many elements was far from the “Wow” we were counting on.
In order to somehow solve this, we had to change the Vertical Slice. I understood that there would be no chance yet in time. Catching time on time means that you’ll go out simultaneously with a big release on American TV. It was a real deadline that could not be moved.
At this moment the programmer comes to me and says that you need to stop any work and throw out 80% of the entire code base. What is the only solution. We both understand that this is true. At best, if we rewrite the current complex elements in the simplest form, for example, full random, instead of the current complex AI, we will do it all in 2 weeks. We will return to the working build in 2 weeks. We understood that we would return to the previous point soon after a month. And it only allows us to make at least some changes to the project.
Remember, I recently talked about the fact that the project is like a play , where the manager's task is to “cram in unshaken” and make everyone a winner.
My project became a TRAGEDY. My personal tragedy. The tragedy of the team and our entire company.
I clearly realized that we were late for minimum by 2 months and that we were actually at the beginning of pre-production.
Transferring the deadline for 2 months, for a big brand, where we have big guarantees for revenue, was a CATASTROPHE.
LIFE AFTER A CATASTROPHE
It was only when realistic terms returned to the project, when we clearly realized that we were at the beginning of pre-production, when a very experienced and robust team sitting in the same room took up the project, and the project began to move at a very fast pace. Ideas were generated not by one designer, but by the whole team, that is, PM, designer, artists, programmers, and later - qa. Changes were made very quickly. The project was changing very quickly.
Literate lead programmer Dimusik then read the Perfect McConnell Code and was able to effectively implement the ideas outlined there, a very clear and very simple architecture of 3 state (instead of complicated and confusing 10-20 state) was formulated. These 3 states became the core of the engine. Everyone, starting with me, ending with QA, knew this state.
For the first time, I saw for myself that by correctly asking questions to the customer (designer) you can seriously help the designer to better understand what he wants, you can help find a solution that is much simpler and better.
We have implemented a scripting system on many levels of projects, a real data driven approach. What took 2 weeks and 10 iterations of the work of the designer-programmer, was now done in 3 days by the programmer, right away and almost without bugs, and then the designer himself very easily and quickly set up the feature in a couple of days. This was a real shock to me.
Later we learned that Nival does the same for scripting magic in Allods Online. Hardcode 200 spells in heroes 5 - it was the old inefficient approach, from which we also left.
It was 10x. We all know that the best people are 10x medium. Now I could measure it very accurately, and indeed it was 10x. It was a shock to me again.
DARK SIDE AGILE
I knew before well what evil Agile can be. On the project, I saw with concrete examples, how big an Evil can be Agile in the wrong hands. Some features we did in the agile style, that is, to be honest, not thinking much what we want and what will change (there will be iterations for changes later).
When at a certain moment we had to make a big next changelog, the lead programmer Dimusik behaved quite differently. We both realized that there must be a better way.
Dimusik for 2 days did not write a single line of code. He talked with the designer a lot and for a long time, they discussed and thought through various options, Dimusik suggested another way to implement the feature, which was radically simpler, gave rise to noticeably fewer problems, but, in essence, features did the same. Dima drew the shemki, drew tables of different interaction options, worked through all the details in great detail. Fortunately, I had the sense to understand that this is not “2 days of doing nothing”, but our future success.
After these 2 days, Dima implemented this already simple feature in 1 day. And when she earned from the very first iteration, completely without bugs (except with only 2 very small blots, which were rather completions) the whole team was in shock. It was fixed for 3 days, of which only 1 day the code was written, it was 1 iteration, instead of 2-3-4 weeks and 3-5-10 iterations.
It became obvious to everyone how much better the good old forgotten (?) All (?) Waterfall approach could be. Only after that I truly, on my own blood, realized how important it is to always adhere to the barely perceptible balance between Agile and Waterfall approaches. I talked about this before, at the last CWI , for example, but only now I saw such a glaring example.
“Keep it simple” is Dimusika’s slogan, tough and very simple architecture, the abandonment of Agile in many elements of the project, scripts and data driven approach in almost all elements of the project - this was the foundation of the revival of the project.
WHERE ARE WE
I already spent 8 months and 50+ person-months on a portable project for many platforms (multiply by the cost of a person-month and get the cost). The project employs the best people of the company, the maximum of people I could “effectively occupy” - these are 4 programmers, 4 artists, 2 dedicated teams.
Now 23:29, Thursday February 12th. Yesterday we received 6 devastating alpha play tests. Never before in all 5 years and 20+ projects play-tests were not so bad. All players are united in the fact that “nothing is clear about what is happening on the screen, the goal is not clear, it is boring to perform the same actions”.
It is ridiculous to say, but we actually did not pass the alpha, we have already done a lot on the beta, I have 3.5 weeks or 21 working days left. We are in a hurry to complete submission of materials on all key Tier1 American channels, otherwise we will simply miss “our window” to launch.
I know we'll be late. It is impossible to hand over a project in 3.5 weeks when the alpha stage has not yet been completed. We will be late for 1 to 3 weeks. We will fight for every day, do everything to bring the release at least for a day, for each of the days.
TODAY FULFILLED DREAMS
Four hours ago, Vovcheg did something very important for him - he bought a good apartment in Minsk, and now, when I write these lines, Vovcheg writes a new pack of changes at home, which we conditionally call “Death Changes” (before that there was “Mad changes pack "), Which will be cheap and that will greatly improve the game. Despite the devastating alpha review, we believe that there are changes that will be good and cheap.
Our faith is our little ray of hope, this is our vision, we have come a long way in search of a goal, and we feel that we are already very close. We are almost here. Our intuition tells us that we are already very close. It seems we are almost happy.
Is there life after the deadline? I dont know.
But I know for sure that I will be proud of our Game, proud of the team that has gone this way, I know that when Vovcheg and I return from San Francisco after listening to Kajim and his "Solid Game Design: Making the 'Impossible' Possible" , we will make our 'Impossible' Possible - we will make our new project much better than the current one.
Denis Voikhansky,
Executive Producer, where, at Reaxion, we create games we want to play ourselves.
PS Perhaps after we announce our game, you will have a chance to evaluate what we have been creating all these months.
Source: https://habr.com/ru/post/51929/
All Articles