The complexity of multi-storey programming

Hello dear Habrachiteli and programmers!

I must warn you that my post today will be more like a question than an article. The purpose of the post is to search for like-minded people who are passionate about the excitement of people who want to create high-quality and competitive applications. I will start from far away, pour some silent water so that the post looks like something that is not immediately removed from the habr.

For many years I have been dreaming and still dreaming of creating high-quality applications. Every time I started writing my new program, I was 100% convinced that it would be the best program of its kind. The beginning of any application was inspiring, delightful and inspiring hope for great prospects. On such days I was immersed in work with my head, forgetting everything in the world. Every day new functions appeared, new successful solutions were applied, everything promised unprecedented progress and the golden mountains if everything continued at the same pace. I wanted to mentally extend this growth chart, and it seemed that nothing could stop him. I think each of us had this happen.
')
Over time, the program grew as a high-rise building floor by floor, sometimes turning into catacombs, for which you had to wander for hours, renewing the author's entire idea, or looking for where you can insert another crutch.

Work on finalizing the program gradually became harder and harder. Each new glitch was more insidious than the previous one, and it so happened that the goal of the program’s development was replaced by the goal of ridding it of errors. It is difficult to write new functions, knowing that there is something wrong with the old behavior. At such moments, you realize how difficult software development is. All the sins and miscalculations in the beginning, the flaws in the architecture and the crutches inserted came back to me like a recurring nightmare. Besides the fact that these were extremely unpleasant imperfections of the program, they also forced me to work for hours in an attempt to get rid of this nightmare. In the end, there was a choice: either wake up from a nightmare and start the program again, in the hope that the second (third, fourth) time I will be able to foresee all the pitfalls, and this time I will make a competent architecture, divide everything by concepts, encapsulate the complexities and I will carry out the repeating code in separate places; or preserve the program in the state in which it was at that moment, eliminate all the most visible errors and wait for better times, new inspirations for this program or new ideas.

This pattern of development of events was repeated every time, with each new project, every year several times, regardless of the level of my skills, regardless of whether I wrote programs for myself or it was money orders from other people. First, the spark in the eyes of the ease with which new functions appear, then hard work (comforting yourself, well, you can’t easily move anywhere), then a multiseries nightmare and death pangs, and finally the death of the project. Death in the form in which it existed before, and the birth of a new, written from scratch version of the same application.

Over time, it became clear how much of the code can live for a long time, and which part will die after several attempts to modify it. Just look at the part of the code filled with conditions that does not fit either on the monitor screen or in the human brain, and it becomes clear that this code is close to being abandoned. He certainly can work. And if it works well, there are no errors in it and the performance does not cause problems, then the best thing you can do is not to touch it. You can look at the class, the names of the methods of which say that it is a mixture of some heterogeneous concepts, and it becomes clear that you do not want to use it a second time. It reached the limit of its complexity, the building grew to such a height that another floor and base could not stand and everything would be overwhelmed. If my brain does not stand up, in an attempt to describe the entire set of combinations of states of this object, the concept becomes too complicated to understand. This was well manifested when I investigated the reasons for the wrong behavior of such classes. It was necessary to change something.

Rewriting the program gave good results, but spent a large number of forces. I could not always decide on this step. Almost never. I rummaged, realized the amount of work and my hands dropped. I know, I am not alone in this memory, because it cannot be otherwise. Users say it would be nice to add just such a function, and you understand that you have already added too many of these functions and the time has come to rewrite everything from the beginning. Users (or customer) do not understand: is it really so difficult to add another single function? They do not understand that you need to redo everything, rewrite, eliminate architectural blunders, redo everything from scratch, because when we added the last ten new features, we already told ourselves that each of these features would be the last.

The increase in the complexity of the program is inevitable, and the more inevitable it becomes, the more people work on the project. This is a real programming curse that overtakes any program, sooner or later, a matter of time. And it is precisely the skills to resist this snowman of the growing code that distinguish successful projects from the mass of frozen crafts.

It is conditionally possible to distinguish three phases of the situation in the program:

1. Green phase or childhood project. The code seems simple, there is not much of it, the hands are untied and I want to program. :)
2. Yellow phase or workdays. The project is large, the work on it is difficult, but it seems quite justified. New functionality is added carefully, but regularly. Many would be content to remain in this phase forever, but there’s no. : |
3. The brown phase or the catacombs. In the project a lot of incomprehensible. The same things are done in several different ways. Rewriting the individual parts lacks the spirit, since it is not clear what it can affect. Too many unsuccessful decisions, nothing can be changed, and is it worth it to get involved? Alteration of individual sections entails a long period of error trapping in various parts of the program. Crutches are inserted. The amount of forces expended far exceeds the result. The development of the program is not possible. There is a desire to do something else. :(
   

I have an example from life. I know one company that always programs in the green phase. Programmers in this company seem contented and optimistic, people who have not experienced difficulties. It seems that they have a knee-deep sea. Wonderful, isn't it? What is the matter? Her brave manager has long understood about the natural phase change in the development process and sells the project every time a build-up of complexity begins to be felt. He sells startups. Growth charts are attached. The reports on the work done are excellent. The deadline is fascinating. Prospects seem impressive. Naive investors believe that the program is a product, like any other ordinary business, bought, put salespeople and accountants in place, and make a profit. They often do not understand that they are buying a delayed-action headache, which they will throw out after a year, when programmers start to climb walls, so that everything will stand still. All the problems of the brown phase remain at the share of the buyer, and the company that sold the startup feels all the delights and optimism of the green.

The exit is certainly original, but obviously not the only one. There are after all Microsoft, Apple, Oracle which are able to overcome these difficulties of growth. How do they do it? What are the methods to save the green zone?

I suggest you write your answer in the comments. I will also state my thoughts.

Thank you for your comments.