Hellish project

A few years ago, I was invited as a consultant for a single software project for a large French technology company. What I saw goes beyond everything I could imagine in the design. The mere lack of professional competence was not the worst. Much worse was the extreme contempt for human dignity, which seemed to me comparable to a prison as I imagine it. Here is the list, check for yourself.

Scale

• Software development for a government agency.
• The difficulty is low, with a few twists and turns.

The government pays several million euros in advance. Development plan for two to three years. To begin with, the company hires several developers and continues to double the size of the team every three months or so, mastering the budget as funds become available.

7 years later, the project has not yet taken shape. They incur fines of several thousand euros per day. Management decides to cut costs and makes layoffs of experienced employees, hiring people with little or no work experience.
')
10 years later, given the disastrous state of the project, middle management decides to hire some people with development experience to return to the schedule. Average turnover among beginners: three months. This is the minimum term to have the right to resign in France.

12 years later, the project is not finished yet. The company again falls on daily fines, issuing new bills to the government for the ever-increasing flow of change requests. 2008 is coming.

Numbers

• 6 million lines of code
• Based on C ++
• Over 50,000+ grades
• The specific version of C ++ is outdated, but tied to a compiler that is distributed only with one (not supported) operating system.
• Based on CORBA
• DBMS from bankrupt company
• Multiple layers on top of each other for GUI processing, none of which is actually supported by the authors.
• Assembly takes 48 hours on 32 parallel machines
• 40 to 50 simultaneous processes to run only user interface
• Lack of dynamic linking libraries: executable file sizes start at several hundred megabytes
• Program start time: about 15 minutes
• Average time between failures: from 30 seconds to 30 minutes

No developer can say that C ++ is a simple language. On the contrary, one of the most difficult. In fact, it is so complicated that even its creators admit that they have not yet mastered everything. See the recognition of Straustrup in his famous "interview" , which was composed as a rally.

Faced with such an incredible maze of bottomless complexity, people react differently. All upstart geeks have heard of C ++ and want to prove that they can also program on it. They are fearlessly immersed - and injured beyond recognition. They spend countless hours trying to figure out why a bunch of gibberish endlessly crashing for no apparent reason. Who has a mind - they quickly go to other languages ​​and other projects. Life is too short.

Maintaining a large project is difficult regardless of language. Just imagine that employees need to maintain 6 MILLION lines of code - and you get an idea of ​​how far insane development can go. Six million is a big number. If you read one line per second, then you sit in front of the screen for seventy days without a break.

To understand the scale will tell two cases.

One developer was tasked with checking why a right click in the interface completely hangs the application. After several days of careful study and an incredible amount of patience, he found that right-clicking is working fine. Just the context menu appears after about 45 minutes. All menus are dynamically generated from a heap of (static!) Content with each click of the right mouse button in the main window.

At some point, users report that the option “Download data from CD-ROM” does not work at all. It took a few weeks to figure it out. But in the end, the bug report was marked as “Solved Earlier” because the data was loaded correctly. Unless loading 700 MB occupied seven days. But patience is a virtue.

Version control went hand in hand

Several years passed before one clever guy in the team had the idea to use version control tools. The first attempt was not very successful, so the team switched to another system. A couple of years - on the third, losing the whole story with each transition.

In the end, we chose a disaster tool with GUI. An abomination emanating directly from Sweden. I had to allocate a full-time team of four people to perform most tasks in this system. It looked like this:

• For the first checkout, it was necessary to make an appointment with the version control team. The meeting was usually scheduled after a week.
• Editing files was not allowed without the permission of middle managers. You must tell your manager in advance which files you want to edit, and then send a formal request for permission, which the version control team can review within a few days.
• Each code modification starts a branch. This means that then you need to combine all the modifications. With so many files in the repository, you might think that you’ll rarely see two people working on the same file. But it turned out that most of the work goes on the same 100 files or so.
• The check-in goes through a painful procedure when your code is checked by an automatic error detection program, and then by managers. Needless to say, with this approach, mistakes multiply faster than they manage to correct. A close look at the number of recorded errors showed that for each correction there were two new errors.
• Versioning is generally simple. The old program has the first version, the today's one has the second, the future one has the third. No one can tell which version they sent to the customer.

At some point, the official release date was set, completely cut off from the plans of the team. When the day came, the client was actually sent an empty CD with installation instructions, because no one was able to build software for a few weeks. The client found out that they sent him an empty disk, officially complained - and got the old version. He found out about it because the displayed date in the “About the program” field turned out to be the same as last year.

Frames

Pay crumbs - you get idiots

With such a large number of people without any software development experience, is it any wonder that mistakes were breeding in huge numbers? Some highly gifted manager found out that personnel costs are the main costs in a clean software development project. Not at all frightened by this extraordinary discovery, he decided to dismiss all people with at least some experience, but to retain all managers. It was not uncommon to see the books “C ++ for dummies” on the tables of many employees.

Meet the team

55 people in a team: 20 developers, 35 managers. You were not mistaken: there are more managers than real developers.

Managers continue to organize meetings, where they again and again show the same PowerPoint AD presentation to disgust, while developers kill time while chatting in a huge open office.

Few managers have experience in the software industry. At that time, SCO sued IBM for Linux. Even if everything was a bluff, it really worked on all these people, who understood that they would soon have to pay for free software. None of them ever mention “Software Libre”, but they all know about “Software Gratuit”. Needless to say, the project is crammed with GNU libraries, and these guys have no idea that this is how the project becomes compatible with GNU. Although, okay, given the terrible quality of this code, no one will ever insist that they open source.

Technical knowledge below the plinth. Few people know about the internet. Those who know thinks that it is made for porn. Mention that you have seen something on the Internet - and you will start to wink and smile.

welcome to Hell

All this would be funny if the top leadership did not behave like the Nazis in a concentration camp. Just give a few examples.

• It is forbidden to come to work after 9 am. One day, the senior manager stood in front of the main gate and immediately dismissed everyone who came after 9:01. Under the distribution came including a number of managers and salespeople.
• Smokers take more breaks, so they work less. The management attempted to force everyone to quit smoking on orders. Did not work out.
• Coffee machines regularly fail for a few days. Those who drink coffee do less work than those who sit at their desk, gathering precious lines of code.
• The same coffee machines are turned off whenever officials come to the office to create the impression that everyone is at work.
• Toilets are the most disgusting, which I have seen in my life. Probably, the idea was to increase productivity: the less time you spend in the toilet, the more (and better) you work.

Perhaps you are wondering why people continued to come to work in such a place. The first and main reason was the deep economic crisis that France was experiencing at that time (and is still experiencing, to a certain extent). Having a job and a salary is considered a privilege, regardless of conditions.

Another reason was that for many it was the first job. Without some comparison, it is impossible to estimate how your job sucks. Most newcomers were considered perfectly normal, which they had to come at exactly 9:00 or be fired when absolutely nothing imposed such a restriction, except for the manager's sick mind.

As for how the government allows this: we all know how it works. The guys responsible for the budget in the ministry, are friends with top managers in a number of companies. In a country like France, corruption is not uncommon at this level; it is mostly not disclosed and is rarely prosecuted. Apparently, this applies not only to France. I heard the same stories from other places in Europe and from the USA.

The next time you think that your job sucks - think again.

Posted June 24, 2008.