7 most unpleasant problems in programming

It is known that on old maps, in uncharted territories, they often put an ominous warning: "Dragons live here." Probably , the meaning of this warning was that you should not enter this space of the world, not being ready to fight with an awesome adversary. Anything can happen in these mysterious spaces, and often such “anything” can end very badly.

Programmers may be somewhat more civilized than medieval knights - but this does not mean that the modern technical world does not have its own technical dragons waiting for us in unforeseen places: complex problems that emerge precisely when the deadline for work moments of particularly high loads and responsible work; competitors who have read the manual and know that it is not well implemented; Evil "demons" who know how to use the bugs and program defects that have not been completely removed until the end, and often find out immediately after the program is put into use.

There are people who sleep peacefully at night, warmed by their naive confidence that computers are completely predictable and tirelessly give out the right answers. How little they know! Despite the most serious efforts of chip developers, creators of languages ​​and millions of programmers, there are also dangerous thickets in programming that can destroy even the most advanced programmers and developers.
')
Here are seven of the frightening corners of the programming world, where you can easily write: "Dragons live here."

Multithreading

Looks like a good idea. Split your program into independent sections and allow the operating system to process them as separate small programs. If a processor has four, six, eight, or more cores, then why not write a program to have four, six, eight, or more threads that use all cores independently?

The idea works - when all parts are, in fact, completely separate and in no way connected with each other. But if they need to access the same variables or write bits to the same files, the game is over. One of the streams will come to this data first, and it is impossible to predict which one.

Therefore, we create synchronizer monitors, semaphores, and other means to streamline multi-threading confusion. When they work, then it goes. They simply introduce another level of complexity and turn the data storage action into a variable into an operation that requires some additional intellectual effort.

When and if they do not work, chaos reigns. The resulting data loses its meaning. Columns do not fit. Money disappears from the accounts with a whistle. And that's all - just bits in memory. And great luck, if you can identify any of them. In most cases, developers end up blocking large areas of the data structure so that only one thread can interact with them. This allows you to stop the chaos, but only due to the elimination of the majority of the positive aspects of having several streams running on the same data. You could just rewrite such a program as "single-threaded."

Closures

Somewhere along the way, someone decided that it would be useful to transfer functions as if they were data. This worked well in simple cases, but programmers began to realize that problems arose when functions began to go outside on themselves and access other data, often called "free variables." Which version is correct in this case? Data at the time of function call initialization? Or at the time of her actual work? This is especially important for JavaScript, where these moments can be greatly separated.

The solution - “closure” - is one of the biggest causes of headaches for JavaScript programmers (and now Java and Swift). Beginners and even many experienced programmers cannot understand what is being closed and where there could be boundaries of this so-called closure.

The name does not help - it does not seem like a warning about the permanent closure of access, like a suggestion for pub visitors to make the last order (before closing it). In any case, access is open, but only through the wormhole in the data-time continuum - a strange mechanism of time shift, which must have been inspired by science fiction TV shows. But the names “complex access mechanism to the stack” or “data management manipulation system” seem to be too long, so “closure” was chosen. And let's not start a conversation about who should pay for non-free variables.

Data too large

When RAM begins to overflow, everything falls. It does not matter whether you are performing a sophisticated statistical analysis of consumer data or working with a familiar old spreadsheet. When a car goes beyond RAM, it refers to the so-called. virtual memory, which operates on an extremely slow (compared to RAM) hard disk or solid-state drive. This, of course, is better than a complete stop or completion of the task, but the work slows down in any case.

The problem is that the speed of hard drives is 20-30 times less than that of RAM, and the drives sold in large quantities on the market are even slower. If some other process is also trying to write to or read from it at this time, then the situation becomes simply dramatic, since these drives can perform only one task at a time.

Activating virtual memory exacerbates other, hidden problems in your software. If there are any streaming defects, they begin to manifest themselves much more strongly, since the streams entering the virtual memory on the hard disk are processed much more slowly than other streams. Other threads "hang" only for the time when the loser flow is in this memory. If the program is well done, the result is a noticeable slowdown. If the program has defects, then they quickly lead to disaster. And this is just one small example.

Managing such a situation is a serious challenge for programmers operating with large data sets. Anyone who relaxes a little while designing large dataset structures will inevitably end up with an unacceptably slow program. She will be able to pass several tests normally, but real loads will send her with a corkscrew into failure.

NP-complete task

Everyone who has a university degree in computer science knows about mysterious problems hidden behind a certain abbreviation, which is rarely explained: “the search and decision problem, which is polynomial for a non-deterministic Turing machine, or NP-complete problem. A detailed study of them takes an entire semester of a computer science course, but even in this case, many students come out with a vague presentation. Mostly that no one can solve these problems because of their exceptional difficulty.

The problems of the NP-complete problem are often quite complex - if you try to solve them using just a rough, forceful approach. For example, the duration of the solution to the " traveling salesman problem " grows exponentially as the number of route points increases. The solution of the “ backpack problem ” by finding a subset of numbers that come closest to a certain value of N requires going through all the possible subsets, the number of which is extremely large. Everyone fearfully tries to get away from these problems, because they are an example of one of the worst computer monsters: unscalable algorithms.

The caveat is that some problems of NP-complete problems are easily solved with some approximate solution. The corresponding algorithms do not promise an exact solution, but give a result that is rather close to exact. They may not give the perfect route for a traveling salesman, but their solution will differ from the correct one only by a few percent.

The existence of such pretty good decisions makes the "dragons" only more mysterious. No one can be sure in advance whether these problems are really difficult or they can be solved relatively easily - if you are satisfied with an answer that will be just good.

Security

“There is known knowledge — something that we know we know,” said Donald Rumsfeld, defense secretary of the second administration of US President George W. Bush, at a press conference once . “There is also known ignorance - that which we know, that we do not know. But there is also unknown ignorance - this is something we don’t know, we don’t know. ”

Rumsfeld talked about the war in Iraq, but the same is true for computer security. The biggest problem is the hole, about which we do not even know that they are possible. Everyone understands that it is necessary to make your password difficult to solve; this is known knowledge. But has anyone ever said that your network equipment has its own software level, hidden deep inside? The opportunity not to engage in hacking your OS, and instead aiming at this “secret” level is unknowable ignorance.

The possibility of such penetration can now not be considered as "unknown", but there may be others? We have no idea whether it is possible to repair holes if we don’t even know if they exist. You can carefully protect passwords, but there are ways of hacking that we cannot even imagine. Working with computer security delivers, of course, a lot of fun. And when it comes to programming, security-oriented thinking becomes more and more important. You can not shift on the professionals involved in security, clearing the confusion created by you.

Encryption

Encryption seems to be powerful and unapproachable when law enforcement officers stand in front of Congress and ask for official entrances to their programs to overcome it. The problem is that the bulk of encryption is built on a hazy cloud of uncertainty . What mathematical proofs do we have for unclear assumptions, such as the fact that it is difficult to factor into really large numbers or to carry out discrete logarithmization?

Are these problems really difficult? No one has publicly described any algorithms for solving them, but this does not mean that there is no such solution. If you could find a way to eavesdrop on every conversation and penetrate any bank, would you immediately tell the whole world about this, that you could help fix the holes? Or would you be silent?

The real challenge is to use encryption in your own code. Even if the basic algorithms are safe, we believe, there is a lot of work to be done in handling passwords, keys and connections. If at least one mistake is made and some password is left unprotected, then everything will be in vain.

Identity Management

Many people like a picture published in the newspaper “The New Yorker” in 1993, with the inscription: “On the Internet, no one knows that you are a dog.” It even has its own Wikipedia page with four detailed sections . (On the Internet, few people know the old joke about analyzing humor and dissecting frogs.)

The good news is that anonymity can be liberating and helpful. The bad thing is that we have no idea how to do anything other than anonymous communication. Some programmers talk about “two-factor identification and authentication,” but more advanced ones rush toward “N-factor”.

In addition to the password and, possibly, a text message on a cell phone, we have quite a bit of what you can rely on. Fingerprint readers look impressive, but many seem ready to tell and show how such an identification can be cracked (see here , here and here for beginners).

Little of this matters for the world of idle talk on Snapchat or Reddit, but the many hacked Facebook pages are a little discouraging. There is no easy way to deal with serious issues online, such as property, money, health care and to a large extent everything else in life, except for "secular" talk on general topics.

Bitcoin fans love to express how reliable a blockchain can be, but somehow Bitcoins continue to steal (see here and here ). Here we have no real method for identification.

Measurement reliability

Of course, when it comes to programming, the question arises - is there at least a way with which we can measure the complexity of the problem? No one really knows that. We know that some problems are easy to solve, but it’s quite another thing to determine how difficult it is to do. NP-completeness is only one part of a complex attempt to codify the complexity of algorithms and data analysis. The theory is useful, but it can offer no guarantees. It is tempting to say that it is difficult to even know if the problem is difficult, but this is, of course, a joke.