Misunderstanding about the omnipotence of the “Manhattan Projects”: why technology companies do not solve “real problems”

Recently, DeepMind, a division of Google Inc., announced its victory over the world champion in go thanks to a new AI system. This is an important technical achievement, the value of which cannot be underestimated, especially considering how far current breakthroughs do not correspond to the former predictions and expectations. But this essay does not address this problem, since there are more qualified people to discuss the scientific and technical implications.

There is a temptation to drop the go-go AI as an insignificant phenomenon; a computer that plays a board game instead of dealing with “really serious” problems. Some people, the reactions of which I have observed, speak in exactly this vein, but the majority positively assess the situation. Outside of the game of go, the same argument is often used about the modern technology industry. Critics say she is frivolous. " The best minds of the generation are working to get people to click on ads ." The argument is familiar, it is annoying and ubiquitous. The best minds of the generation spend time on bullshit, toys and commerce, and not on really important things. When will techies deal with real problems? - Critics ask.

Real problems?

One vivid example of such an argument can be found in an article lamenting the fact that “Silicon Valley” does not solve “big” problems. Even if this complaint is sincere, the author misses the most important thing. What are “real problems”?

The author explains:

Silicon Valley must deal with big, unpleasant and complex problems to improve the world. Problems unpleasant, tightened, threatening failures and shame. They have no ready-made markets, they treat both the poor and the rich, and problem systems. Not “what mom will do to improve my life,” but “what mom will be proud of.” They require more than just writing a check - you need to bite into them and work for years.

How can we understand this statement?
')
This is an ornate way to say that the author wants to designate Google, Facebook, Apple and other agents to solve social problems. Borrow them to fix imperfect systems. Make mom proud. Few people who spread such statements explained why this line of work would be of practical value, would be useful, and that entitles non-governmental organizations that report to shareholders or go to the stock exchange to assume the functions of government and legislators. Do we really want technology companies to be distracted from victories over board game players and business products to solve social problems? Such a question must be emotionally answered "no." The question has been turned upside down.

This essay proves the following.

1. Requirements for techno-companies to solve social problems ignore the history and empirically observable characteristics of engineering developments “from top to bottom”, and grow from a naive belief to the omnipotence of technical rationality [the theory of technical rationality asserts that after a certain technology is introduced into society, it changes concepts of what is considered rational for this society. The term was first proposed by the theorist Herbert Markus in 1941 in his book “Some Consequences of Modern Technologies” - approx. trans.]. This rationality has not only a long and very unstable history of solving social problems; it has pathological characteristics when it is used in bureaucratic organizations in an attempt to circumvent the rules. There is no clear idea about how exactly techno companies should work on these tasks, only a vague belief that they should do this, based on the worship of technical rationality (TR).

2. Technological companies are not adapted to perform quasi-government or real government functions, and they are unlikely to succeed in avoiding the problems of TR better than government agencies. And although techno companies can make a significant contribution in certain areas and with certain mechanisms, their greatest contribution is in creating the tools with which we can solve our problems ourselves. Why should we rely on technocompany in solving social problems, if it is much easier for them, both from a regulatory and a practical point of view, to give us tools that strengthen us?

The Misconception of the Manhattan Project I: The Shadow of Technical Rationality

Technocritics need to be careful with their desire to entrust the technology companies with solving "real" problems. They, in fact, require technology companies to come up with solutions to social problems, and more specifically, they require the development of top-down solutions from experts, bureaucrats, scientists and engineers. This process of rational design usually does not improve complex social problems in imperfect systems, but only serves as a source of new ones.

Loud calls for the best minds of California with the requirements to eliminate all our problems can be understood. Silicon Valley is considered a magical place. But people's love for Silicon Valley, growing from the image of the pioneers of Palo Alto, acting as technical wizards, is simply a relic of a previous love for government bureaucrats, experts, scientists and engineers. The remnants of this adoration still remain in perpetual appeals for the creation of the next " Manhattan Project " for various social, political and economic problems.

If there is something in modern American science and technology that comes close to the myths from comics, it is an establishment of researchers from the Second World War. Teams of researchers from different fields of science worked together on everything from the optimization of strategic logistics to the creation of the atomic bomb. The Second World War and the Cold War led to the emergence of a new concept of scientific progress, described in detail by the historians of science and technology. A small group of experts takes on an uncompromising problem, and combines their knowledge and expert opinions to produce a fantastic result. Therefore, such a cliché was the statement “we need a Manhattan project for ____”. People want to believe that the same joint work of scientists, which led to the creation of the atomic bomb, can be generalized to solve other complex problems (especially social ones), and this logic has a seductive appeal.

The naive call for techies to tackle the “big problems” can be categorized as a subset of a more general annoying trend: constant calls for the creation of the next Manhattan Project (MP) to solve the social, political and fundamental problems of humanity. But the real MP did something far less common and ambitious: it created an atomic bomb. The calls for the creation of a new MP reflect the American faith in TR, which I described in my previous essay . It is a belief in experts, projects, plans, and science with technology. She does not reckon with historical facts, since she is inherently messianic. MP is the equivalent of the immaculate birth of Christ.

Unfortunately, MP-like enterprises are not transferred to social and political problems. Firstly, the MP itself was sui generis [unique]. It makes no sense to believe that this model can be extended beyond the basic and applied sciences. Most of the research model of the Second World War and the Cold War is essentially combinatorial. These were experiments with different combinations, parameter settings, and lower-level component configurations in an attempt to create an engineering whole. It is not by chance that the MP led to the birth of statistical calculations in the form of stochastic simulations, or that the idea of ​​" bounded rationality " grew out of the task of using limited computational resources to solve optimization problems of Air Force logistics. Such methods of solving problems are suitable only for a narrow range of tasks, and are not suitable for others.

It should also be noted that although the MP solved one problem, it gave birth to another. If the problem of creating an atomic bomb attracted the attention and resources of the scientific community during World War II, it is not surprising that this same community took up the task of adapting such a destructive innovation to political, military and intelligence consequences. Today we live with the heritage of these events, and we cannot imagine a world where they do not exist.

In addition to the impossibility of translating the MP model to solving social problems, history indicates that attempts at rational search for solutions to these problems are, at best, a futile venture, and at worst a path to disaster. There was ample evidence of this in the past century, but the Vietnam War is the best warning for geeks with good intentions. So the main, though not the only, problem of using TR for solving social problems will be the inability to find an engineering solution for them. As theologian Eric Voeglin warned [Eric Voeglin], there is no need to make the end of the world inevitable. Unfortunately, we ignored it.

Historian Nils Gilman wrote the book Mandarins of the Future on the legacy of the theory of modernization — one of the many examples of the Cold War era, when intelligent TR turned into a terrible failure. Modernization theorists believed that societies are developing in certain linear stages of political, social and economic development. The theory of modernization is a small part of the set of attitudes known as “the highest modernism ”, claiming that science and technology have unlimited possibilities for the reorganization of the world. And although the highest modernism and the theory of modernization are different (the first is a superset of the second), they share the belief in top-down planning opportunities, finding solutions through rational design, and the superiority of scientific and technical experts. It was by such means that utopia was to be achieved.

In the USSR, the twins of the Gilman mandarin bureaucrats used similar methods to analyze and optimize a planned economy. In both cases, computers played a large, if not dominant, role. Computers could calculate schedules, find effective methods, and simulate the results of scenarios with a large number of interacting variables. Computational non-compliance became the most serious technical and scientific obstacle, due to which it was necessary to use complex approximate methods. Of course, the difference between engineering and “social engineering” is that the latter requires large-scale manipulations of human behavior, and not schedule or numerical parameters. Before computers started programming with modern methods, the concept of “programming” was related to a set of steps needed to bring an organization from its current state to the desired one.

To say that TR attempts to solve such problems from the top down failed, so to say nothing. As James C. Scott argued , the biggest assumption of these attempts was the ability to present society in a way that was understandable to experts and machines. But as any programmer who works with unstructured and unpurified data will immediately understand, the necessary data clarity is often difficult to achieve, and the cost of trying to bring them to such clarity may exceed the benefits of using the results. The same is true for large-scale social events. You can find many examples of how the efforts of smart people who tried to solve social problems through TR led to terrible consequences, from the farce program “ Millennium Development Goals ” to the horrors of Soviet collectivization . But as a vivid example, I will use the Vietnam War .

There is a reason why the war in Vietnam is considered the highest point of the theory of modernization and the highest modernism in general. Enormous computing power was thrown at solving the problem of preserving the South Vietnamese regime in the struggle against communism. Collected data for various indicators. In contrast to the myth that the United States neglected punitive actions to suppress the uprisings in favor of major battles, a large and complex bureaucracy tried to win the war "in the heads" of civilians through reform and development. But it was impossible to ignore the battles, and therefore most of the analytical resources were looking for opportunities to win over the enemy. And none of these goals was achieved.

Something went wrong? A lot of things, unfortunately.

Vietnam showed no misconceptions about the value of measurements. Measurement is a key part of any enterprise, scientific or not. But the key characteristic of many social problems is the fundamental disagreement about what to measure and how. According to the observations of the historian Gregory Daddis , such a problem constantly arose during the Vietnam War. Disagreements were not a purely professional issue. The numbers became a political weapon that justified certain political and strategic goals and actions. And when the numbers began to indicate that the underlying assumptions and assumptions were wrong, high-ranking comrades simply ignored this and mastered them. It should be noted that in addition to these problems with numbers, they turned out to be unreliable. The United States relied on its allies from South Vietnam to collect data, and they, in turn, were happy to supply data that supported such US government conclusions as they needed.

While the Americans fought and died in Vietnam, the same people who stood behind the schemes of using TR in the war began to apply them in their own country. The fight against drugs and poverty are vivid examples of the failure of this trend, despite the fact that the former has already become a real war against some segments of the American population.

Misunderstanding of the Manhattan Project II: bureaucratic rationality in the role of evil superintelligence

As shown in the last section, TP often exacerbates complex social problems rather than fixing them. And in many cases, and creates new ones! However, belief in experts and rational planning is only one part of the TR. TR is associated with organizations. Corporations, governments and other composite entities are often legal, and sometimes conceptually personalized. When we talk about them, we treat them as group agents. “The Russians did ____”, “General Motors wants to do ____”, and so on.

In this regard, it is amusing that the call for techies to save all of us occurs simultaneously with the popular panic on the topic of artificial intelligence. Ilon Musk, Stephen Hawking and others are worried if the AI ​​will not fly off the coils. Others represent a machine that rationally optimizes some objective function, regardless of the insanity or absurdity of this action. The AI ​​horror story in such scenarios is a kind of hyper-rational psychopath armed with an incredible computing power and smarter than any human.

My readers will be shocked to learn that I came to the conclusion that Musk, Hawking and their gang were right, and I was wrong. There is a danger posed by a super-intelligent, hyper-rational AI, which optimizes everything. I do not agree with them only that the appearance of this creature threatens us in the future. It has already taken control of everything. I have already met him - because of my limited military service, it can call me in case of military necessity. I had to demonstrate my driving skills to him in order to get a driver's license. After filling in the mystical scroll called " Defense Travel System form ", I already informed this all-powerful creature about the dates on which I will present my scientific work at a defense conference in London. And this artificial superintelligence subtracts a piece of each person’s income each year under pain of prison. Yes, dear reader, I am talking about the federal government. Uncle Sam. Star-striped. The feds.

Of course, I do not claim that the federal government is bad in its essence. It performs an important function, and without it we would be worse. I am simply saying that the federal government is a huge impersonal bureaucracy. The bureaucratic dimension of technical rationality can be seen as a sub-AI, since most of the people's fantasies about omnipotent, super-intellectual and hyper-rational entities describe what already exists in huge impersonal bureaucracies. TR Pathologies is a nightmare scenario that Musk and Hawking fear. Their great nightmare - a super-intellectual and hyper-rational, but at the same time harmful and psychopathic essence controlling us - has already been realized many times.

All the signs of the speculative super-intelligent machine Mask and Hawking are already present in the composite bureaucratic group agents. They receive intelligence from a combination of pure processing power (resources exceeding the resources of individuals) and algorithms (bureaucratic programs and procedures). And, just like the horror story of AI opponents, the bureaucracy is rational in the sense of finding effective ways to achieve goals, but it often lacks what philosophers, artists, and statesmen call “common sense” or “reason.”

Unfortunately, I can not boast of the authorship of this idea - Joanna Bryson wrote about it in various posts. But there are earlier descriptions of this idea. Computers and bureaucracies approach each other so much, because bureaucratic organizations can be viewed as a type of technology. And this is not a metaphor. According to sociologist Max Weber , the ideal bureaucracy has the following characteristics:
1. The hierarchy of power.
2. Impersonality.
3. Prescribed management rules.
4. Promotion based on achievements.
5. The division of labor.
6. Efficiency.

Some of them can be called computational in essence. Computer science often represents complex systems that it models or creates as hierarchical abstractions with special rules that allow the system to move from one discrete state to another. Weber and Herbert Simon [ Herbert Simon ] suggest that complex social artifacts are also result-oriented and made so that their internal structure and organization of behavior is built to achieve certain goals, taking into account the needs of the external environment. Bureaucracies are also a collection of people and machines working together to achieve these goals. Therefore, the problems of TR grow not only out of arrogance. The disadvantages of TR appear due to the pathologies of rationalization and its dominance in social life. Weber suggests that in the era of dominating rationalization processes, the dominance of computing will become the motivation and cause of social action.

If the singularity of the Mask and the company has long occurred, then we live in its consequences. And they, to put it mildly, are not too encouraging. Think of all the sufferings of people that occurred because of this bureaucratic process, working from top to bottom, from trivial (the queue in the department of vehicle registration) to the most terrible (the number of deaths in communist countries caused by the mistakes of planned economies during the Cold War) . Or think about the phenomenon that is called "Kafkian." Kafka described an impersonal, insensitive, opaque, and tireless bureaucracy, which in every way was also insane. The massive power, impersonality and opacity of the bureaucracy ensures the victim’s inability to resist, but the madness of this bureaucratic machine makes it a victim of man.

Misunderstanding of the Manhattan Project III: a false rumor about technology and real problems

This retreat has far removed us from go, deep learning, AI, and what technology companies should and should not do. Explaining in detail why the idea that technology, rational design and experts can solve complex social problems is wrong, I will now explain why the constant requests from the technical world to do something — anything — and use technology, rational design and experts. to solve complex social problems is not just wrong, but incredibly stupid. A very large part of the social problems of the world stems from the fact that technical rationality and bureaucratic "technology" have gone astray. Even more of the same is not a solution, but part of the very problem that technical rationality is supposed to solve.

The idea that tech companies should devote most of their efforts to solving the big social problems that governments and international organizations are about to take is crazy. It is very difficult to understand how the objective history of the twentieth century can support the belief that all problems can be solved, and that only a few intelligent people are needed for this. It is also difficult to understand how it is possible not to conclude that top-down planning by scientists and technology experts in social and political areas has led, to put it mildly, to undesirable consequences. But this is exactly what follows from the complaints that the technoindustry does not deal with big problems. Why are people so eager to see how engineers and bureaucrats will use computers to solve problems created by another team of engineers and bureaucrats with computers? But it does not even begin to explain the idiocy of such a meme.

Critics are constantly asking, “why aren't techies doing really important tasks?”, Without any understanding of what they are actually asking. Technocompanies perfectly cope with technological and business things, such as management, logistics, production, marketing, research and development, software development, support for software platforms. And the only thing that makes them attractive tools for solving problems is the feeling of their expertise, technological competence and resources. “If they are so smart that they did [the technology business thing], why can't they do [a political / social thing that is not related to it]?” With this approach, you can ask your native language teacher to build a bridge, your postman to calculate your taxes, or your therapist to teach you kung fu.

I do not understand why it is difficult to understand. Many people know that government is not a business. For many reasons, the Ministry of Defense or the Foreign Ministry will not turn into flexible startups in which young and hungry entrepreneurs hunt for investors' money. And the problems of the government almost do not overlap with the problems of business. Example? If the Facebook team sends you an ad you don’t like, you can remove it from your feed with one click. If, due to incorrect procurement decisions made by the Defense Ministry, the tank will not be able to fire, a noisy crowd of young Americans will disappear when the enemy fires back. The objectives, motivation and competence of technology companies are different from government, and they are not adapted to perform the functions of government.

Techno-companies can benefit society and play a valuable role in social action. The question is exactly what role. Google has something to offer for solving social problems, and they are already working in this direction with such things as Google Ideas. Having been at the joint conference of the Council on Foreign Relations and Google Ideas, I became a supporter of this model of partnership. Both sides have the experience, knowledge and the opportunity to make a contribution. The technological world can offer a lot of things that the government and non-profit sector do not have.

But the inconvenient truth is that people who call on technology to solve big problems unreasonably believe that experience in technology and business helps in solving social problems and is transferred outside the areas of expertise in technology and business. And also there are no ideas about how exactly Google, Facebook or someone else can start solving these problems. The abstract idea that they are doing this is attractive, but few can say something more concrete. Blurred language about changing the world and doing really important things is not enough.

And finally, a vague complaint about the fact that “technologies must change the world” reminds one more such stupid meme: anything has the right to life if it is worthy of discussion. For example, there will always be research, development and financing of projects with an undeniable "influence on society." In the case of the Pentagon, the influence will consist in sending warheads against enemy targets. And if you do not mean this under the influence of society, these are your problems. The complaint is that techies do not deal with complex issues. Well, even if it is, in your opinion, a not particularly worthy difficult problem, then the preservation of US military superiority is a really difficult problem!

Have you seen briefings on strategic balance in the Pacific? In my opinion, this is quite difficult, and the impact on society is evident. And as a person who often works in the defense and security sectors, I think this is a worthy problem. And although I specifically add comic to the situation, my thought is very serious. How techies change the world, or work with serious problems, is also important, and only that a certain tech company or agency that finances a technological project works on solving a complex and important problem does not make them automatic. Good.

The Misconception of the Manhattan Project IV: Let Him Play

Perhaps the most mysterious thing about “technicians do not solve Complicated Problems” in a meme is that critics demand technical companies to devote the time of their engineers to solving social problems - and it is for these very same critics who would hate these very companies if their desire come true. However, they ignore the simple way technology can serve as a tool for achieving goals. TR served the appearance of serious problems in our world. But she did a lot to improve our lives. The presence of good tools allows us to use them in the way we consider necessary to solve complex problems independently, and not for passive-aggressive requirements for technical companies to solve these problems for us. And good tools give us the opportunity to use the positive side of TR and get rid of the negative.

One example of using technology as a resource for solving social problems is the use of drones to help Aboriginal people to protect their land, support in dealing with the effects of natural disasters and protecting archaeological values. All of these problems existed before the advent of inexpensive commercial drones, and people will use other technologies to solve them for a long time after the disappearance of drones. But the appearance of drones really changed the situation. Of course, even the most efficient technology is a temporary solution. But a temporary solution is better than nothing.

Access to free software, such as libraries for working with data from R or Python, will not solve the problem of hunger. But the necessary part of solving these problems will be the possibility of conducting a statistical analysis of the numerical components of the problem. In the past, this task required paying for expensive licenses to use Stata, SPSS, Eviews, SAS, and other such tools. Now people can do it for free with free software and share the results through the same open source tools like Jupyter Notebooks or Github. Because it reduces the cost of using analytical methods for solving social problems, it is much better than if we did not have libraries for R or Python.

But there is no free lunch either. Both money and high-quality and constantly updated technical systems do not grow on trees. You can't get something for nothing. People do not create and do not support complex technical innovations out of the goodness of their soul and because of the noble desire to help the oppressed inhabitants of Zambuistan. There are exceptions, but how long do they last? People do something of self-interest, whether it be material reward or fame and recognition. Even non-profit, non-governmental organizations are competing for funding, resources, and members.

Now, let's go back to the go-go program I mentioned at the beginning of this essay. The real expectations of AI are that they will give us tools suitable for use. Companies like Google can already help solve social problems by producing tools that people can use to solve them. Tensorflow in the hands of a specialist using it correctly will be useful. Having good tools is much better than not having them. So the criticism of techno-companies for concentrating on supposedly trivial things is directed in the wrong direction.

Of course, there is some kind of childishness in wondering how computers can beat people in board games. So what?How many people went to the army because they were playing with soldiers in childhood? How many went to science because they listened to popularizers of science? People need entertainment and motivation to do something complicated. Winning a person with go is a very difficult task due to the mathematical complexity of the game. But Google’s machines have learned this. Hurray them. And while scientific statements about the wider capabilities of the system playing games will be confirmed, there is an opportunity to use them for something other than the game of go. And what's wrong with that?

Is there any urgent social problem that Google, unlike the United Nations, Amnesty International or the US government, has unique capabilities to solve, and which the company ignores, creating cool machine learning algorithms instead for go? I do not think.But any of these public organizations, armed with Google technologies, will receive at least new opportunities for work. And these opportunities might not appear if Google didn’t have enough motivation to develop technologies by trying to beat people in board games. Does the possession of improved technology mean that the problem will change? Not.But the possession of such technologies simplifies problem solving. So let Google play go, or spend a lot of money and man-hours on developing anything else. If this leads to what other people can use, great. If not, I don’t see what losses it will cost us.