The second part ofTranslator's note: This article is a translation of the publication “The AI Revolution: The Road to Superintelligence” . The original article was written for a wide audience, so many terms used in it may not be accurate or not at all scientific. When translating, I tried to keep the spontaneous spirit of the article and the humor with which the original was written. Unfortunately, it did not always work. The translator does not agree with everything that is written in this article, but the edits to the facts and his opinion were not added even in the form of notes or comments. There may be mistakes and typos in the text, please report them to your private messages, I will try to correct everything as quickly as possible. All links in the text are copied from the original article and lead to English-language resources.It is a life of peace on earth. - Vernor Vinge
We are on the verge of comparable changes, except with the very birth of humanity. Vernor Vinge.
')
How does it feel to be here?

Fascinating, is it true? But now we do not see the right side of the graph.
And it seems to us that we are here.

Not so interesting ...
The distant future is not so distantLet's imagine that we have a time machine, and we are transported to the 1750th year, to a world in which electricity was still unknown, where sending messages over long distances was only shouting over the field or shooting into the air from guns, and the only fuel for transport was hay. Take from there the first passer-by and are transported back to 2015, after which, walking with him through any modern city, we observe his reaction to things we are used to. We can’t even imagine what he will experience by watching sparkling capsules sweeping past the highway, talking to people still in the morning on another continent, watching hockey players playing thousands of kilometers away from the stadium, listening to a piece of music that the orchestra played 50 years ago or considering my magic rectangle that allows you to draw a realistic portrait of any person in one instant, look at the map and see your location in real time, or talk with the person on the other end of the country, and other things of magical origin. And this is before you tell him about the Internet, the ISS, the Large Hadron Collider, nuclear weapons or the general theory of relativity.
This experience will be shocking for him, or even a “roof-breaking” and this only partially describes his impressions. He may even die of shocked shock.
But what will happen if this person, upon returning back to his native 1750th year, wants to repeat our experiment, sits in a time machine and transfers to the same 250 years ago in the 1500th year, takes the first passer-by in his time and shows is his life in 1750? A man from the 1500th will be surprised at the progress and many unfamiliar things in the 1750th, but this is unlikely to be enough to kill him. His feelings will not be such a "roof-demolishing." The differences between life in the 1500th and in the 1750th are much smaller than between life in the 1750th and in 2015. A man from 1500 learns many amazing things about space and physics, will be surprised by the scale of European Empires, and of course, he will have to make strong amendments to his view of the world map. But the differences in everyday life in the 1500s and 1750s, transportation, communications, and other things, will definitely not be enough to kill him.
In order for our acquaintance from the 1750s to have the same fun as we and he, he will have to travel much further, approximately 12,000 years before our era, in the times before the First Agrarian Revolution, which led to the creation of the first cities, and gave the opportunity for the development of civilization. If someone from the hunting-gathering society, from the time when man was essentially just another animal species, would have seen huge empires of 1750, magnificent churches, ocean-going ships, huge communes, and human knowledge accumulated by mankind, is dead.
But what if, after death, he also wanted to repeat our experiment? If he is transferred from 12000 BC to 24000 BC, takes a person living there and returns with him back to 12000 BC, shows him life in his time, then the person will probably say that something like: “ok, and what has changed?”. For a person from 12000 BC, in order to entertain himself, you will have to travel back over 100,000 years to find someone whom he can surprise with the fire and existence of a language.
In order to die from the experienced shock of traveling to the future, you need to move a sufficient distance to reach the “killing level of progress”, or the Killing Progress Unit (EGD). Since the EDP took 100,000 years for a hunting-gathering society, for society after the agrarian revolution, one EDP took only about 12,000 years, the Postindustrial world advanced so fast that for a person from 1750-year you need to move only a couple of hundred years ahead get one EDP.
The basic idea is that progress is accelerating all the time. This is what futurist Ray Kurzweil calls the Law of Accelerating Return. This is because a more developed society tends to grow faster than a less developed one. In the 19th century, humanity had much more knowledge and more advanced technologies than in the 15th century, so it is not surprising that humanity made much more discoveries in the 19th century than in the 15th century.
It also works on a smaller scale. The film “Back to the Future” was released in 1985, and the past was represented in it in 1955. In the film, when the hero Michael J. Fox set off in 1955, he was surprised that television was still a wonder, the price of soda of that time, lack of love for guitar drums and unusual slang. It was a completely different world, but if the film were filmed today, and the past would have been presented in 1985, the film would have been perhaps more funny and would display a much larger difference between the worlds. The hero would be in time to personal computers, the Internet, mobile phones, and social networks. The teenager, born in the late 90s, would feel much more alien in 1985 than Marty McFly in 1955.
The reason for this is the same law of accelerating returns. The average rate of change between 1985 and 2015 was much higher than the rate of change between 1955 and 1985, thanks to the fact that the world in 1985 was much more developed, which allowed much more changes to occur in recent years. 30 years than in the previous 30 years.
So the changes are getting bigger and their speed is increasing. It tells us that an interesting enough future awaits us, doesn't it?
Kursweil believes that the equivalent of progress of the entire 20th century would have been achieved in 20 years, while maintaining the pace of development of the year 2000. That is, the pace of development in the year 2000 was five times higher than the average growth rate for the 20th century. Ray also claims that the equivalent of progress for the entire 20th century has already been achieved between 2000 and 2014, and that the next equivalent of progress of the 20th century will be achieved by the year 2021.
If Kurzweil is right, then we can be just as shocked by 2030, as our friend from 1750 was shocked by 2015. And the next EDP will be reached in just a couple of decades, and the world in 2050 will be so different from today that we could hardly recognize it at all.
And this is not science fiction. This is what the best minds of today believe in, who are much smarter than you and me and have great knowledge in this field. Looking back at the history, their predictions start to look quite logical.
So why when I say “the world in 35 years may turn out to be unrecognizable for us”, you think “cool ... but hardly” Three reasons for skeptical attitude to the “roof-blowing” predictions of the future:
1. It seems to us that history is linear. Trying to imagine the progress of the coming 30 years, we look back at a similar period of time to imagine the scale of the possible changes. Predicting how the 21st century will change the world, we take progress for the 20th century and add it to the 2000th year. It was this mistake that our acquaintance of 1750 made, having chosen a person from 1500, as an experimental one, expecting that the impressions from 1750 would be comparable to his own from traveling the same time forward. We are accustomed to think linearly, whereas here it is necessary to think exponentially. A much more logical method of predicting future changes would not be to look back 30 years, but to draw conclusions based on the current speed of development. Such a forecast will be much more accurate, but still not enough. To predict the future more precisely, it is necessary to take as a basis an even greater rate of progress.
2. Recent events do not give us an understanding of the overall picture of what is happening. First, if you take a small enough piece of even a very steep exponential curve, it will appear straight, just as if you take a huge circle and look at it closely enough, it will seem quite straight. Secondly, exponential growth, in fact, not smooth. As Kurzweil explains: progress actually takes place along an S-shaped curve.

The S-shaped curve is born when a new paradigm only appears. The curve goes through three stages:
Slow growth (initial stage of exponential growth).
Rapid growth (late, explosive stage of exponential growth).
Alignment is already a new, ripe paradigm.
If we take into account only recent events, then that part of the S shaped curve in which we are in can distort our understanding of the speed of technology development. The length of time between 1995 and 2007 was extremely intense, he found: the explosive growth of the Internet, the rise of Microsoft, Google and Facebook, the birth of the phenomenon of social networking, the massive emergence of mobile phones and later smartphones. This was the second phase of the S-shaped curve, and the period 2008-2015 was no longer so impressive, at least in terms of technology. And making a development forecast taking as a basis only the speed of changes in recent years, it is impossible to get an accurate idea of what awaits us, since in this case we lose sight of the overall picture. Perhaps we are already standing on the threshold of a new large second phase S-shaped curve of exponential growth.
3. Our experience turns us into stubborn old people when we think about the future. Our understanding of the world is based on our experience, and it defines our understanding of the speed of development, based on current events. We are also limited by our imagination, which builds assumptions based on our experience, but our knowledge does not allow us to make accurate predictions of the future. And when we hear predictions that do not work out into our picture of the world, we tend to think of them as stupid and unsubstantiated. When later in this article I’ll tell you that you’ll probably live 150-250 years, or you don’t die at all, your subconscious will say something like “This is just ridiculous, the only thing I am sure of is that everyone will die sooner or later. ” Yes, of course, none of those who ever lived on earth has yet been able to survive his own death. But no one flew on airplanes until they were invented.
Logically, then we should expect a repetition of historical patterns, that is, we expect much greater changes in the coming decades than our intuition tells us. It is also logical to assume that if the most advanced creatures on our planet make more and more breakthroughs, with an ever-increasing pace, then sooner or later they will make a breakthrough so huge that it will change the whole world and themselves. In much the same way as evolution made a breakthrough after a breakthrough on the path of creating a mind, until it created a person who completely changed the planet and the life of all creatures on it. And if you look closely at what is happening now in science and technology carefully enough, you will see signs of a coming breakthrough that will once again completely change our world.
The Path to the OvermindWhat is artificial intelligence (AI)?For me, I think, as for you, artificial intelligence, until recently, was just an invention of science fiction, but recently everything changed when quite respected people began to talk about it, and you probably don’t understand what caused it.
There are three reasons why most people do not fully understand the definition of “artificial intelligence”:
1.and associated with movies. Star Wars, Terminator, Space Odyssey 2001, and even the Jetsons. All this is fantastic, as well as heroes-robots. Therefore, AI sounds too fantastic.
2. And too broad a concept. The spread of forms of AI is too large: AI varies from calculators to unmanned vehicles. The term AI applies to too many things, which makes understanding it quite difficult.
2. Even when we use AI, we are not always aware of this. John McCarthy, who proposed the term "AI" in 1956, remarked: "as soon as he starts working, no one else calls him artificial intelligence." It is for this reason that AI is often perceived as an absurd concept from a very distant future, and not as something real. And at the same time, it sounds like a pop concert from the past, which we will never visit. Ray Kurzweil says that he heard from a large number of people that the AI is something that died already in the 80s, which, according to his own statement, is equivalent to the statement that the Internet died during the dotcom crash at the beginning of zero.
For a better understanding of AI, we first need to stop imagining humanoid robots, a robot is just a shell for AI, in fact, AI is a computer inside a robot, unless of course this computer was intentionally inserted into a robot. In the same way, the software is responsible for the operation of Siri. This is AI, and a pleasant female voice, this is just a form with user interaction, and no robot is involved here.
Secondly, you are probably familiar with the concept of singularity or technological singularity. Singularity is a concept from mathematics describing a situation in which customary rules stop working. It is also used by physicists to describe the incredibly dense matter of black holes or to describe the whole matter of the universe until the Big Bang, again the situation to which the laws of physics we are used to do not apply. Vernor Vinge in his
famous essay used the term technological singularity by the time in the future when the capabilities of AI exceed our own. According to him, at this moment our world will be changed so much that the rules we are used to will no longer apply to it. Later, Ray Kurzweil defined the technological singularity as the moment in which, according to the law of accelerating returns, the speed of development will tend to infinity, and in which, our world will change drastically. I noticed that many people associated with AI for some reason stopped using this term, so I will not use it too much in this article (despite the fact that it will be a question of technological singularity). And last, despite the fact that the concept of AI is too vague, all AI can be divided into three categories according to their capabilities.
1. AI narrow application. It is also sometimes called weak AI. This AI is specialized in some strictly defined area. Such an AI can beat the best chess players in the world, but this is where its capabilities end. He is not capable of anything more.
2. AI of general use. He is a strong AI, or human-level AI. This is an AI that corresponds to the level of human intelligence in all areas, a machine capable of performing all the same tasks as a human being. The task of creating a strong AI is incomparably more difficult than the task of creating a weak AI, and we have yet to solve it. Professor Linda Gottfredson describes the mind as "a general mental activity, capable of, among other things, awareness, planning, problem solving, abstract thinking, understanding complex ideas, fast learning, and learning from one's own experience." A strong AI will be able to do all this as easily as any person.
3. Artificial Overmind. Nick Bostrom, philosopher and chairman of the World Transhuman Association, gives the following
definition to the Overmind: “a mind that transcends the best minds of humanity in all areas, including general knowledge, and social skills”. Artificial Overmind is a rather broad concept, which differs from a mind only slightly superior to a man, to a mind of a superior man in all areas trillions of times. The artificial Supermind itself is the reason for such a heated discussion around AI, as well as the fact that the words immortality and extinction in AI discussions are so common.
To date, mankind has already built and uses many weak AI.
AI revolution is the path from a weak AI to an artificial Overmind, running through the creation of a strong AI. Let's take a closer look at what the best AI theorists and scientists think and why a revolution can happen much faster than we expect.AI already governs our world ofnarrow-range AI — whose AI is equal or superior to human capabilities in a certain area. A few examples:- : , , , , - . , Google, , .
- : - Amazon, . Facebook - , , . , , , , , , . “ ...”, , , .
- Google Translate — , , . — . , , , , .
Narrowly used AIs that exist now present no particular danger to us. In the worst case, they can provoke a small catastrophe that does not affect most of the population of the planet or even a country. For example, rolling blackouts, problems with equipment at a nuclear power plant or a catastrophe on the exchange (for example, black Thursday, May 6, 2010, when AI crashed brought down the market, taking more than a trillion dollars of market value with itself, after restoring the system lost money).So far, AI does not have enough power and capabilities to represent a danger to life on the planet, but we are already seeing an increasing network of AI, which are the precursors of other AI that can drastically change our world. According toAaron Saenz (Aaron Saenz), the existing AI are similar to the amino acids in the primary soup, from which life once originated on earth.The path from weak to strong AIWhy is it so difficult?Nothing will make you admire the human intellect more than knowing how difficult it is to create a computer equal in its abilities. The construction of skyscrapers, the conquest of space and a detailed understanding of the events that took place after the big bang cannot be compared with the understanding of the work of our own brain. Today, the human brain is the most complex phenomenon of all that we know.Interestingly, when building a strong AI, problems do not arise where you would expect. It is easy to build a computer that can multiply two ten-digit numbers in less than a second, but to build a computer that distinguishes a dog from a cat is very difficult. We have already built a computer that can beat the best chess players. But to build a computer that could understand the children's book, and not just recognize the words, is still very difficult. Google is investing billions nowdollars to create such an AI. Complex things such as computing, trading strategy, translation from one language to another, are given to computers very easily. But such light things as sight, movement and perception are incredibly complex. As the scientist Donald Knut said, “AI today has succeeded in everything that requires thinking, but they are not given what the people and animals do unconsciously.” If you think about it, then rather quickly comes the understanding that things that seem to us to be light are in fact incredibly complex, and they seem to be easy to us only because these skills have been optimized over millions of years of evolution. When we stretch our hand to an object, our muscles, tendons and bones in the shoulder, elbow and wrist make a huge number of movements, coordinate these movements with the information obtained through vision,so that we can make a clear, straight arm movement. It seems that all this is happening without any tension, and it seems very easy, but this is only because our brain is equipped with perfect software, developed and optimized by evolution specifically for these operations. On the other hand, the multiplication of long numbers or the game of chess are new challenges for biological creatures, and we did not have enough time to optimize these skills, so it is much easier for the computer to defeat us in this field. Consider which program you would most like to write: a program that can multiply long numbers, or one that could recognize the letter “B” in all variations of fonts and even in all the variety of handwritten spelling variants?that all this is happening without any tension, and it seems very easy, but this is only because in our brain there is a perfect software, developed and optimized by evolution specifically for these operations. On the other hand, the multiplication of long numbers or the game of chess are new challenges for biological creatures, and we did not have enough time to optimize these skills, so it is much easier for the computer to defeat us in this field. Consider which program you would most like to write: a program that can multiply long numbers, or one that could recognize the letter “B” in all variations of fonts and even in all the variety of handwritten spelling variants?that all this is happening without any tension, and it seems very easy, but this is only because in our brain there is a perfect software, developed and optimized by evolution specifically for these operations. On the other hand, the multiplication of long numbers or the game of chess are new challenges for biological creatures, and we did not have enough time to optimize these skills, so it is much easier for the computer to defeat us in this field. Consider which program you would most like to write: a program that can multiply long numbers, or one that could recognize the letter “B” in all variations of fonts and even in all the variety of handwritten spelling variants?developed and optimized by evolution specifically for these operations. On the other hand, the multiplication of long numbers or the game of chess are new challenges for biological creatures, and we did not have enough time to optimize these skills, so it is much easier for the computer to defeat us in this field. Consider which program you would most like to write: a program that can multiply long numbers, or one that could recognize the letter “B” in all variations of fonts and even in all the variety of handwritten spelling variants?developed and optimized by evolution specifically for these operations. On the other hand, the multiplication of long numbers or the game of chess are new challenges for biological creatures, and we did not have enough time to optimize these skills, so it is much easier for the computer to defeat us in this field. Consider which program you would most like to write: a program that can multiply long numbers, or one that could recognize the letter “B” in all variations of fonts and even in all the variety of handwritten spelling variants?therefore, it is much easier for the computer to defeat us in this field. Consider which program you would most like to write: a program that can multiply long numbers, or one that could recognize the letter “B” in all variations of fonts and even in all the variety of handwritten spelling variants?therefore, it is much easier for the computer to defeat us in this field. Consider which program you would most like to write: a program that can multiply long numbers, or one that could recognize the letter “B” in all variations of fonts and even in all the variety of handwritten spelling variants?Interesting fact: If you give this picture to the computer and the person, then both will be able to understand that this is a rectangle composed of squares of two different colors.
But if you remove the black part and open the picture completely ...
It’s not difficult for a person to recognize all these cylinders, different planes and three-dimensional angles, but for a computer this task will be overwhelming, he will see only two-dimensional shapes and shadows that he cannot correctly interpret. Our brain performs a huge number of operations to interpret these shapes and shadows into three-dimensional objects. Similarly, for the picture below, the computer on it will see only a two-dimensional black-and-white picture, when a person is able to understand that this is a photograph of a black three-dimensional stone.
Credit: Matthew LloydAll our examples were limited to the simplest frozen pictures. In order to be called a strong AI computer it will be necessary to be able to recognize facial expressions, and understand the difference between different emotions.So how do we still achieve this?The first step to creating a strong AI: increase computing powerThe increase in computing power is a necessary element in creating a strong AI. If an AI should be as smart as a person, then the computational capabilities of its hardware should be close to the computational abilities of the human brain. One way to determine the processing power of the brain is to count the number of operations performed per second. To do this, it is necessary to calculate the maximum number of operations per second of different parts of the brain and add up the resulting figures. Ray Kurzweil came up with a simpler way to estimate this indicator. He took the approximate number of operations per second performed by the part of the brain counted by the scientists, and the ratio of the mass of this part to the mass of the whole brain, and multiplied them. It sounds doubtfulbut he did this operation several times taking as a basis the data obtained by different scientists and always came to approximately the same result - 10 ^ 16 (10 quadrillion) operations per second. To date, the most powerful super-computer, ChineseTianhe-2 , has already surpassed this figure, giving out about 34 quadrillion operations per second. But Tianhe-2 is a huge machine, occupying 720 square meters of space and consuming 24 megawatts of energy (our brain consumes only 20 watts ), and its construction cost 390 million dollars. This is certainly not a mass product, at the moment he did not even find any commercial use. We rate computers by how much computing power you can buy for $ 1,000. Kurzweil is confident that strong AIs will get their chance to become a mass product, when for $ 1,000 you can buy a computer capable of issuing 10 ^ 16 operations per second.
According to Moore's Lawwhich runs throughout the lifetime of computers, the maximum computing power doubles approximately every two years. That is, the development of computers, as well as human development, has an exponential character. If you look at the chart at the top, then we are now exactly where Kurzweil expected. That is, at around 10 trillion operations per second for $ 1,000. That is, modern computers at the cost of $ 1,000 have already surpassed the capabilities of the mouse brain. It doesn't sound very impressive, but if we recall that in 1985 we were at the level of one trillionth of the human brain, one billionth in 1995 and one millionth in 2005, then in 2015 it is already one thousandth. This progression easily fits the creation by 2025 of a computer of equal computational capacity to the human brain worth $ 1,000.To date, iron has already allowed us to create a strong AI and make it a mass product for 10 years. But computing power alone does not make a computer a strong AI. So the next task is to unleash the potential of this computer.The second step to creating a strong AI: Make it smartThis is quite a difficult part. No one really understands how to make a computer smart. So far, disputes about how to ensure that the AI could understand the word, no matter how it is written, or distinguish a dog from a cat, are not abating. But today there are already several strategies for creating a strong AI, and sooner or later one of them will be successful and will allow you to create a strong AI. A little more detail on the existing strategies:1.Copying of the brainIt is as if scientists are sitting next to a child, to whom everything is given so easily and naturally, that he continues, despite all the efforts of scientists, to get higher marks for checking that they at some point think: hell with all, why not just write him off. ” And it makes sense. We are trying in vain to build a super complex computer, but we are lucky to have a wonderful prototype in the form of our own brain, so why not take advantage of it. Scientists around the world are trying to unravel the principles of the brain, and how evolution managed to create such a cool thing. According to optimistic forecaststhey will succeed around 2030. As soon as scientists solve all the secrets of the brain, we will understand how he manages to be so powerful and effective, and we can use this knowledge to create a more powerful AI. One way to copy the work of the brain to create a computer is to use artificial neural circuits. In the beginning, this is simply a network of interconnected transistors of “neurons,” which, like an infant’s brain, do not possess any knowledge. Then this “brain” learns, for example, trying to recognize a handwritten text, simply gives random letters from the beginning, but after it guesses the first letter, the neural connections that issued this answer are amplified (fixed), if the answer was incorrect, the connections are weakened. This process is repeated a great number of times.in this way, the machine learns and is optimized for the performance of a specific task. The brain works on a similar principle, and we continue to study the creation and weakening of neural connections, thus discovering all the new ways of teaching artificial neural circuits. A more complex way to emulate the work of the brain is to completely recreate its structure. To do this, the brain is taken, cut into thin layers, scanned and then using its powerful software, its virtual model is built, which is run on a powerful computer. As a result, we get a computer that is capable of all the operations that the human brain is capable of; it only remains to train it. If scientists can develop an ultra-accurate way to copy the brain, then we can even getthat all memories and even the identity of the brain master will be transferred to the computer. So if the brain belonged to Jim right before he died, then our computer will wake up Jim (? ), and we get a strong AI, capable of all the same what a person is capable of, then we can turn him into the Overmind, which Jim will surely like.But how far are we from the complete emulation of the brain? Recently , the work of a 1mm slice of the brain containing only 302 neurons was reconstructed. There are about 100 billion neurons in the human brain. If you are not impressed with the result, then it's time to recall the exponential growth. Now we can recreate the work of the brain of the worm, then, after a long enough period, we will recreate the work of the brain of the ant, then the mouse and suddenly this idea will already seem much more plausible.2. Making evolution work for usIf we come to the point that it is too difficult to write off our prodigy, you can try to copy his exam preparation technique. To build a computer of equal capacity in our brain is possible, the evolution of the brain is proof of this. And if recreating the brain’s work is too difficult, why not imitate evolution? As practice shows, accurate copying of biological prototypes is not always the best solution, and an airplane wing is a vivid example.So how to make evolution build us a strong AI? The answer to this question is a genetic algorithm that works as follows: each time a task is completed, an assessment of the results is carried out, this happens time after time (in the process of evolution, the “productivity” of a living being was measured according to whether it could transfer its genes to the next generations or not) . A group of computers will work on solving one problem, and the most successful will “cross over”, transferring half of their software to a new machine. After a long process of such selection, more and more productive computers will be created. The biggest problem here is the creation of an automatic system for evaluating results and “crossing”. The disadvantage of this approach is that such an “evolution” can take millions of yearsand we need the result over the next decades.But we have an advantage over evolution, in the first place evolution evolves haphazardly, and for every useful mutation there are far more useless or even harmful mutations. If we control the process, retaining only useful mutations and even pushing computers towards them, we can significantly speed up the process. Secondly, evolution does not have a goal, it can choose a way to reduce the “performance” of the brain, because it is a luxury for an animal to hold a large brain, it consumes too much energy. Third, in order for evolution to create a more productive brain, it needs to solve the problem of supplying energy to the cells, but we do not have such a problem, since we will use pure electricity. Without a doubt, we will cope with this task much faster than evolution, but it remains an open question whether it will be fast enoughthat the game was worth the candle.3. Making AI create itselfWhen scientists come to a dead end, they try to create an experiment that would conduct itself. And this is probably the most promising way.His main idea of this method is to create a weak AI with a specialization in the study of AI and making changes to his own code, which will allow him not only to learn, but also to create himself. We will teach the computer to be a programmer, and thus the computers themselves will help us improve them. And their main task will be to develop a way to improve them themselves.And all this is the matter of the near future.Fast progress in the field of equipment, and various experiments with software, can bring strong AI into our life very quickly. And there are two reasons for this:1. The exponential growth has reached such speeds that even what seems to be moving very slowly seems to lead to explosive growth in a short period of time. Gif at the bottom illustrates this process well:
A source2. As far as software is concerned, development may seem slow, but one discovery can drastically change the situation (just as science was treading on the spot, trying to describe the movement of celestial bodies in a geocentric system, but when people took the heliocentric system of the world as a basis, everything it became much easier). Similarly with a self-developing computer. Yesterday it seemed that he was very far from creating a strong AI, but one “mutation” in his software can make it 1000 times more effective, and now he is on the threshold of the human level of mind.The Path from the Strong AI to the OvermindWhen we reach the AI of the human level, we will live in peace and equality with them.Of course not.A strong AI with the same level of intelligence as a person actually has tremendous advantages:Hardware:- . 200 , 2 , 10 , , . 120 / , .
- Reliability. , . , , ( ). , 24/7.
Software:- Humanity has bypassed all other types of animals in regard to the interaction between individuals. It began with the creation of language and the formation of large communities, then this ability was strengthened by the invention of writing and printing, and in recent times, with the development of the Internet, received a huge impetus in its development. And thanks to our collective intelligence, we were able to go so far in terms of development level from all other animal species. Computers in turn will bypass us in this. Thanks to the Internet, AIs will be able to synchronize their knowledge, and thus each of them will have access to all the knowledge gained by any of the AIs. They will also be able to work more effectively than people on one task, since they will not have a conflict of interests, bad mood, problems with motivation and other problems peculiar to a person. An AI that can reach the human level through self-programming will look at the “human intelligence level” as nothing more than an important milestone in its development, and, of course, nothing will make him stay at this level, he will slip it and increase it with huge leaps their “mental abilities”, using the whole arsenal of their advantages over man. And sooner or later it will become an artificial Overmind.
And it certainly will scare mankind, and there are reasons for this, a) We know all the animals inhabiting the Earth and understand their level of intelligence precisely because their level is far below ours and B) An intelligent person seems to be an order of magnitude smarter than a stupid person, we see it like this:

When the AI develops, we will see that it becomes intelligent enough for the animal. Then, as soon as he reaches the lower limit of human intelligence, Nick Bostrom calls it “the level of a village idiot”, we will think “oh, how nice, he's like a village idiot”. There is one mistake. From the point of view of the development of the mind, the whole spectrum of human intelligence, from the village fool to Einstein, takes a very small gap, that is, immediately after reaching the level of the human fool and getting the title of AI of the human level, our AI suddenly and very quickly become smarter than Einstein, and we We do not imagine where the boundaries of its development may be.

What will be next?
The explosive development of AIFrom this point on, the article becomes pretty scary, I hope you enjoyed the “normal” part of the article. I want to warn you that what I am going to tell next is not science fiction and not a horror story, but quite real predictions from a large number of scientists. Just remember this while you read the final part of the article.
As mentioned above, most of the methods for creating a strong AI imply self-development of AI to this level. And even if a strong AI will be created in a different way, it will be foolish enough for him not to engage in self-development. And here we come to what is called recursive self-improvement. It will happen like this:
Development to the lower boundary of the human level of intelligence will take decades from the AI. A computer will understand the world around it in much the same way as a 4-year-old child. Then, unexpectedly, in about an hour, the system will issue the Theory of Great Unification, which will unite the general theory of relativity and quantum mechanics, that is, do something that no single person is capable of. 90 minutes later, the AI will become an artificial Overmind 170,000 times smarter than a man.
The supermind of such power will be for us to be as incomprehensible as for the bumble-bee Keynesianism. In our world, to be smart means to have an IQ above 130, to be stupid is to have an IQ of about 85, we do not even have a name for IQ 12,952.
The pledge of human domination on the planet lies in its relative rationality in comparison with other living beings. From the moment we create the artificial Supermind, it will become the most powerful “being” on our planet, and all life, and even the very existence of life, will depend entirely on the whim of this Supermind.
And if humanity with its meager mind could invent WI-FI, then for such an intelligent “being” it would hardly be a problem to manage all the atoms and their movement as he pleases. Everything that we considered magic, any super abilities that we have ever attributed to any of the most powerful gods, will be at his disposal. All that we could ever dream of: the cure of any disease, the victory over hunger or even death itself, the management of the weather to protect life on earth, all this will become possible. As possible as the complete destruction of life on earth. When we create this Supermind, it will have no equal on the planet, and there will be only one question: Will it be good?