Richard Hamming: The Basics of the Digital (Discrete) Revolution

“The Buddha said to his disciples,“ Do not believe anything, no matter where you read it, or who said it, even if I said it, if it is not consistent with your own reason and your own common sense. ” I say the same thing - you must take responsibility for what you believe. "

Hi, Habr. Remember the awesome article "You and your work" (+219, 2112 bookmarks, 335k reads)?

So Hamming (yes, yes, self-checking and self-correcting Hamming codes ) has a whole book based on his lectures. Let's translate it, because the man is talking.

This book is not just about IT, it is a book about the thinking style of incredibly cool people. “This is not just a charge of positive thinking; it describes the conditions that increase the chances of doing a great job. ”
')
We have already translated 3 chapters (albeit in the order of subjective interests):

• Preface and Introduction
• Chapter 24. Quantum Mechanics
• Chapter 25. Creativity
• Chapter 30. You and Your Study

Today - Chapter 2. Basics of the digital (discrete) revolution.
(For the translation, thanks to Daniko Ikhoshvili, who responded to my call in the "previous chapter.")

Who wants to help with the translation - write in a personal or mail magisterludi2016@yandex.ru

Chapter 2. Basics of the digital (discrete) revolution

We are approaching the finale, the logical conclusion of the revolution, when the use of
continuous signals for data transmission is replaced by discontinuous (discrete), and,
probably for this purpose we will abandon the use of impulses in favor of secluded
waves. In nature, many signals are continuous (provided that the apparent discrete structure of things built of molecules and electrons is ignored).

As an example of continuous signals, voice transmission over telephone, musical sounds, heights and masses of people, distance traveled, speed, density, etc. can be considered. We almost instantly convert the continuous signal to discrete, while sampling is usually performed at equal time intervals, and the signal volume is broken down to a relatively small number of intervals. Splitting the signal into intervals, in other words quantization, will not be considered in these chapters, although in some situations this is important, especially in computing large amounts of data.

So what were the causes and prerequisites of this revolution?

1. When transmitting continuous signals, it is often necessary to compensate for the natural losses by amplifying the signal. Any mistake made at one stage, before or during amplification, is exacerbated at the next stage. For example, a telephone company sending voice across the continent may have a gain of 10 ¹²⁰ . This coefficient may seem very large, so we will quickly perform calculations on a napkin (they were discussed in the 1st chapter) to see if this is reasonable. Consider the system in more detail. Suppose that each amplifier has a gain of 100, and they are separated every 80 kilometers. The actual signal path can be more than 4800 kilometers, therefore, about 60 amplifiers are placed on the path, so the above factor seems reasonable, now we have seen how it can occur. It should be obvious that such amplifiers should have been built with extreme precision if the system were suitable for human use.

Compare with the transmission of discrete signals. There is no need to amplify the signal at each stage; repeaters are used instead. In this scheme, the noise detected at one of the stages is automatically removed at the next stage of signal transmission. Thus, the voice signal can be transmitted with high accuracy, and the requirements for equipment and the accuracy of its setting are not so high. If necessary, we can use error detection and correction codes to further eliminate noise. We will look at these codes later in chapters 10-12. Along with this, we have developed the field of digital filters, which are often much more versatile, compact and cheaper than analog filters, as will be discussed in Chapters 14-17. It should be noted that the transmission of a signal through space is similar to the transmission of a signal through time (ie storage).

Digital computers can take advantage of these features and perform very deep and accurate calculations that are not available for analog computing. Analog computers have probably reached their peak of performance, but they should not be discarded lightly. They have some functions that, if they do not require greater accuracy or deep calculations, make them ideal in some situations.

2. The invention and development of transistors and integrated circuits, especially digital ones, contributed significantly to the digital revolution. The problem of soldered joints was the main one when creating a large computer, and the chips helped to cope with most of this problem, although solder joints still remain problematic. In addition, the high density of the components of integrated circuits means lower cost and higher computation speed (the parts must be close to each other, because otherwise the signal transmission time will slow down the computation speed significantly). The steady decline in both the voltage level and current contributed to a partial solution to heat transfer. In 1992, it was estimated that interconnect costs are approximately:

• Interconnection on the chip: $ 10 ^-5 = 0.001 cents.
• Interchip communication: $ 10 ^-2 = 1 cent.
• Inter-board connection: $ 10 ^–1 = 10 cents.
• Interframe link $ 100 = 100 cents.

3. The society is steadily moving from the material goods society to the information services society. During the American Revolution (around 1780), more than 90% of the population of the future United States was mostly farmers - now farmers make up a very small percentage of workers.

Similarly, before World War II, most workers were in factories — now
they are less than half. In 1993, there were more people in the government (except for the military) than in
production! What will be the situation in 2020? As I understand it, less than 25% of people in
civilian labor will handle things; the rest will process information in one form or another. When you produce a movie or a television program, you do a lot of actions, although, of course, the final product has a material form, containing the result of organizing and processing information. Information, of course, is stored in a material form, if, for example, a book is considered (information is the essence of a book), but information is not a good material for consumption, like food, a house, clothes, a car or a plane ride.

The information revolution arises from the above three points plus their synergistic interaction, although the following points have an influence.

4. Computers allow robots to do a lot of things, including most operations in manufacturing plants. Obviously, computers will play a significant role in enterprises, although you need to be careful not to require the standard von Neumann computer to be the only control mechanism, most likely neural network computers, fuzzy dial logic and variations will perform most of the control. Leaving aside the child’s view of a robot as a machine resembling a human, but, thinking of it as a way of controlling and controlling things in the material world, the robots used in production do the following:

A. Produce the product under more serious quality control conditions.
B. Produce a cheaper product (lower in cost).
C. Produce a new product that was previously difficult to produce or impossible due to human capabilities.

The last point requires special attention.

When we first switched from manual to machine reporting, for economic reasons, we found it necessary to change the reporting system somewhat. Likewise, when we moved from strict handicraft to making a machine, we moved from screws and bolts mainly to rivets and welding.

In practice, it was hard to get the exact same product by hand, but the machines allow it.

Indeed, one of the basic elements of the transformation from manual to machine products is the creative redesign of the equivalent product. Thus, the mechanization of a large organization may not work if you try to keep the product in detail exactly the same, but rather a major compromise must be adopted if we want to ensure significant success for ourselves. You have to understand the essence of production, and then develop an algorithm for the production of a product, its new form, and not try to mechanize the current version and current production realities - if you want to achieve significant success in the long term.

I need to emphasize this point; mechanization requires you to produce an equivalent product, not identical to the old one. In addition, in any design, it is now necessary to take into account the specifics of product servicing in the field; in the long term, maintenance often covers other expenses. The more complex the system developed, the more its maintenance should be central to the final design. Only if field service is part of the original design, can it be safely monitored; it is inappropriate to try to transfer it later to another environment. This applies to any organizations mechanized and using manual labor.

5. The influence of computers on science is very large and is likely to continue over time. My first experience with large-scale computing was to develop the original atomic bomb at Los Alamos. When developing it was not possible to conduct an experiment on a small scale to check whether you have a critical mass or not, and, therefore, at that time computing facilities were the only practical approach. We modeled on IBM's primitive accounting machines various proposed projects, and they gradually came down to design for testing in the desert in Alamagordo, New Mexico.

After thinking about the experience gained in this project, I realized that computers would allow to model many different types of experiments. I have embodied this vision in the practice of Bell Telephone Laboratories over the years. Somewhere in the mid-1950s, in an address to the president and employees of Bell Telephone Laboratories, I said: “We are currently doing 1 of 10 experiments on computers and 9 in laboratories, but before I leave, 9 of 10 experiments. Then they didn’t believe me, because they were sure that real observations were the key to experiments, and I was just a wild theorist from the Faculty of Mathematics, but you all understand that now we are doing about 90-99% of our experiments on computers, and only the rest is in laboratories. And this trend will continue! It is much cheaper to do simulations than real experiments, simulations are much more flexible in testing, and we can even do things that cannot be done in any laboratory. The trend will inevitably continue for some time. Again, the product has changed!

But you are familiar with the evils of the scholasticism of the Middle Ages - people drew conclusions about what would happen on the basis of the books of Aristotle (384-322), and not looking at the nature and real character of things. The great idea of ​​Galileo Galilei (1564-1642) was the beginning of the modern scientific revolution: the study of phenomena from nature, and not just from books! But what did I say above? We are now looking more and more in books and less and less at nature! Obviously, we will go too far too often, and I expect this to happen in the future. With all the enthusiasm for computer simulations, we must not forget to look back at nature as it is.

6. Computers also greatly influenced Engineering. Not only can we design and build much more complex things than we could manually, we can explore many more alternative projects. We also now use computers to manage situations, for example, on modern high-speed aircraft, where we build unstable structures, and then we use high-speed detection and computers to stabilize them, since the pilot deprived of outside help simply cannot fly with them directly. Similarly, we can now conduct unstable experiments in laboratories using a fast computer to control instability. The result will be that the experiment will measure something very precisely right on the edge of stability.

As noted above, engineering is approaching science, and, therefore, the role of modeling in unexplored situations is rapidly increasing both in engineering and in science. It is also true that computers are often an important component of good design / development .

In the past, the “what we can do” approach dominated in engineering, and now the approach is formulated as “what we want to do,” as we now have the opportunity to design almost everything that we want. More than ever before, engineering is a matter of choice and balance, and not just something that can be done. And more and more human factors that will determine good design - a topic that always requires your serious attention.

7. The impact on society is also great. The most obvious illustration is that computers provide the top management with the possibility of micromanagement by their organization, and top management practically did not show the ability to resist the use of this power. You can regularly see in the newspapers articles about large corporations that are engaged in decentralization, but when you observe this company for several years, you see that they simply intended to do it, but did not.

Among other vices of micromanagement, lower management does not get an opportunity to make responsible decisions and learn from their mistakes, but rather because older people finally retire, then lower management turns out to be the top management, not having a lot of experience in management!

In addition, it was repeatedly shown that central planning gives bad results (for example, the Russian experiment or our own bureaucracy). Persons in the field usually have better knowledge than those who are at the top and, therefore, often (not always) make the right decisions if actions and plans do not undergo micromanagement. People from below do not have a global view, but people from above do not have a local view of all the details, many of which can often be very important, so that any extreme results in poor results.

Further, an idea that arises on the ground, based on the direct experience of people performing this work, cannot get into a centrally controlled system, since the managers themselves did not think about it. The “not invented by us” syndrome (NIH) is one of the main curses of our society, and computers that can encourage micromanagement are becoming a significant factor.

The trend of using micromanagement is gradually approaching. There are loose connections between small, somewhat independent organizations. For example, in the brokerage business, one company pledged to sell its services to other small subscribers, such as computer and legal services. This makes brokerage solutions for local client managers close to advanced positions, in other words, very relevant. Similarly, in the pharmaceutical field, some weakly connected companies organize internal trade with their own rules. I believe that you can expect much more from this free association between small organizations as a defense against micromanagement from above, which happens so often in large organizations. Organizations have always had some kind of unit independence, but power from above at the micromanagement level seems to have disrupted conventional lines and decision-making autonomy, and I doubt the ability of most top managers to resist micromanagement for a long time. I also doubt that many large companies will be able to abandon micromanagement; most of them are likely to be replaced in the long run by small organizations without costs (overheads) and top management mistakes. Thus, computers influence the very structure of how society does its business, and for the time being, apparently, for the worse.

8. Computers have already invaded the entertainment area. An unofficial survey shows that the average American spends much more time watching TV than on food - again, information takes precedence over basic nutritional needs! Many commercials and some programs are now either partially or completely created on the computer. How far cars will affect society is a matter of speculation, which opens many doors to problems, if you don’t discuss it openly! Therefore, I have to leave it in my fantasies about what can be done using computers on chips in areas such as sex, marriage, sports, games, “traveling at home through virtual reality” and other human activities.

Since their creation, computers have been mainly used for complex logical calculations, but they have quickly been reoriented towards information retrieval (for example, ticket reservation systems), text processing that is distributed everywhere, symbol manipulation, as many programs do, such as those that can Significantly improve analytical integration in calculus and cheaper than students, as well as in logical areas and solutions, in which many companies use such programs to manage their operations. operations from one point in time.

The future computer intrusion into traditional fields remains to be seen and will be discussed later under the heading “Artificial Intelligence” (AI) in Chapters 6–8.

9. In military affairs, it is easy to observe (for example, in the Gulf War) the dominant role of possession of information, because the inability to use information about their own situation killed many of our people! It is clear that in the war information was in addition to everything else, and this is probably one of the indicators of the future. I do not need to tell you such things, because you all know or need to be aware of these trends. It is up to you so that you can try to anticipate the situation by 2020, when your career will be at its peak. I believe that computers will be almost everywhere, as I once saw a sign that read: "The battlefield is not a place for man." Similarly, for situations that require constant decision-making. Many of the advantages of cars over people were listed at the end of the last chapter, and it is difficult to circumvent these advantages, although they are not all, of course. Obviously, the role of people will be very different from the traditional one, but many of you will insist on the old theories that you taught for a long time, as if at the same time they will be automatically correct in the future. It can also be found in business, much of what is now being taught, is based on the past and ignores the computer revolution and our responses to some of the flaws that the revolution has brought; benefits are generally understood by management, and problems to a lesser extent.

How many trends, predicted in part 6 above, in the direction of micromanagement and beyond, will be widely used and will again be the most suitable topic for you? It is not known, but you will be a fool if you do not give it your and constant attention. I invite you to rethink everything that you have ever learned on this issue, to ask a question about each successful doctrine from the past and, finally, to decide for yourself your future applicability. Buddha said to his disciples: "Believe nothing, no matter where you read it, or who said it, even if I said it, if it does not fit with your own mind and your own common sense." I say the same thing - you must take responsibility for what you believe.

Now I turn to a topic that is often ignored: the rate of evolution of a conditional special activity, an example of which would refer to the “backward envelope computation” process. The growth of most, but not all, fields follows the “S” curve. Everything develops slowly, then the speed increases, and then it smoothes when it reaches certain natural limits.

The simplest growth model assumes that growth rates are proportional to current size, something like a complex interest, unrestrained growth of bacteria and humans, as well as many
other examples. Corresponding differential equation



whose solution, of course,



But this growth is unlimited, and everything must have limits, even knowledge itself, since it must be written in one form or another, and we have (now) said that the Universe is finite! Therefore, we must include the limiting factor in the differential equation. Let L be the upper limit. Then the next simplest growth equation looks like



At this stage, we, of course, reduce it to the standard form, which eliminates constants. We set y = Lz and x = t / kL ² , then we have





as a reduced form for a growth problem, where the saturation level is now 1.

Separation of variables plus partial fractions gives:



And, of course, is determined by the initial conditions, where you set t (or x) = 0. You immediately see the "S" curve shape; when t = - ∞, z = 0; at t = 0, z = A / (A + 1); and at t = + ∞, z = 1.

More flexible growth model (in the above variables)



This is, again, a variable-sharing equation, and also allows for numerical integration, if you wish. We can analytically find the steepest slope, differentiating the right-hand side and equating 0. We get



Therefore in place



We have a maximum slope



Sketch the direction field in Figure 2.I. I will indicate the nature of the decision. It is easy to solve the problem, since the slope depends only on y and not on x, the lines of equal magnetic inclination are horizontal lines, so the solution can slide along the x axis without changing the “shape” of the solution.



For given a and b, there really is only one form of initial conditions that determine where you are looking, not what you are looking at. When a differential equation has coefficients independent of the independent variable, then you have this effect. a=b = 1/2 ^2a . , z=1/2. a=b=1/2



, . a b , , .

, , «S» ( ) , , : (1) (2) , , , , . — , «S» . . .

, , , . , , . «S», 2.II. , , , , , ; .

, : (1) ; (2) , , (3) .
, , ,
, Vacuum Tube Catalog ( ,
, –
approx. .).

. , . -, , . -, . , , , :

1. , .
2. , .
3. , , , .
4. .
5. , , , , . , , , . , , , , .

!

.

To be continued...

Who wants to help with the translation - write in a personal or mail magisterludi2016@yandex.ru