Linux redirection

If you are already familiar with the basics of the terminal, you may already be ready to combine the learned commands. Sometimes it is enough to execute shell commands one by one to solve a certain task, but in some cases it is too tedious and irrational to enter a command by command. In a situation like this, some special characters like angle brackets will be useful to us.



For the shell, the linux command interpreter, these extra characters are not a waste of space on the screen. They are powerful teams that can tie together different pieces of information, share what was previously solid, and do much more. One of the simplest, and, at the same time, powerful and widely used shell capabilities is redirecting standard I / O streams.

Three standard I / O streams

In order to understand what we are going to talk about here, it is important to know where the data comes from, which can be redirected, and where they are going. There are three standard input / output streams for Linux.
')
The first is the standard input stream. In the system, this is stream # 0 (as in computers, the score usually starts from zero). Thread numbers are also called descriptors. This stream represents some information transmitted to the terminal, in particular, instructions that are passed to the shell for execution. Typically, the data in this stream fall during user input from the keyboard.

The second stream is the standard output stream (standard output), it is assigned the number 1. This is the data stream that the shell displays after performing some actions. Typically, these data fall into the same terminal window, where the command that caused their appearance was entered.

And, finally, the third stream is the standard error stream (standard error), it has descriptor 2. This stream is similar to the standard output stream, since usually what gets into it is on the terminal screen. However, it is, in its essence, different from the standard output, as a result, these streams, if desired, can be controlled separately. This is useful, for example, in the following situation. There is a team that processes a large amount of data, performing a complex and error-prone operation. It is necessary that the useful data that this command generates do not mix with error messages. This is realized through the separate redirection of output streams and errors.

As you have probably already guessed, input / output redirection means working with the streams described above and redirecting data to where the programmer needs. This is done using the characters > and < in various combinations, the use of which depends on where, in the end, the data should be redirected.

Standard Output Stream Redirection

Suppose you want to create a file on which the current date and time will be recorded. The matter simplifies the fact that there is a command, successfully named date , which returns what we need. Usually commands output data to standard output. In order for this data to appear in the file, you need to add the > character after the command, before the name of the target file. Before and after > you must put a space.

When using redirection, any file specified after > will be overwritten. If there is nothing valuable in the file and its contents can be lost, it is acceptable to use an existing file in our design. Usually it is better to use in this case the name of a file that does not exist yet. This file will be created after the command is executed. date.txt it date.txt . File extension after the dot usually does not play a special role, but extensions help maintain order. So, here is our team:

 $ date > date.txt 

This is not to say that by itself this team is incredibly useful, however, based on it, we can already do something more interesting. Let's say you want to find out how the routes of your traffic, going through the Internet to a certain end point, change, daily recording the corresponding data. In solving this problem, the traceroute command will help, which provides details about the route of traffic between our computer and the end point specified when the command is called as a URL. Data includes information about all routers through which traffic passes.

Since we already have a date file, it will be quite justified to simply attach the data received from traceroute to this file. In order to do this, you need to use two characters > , put one after another. As a result, a new command that redirects the output to the file, but does not overwrite it, but adds new data after the old one, will look like this:

 $ traceroute google.com >> date.txt 

Now we just have to change the file name to something more meaningful, using the mv command, which, as the first argument, passes the original file name, and as the second, the new one:

 $ mv date.txt trace1.txt 

Redirect Standard Input Stream

Using the < sign instead of > we can redirect the standard input, replacing it with the contents of the file.

Suppose there are two files: list1.txt and list2.txt , each of which contains an unsorted list of strings. Each of the lists has unique elements for it, but some of the elements of the list are the same. We can find the rows that are in both the first and second lists, using the command comm , but before using it, the lists must be sorted.

There is a sort command that returns a sorted list to the terminal without saving the sorted data to the file from which it was taken. You can send a sorted version of each list to a new file using the > command, and then use the comm command. However, this approach will require at least two commands, although the same can be done on one line without creating unnecessary files.

So we can use the < command to redirect the sorted version of each file to the comm command. Here's what we got:

 $ comm <(sort list1.txt) <(sort list2.txt) 

Parentheses here have the same meaning as in mathematics. The shell first processes the commands in brackets, and then everything else. In our example, we first sort the lines from the files, and then what we get is passed to the command comm , which then displays the result of the list comparison.

Standard Error Redirection

And finally, let's talk about redirecting the standard error stream. This may be necessary, for example, to create log files with errors or to merge error messages and data returned by some data commands in one file.

For example, what if you need to search the entire system for information about wireless interfaces that are available to users who do not have superuser rights? In order to do this, you can use the powerful find .

Usually, when an ordinary user runs the find throughout the system, it displays both useful data and errors to the terminal. At the same time, the latter are usually larger than the first ones, which complicates the finding of what is needed in the command output. Solving this problem is quite simple: just redirect the standard error stream to a file using the command 2> (recall, 2 is a descriptor for the standard error stream). As a result, the screen will only get what the command sends to the standard output:

 $ find / -name wireless 2> denied.txt 

What if you need to save the results of the team in a separate file without mixing this data with error information? Since streams can be redirected independently of each other, at the end of our construction, we can add the command to redirect the standard output stream to a file:

 $ find / -name wireless 2> denied.txt > found.txt 

Note that the first angle bracket comes with a number - 2> , and the second without it. This is because the standard output has a descriptor of 1, and the > command implies redirection of the standard output if the descriptor number is not specified.

And finally, if you want everything that the command displays to be in one file, you can redirect both streams to the same place using the &> command:

 $ find / -name wireless &> results.txt 

Results

Here we have analyzed only the basics of the thread redirection mechanism in the Linux command line interpreter, but even the little that you learned today gives you almost unlimited possibilities. And, by the way, like everything else regarding the work in the terminal, mastering the redirection of flows requires practice. Therefore, we recommend that you start your own experiments with > and < .

Dear readers! Do you know some interesting examples of using redirection of streams in Linux, which will help beginners to better master this technique of working in the terminal?