Geekly Articles each Day
📜 ⬆️ ⬇️

Unix shell: absolutely first steps

Why and for whom the article?


Initially, it was a memo for students who are starting to work with unix-like systems. In other words, the article is intended for those who have no previous experience in the unix command line, but for one reason or another want to or should learn to interact effectively with it.

There will not be a retelling of manes (documentation), and the article does not cancel or replace their reading. Instead, I will talk about the main things (commands, techniques and principles) that need to be realized from the very beginning of work in the unix shell, so that work can be done efficiently and pleasantly.

The article concerns full-fledged unix-like environments, with a full-featured shell (preferably zsh or bash) and a fairly wide range of standard programs.
')

What is shell


Shell (shell, also known as “command line”, also known as CLI, also known as “console”, also known as “terminal”, also known as “black window with white letters”) is a text interface for communicating with the operating system (well, speaking, this is a program that provides such an interface, but now this distinction is not essential).

In general, the work through the shell looks like this: the user (that is, you) enters a command from the keyboard, presses Enter, the system executes the command, writes the execution result to the screen, and again waits for the next command to be entered.

Typical shell type:


Shell is the main way to interact with all Unix-like server systems.

Where do command line systems meet?


Where you can wait for the unix shell, popular options are:


What tasks it is reasonable to solve with a shell?


Natural tasks for which the shell is suitable, useful and indispensable:


Absolutely first steps


Getting started: log in and log out


Make sure you know exactly how to run the shell and how to get out of it.

If you are working behind a machine on which Ubuntu is installed, you need to start the Terminal program. At the end of the work, you can simply close the window.

On MacOS, also run Terminal.

To access a remote server, use ssh (if you have MacOS, Ubuntu or another unix-like system putty ) or putty (if you have Windows).

Who am I, where am I?


Run the following commands:


Command history (history)


An important feature of a full command line is the command history.

Execute several commands: hostname , ls , pwd , whoami . Now press the up key. The previous command appeared in the input line. Use the up and down keys to move back and forth through the history. When you go to hostname , press Enter - the command will be executed again.

Commands from the history can not just be re-executed, but also edited. Add history to the ls , add the -l key to it (it turned out to be ls -l , there is a space before the minus, but not after). Press Enter - the modified command will be executed.

Scrolling through history, editing and re-executing commands are the most common actions when working on the command line, get used to it.

Copy-paste


The command line is very text-centered: commands are text, input for most standard programs is text, and the result of the work is often text too.

A great feature of the text is that it can be copied and pasted; this is also true for the command line.

Try the date +"%y-%m-%d, %A" command
Did you enter it entirely with your hands or copied from the article? Make sure you can copy it, paste it into the terminal and execute it.

After learning how to use man , make sure that you can copy and execute sample commands from the help. To check, look in the help program for the date section EXAMPLES , copy and execute the first example given (just in case: the dollar sign is not part of the command, it is a conditional image of the invitation to enter).

How exactly to copy text from the terminal and insert it into the terminal depends on your system and on its settings, therefore, unfortunately, it will not work to give a universal instruction. On Ubuntu, try this: copying is just a selection with the mouse, insertion is the middle mouse button. If it does not work, or if you have another system, look on the Internet or ask more experienced friends.

Keys and options


When examining the history of commands, you have already come across the fact that the ls command has at least two options. If you simply call her, she displays a simple list:

 [22:26]akira@latitude-e7240: ~/shell-survival-quide> ls Makefile shell-first-steps.md shell-first-steps.pdf shell-survival-quide.md shell-survival-quide.pdf

If you add the -l , detailed information is displayed for each file:

 [22:28]akira@latitude-e7240: ~/shell-survival-quide> ls -l total 332 -rw-rw-r-- 1 akira akira 198 Feb 13 11:48 Makefile -rw-rw-r-- 1 akira akira 15107 Feb 14 22:26 shell-first-steps.md -rw-rw-r-- 1 akira akira 146226 Feb 13 11:49 shell-first-steps.pdf -rw-rw-r-- 1 akira akira 16626 Feb 13 11:45 shell-survival-quide.md -rw-rw-r-- 1 akira akira 146203 Feb 13 11:35 shell-survival-quide.pdf

This is a very typical situation: if you add special modifiers (keys, options, parameters) to the command call, the command behavior changes. Compare: tree / and tree -d / , hostname and hostname -f .

In addition, commands can take file names, directories, or just text strings as parameters. Try:

 ls -ld /home ls -l /home grep root /etc/passwd


man


man - help on the commands and programs available on your machine, as well as on system calls and the standard C library.

Try: man grep , man atoi , man chdir , man man .

Scrolling back and forth is done with the buttons "up", "down", "PageUp", "PageDown", exit from the help view - with the q button. Search for specific text in the help article: press / (straight forward slash), enter text to search for, press Enter. Move to the next occurrences - the n key.

All reference articles are divided into categories. The most important:

Specify which category you need to show a certificate, you need in cases of matching names. For example, man 3 printf describes a function from the standard C library, and man 1 printf is a console program with the same name.

You can view a list of all the help articles available on the machine with the help of the man -k . command man -k . (the point is also part of the team).

less


When in a small terminal window you need to look at a very long text (the contents of some file, a long man, etc.), use special pager programs (from the word page / page, that is, page-readers). The most popular leaflet is less , and it is this that provides you with scrolling when you read mans.

Try and compare the behavior:

 cat /etc/bash.bashrc cat /etc/bash.bashrc |less


You can transfer the file to the page directly in the parameters:

 less /etc/bash.bashrc


Scroll up and down - the buttons "up", "down", "PageUp", "PageDown", exit - button q . Search for specific text: press / (straight forward slash), enter text to search, press Enter. Move to the next occurrences - the n key. (You will learn the instruction about man ? No wonder, less is also used to display help.)

The rights


A set of “rights” is associated with any file or directory: the right to read the file, the right to write to the file, the right to execute the file. All users are divided into three categories: file owner, file owner group, all other users.

You can view file permissions using ls -l . For example:

 > ls -l Makefile -rw-r--r-- 1 akira students 198 Feb 13 11:48 Makefile

This conclusion means that the owner (akira) can read and write the file, the group (students) can only read, all other users can also read.

If during work you get the message permission denied , this means that you have insufficient rights to the object with which you wanted to work.

Read more in man chmod .

STDIN, STDOUT, pipelines


There are 3 standard data streams associated with each executing program: the STDIN input data stream, the STDOUT output data stream, the STDERR error output stream.

Run the wc program, enter the text Good day today , press Enter, enter the text good day , press Enter, press Ctrl + d. The wc program will show statistics on the number of letters, words and lines in your text and will end:

 > wc good day today good day 2 5 24

In this case, you submitted two-line text to the STDIN program, and in STDOUT received three numbers.

Now run the command head -n3 /etc/passwd , it should look like this:

 > head -n3 /etc/passwd root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x:2:2:bin:/bin:/usr/sbin/nologin

In this case, the head program did not read anything from STDIN , and wrote three lines in STDOUT .

One can imagine this: a program is a pipe into which STDIN flows, and STDOUT flows out.

The most important feature of the unix command line is that the “pipe” programs can be interconnected: the output ( STDOUT ) of one program is transmitted as input data ( STDIN ) to another program.

Such a construction of the combined programs is called in English pipe (pipe), in Russian - a pipeline or pipe.

Combining programs into a pipeline becomes a symbol | (vertical bar)

Run the command head -n3 /etc/passwd |wc , you get something like the following:

 > head -n3 /etc/passwd |wc 3 3 117

This is what happened: the head program sent three lines of text to STDOUT , which immediately went to the wc program, which in turn counted the number of characters, words and lines in the received text.

In the pipeline you can combine any number of programs. For example, you can add another wc program to the previous pipeline, which will calculate how many words and letters were in the output of the first wc :

 > head -n3 /etc/passwd |wc |wc 1 3 24


Drawing pipelines is a very frequent thing when working on the command line. An example of how this is done in practice can be found in the “Drawing up a one-line conveyor” section.

I / O Redirection


The output ( STDOUT ) of the program can not only be transferred to another program through a pipeline, but also simply written to a file. Such redirection is done with the help of > (“more” sign):

 date > /tmp/today.txt

As a result of this command, the file /tmp/today.txt will appear on the disk. View its contents with cat /tmp/today.txt

If a file with the same name already existed, its old contents will be destroyed. If the file did not exist, it will be created. The directory in which the file is created must exist before the command is executed.

If you need not to overwrite the file, but add the output to its end, use >> :

 date >> /tmp/today.txt

Check what is now recorded in the file.

In addition, instead of STDIN , the program can transfer any file. Try:

 wc </etc/passwd


What to do when something is not clear


If you encounter a system behavior that you don’t understand, or want to achieve a certain result, but don’t know how, I advise you to act in the following order (by the way, this applies not only to shells):


If none of the above helped - seek advice from a teacher, experienced colleague or friend. And do not be afraid to ask "stupid" questions - do not be ashamed not to know, ashamed not to ask.

If you have dealt with a difficult problem (on your own, with the help of the Internet or other people) - write down your decision in case the same problem arises again for you or your comrades. You can record in a simple text file in Evernote, publish in social networks.

Working methods


Copy-and-paste - from man-s, from articles on StackOverflow, etc. The command line consists of text, use this: copy and use examples of commands, write successful finds for memory, publish them on twitter and blogs.

Read man . Nuff said.

Pull out the previous command from the history, add another command to the pipeline, run, repeat . See. also the section “Drawing up a one-line conveyor”.

Basic commands




Analytics




System diagnostics




You may not have some programs, they must be installed additionally. In addition, some options of these programs are available only to privileged users ( root 'y).

Mass and semi-automatic execution


At first, skip this section; you will need these commands and constructions when you get to simple shell scripting.



miscellanea




Drawing up a one-line conveyor


Let's look at an example of a real task: you need to kill all the processes of task-6-server running on behalf of the current user.

Step 1.
Understand which program provides approximately the necessary data, even if not in its pure form. For our task it is worth getting a list of all processes in the system: ps axuww . Run.

Step 2.
Look at the data obtained by the eyes, come up with a filter that will throw out some of the unnecessary data. Often this is grep or grep -v . Use the "Up" button to pull out the previous command from the history, assign a filter to it, run it.

 ps axuww |grep `whoami`

- only current user processes.

Step 3.
Repeat point 2 until the desired data is clean.

 ps axuww |grep `whoami` | grep '\<task-6-server\>'

- all processes with the desired name (plus, perhaps, extra ones like vim task-6-server.c, etc.),

 ps axuww |grep `whoami` | grep '\<task-6-server\>' | grep -v vim ps axuww |grep `whoami` | grep '\<task-6-server\>' | grep -v vim |grep -v less

- only processes with the desired name

 ps axuww |grep `whoami` | grep '\<task-6-server\>' | grep -v vim |grep -v less |awk '{print $2}'


- pid-s of the necessary processes, p. 3 is completed

Step 4.
Apply the appropriate final handler. Use the "Up" button to pull out the previous command from the history and add processing, which will complete the solution of the problem:



Tasks for training


Want to practice new skills? Try the following tasks:


Hint: you will need find , grep -o , awk '{print $1}' , regular expressions in grep , curl -s .

What to study next?


If the command line begins to please you, do not stop, continue to improve your skills.

Here are some programs that will definitely come in handy if you live on the command line:


In addition, over time, it is worth mastering some scripting language, for example, perl or python , or even both of them.

Who needs it?


Is it worth it to study the command line and shell scripting today? Definitely worth it. I will cite just a few examples from Facebook's requirements for candidates who want to go to work at FB.

Data Scientist, Economic Research : Unix core tools; preferred: adeptness with a scripting language such as Python, or previous software engineering experience.

MySQL Database Engineer : High degree of proficiency in Shell scripting (Bash, Awk, etc); high degree of proficiency in Linux administration.

Manufacturing Quality Engineer, Server : Scripting skills in Bash, Perl, or Python is desirable.

Data Platform Engineer : 2 years experience with Unix / Linux systems.

DevOps Engineer, Data : 2 years experience with Unix / Linux system administration and programming.

Questions?


If you have questions on this article or even on the work in the unix command line, ask them in the comments or email liruoko (at) yandex (dot) ru .

Some useful and interesting links.


15 interesting Linux commands

Survival guide for Unix newbies

Interesting bash programming techniques
original: Better Bash Scripting in 15 Minutes

Shell programming with bash: by example, by counter-example

Simple ways to make the console utility more convenient.

Debug your programs like they closed source!

Source: https://habr.com/ru/post/267825/

More articles:


All Articles