A dozen Linux tricks that really save a lot of time

One evening, re-reading Jeffrey Fridle , I realized that even with all the available documentation, there are many tricks tailored for themselves. All people are too different. And the techniques that are obvious to some may not be obvious to others and look like some kind of magic to others. By the way, I already described several similar moments here .

The command line for the administrator or user is not only a tool with which you can do everything, but also a tool that is customized for your beloved person indefinitely. Recently I ran a translation on the topic of convenient receptions in the CLI. But I got the impression that the translator himself did not use much advice, which is why important nuances could be missed.

Under the cut - a dozen command line techniques - from personal experience.

A small digression - in real life, I use a variety of techniques in which the names of real servers or users may accidentally meet, which may fall under the NDA, so I couldn’t save and paste and specially rewrote and simplified all the examples in the article as much as possible - and if you think that some method in this context is completely useless - perhaps this is precisely the reason. But in any case, share your ideas in the comments!
')

1. Splitting a string with variable expansions

Often use cut or even awk to simply get the value of a single column.
If you want to take the first characters in the string, use $ {VARIABLE: 0: 5}.

But there is a very powerful tool that allows you to manipulate strings with #, ##,% and %% (bash variable expansions) - with their help, you can cut off the unnecessary by the pattern from any side.
The example below shows how from the string “username: homedir: shell” you can get only the third column (shell) using cut or using variable expansions (we use the *: mask and the ## command, which means cutting all characters to the last found colon):

$ STRING="username:homedir:shell"

$ echo "$STRING"|cut -d ":" -f 3
shell
$ echo "${STRING##*:}"
shell

(cut), , . bash- windows, , .

Ubuntu — 1000

$ cat test.sh
#!/usr/bin/env bash

STRING="Name:Date:Shell"

echo "using cut"
time for A in {1..1000}
do
  cut -d ":" -f 3 > /dev/null <<<"$STRING"
done

echo "using ##"
time for A in {1..1000}
do
 echo "${STRING##*:}" > /dev/null
done

$ ./test.sh
using cut

real    0m0.950s
user    0m0.012s
sys     0m0.232s
using ##

real    0m0.011s
user    0m0.008s
sys     0m0.004s

— !

, — . . . cut /etc/passwd. ##, read. , ?

$ cat test.sh
#!/usr/bin/env bash

echo "using cut"
time for count in {1..1000}
do
  cut -d ":" -f 7 </etc/passwd > /dev/null
done

echo "using ##"
time for count in {1..1000}
do
  while read
  do
    echo "${REPLY##*:}" > /dev/null
  done </etc/passwd
done

$ ./test.sh
$ ./test.sh
using cut

real    0m0.827s
user    0m0.004s
sys     0m0.208s
using ##

real    0m0.613s
user    0m0.436s
sys     0m0.172s

$ VAR="myClassName = helloClass"
$ echo ${VAR##*= }
helloClass

$ VAR="Hello my friend (enemy)"
$ TEMP="${VAR##*\(}"
$ echo "${TEMP%\)}"
enemy

2. bash TAB

bash-completion , /etc/bash.bashrc /etc/profile.d/bash_completion.sh, . TAB — , .
, , , , , , , , . .
PATH, — .profile .bashrc.
*nix lowercase , uppercase — :

$ alias TAsteriskLog="tail -f /var/log/asteriks.log"
$ alias TMailLog="tail -f /var/log/mail.log"
$ TA[tab]steriksLog
$ TM[tab]ailLog

3. bash TAB — 2

, $HOME/bin.
.
PATH, .
LastLogin .profile ( .profile):

function LastLogin {
  STRING=$(last | head -n 1 | tr -s " " " ")
  USER=$(echo "$STRING"|cut -d " " -f 1)
  IP=$(echo "$STRING"|cut -d " " -f 3)
  SHELL=$( grep "$USER" /etc/passwd | cut -d ":" -f 7)
  echo "User: $USER, IP: $IP, SHELL=$SHELL"
}

( , , , alias)
:

$ L[tab]astLogin
User: saboteur, IP: 10.0.2.2, SHELL=/bin/bash

4. Sensitive data

, bash history, , — , echo «hello 2» :

$ echo "hello"
hello
$ history 2
 2011  echo "hello"
 2012  history 2
$  echo "my password secretmegakey"
my password secretmegakey
$ history 2
 2011  echo "hello"
 2012  history 2

:
export HISTCONTROL=ignoreboth

, git ansible, external bash file , 600, .gitignore , source:

secret.sh
PASSWORD=LOVESEXGOD

myapp.sh
. ~/secret.sh
sqlplus -l user/"$PASSWORD"@database:port/sid @mysqfile.sql

— Tanriol, , , , .
, ssh , . wget — --password, , wget, .

— ( , , SQL) openssl .
openssl , . :

secret.key :

$ echo "secretpassword" > secret.key; chmod 600 secret.key

aes-256-cbc, c :

$ echo "string_to_encrypt" | openssl enc -pass file:secret.key -e -aes-256-cbc -a
U2FsdGVkX194R0GmFKCL/krYCugS655yLhf8aQyKNcUnBs30AE5lHN5MXPjjSFML

-, .git — secret.key .
, -e -d:

$ echo 'U2FsdGVkX194R0GmFKCL/krYCugS655yLhf8aQyKNcUnBs30AE5lHN5MXPjjSFML' | openssl enc -pass file:secret.key -d -aes-256-cbc -a
string_to_encrypt

, , , secret.key .

5. grep

tail -f application.log | grep -i error

tail -f application.log | grep -i -P "(error|warning|failure)"

-v, — , — , exception , ( , info, ):

tail -f application.log | grep -v -i "info"

:

-P, grep basic regular expression, PCRE, , .

"--line-buffered", "-i".

, --only-matching, . . grep, // .

6.

, , , *nix, , iNode.
, , , ( ). , , .
( , , )

( , ):

> application.log

( ):

truncate --size=1M application.log

, , , .

, 1000 :

echo "$(tail -n 1000 application.log)" > application.log

Himura .
P.S. — - , , log4j, rotatelogs.

7. watch !

, - . , ( who ), - ftp ( ls ).

watch <>

, 2 , Ctrl+C. .
,

8. bash

, :

for srv in 1 2 3; do echo "server${srv}";done
server1
server2
server3

for srv in server{1..3}; do echo "$srv";done
server1
server2
server3

seq, . :

for srv in $(seq -w 1 10); do echo "server${srv}";done
server01
server02
server03
server04
server05
server06
server07
server08
server09
server10

, . basename:

for file in *.txt; do name=$(basename "$file" .txt);mv $name{.txt,.lst}; done

, %:

for file in *.txt; do mv ${file%.txt}{.txt,.lst}; done

«rename»

java :

mkdir -p project/src/{main,test}/{java,resources}

project/
   !--- src/
        |--- main/
        |    |-- java/
        |    !-- resources/
        !--- test/
             |-- java/
             !-- resources/

9. tail,

multitail, . - , .

tail:

tail -f /var/logs/*.log

, «tail -f».
, , , , tail -f. - .
, «tail -f», . , , .

, PID PID , tail :

alias TFapplog='tail -f --pid=$(cat /opt/app/tmp/app.pid) /opt/app/logs/app.log'

. , tail, .

10.

dd if=/dev/zero of=out.txt bs=1M count=10

fallocate:

fallocate -l 10M file.txt

, (xfs, ext4, Btrfs...), , dd.

11. xargs

, , .
— .

-n:

$ #    5 
for string in string{1..5}; do echo $string >> file.lst; done
$ cat file.lst
string1
string2
string3
string4
string5
saboteur@ubuntu:~$ cat file.lst | xargs -n 2
string1 string2
string3 string4
string5

— , . , xargs , 2 :

cat file | xargs -n 2 -P 3

, nproc, :

cat file | xargs -n 2 -P $(nproc)

12. sleep? while? read!

sleep while, read, , :

read -p "Press any key to continue " -n 1

, :

read -p "Press any key to continue (autocontinue in 30 seconds) " -t 30 -n 1

REPLY=""
until [ "$REPLY" = "y" ]; do
  # executing some command
  read "Press 'y' to continue or 'n' to break, any other key to repeat this step" -n 1
  if [ "$REPLY" = 'n' ]; then exit 1; fi
done

, .
— !

P.S. Update:
1. — !
2. `` $() — !
3. , .

, — , =), + : khim, Tanriol, Himura, bolk, firegurafiku, dangerous3, RPG, ALexhha, IRyston, McAaron

Source: https://habr.com/ru/post/340544/

All Articles