"Examples of effective use of GNU utilities in Linux"

Hello to the whole community Habra.
This is my first post and hopefully not the last. Therefore, all kinds of shortcomings, bugs and some wateriness of the text are inevitable, so please do not judge strictly :)
By writing this post, I was inspired by the topic “Console for a beginner.” Habrahabr.ru/blog/linux/46610.html , where ISVir raised a sharp, in my opinion, topic — how to use it in an accessible language to tell newbies about the console complexity.

I am not going to take parity from ISVir , I’ll only tell you about the practical use of several of the most basic utilities “for every day”, without which working in the console is impossible for me.

So, what we have:
')
find - find files. allows you to search for files, directories, symlinks and other file objects. find Allows you to specify a variety of search options such as:

• search by mask (in name)
• control the depth of search nesting
• search for specific file types (directory, symlinks, sockets)
• search by file creation / modification time
• You can set the size (from and to in the range) file
• performing actions with each file found

in the mane to find you can read about the other options and parameters.

So, right off the bat - search for all files in the / etc / directory that have been changed in the last 24 hours:

$ find / etc / -type f -mtime -1

analyze what we wrote:

The first parameter is always the starting directory for the search.
The -type option with the f parameter tells find to only look for ordinary files.
The -mtime option with the -1 parameter tells find to search for files that have changed in the last 24 hours.
"-" before 1 sets the upper limit of the range, i.e. "Everything has changed in the last 24 hours"

if we specified "+" before 1 , then find would find all files that changed from 01/01/1970 to yesterday (more than a day ago)
You can also specify the exact date by putting a number without a modifier.

performing actions with the found files.
The -exec option accepts a line with a command that will be executed for each file found.
the parameter passed to the command is denoted by {}
line must end with "\;"

Consider an example:
* find in the / tmp directory all files that have changed over the last month and copy them into the directory
/ tmp / backup /

$ find / tmp -type f -mtime -30 -exec cp {} / tmp / backup \;

* delete all directories (recursively) with the name logs that have changed in the last day in the directory
/ var / www

$ find / var / www -type d -mtime 0 -name logs -exec sudo rm -fr {} \;

The d parameter in the -type option indicates a search for directories only.
The -name option specifies a search by name.
here it is worth adding that deleting files in this way is not optimal (slow).
for deletion, find has a built-in option -delete, which works an order of magnitude faster.

not tired yet? go ahead :)

Consider the awk utility.
awk is a programming language for handling files. Purpose of it
development - to facilitate the formulation and solution of many problems associated with the processing of textual information. Essentially, awk is a utility available from the console.
For obvious reasons, I will not consider here how to write awk code - I’ll tell you about one important technique for us.

first, awk can get data from STDIN: $ echo "test" | awk ...
secondly, awk is effective when writing one-liners to the console, since executes the code given to it as a parameter:

$ echo "test" | awk "..."

awk splits the input stream into fields and places these fields in variables of the form $ 1, $ 2, .. $ N
By default, the field separator is a space, but with the -F "_ separator_" option this can be overridden:

$ head -4 / etc / passwd
root: x: 0: 0: root: / root: / bin / bash
daemon: x: 1: 1: daemon: / usr / sbin: / bin / sh
bin: x: 2: 2: bin: / bin: / bin / sh
sys: x: 3: 3: sys: / dev: / bin / sh

$ cat / etc / passwd | awk -F ":" '{print $ 1}'
root
daemon
bin

For example, we have several sites in the / var / www directory. for each site there is a logs directory where apache logs are written (for this site). and now we want to find out the total volume of these logs, as well as find all logs greater than 100Mb.

here we go:

1. search for large logs:

$ find / var / www -type f -name "access.log *" -size + 100M
/var/www/site1/logs/access.log
/var/www/site2/logs/access.log.1.gz

2. calculate the total volume of logs:

find / var / www / -type f -name "access.log *" -exec du -k {} \; | awk '{s + = $ 1} END {print s}'
5071604

so, do not be afraid - right now all will explain :)

find finds all files by mask (access.log *) and runs du command for everyone.
du command prints file size. the -k option makes output in kilobytes.
then the awk processor starts, simply summarizes the first field of strings (numbers) into the variable s and displays the value of the variable on the screen.

another example: find all the files and directories belonging to the user test1 in the system and calculate the total volume.

#find / -user test1 -exec du -sm {} \; | awk '{s + = $ 1} END {print s}'

those. here, using the -user option , find searches for files owned by the user test1 and for each file / directory we calculate its size (command du)
further through the pipeline, this data takes awk and, as we did above, considers their sum in Kb.

OK. I think that's enough for today.
the post was quite large, apparently out of habit :)

I want to say at once - I didn’t set a goal to just talk about using find and awk, but to give examples of practical applications in real situations.
If you like the article, I will continue to write in this direction.

Thanks for your time.