Grep all you can

About grep they know, if not all, then many readers of Habr , but few know his numerous relatives.

Let's find out how to grill everything that hides in itself even a grain of text.

pgrep

The pgrep command lists the executable processes.

 (5:573)$ pgrep bash 1772 11003 20678 

With the -a , the command will also output the entire command line.

 (5:574)$ pgrep -af bash 1772 -bash 11003 /bin/bash 20678 /bin/bash 23567 -bash 

If you correctly combine ps and pgrep , the result will probably be more convenient than ps -ef |grep process_name , since it does not contain the grep command itself.

 (5:575)$ ps wup $(pgrep bash) USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1772 0.0 0.0 21572 3020 tty1 S+ 19 0:00 -bash user 11003 0.0 0.0 21384 3772 pts/3 Ss 28 0:00 /bin/bash user 20678 0.0 0.0 21396 3604 pts/1 Ss+ 24 0:00 /bin/bash root 23567 0.0 0.0 21364 3016 tty2 S+ 21 0:00 -bash 946/7720MB 0.48 0.94 1.00 1/352 20403 

zipgrep, zfgrep, bzfgrep, xzfgrep

You can search for a keyword, a pattern directly from the archive using these commands.

 (5:576)$ zipgrep -i exception apptrace.zip jboss.stderr:java.lang.NumberFormatException: For input string: "4718-4e99-bc84-828a7bf7f254" jboss.stderr: at java.lang.NumberFormatException.(NumberFormatException.java:65) (5:576)$ (5:576)$ bzfgrep -w bugs /usr/share/doc/coreutils-8.25/README.bz2 that distribution and found and reported bugs. <http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/12292/focus=12318>. see <http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/12015/>. Reporting bugs: subtle bugs. When reporting bugs, please include in the subject line both the package 

Utilities support extended regular expressions and have their own egrep .

What packages do they belong to?

 (5:777)$ equery belongs $(which zgrep zipgrep bzgrep xzgrep) * Searching for /usr/bin/zgrep,/usr/bin/zipgrep,/usr/bin/bzgrep,/usr/bin/xzgrep ... app-arch/bzip2-1.0.6-r7 (/usr/bin/bzgrep) app-arch/gzip-1.8 (/usr/bin/zgrep) app-arch/unzip-6.0_p20 (/usr/bin/zipgrep) app-arch/xz-utils-5.2.2 (/usr/bin/xzgrep) 

We compare the ability to parse extended regulars with grep and rejoice at the result.

 (5:578) grep -oE '\b[0-9]{1,3}(\.[0-9]{1,3}){3}\b' /var/log/emerge.log |tail -n 3 7.2.6.8 213.180.204.183 213.180.204.183 (5:579) (5:579) bzegrep -o '\b[0-9]{1,3}(\.[0-9]{1,3}){3}\b' /tmp/emerge.log.bz2 |tail -n3 7.2.6.8 213.180.204.183 213.180.204.183 

pdfgrep

The program comes with the same package.

 (5:580)$ eix pdfgrep [I] app-text/pdfgrep  : 1.3.2 1.4.1-r1 {+pcre test unac}  : 1.4.1-r1(14:10:46 29.11.2016)(pcre -test -unac)  : http://pdfgrep.org/ : A tool similar to grep which searches text in PDFs 

The command parses pdf files and does all that grep .

 (5:581)$ pdfgrep -i outdiscards ebook/linux_netw.pdf IP6_INC_STATS(net, ip6_dst_idev(dst), IPSTATS_MIB_OUTDISCARDS); 

Pdfgrep does not support extended regular expressions , however it is a very useful program . I often open pdfs and look for text in them until I remember which book was what I needed and so I lose a lot of time. Now I know.

dgrep

Debian users may be familiar with the dgrep that comes with the debian goodies package . The command is invoked in the same way as regular grep , only the package name is indicated instead of the file.

The top three are searched in text files, and dzgrep - in archive files.

msggrep, mboxgrep

This is a very narrowly specialized gizmo to parse localization directories. Comes with the gettext package. The program is not from the category of user, but if you really need, you can run from the command line.

 (5:752)$ msggrep -K -e help /usr/share/locale/ru/cups_ru.po 

 msgid "" msgstr "" "Project-Id-Version: CUPS 2.0\n" "Report-Msgid-Bugs-To: http://www.cups.org/str.php\n" "POT-Creation-Date: 2015-07-20 14:24-0400\n" "PO-Revision-Date: 2015-01-28 12:00-0800\n" "Last-Translator: Aleksandr Proklov\n" "Language-Team: PuppyRus Linux Team\n" "Language: Russian\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=utf-8\n" "Content-Transfer-Encoding: 8bit\n" msgid " --help Show help." msgstr " --help  ." msgid " --help Show this help." msgstr " --help   ." 

The next exhibit is the mboxgrep mailbox parser . The project did not take off, its development is terminated. In theory, he had to find patterns in letters and process the output as if they were separate files. However, to begin with, he must be able to find these patterns.

 (5:753)$ grep Inomics docs/PocoMail.bak/Mail/Trash.mbx From: Inomics <inomics-alert@inomics.com> Reply-To: Inomics <inomics-alert@inomics.com> Subject: Inomics Alert Service following is a list of new job openings in Inomics, the= Inomics! You can always unsubscribe from the Inomics Job Alert service by= plying to this mail with the words "Inomics-Job-Unsub#1avwnr55di"= 

And he does not find.

 (5:754)$ mboxgrep Inomics docs/PocoMail.bak/Mail/Trash.mbx (5:755)$ 

Strangely, the read system calls are the same all the time, regardless of the search.

 (5:779)$ strace -e trace=read -o strace_inomics mboxgrep Inomics Mail/Trash.mbx (5:780)$ strace -e trace=read -o strace_freenet mboxgrep freenet Mail/Trash.mbx (5:781)$ diff strace_inomics strace_freenet;md5sum strace_* 14b7de546a2a776006ad2a6440b680fd strace_freenet 14b7de546a2a776006ad2a6440b680fd strace_inomics 

It would be curious to find out if this program was successfully launched from someone?
Well, okay, we got carried away, and the family is not fully inventoried.

ssgrep

You need to install the Gnumeric package. Able to wool spreadsheets, including proprietary Excel format. Swears at him, but does.

 (5:782)$ ssgrep Date files/*.xlsx     "43"     "41"     "44"     "42" Unexpected element 'workbookProtection' in state: workbook  /Date of Birth (5:783)$ grep Date files/*.xlsx (5:784)$ 

Extended regular expressions are not supported. It is a useful thing, I recommend .

ngrep

Another abandoned SourceForge project . The program is a hybrid of tcpdump and grep , and the first is much closer than the second.

Monitor network traffic on the syslog port and keyword.

 $ ngrep -d any 'error' port syslog 

Monitoring network traffic on ftp port and case-insensitive keywords, compare whole words.

 $ ngrep -wi -d any 'user|pass' port 21 

On-line output http traffic.

 $ ngrep -W byline port 80 interface: eth0 (64.90.164.72/255.255.255.252) filter: ip and ( port 80 ) #### T 67.169.59.38:42177 -> 64.90.164.74:80 [AP] GET / HTTP/1.1. User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686) Opera ... Host: www.darkridge.com. Accept: text/html, application/xml;q=0.9, application/xhtml+xml;q=0.9 ... Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1. Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0. Cookie: SQMSESSID=5272f9ae21c07eca4dfd75f9a3cda22e. Cookie2: $Version=1. Cache-Control: no-cache. Connection: Keep-Alive, TE. TE: deflate, gzip, chunked, identity, trailers. 

And then there is a killer feature . You can specify a hex string with which ngrep maps the binary data of the package. For example, you can set the signature of the corrupted gifs to then configure the firewall for early detection.

 $ ngrep -xX '0xc5d5e5f55666768696a6b6c6d6e6' port 80 interface: eth0 (64.90.164.72/255.255.255.252) filter: ip and ( port 80 ) match: 0xc5d5e5f55666768696a6b6c6d6e6 

It is a pity that the development of the project is terminated, it may eventually turn out quite suitable self-locomotive-helicopter parser and network traffic analyzer.

deepgrep

In the end, you can remember non-uniclass search aggregates, such as KDE's deepgirep strigi-utils .

The program can to shuffle search in format files:

• tar
• ar / deb
• rpm (but not cpio )
• gzip/gz
• bzip2/bz2
• zip , and therefore jar / war as well as OpenOffice.org/LibreOffice documents
• MIME messages (i.e. attachments in email)

Able to search in archives nested in each other .

 $ deepgrep bar foo.ar foo.ar/foo.tar/foo.tar.gz/foo.zip/foo.tar.bz2/foo.txt.gz/foo.txt:foobar foo.ar/foo.tar/foo.tar.gz/foo.zip/foo.tar.bz2/foo.txt.gz/foo.txt:bar 

Of the shortcomings.

• Virtually no command line options.
• Poorly documented.
• The return code is not related to the search result.
• There is no support for compression using the LZMA algorithm: lzma, xz, lzip, 7z and also lzop, rzip, cab. cpio, xar, rar, .Z lzop, rzip, cab. cpio, xar, rar, .Z formats.

Used materials

• grep everything
• deppgrep: grep nested archives with one command
• ngrep - network grep