Pitfalls shell scripting

Despite the widespread use of graphics, the shell does not lose its relevance to this day. And sometimes it allows you to perform operations much faster and easier than in a graphical environment. However, there are many things that most do not even suspect.
I would not want to be attached to any particular shell, however not every one of the features discussed below can be POSIX compatible, but it will be guaranteed to work in ksh / bash / zsh.

1. Variables and test

It's no secret that in the shell you can compare strings, numbers, and even variables. (:

[[ 2 -eq 3 ]] [[ "test" == "test" ]] [[ $VAR -eq 3 ]] 

But here's the last option, after a random typo (forgotten $ before a VAR), made me study the behavior in this case in more detail, because To my surprise, the construction worked without errors and the value of VAR was substituted as if I had not forgotten $. Moreover, if in VAR to refer to another variable, then everything works fine.
The developers didn’t forget to protect themselves from recursion:

 $ V=V ; [[ V -eq 12 ]] -bash: [[: V: expression recursion level exceeded (error token is "V") 

2. Get variables from a subshell without third-party utilities

Suppose we have a simple construction:

 do_something | while read LINE ; do export VAR_N=${LINE##%*} ; done 

Beginners often have to wrestle with their head, why VAR_N is not available after the completion of construction. The fact is that for a while loop, a subshell is created from which the variables do not reach the parent anymore. To get the variables we need from the cycle we have to sweat a lot. All options offered on the Internet, as a rule, are reduced either to memorizing the output into a variable and repeated parsing:

 VAR=$(cycle) VAR_N=$(echo "$VAR"|sed 'magic_sed') 

works, but somehow ugly. Yes, and all sorts of sed'y, perl'y and other pleasures of life have to repeatedly cause, which is clearly not for the better affects the performance. And why are they all, if you can do without them?
You just need to correctly form the output with a beautiful separator. For example:

 OLD__IFS="$IFS" IFS='~' #    set -- $(cycle) VAR_N="$1" VAR_NN="$2" IFS="$OLD__IFS" 

And much clearer, and correct if something is much simpler than reworking the regulars each time.

3. Hide some data from the pipe

Sometimes a situation arises when part of the data needs to be hidden from one of the pipes, and then connected back to the whole stream. In such a situation, the hidden data can be redirected to stderr, and then returned from stderr back to stdin:

 $ ( { echo DATA ; echo HIDDEN_DATA >&2 ; } | sed 's/^/MODIFIED_/' ) 2>&1 | sed s/$/_CATCHED/ HIDDEN_DATA_CATCHED MODIFIED_DATA_CATCHED 

4. I want to parse the history of commands

Along with lovers to parse the output of ls, there are also lovers to parse the history of commands (for further automatic processing of the result), but they don’t even know about anything other than history, moreover, they don’t completely take into account that the output of history is a great success Here you can ask a backfill question: “what else can you see history?” (I'll hide the answer specifically for the spoiler) (:

5. I remember that if .. then .. else .. fi is different from .. && .. || .. .. && .. || ..

Some fans of refactoring rush to make the code as compact as possible. It is especially amusing to observe the interchange of the above-mentioned constructions with subsequent debugs.
These two designs are not just different, they are completely different, although outwardly it may seem that they do one thing.

 if $(condition); then com_1; else com_2; fi condition && com_1 || com_2 

The obvious difference is that in the first case com_2 will be executed only in one case: condition returns false.
In the second case, com_2 will be run if condition returns false, and also if com_1 returns an error. Do not believe? see for yourself:

  $ if true ; then false ; else echo 'hello' ; fi $ true && false || echo 'hello' hello 

Well, perhaps that's all. I wanted to put in a few words about my beloved sed, but already somehow it happened a long time. Maybe another time (: