Running external processes in Scala

Introduction

In one of my home projects, I had to write a small external process manager. The application should have been able to start an external daemon, periodically monitor its state when it is necessary to turn off, turn on, change settings, etc. The existing functionality in Java for such tasks is very poor, and since I was also dealing with Scala at the same time, I decided to see how it deals with this. And I was pleasantly surprised: Scala offers in my opinion a good API for working with external processes.
In this article I would like to tell about it in more detail.

Running processes

At the heart of working with processes are two traits: this is scala.sys.process.Process and scala.sys.process.ProcessBuilder .
Process allows you to work with an already running process, and ProcessBuilder allows you to configure startup parameters.
These entities are located in the scala.sys.process package. To run a simple example, run the code:

scala> import scala.sys.process._ scala> val process: Process = Process("echo Hello World").run() scala> println(process.exitValue()) 

The run method is the main method for starting a process, whose declaration is located in the processBuilder track . Returns a reference to an object of type Process . The running process runs in the background, data is output in the console. There are two methods declared in Process :

• exitValue() - waits for the completion of the process and returns a completion code;
• destroy() - destroys the running process.

This trait is very similar to the standard Java class java.lang.Process.
There are more specialized methods for running processes in the ProcessBuilder process. I will give a brief description of the main ones:

• ! - starts the process, waits for completion of the execution, displays the data to the console, and returns the exit code of the process as a result;
• !! - starts the process, waits for completion of the execution, displays data in the console, if the completion code is different from zero - throws an exception, as a result returns the output of the process as a string;
• lines - starts the process, returns Stream [String]. This thread allows parallel execution of the process to read the process data. If the information is not available, the Stream is blocked and will wait until the information reappears, or the process completes execution. If the exit code of the process is non-zero, the method will raise an exception. To avoid an exception, call lines_ !;
• run - starts the process and returns a link to Process .

In my project, I did not need to store references to external processes, so I almost did not use the run method. But the method ! just fit for me.
The previous example can be rewritten as:

 scala> Process("echo Hello World!").! Hello World! res1: Int = 0 scala> Process("echo Hello World!").!! res2: String = "Hello World!" scala> Process("echo Hello World!").lines res3: Stream[String] = Stream(Hello World!, ?) 

Implicit type conversion

There are methods for implicitly converting strings ( java.lang.String ) and sequences ( scala.collection.Seq ) to the ProcessBuilder treyt .
We can write our code like this:

 scala> "echo Hello World!".! Hello World! res2: Int = 0 

or so:

 scala> Seq("echo", "Hello", "World!").! Hello World! res3: Int = 0 

To a large extent, this reduces and simplifies writing, and the code becomes more understandable.
And this in turn reduces the number of errors in the future.

Process Combining (Pipe)

Process calls can be combined in chains, similar to command chains in linux.

 scala> "ls".! 11.txt 1.txt 2.txt 3.txt res2: Int = 0 scala> ("ls" #| "grep 1").! 11.txt 1.txt res6: Int = 0 

The output from the ls command was sent to the grep input. Grep filtered the received information on entry 1.
You can perform conditional operations, for example:

 scala> ("find . -name *.txt -exec grep 0 {} ;" #| "xargs test -z" #&& "echo 0-free" #|| "echo 0-exists").! 0-exists res23: Int = 0 

Here, if there are files with the extension * .txt in the directory and in some of them, there is 0 in the text - it will output 0-exists to the console, otherwise 0-free.
#&& - executes the following command, if the previous one was executed correctly;
#|| - executes the following command, if the prediction is executed with errors.
I like this functionality most of all; it allows you to use linux-like pipe inside Scala and write small sh scripts right inside your code.
')

Override I / O streams

All our code is inconvenient and useless without the functionality of redefining the input / output of external processes.
It is often necessary to monitor the output of information in order, for example, to decipher the error that has occurred, or to make sure that everything works correctly.
In each processBuilder method, in each of the methods run,!, !!, lines, you can pass an instance of a treyt ProcessLogger , which allows you to redirect the program's output streams to a file or line.
Here's how using ProcessLogger you can calculate the number of lines printed by the process:

 scala> var normalLines = 0 normalLines: Int = 0 scala> var errorLines = 0 errorLines: Int = 0 scala> val countLogger = ProcessLogger(line => normalLines += 1, | line => errorLines +=1) countLogger: scala.sys.process.ProcessLogger = scala.sys.process.ProcessLogger$$anon$1@459c8859 scala> "ls" ! countLogger res0: Int = 0 scala> println("normalLines: " + normalLines + ", errorLines: " + errorLines) normalLines: 4, errorLines: 0 

ProcessLogger allows you to override output streams. To override both input and output, the class scala.sys.process.ProcessIO is also used.
A small example:

 Seq("grep", "1") run new ProcessIO((output: java.io.OutputStream) => { output.write("1.txt\n2.txt\n3.txt\n11.txt".getBytes) output.close() }, (input: java.io.InputStream) => { println(Source.fromInputStream(input).mkString) input.close() }, _.close()) 

The first parameter is the input stream to the process: here we write the initial data.
The second parameter is standard output, and the last is output for errors.
Parameters are functions that process the necessary threads.
I said earlier that the execution of external commands can be combined, in addition, using the same recording form, you can transfer data to the process, or read them from there.
You can transfer data from a file to a process using the #< method, and write it down using the #> method:

 scala> ("echo -e 1.txt\\n2.txt\\n3.txt" #> new java.io.File("1.txt")).! res21: Int = 0 scala> ("grep 1" #< new java.io.File("1.txt")).!! res22: String = "1.txt" 

In the same way, for example, you can copy information from one file to another:

 scala> (new java.io.File("1.txt") #> new java.io.File("2.txt")).! res23: Int = 0 scala> "cat 2.txt".! 1.txt 2.txt 3.txt res24: Int = 0 

Conclusion

In the article, I talked about the basics of working with external processes in Scala. In Java, to implement this, I would have to write a bunch of wrappers, and in the end, I still would not be able to come close to such simplicity. You can read more about the API at http://www.scala-lang.org or delve into the source code (which I did, for example, took some examples from there). In jdk1.7, we slightly expanded the java.lang.ProcessBuilder class, and Java has become easier to run and execute external commands. But the simplicity of Scala, jdk is far away.