We write shell scripts in Python and is it possible to replace them with Bash

This small article will discuss whether it is easy to use Python to write scripts instead of Bash / Sh. The first question that will arise from the reader, perhaps, is why, in fact, not to use Bash / Sh, which were specifically created for this? They were created quite a long time ago and, in my opinion, have a rather specific syntax, not much like other languages, which is difficult to remember if you are not the administrator of 50+ level. Do you remember how to write a simple if on it?

if [ $# -ne "$ARGCOUNT" ] then echo "Usage: `basename $0` filename" exit $E_WRONGARGS fi 

Elementary truth? Intuitive syntax. :)

However, in python these constructs are much simpler. Every time I write something on a bash, I’m sure to go to a search engine to remember how to write a simple if, switch, or something else. I have already memorized the assignment. :) In Python, everything is different. Although I do not write on it round the clock, but I never had to go and watch how to make a simple cycle there, because the syntax of the language is simple and intuitive. Plus, it’s much closer to other mainstream languages ​​like java or c ++ than Bash / Sh.
')
Also in standard and other Python libraries there are much more convenient libraries than console utilities. Let's say you want to parse json, xml, yaml. Do you know what I recently saw code in bash to do this? Right:

 python -c "import json; json.loads..." :) 

And it was not my code. This was the code of the bache / python neutral man.

The same with regex, sed is undoubtedly a handy utility, but how many people remember how to use it correctly? Well, except for Lee E. McMahon, who created it. Yes, in principle, many people remember, even I remember how to do simple things. But, in my opinion, in Python the re module is much more convenient.

In this small article I would like to introduce you to the Python dialect called shellpy and serves to replace Bash with python in scripts as far as possible.

Wellcome under cat.

Introduction

Shell python is no different from simple Python except for one detail. The expressions inside the grave accent characters (`), unlike Python, are not eval, but indicate the execution of a command in a shell. for example

 `ls -l` 

will execute ls -l as a shell command. It is also possible to write all this without the `at the end of the line

 `ls -l 

and this will also be the correct syntax.

You can execute several commands at once on different lines.

 ` echo test > test.txt cat test.txt ` 

and multiple line commands

 `echo This is \ a very long \ line 

Execution of each expression in shellpy returns an object of class Result

 result = `ls -l 

This can be either Result or InteractiveResult (Links to the github with documentation, and then you can see :)). Let's start with a simple result. From it, you can easily get the return code of the executed command.

 result = `ls -l print result.returncode 

And the text from stdout and stderr

 result = `ls -l result_text = result.stdout result_error = result.stderr 

You can also run through all stdout lines of a command executed in a loop

 result = `ls -l for line in result: print line.upper() 

and so on.

For the result, there is also a lot of syntactic sugar. For example, we can easily verify that the return code of the command being executed is zero.

 result = `ls -l if result: print 'Return code for ls -l was 0' 

Or a simpler way to get text from stdout

 result = `ls -l print result 

All of the above is a syntax overview in brief, to simply understand the basic idea and not load you with all the details. There is much more to interact with the commands being executed, to control the execution of commands. But these are all the details that you can plunge into the documentation (in English), if the idea itself seems interesting to you.

This is not a valid Python syntax, how does it work?

Magic of course, how else :) Yes, my friends, I had to use preprocessing, I confess, but I could not find another way. I've seen other libraries that do something like this without breaking the syntax of a language like

 from sh import ifconfig print(ifconfig("wlan0")) 

But this syntax did not suit me, because even despite the difficulties, I wanted to get the best user experience ©, but for me it means as much as possible the writing of commands that is simple and close to His Majesty Shell.

A reader familiar with the topic asks what the IPython didn’t suit you, there you’re almost like you just have to put another icon, can you just be a cyclist who is too lazy to look at the search engine? And it really looks like this:

 lines = !ls -l 

I tried to use it but met a couple of serious problems that I could not get along with. The most important of them, is that there is no simple import like in Python. That is, you cannot write any code on the ipython itself and it is easy to reuse it in other places. Impossible to write for your ipython module

 import myipythomodule 

and that all at once earned like a fairy tale. The only way to reuse the script is to execute it. After performing in the environment, you have all the functions and variables declared in the executable file. Not kosher in my opinion.

In shellpy, the code is reused easily and imported in the same way as in regular python. Suppose we have a common module in which we store a very useful code. Look in the directory with this module.

 ls common/ common.spy __init__.spy 

So, what we have here, well, firstly init , but with the extension .spy. This is the hallmark of the spy module from the usual. Let's also look inside the common.spy file, what's interesting there

 def common_func(): return `echo 5 

We see that a function is declared here that internally uses shellpy syntax to return the result of the execution of `echo 5. How is this module used in the code? That's how

 from common.common import common_func print('Result of imported function is ' + str(common_func())) 

Do you see? As in normal Python, they just took it and imported it.

How it all works. This works with PEP 0302 - New Import Hooks . When you import something in your code, Python first asks the hook if there is something yours, the hook looks at PYTHONPATH for * .spy files or shellpython modules. If there is nothing, then it says: "There is nothing, import it yourself." If he finds something there, then the hook imports on its own. Namely, it makes the file preprocessing into regular python and puts all this stuff into the temp directory of the operating system. Writing down a new Python file or module, he adds it to PYTHONPATH and the most ordinary import is taken into account.

Let's take a quick look at some example.

This script downloads a Python user avatar from Github and puts it in the temp directory

  import json import os import tempfile #   curl      answer = `curl https://api.github.com/users/python #         if answer: answer_json = json.loads(answer.stdout) avatar_url = answer_json['avatar_url'] destination = os.path.join(tempfile.gettempdir(), 'python.png') #       result = `curl {avatar_url} > {destination} if result: #    ,    p`ls -l {destination} else: print('Failed to download avatar') print('Avatar downloaded') else: print('Failed to access github api') 

Beauty...

Installation

Shellpython can be installed in two ways: pip install shellpy or by pip install shellpy repository and running setup.py install . After that, you will have a shellpy utility.

Run something

After installation, you can test shellpython with examples that are available directly in the repository.

 shellpy example/curl.spy shellpy example/git.spy 

There are also allinone examples here, which are called like this, because they are testing all-all functions that shellpy has. Take a look there to better find out what else is there, or just follow

 shellpy example/allinone/test.spy 

For the third Python, the command looks like this.

 shellpy example/allinone/test3.spy 

Compatibility

It works on Linux and should work on Mac for Python 2.x and 3.x. It does not work on Windows yet, but there are no problems for work, since everything was written using cross-platform libraries and there is nothing platform-specific in the code. Just did not reach the hands yet to test on windows. I don’t have a poppy either, but it seemed to work for a friend :) If you have a poppy and you are fine, please tell me.

If you find problems - write in the comments, or here either wire sometime :)

Documentation (in English)

Wiki

Is it possible to translate

Of course :)

Doesn't it break me in production?

Now version 0.4.0, this is not a stable and production process, so far it is better not to tie it to a script, after waiting for everything to work out. But in development, CI can be used completely. All this is covered in tests and works :)

Ps

Write your feedback about the idea as a whole and about the implementation in particular, as well as about problems, wishes, everyone is happy to hear :) Start Issues still in the githaba, there are already a lot of them :)