Creating a script for publishing
From time to time the task appears: to make a script for the publication, which needs to be updated but cannot be changed. For example, it can be an initialization script, wired inside a virtual machine image or a script to install the site engine (published by the engine developer).
In the article I will talk about the techniques that I use to create such scripts, they will help to avoid some rakes, to preserve the simplicity and flexibility of scripts. The approach is suitable for those scripts whose behavior should change depending on the needs of the author, updates, etc. The approach is NOT suitable for scripts that should work autonomously (without communication with the author's system).
I use this approach in bash scripts, but the general principle can be applied regardless of the language.
A few years ago I began to make templates of VDS-servers for mass use by clients. After creating a template, changing it is no longer possible; the only way to correct the error is to publish a new template, which costs time and a lot of gigabytes of space (template + copies on all servers). In these few years, it has turned out to make some uncomfortable mistakes, and as a result, now many more years will have to maintain patterns with uncomfortable behavior at the start.
')
A similar situation can happen with the installation / configuration scripts of control panels, site engines, just programs that need to be downloaded, configured and configured one at a time. Despite the fact that the program itself can be updated, and everyone who downloaded it already has no access to the installation script fix.
Some of what I use was seen in similar scripts, for example, installers ispsystem, brew. The part was prompted by colleagues and the part is acquired by their own bitter experience.
Every time, download and execute all the code from your server.
Published script - only loads the first file of the executable code, nothing else is done.
The first file of the executable code - immediately does what it needs (in simple cases) or determines something in common and loads new files.
The rest of the files are organized in any convenient manner, it can be changed already in the process of work.
The published script in no case should perform the task for which it is published and should not even have a hint of its solution. The only task of this script is to find a way to connect to the developer’s server and download the code to execute from there. This code should be as simple as possible with a minimum of external dependencies, since they will have to be maintained throughout the life of the script.
I put this code in a template with a previously known environment, in particular, I know for sure that curl is there, but many attempts are necessary since when the server starts, the network may not work or the http server may temporarily generate an error. Other scripts may work in different environments and there may not be curl there. This is a good place to try to connect to the server in many ways, if necessary many times.
It is necessary to check that the code for execution has been loaded entirely - a long line with if does exactly the check: it checks that the script starts with #! / Bin / bash and ends with #BashScriptEnd. So you can be sure that the HTML error code or half-script will not be executed by cutting rm -rf / tmp / my-downloads to rm -rf /
I didn’t intentionally do more complex checks in this place - TCP in this case provides reasonable protection against data damage, and then any complication of the external interface will have to be maintained forever.
This script has only one external dependency - the URL. In the future, even then he had to change - with the improvement of the organization of the code, but the dependence is so simple that it is simple to maintain it. Moreover, in the new scripts, this is also the path used that has developed historically, and not the new “correct” one. Because in case of changes in the future, we would have to support two URLs, etc.
The script should not have any attempts to define the environment and load, for example, init_linux.sh or init_freebsd.sh instead of init.sh - there was also such an attempt, it turned out that it was inconvenient and now it is necessary to support stubs for the old version of scripts.
Here the freedom can be changed more without touching the already published part. So if everything is simple - here you can immediately put the code that will be executed. If something becomes complicated - it is easy to change in the future.
If the complexity of the executed code is more than 1 file, I recommend placing it here:
1. The function to download new files. It will redefine how to connect to the server and in all the scripts you need to use it. It is not suitable for the general library, since it is not loaded yet.
2. Call this function one / several times to connect and execute all the necessary files: general code library, specific code for the environment found, etc. It may be possible to see which command is passed in the arguments and load the code for executing this command.
In the article I will talk about the techniques that I use to create such scripts, they will help to avoid some rakes, to preserve the simplicity and flexibility of scripts. The approach is suitable for those scripts whose behavior should change depending on the needs of the author, updates, etc. The approach is NOT suitable for scripts that should work autonomously (without communication with the author's system).
I use this approach in bash scripts, but the general principle can be applied regardless of the language.
Short prehistory (you can skip):
A few years ago I began to make templates of VDS-servers for mass use by clients. After creating a template, changing it is no longer possible; the only way to correct the error is to publish a new template, which costs time and a lot of gigabytes of space (template + copies on all servers). In these few years, it has turned out to make some uncomfortable mistakes, and as a result, now many more years will have to maintain patterns with uncomfortable behavior at the start.
')
A similar situation can happen with the installation / configuration scripts of control panels, site engines, just programs that need to be downloaded, configured and configured one at a time. Despite the fact that the program itself can be updated, and everyone who downloaded it already has no access to the installation script fix.
Some of what I use was seen in similar scripts, for example, installers ispsystem, brew. The part was prompted by colleagues and the part is acquired by their own bitter experience.
General essence
Every time, download and execute all the code from your server.
General code structure
Published script - only loads the first file of the executable code, nothing else is done.
The first file of the executable code - immediately does what it needs (in simple cases) or determines something in common and loads new files.
The rest of the files are organized in any convenient manner, it can be changed already in the process of work.
What should and should not be done by the published script
The published script in no case should perform the task for which it is published and should not even have a hint of its solution. The only task of this script is to find a way to connect to the developer’s server and download the code to execute from there. This code should be as simple as possible with a minimum of external dependencies, since they will have to be maintained throughout the life of the script.
Here is the code I came to in the end.
function try() { # , - , ... , } # Execute init.sh from panel function execute_init() { local EVAL_CODE=`curl http://panel.1gb.ru/minimal/init.sh` if [ "${EVAL_CODE#\#\!/bin/bash}" != "$EVAL_CODE" ] && [ "${EVAL_CODE%\#BashScriptEnd}" != "$EVAL_CODE" ]; then eval "$EVAL_CODE" return 0 else return 1 fi } try 100000 execute_init I put this code in a template with a previously known environment, in particular, I know for sure that curl is there, but many attempts are necessary since when the server starts, the network may not work or the http server may temporarily generate an error. Other scripts may work in different environments and there may not be curl there. This is a good place to try to connect to the server in many ways, if necessary many times.
It is necessary to check that the code for execution has been loaded entirely - a long line with if does exactly the check: it checks that the script starts with #! / Bin / bash and ends with #BashScriptEnd. So you can be sure that the HTML error code or half-script will not be executed by cutting rm -rf / tmp / my-downloads to rm -rf /
I didn’t intentionally do more complex checks in this place - TCP in this case provides reasonable protection against data damage, and then any complication of the external interface will have to be maintained forever.
This script has only one external dependency - the URL. In the future, even then he had to change - with the improvement of the organization of the code, but the dependence is so simple that it is simple to maintain it. Moreover, in the new scripts, this is also the path used that has developed historically, and not the new “correct” one. Because in case of changes in the future, we would have to support two URLs, etc.
The script should not have any attempts to define the environment and load, for example, init_linux.sh or init_freebsd.sh instead of init.sh - there was also such an attempt, it turned out that it was inconvenient and now it is necessary to support stubs for the old version of scripts.
What to put in the downloadable script
Here the freedom can be changed more without touching the already published part. So if everything is simple - here you can immediately put the code that will be executed. If something becomes complicated - it is easy to change in the future.
If the complexity of the executed code is more than 1 file, I recommend placing it here:
1. The function to download new files. It will redefine how to connect to the server and in all the scripts you need to use it. It is not suitable for the general library, since it is not loaded yet.
2. Call this function one / several times to connect and execute all the necessary files: general code library, specific code for the environment found, etc. It may be possible to see which command is passed in the arguments and load the code for executing this command.
What to look for
- Always check loaded code before executing. Otherwise, you can perform a half-script or some kind of error.
- In the published script, the minimum (ideally one) external dependency is the address from which the main code is loaded. Always the same
- Always make several attempts to download each file. Even if the work goes on the local network - there may be temporary server errors when it returns something wrong, such as an error. Or even the connection will not accept. If you have one downloadable file and the commands are executed manually, it may not be a problem. But if the commands are built into the automation and there are a lot of connected files, the probability that the error will happen is very real. The loading of a file is not entirely real either.
- Before publishing the script, be sure to check it - it is a shame when it was sealed in one character and because of this you need to redo a lot of work on preparing / checking the template or interact with those who recently gave the script
Disadvantages and point of view on them
- The need to connect to the network (or the Internet). In the context of applying this approach, you still have to download something from the developer’s server (including from your own if the system is used internally) and you still need the network. The script will not be able to configure the local network if it does not work or set an access password if it cannot get it from the network. Also, the script will not be able to deploy software / website from the developer’s server if it is not available.
- Execution of undefined code. If this is your code, then everything is clear. If this is someone else's code, it will still be launched / executed at that level of trust that is supposed to be. If this is a client application installation - in the client environment and anyway, the developer inside his software can do anything and no one will check the entire code (if someone does - they will hardly use auto-deployment from third-party resources at all - only their own codes from their servers )
Source: https://habr.com/ru/post/230285/
All Articles