Creating CloudFoundry / IBM Bluemix buildpack or Awk web service (gawk)

gawk - stare gaping, goggle



No further, I’ll really describe how to run an Awk (Gawk) web service on IBM Bluemix.
')
CloudFoundry and the IBM Bluemix platform based on it support many different programming languages ​​and web frameworks. All this thanks to the support of buildpacks (my translation of the word buildpack). Buildpack can be considered as a plugin that is responsible for installing and setting up the application execution environment. Build a pack usually in two cases.

Why write buildpack?

In the first case, the language or framework is supported, but the application requires third-party libraries, such as OpenCV, which cannot be installed as a dependency manager standard for this language (npm, pip, bundler). In this case, the existing buildpack is modified to which the installation of the necessary libraries is added. Most buildpacks are on github, a fork is made with which you can deploy the application.

The second case is more rare - the language is not yet supported, i.e. No one has created a buildpack yet. In this case, you will have to make it from scratch, which will be described below. I was looking for a long time what language is not yet supported by CloudFoundry, the choice fell on Brainfuck. Unfortunately, when I added the buildpack, I realized that I could not quickly write a web server in this language. The search showed that others had difficulties with this task.

To make it a little clearer, the problem is that the buildpack sets up the environment into which the user application is then placed, the port and the host name are transferred to it through the environment variables, where the application should start processing HTTP requests. Those. simple hello world is not enough. I continued searching and found the implementation of the web servers in PostScript and Awk (more precisely, gawk) . In the end, I chose Awk and started writing a new buildpack.

Ok, what do we write?

The documentation describes well the general concepts, you can also see the finished buildpacks. In the simplest case, you need to create a bin folder and place 3 executable files in it:

detect - determines whether this buildpack can be applied to the application. If so, the script should return 0 and type the name of the language or framework. In our case, we check the availability of app.awk in the application root, we introduce such an agreement.

#!/usr/bin/env bash # bin/detect <build-dir> # Any awk web app must have app.awk file if [ -f $1/app.awk ]; then echo "gawk" exit 0 fi exit 1 

compile should install everything you need and compile a custom application, if required. In our case, you need to download and install gawk. To save time during the deployment of the application, it was possible to build it manually for different versions of Ubuntu and select the appropriate one at this step. But building gawk is not such a costly process, and the cache is also supported, so for most projects the compilation will be done once. But we will definitely get the file compiled for the required OS version and architecture.

 #!/usr/bin/env bash # bin/compile <build-dir> <cache-dir> set -e build_dir=$1 cache_dir=$2 awk_cache=$cache_dir/gawk-4.1.3 awk_url="http://ftp.gnu.org/gnu/gawk/gawk-4.1.3.tar.gz" if [ -f $awk_cache/gawk ]; then echo "Using cached gawk" else # download and compile gawk mkdir -p $awk_cache curl -o $cache_dir/gawk.tar.gz "$awk_url" cd $cache_dir tar zxvf gawk.tar.gz cd $awk_cache ./configure make fi # copy gawk to build dir cp $awk_cache/gawk $build_dir # set path and test gawk PATH=$build_dir:$PATH gawk -V # create a launcher to wrap env vars passing echo "#!/usr/bin/env bash\n" > $build_dir/awk-start echo "/app/gawk -v PORT=\"\$PORT\" -v HOST=\"\$VCAP_APP_HOST\" -f \$1" >> $build_dir/awk-start chmod +x $build_dir/awk-start 

The first argument is the place to build the files (build_dir), the second is the cache. Our task is to put the necessary files to run in build_dir, in our case it is gawk. In addition, the Awk script does not have access to environment variables, and we need to know which port CloudFoundry allocates to us for launching (the PORT environment variable). So I made a little wrap for easy launch.

release - passes the metadata needed to launch the application. Typically, these are environment variables and the default launch string. It will be used if the application does not provide this information, which is usually done in the Procfile file.

 #!/usr/bin/env bash # bin/release <build-dir> cat <<EOF config_vars: PATH: /app:$PATH default_process_types: web: /app/awk-start app.awk EOF 

We check the work of scripts locally, then we post everything on github, remember the address https://github.com/hashmap/gawk-buildpack

Understand how to run everything?

Now we write application. Create a folder gawktest, in her file app.awk

 # A Web Server in Awk # a simple, single user, web server built with gawk. # based on code form http://awk.info/?tools/server by Michael Sanders BEGIN { host = "/inet/tcp/" PORT "/0/0" # host string status = 200 # 200 == OK reason = "OK" # server response RS = ORS = "\r\n" # header line terminators doc = Setup() # html document len = length(doc) + length(ORS) # length of document print "Staring AWK server" while (1) { print "HTTP/1.0", status, reason |& host print "Connection: Close" |& host print "Pragma: no-cache" |& host print "Content-length:", len |& host print ORS doc |& host close(host) # close client connection host |& getline # wait for new client request } } function Setup() { tmp = "<html>\ <head><title>Simple gawk server</title></head>\ <body>\ Hello from awk!\ </body>\ </html>" return tmp } 

We check application work locally:

  gawk -v PORT=8080 -f app.awk 

Now you can send to Bluemix, do the standard cf push, but also specify the address of our buildpack:

 gawktest cf push gawktest -b https://github.com/hashmap/gawk-buildpack Updating app gawktest in org xxx@gmail.com / space dev as xxx@gmail.com... OK Uploading gawktest... Uploading app files from: /Users/alex/projects/gawktest Uploading 32.3K, 3 files Done uploading OK Stopping app gawktest in org xxx@gmail.com / space dev as xxx@gmail.com... OK Starting app gawktest in org xxx@gmail.com / space dev as xxx@gmail.com... -----> Downloaded app package (8.0K) -----> Downloaded app buildpack cache (11M) Cloning into '/tmp/buildpacks/gawk-buildpack'... Using cached gawk GNU Awk 4.1.3, API: 1.1 Copyright (C) 1989, 1991-2015 Free Software Foundation. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/. -----> Uploading droplet (724K) 1 of 1 instances running App started OK App gawktest was started using this command `/app/awk-start app.awk` Showing health and status for app gawktest in org xxx@gmail.com / space dev as xxx@gmail.com... OK requested state: started instances: 1/1 usage: 64M x 1 instances urls: gawktest.mybluemix.net last uploaded: Wed Jul 15 12:21:13 UTC 2015 stack: lucid64 buildpack: https://github.com/hashmap/gawk-buildpack state since cpu memory disk details #0 running 2015-07-15 03:22:08 PM 0.0% 1.1M of 64M 1.9M of 1G 

Final Captions

Everything works and is available , at least for now, do not forget that the server is single-user. Since this is not the first run, the already compiled gawk was simply copied from the cache. When you first launch a new application, you will see how it is downloaded, unpacked and assembled. Also for debugging, the gawk version and its license are printed.

If anyone writes a web framework on Brainfuck - let me know.