Phoenix Framework - Webpack instead of Brunch, deploy using Distillery and a little systemd

This article is an attempt by the author to bring together in the form of a small guide several topics that, in one way or another, almost all web application developers face, namely work with static files, configurations and application delivery to the server. At the time of this writing, the last stable branch of the Phoenix Framework was the 1.2.x branch.

Who cares why not Brunch and how to combine migrations with Distillery - I ask for cat.

Phoenix for working with JS-code and assets uses Brunch by default - perhaps a very simple and fast bandler, but certainly not the most common and not the most powerful in terms of ecosystem capabilities and size (and answers to StackOverflow, of course). Therefore, Brunch was replaced on the Webpack , de facto the number one bandler in the current world of the front end.

But to solve issues of deployment, the framework provides almost nothing but the ability to put different configurations for different environments. Judging by the answers of various developers on forums and other sites, many of them deploy their applications by installing development tools directly on the combat server and compiling and running the application using Mix. For a number of reasons, I consider such an approach unacceptable, therefore, having tried several options for packaging an application in a self-contained package, I settled on Distillery.

Since Since an article is a tutorial, an absolutely unnecessary application will be developed as an example, displaying a certain list of certain users. All code is available on GitHub , each step is fixed as a separate commit, therefore I recommend watching the change history. Also, I will give links to commits on certain steps, so that, on the one hand, it was clearly visible on the diff, what changes were made, and on the other - not to clutter the text with listings.

Training

So, create a template for our project, with the indication that we will not use Brunch:

$ mix phoenix.new userlist --no-brunch 

Nothing interesting happens here. We need to go inside the new project, correct the database settings in the config / dev.exs file , start the creation of the Ecto repository and migrations ( commit ):

 $ mix ecto.create && mix ecto.migrate 

In order to make an example at least a little clearer, I added a model of User entity, containing two fields - a name and a binary sign, whether the user is active or not ( commit ):

 $ mix phoenix.gen.model User users name active:boolean 

Further, in order to fill the database with at least some data, I added three instances of "users" to the priv / repo / seeds.exs file , which is used for such purposes. After that, you can perform the migration and insert the data into the database:

 $ mix ecto.migrate && mix run priv/repo/seeds.exs 

Now we have the migration in priv / repo / migrations / - it will be useful to us later, but for now, we must also add the http API, according to which the application can pick up the list of users in the format of a JSON object ( commit ). I will not clutter the text with listings, the diff on the GitHub will be more obvious, let me just say that a controller has been added, I twist and change the routing so that we have an "http-handle" along the path / api / users , which will return JSON with users.

That's all with the preparations, and at this stage the application can be started with the command

 $ mix phoenix.server 

and make sure everything works as intended.

Static files and JS

Now let's pay attention to the structure of the project directories, namely, two of them - priv / static / and web / static / . The first of them already contains the files that are needed to display the phoenix "Hello, World!" pages, and this directory is used by the application, when it is running, to upload static files. The second directory, web / static / , is enabled by default during development, and Brunch (in projects with it), roughly speaking, puts files from it into priv / static , processing them simultaneously ( article in the official documentation about this ).

Both of the above mentioned directories are under the control of the version control system, files can be added to both of them, only if you add the files directly to priv / static / , they will not be processed by Brunch, and if they are added to web / static / , , but if you put the file in the web / static / assets / , then again will not ... It seems to me that something went wrong here, because I propose a more rigorous approach, namely:

• the contents of the priv / static / directory never appear there as a result of some manual actions, only as a result of the work of some kind of pipeline. Moreover, this directory is removed from VCS and added to .gitignore ;
• The web / static / directory contains static files that will be copied unchanged into / priv / static by the corresponding pipeline when compiling, building the release package, etc.
• everything else that should appear in priv / static / (js, for example) lies somewhere else in the source tree and enters the resulting directory only through the corresponding pipeler bandler.

So, the next step I cleared priv / static from unnecessary files, and robots.txt and favicon.ico transferred to web / static / - we will return to them later. Also, I cleaned the html layout of the main page and its template ( commit ).

Before adding a Webpack, you need to initialize the NPM itself:

 $ npm init 

I cleaned the resulting package.json, leaving only the most important thing in it ( commit ):

 { "name": "userlist", "version": "1.0.0", "description": "Phoenix example application", "scripts": { }, "license": "MIT" } 

And after that we add Webpack itself ( commit ):

 $ npm install --save-dev webpack 

Now let's add some minimal JS code to the project, for example, like this:

 console.log("App js loaded."); 

For JS files, I created the web / js / directory, where I put the app.js file with the code above. Let's connect it in the web / templates / layout / app.html.eex template by pasting the closing </ body> tag:

 <script src="<%= static_path(@conn, "/js/app.js") %>"></script> 

It is very important to use the macro static_path, otherwise you will lose the ability to load resources with a digest-tag, which will lead to problems with the invalidation of caches by clients and in general, this is not according to the rules.

Create a Webpack configuration — the webpack.config.js file in the project root:

 module.exports = { entry: __dirname + "/web/js/app.js", output: { path: __dirname + "/priv/static", filename: "js/app.js" } }; 

From the code it can be seen that the resulting app.js file will be located in the priv / static / js / directory as intended. At this stage, you can start the Webpack manually, but this is not very convenient, so let's add automation, since the framework allows it. The first thing to do is add the watch shortcut to the scripts section of the package.json file:

 "scripts": { "watch": "webpack --watch-stdin --progress --color" }, 

Now Webpack can be run with the command

 $ npm run watch 

But you don’t need to do this either, let Phoenix do it, especially since the endpoint of your application has the watchers option, which is exactly what is meant for running such external utilities. Change the config / dev.exs file by adding the npm call:

 watchers: [npm: ["run", "watch"]] 

After that, the Webpack in the tracking mode for changes in directories and files will be launched every time along with the main application by the command

 $ mix phoenix.server 

A commit with all the above changes here .

C JS code figured out a bit, but there are still files in the web / static / . I also assigned the task of copying them to the Webpack, adding the copy extension to it:

 $ npm install --save-dev copy-webpack-plugin 

Configure the plugin in the webpack.config.js file ( commit ):

 var CopyWebpackPlugin = require("copy-webpack-plugin"); module.exports = { entry: __dirname + "/web/js/app.js", output: { path: __dirname + "/priv/static", filename: "js/app.js" }, plugins: [ new CopyWebpackPlugin([{ from: __dirname + "/web/static" }]) ] }; 

After these manipulations, our priv / static / directory will begin to be filled with two pipelines - JS processed and static files that do not require it. On top of this stage, I added the display of the list of users with the help of JS ( commit ), visual style for inactive users ( commit ) and a picture-logo for greater visualization of the work of the pipeline ( commit ).

The question may arise - what to do if you need to make pre-processing, for example, CSS. The answer is banal - to put the CSS in a separate directory, add the appropriate plug-ins to the Webpack and set up a pipeline similar to that used for JS. Or use css-loader'y, but this is a separate story.

Build release package. Distillery.

Distillery is the second approach of Exrm in an attempt to make a good tool for packaging and creating release packages for Elixir projects. Errors of the first were taken into account, much has been fixed, it is convenient to use Distillery. Add it to the project, specifying the dependency in mix.exs :

 {:distillery, "~> 1.4"} 

Update the dependencies and create a release configuration template ( commit ):

 $ mix deps.get && mix release.init 

The last command will create a file rel / config.exs of approximately the following content:

 Path.join(["rel", "plugins", "*.exs"]) |> Path.wildcard() |> Enum.map(&Code.eval_file(&1)) use Mix.Releases.Config, # This sets the default release built by `mix release` default_release: :default, # This sets the default environment used by `mix release` default_environment: Mix.env() environment :dev do set dev_mode: true set include_erts: false set cookie: :"Mp@oK==RSu$@QW.`F9(oYks&xDCzAWCpS*?jkSC?Zo{p5m9Qq!pKD8!;Cl~gTC?k" end environment :prod do set include_erts: true set include_src: false set cookie: :"/s[5Vq9hW(*IA>grelN4p*NjBHTH~[gfl;vD;:kc}qAShL$MtAI1es!VzyYFcC%p" end release :userlist do set version: current_version(:userlist) set applications: [ :runtime_tools ] end 

I propose to leave it as it is for now. Specified in the configuration is enough: one release: userlist, it's the same: default, because the first and only release list, as well as two environments: dev and: prod. The release here refers to the OTP Release, the set of applications that will be included in the resulting package, the ERTS version. In this case, our release corresponds to the application: userlist, which is enough for us. But, we can have several releases and several environments and combine them as needed.

Distillery is expanded with plug-ins, so that any additional pipeline can be arranged during assembly. More about plugins here .

Prepare an application for release. First of all, you need to edit the file config / prod.secret.exs , fix the database settings in it. This file is not added to the VCS, therefore, in case of its absence, it is necessary to create it yourself with approximately the following content:

 use Mix.Config config :userlist, Userlist.Endpoint, secret_key_base: "uE1oi7t7E/mH1OWo/vpYf0JLqwnBa7bTztVPZvEarv9VTbPMALRnqXKykzaESfMo" # Configure your database config :userlist, Userlist.Repo, adapter: Ecto.Adapters.Postgres, username: "phoenix", password: "", database: "userlist_prod", pool_size: 20 

The next important step is to correct the configuration of the Userlist.Endpoint in the config / prod.exs file . First of all, replace the host with the required one, and the port from 80 to the PORT parameter read from the environment and add the most important server option, which is a sign that this endpoint will start the Cowboy:

 url: [host: "localhost", port: {:system, "PORT"}], ... server: true 

Next, I added Babel to the JS code processing pipeline, since UglifyJS, used by default in Webpack, is not trained to handle ES6:

 $ npm install --save-dev babel-loader babel-core babel-preset-es2015 

And the Babel configuration section in webpack.config.js after the plugins:

 module: { loaders: [ { test: /\.js$/, exclude: /node_modules/, loader: "babel-loader", query: { presets: ["es2015"] } } ] } 

And the last thing is to add short deploy to the NPM configuration ( commit ):

 "scripts": { "watch": "webpack --watch-stdin --progress --color", "deploy": "webpack -p" }, 

At this stage, you can try to build and run the release:

 $ npm run deploy $ MIX_ENV=prod mix phoenix.digest $ MIX_ENV=prod mix release $ PORT=8080 _build/prod/rel/userlist/bin/userlist console 

With the first command we prepare JS (minification, etc.), copy static files; the second generates a digest for all files; the third directly collects the release for the corresponding environment. Well, in the end - launching the application online, with the console.

After the release, the _build directory will contain the unpacked (exploded) version of the package, and the archive will be located along the path _build / prod / rel / userlist / releases / 0.0.1 / userlist.tar.gz .

The application will start, but when you try to get a list of users, an error will be caused, because we did not apply migration for this database. The documentation for Distillery describes this point , but I simplified it a bit.

Migrations

After the build, the executable file of the application provides us with one of the options called command:

 command <mod> <fun> [<args..>] # execute the given MFA 

This is very similar to rpc, with the difference that the command will be executed on an application that is not running as well. We will create a module with the migration function, remembering that the application will not be running. I placed this file in lib / userlist / release_tasks.ex ( commit ):

 defmodule Release.Tasks do alias Userlist.Repo def migrate do Application.load(:userlist) {:ok, _} = Application.ensure_all_started(:ecto) {:ok, _} = Repo.__adapter__.ensure_all_started(Repo, :temporary) {:ok, _} = Repo.start_link(pool_size: 1) path = Application.app_dir(:userlist, "priv/repo/migrations") Ecto.Migrator.run(Repo, path, :up, all: true) :init.stop() end end 

As you can see from the code, we load, and then run not all applications, but exactly necessary ones - in this case, this is only Ecto. Now all that remains is to rebuild the release (only Elixir, since the rest has not changed):

 $ MIX_ENV=prod mix release 

start migration:

 $ _build/prod/rel/userlist/bin/userlist command 'Elixir.Release.Tasks' migrate 

and run the application:

 $ PORT=8080 _build/prod/rel/userlist/bin/userlist console 

That's all, but there are still a couple of small things left. For example, it is not very convenient to start migrations in this way, specifying the full module name, function. Distillery provides hooks and commands for this (now others).

Hooks and Distillery teams

The concept of hooks and commands is simple - these are ordinary shell scripts that are called at a certain stage of the application’s life (hooks) or manually (commands) and which are extensions of the main executable boot script. Hooks can be of four types: pre / post_start and pre / post_stop.

I added an example of two hooks to the project, see the code , it will best explain how to do this.

In turn, the commands will help to hide unnecessary details, so that, for example, the migrations look like:

 $ _build/prod/rel/userlist/bin/userlist migrate 

If you need manifest.json

When building a release, after executing the phoenix.digest command, all static files get a hash sum in their name (plus compressed versions are added), and a correspondence table is generated between the original file name and the new one, which is in the priv / static / manifest.json file , if you did not change its position in the configuration. If suddenly you need information from it during the execution of the application, then you have two options:

• add it to the list of files that are sent from the directory with statics in lib / userlist / endpoint.ex :

  
  

   only: ~w(css fonts images js favicon.ico robots.txt manifest.json) 

  
  

  after which, it can be picked up by Ajax, for example;

  
  

• if you need it on the backend, or if you want to render it in a template (I don’t know why, but suddenly it’s necessary), you can extend the LayoutView to this:

  
  

   defmodule Userlist.LayoutView do use Userlist.Web, :view def digest do manifest = Application.get_env(:userlist, Userlist.Endpoint, %{})[:cache_static_manifest] || "priv/static/manifest.json" manifest_file = Application.app_dir(:userlist, manifest) if File.exists?(manifest_file) do manifest_file |> File.read! else %{} end end end 

  
  

  so that later, somewhere in the template, write the following:

  
  

   <script> var digest = <%= raw digest() %> </script> 

  
  

Commit with this madness here .

systemd

The last thing I would like to mention is the launch of the application on the combat server. Since systemd appeared in our system, init-scripts writing is not something that has improved, but has simply become elementary.

Assume that we will deploy the archive with the application in / opt / userlist / and run as user userlist. Create a userlist.service file with the following content ( commit ):

 # Userlis is a Phoenix, Webpack and Distillery demo application [Unit] Description=Userlist application After=network.target [Service] Type=simple User=userlist RemainAfterExit=yes Environment=PORT=8080 WorkingDirectory=/opt/userlist ExecStart=/opt/userlist/bin/userlist start ExecStop=/opt/userlist/bin/userlist stop Restart=on-failure TimeoutSec=300 [Install] WantedBy=multi-user.target 

After that, all you need to do is copy it to / etc / systemd / system / :

 $ sudo cp userlist.service /etc/systemd/system 

Include in "autoload":

 $ sudo systemctl enable userlist.service 

And run the application:

 $ sudo systemctl start userlist 

Conclusion

The purpose of this article was to try to gather together isolated information on different topics regarding Phoenix and to give some more or less complete idea about the life cycle of applications written in this wonderful framework. A lot of things are left behind the scenes, there are a lot of topics worthy of individual articles, for example, methods of delivering release packages to a server, etc.

I, as an author, understand perfectly well that I can be wrong, because I apologize in advance for errors or inaccuracies and ask to write about those in the comments.