Recently, a lot of talk has been going on that for C ++ you need your own package manager like pip, npm, maven, cargo, etc. All competitors have a simple and standardized mechanism for connecting a non-standard library. In C ++, everyone acts as they can: someone registers a list of packages for Ubuntu, CentOS and other distributions in README, someone uses git submodule and scripts to build them, someone uses CMake ExternalProject, someone copies all the sources into one giant repository, someone doing the Docker or Vagrant image.

To solve the problem, a startup biicode was even created, but he went bankrupt and his future is unknown. Conan appeared in return , complementing the competitors' zoo — nuget , cget , hunter , cpm , qpm , cppget , pacm, and even gradle for c ++ .

I was not satisfied with any of these methods. I started writing packages for Conan, but I encountered a large number of hacks, an undeveloped API, a lack of guidelines and, as a result, a low probability of reusing other people's packages. And then I remembered that once I really liked the ideas of the package manager in NixOS . And I thought, why bother to produce a package manager specifically for C ++, if the same tasks are solved by a regular package manager? It is only necessary that it be sufficiently flexible and simple in the description of the package. And Nix was perfect for this role.

So, what gave us Nix:

• The ability to get ready for the assembly of the project environment with one command - nix-shell ;
• 7344 ready and supported packages from nixpkgs ;
• The ability to create a derived package from a package from the repository (without copying its code);
• Ability to specify in dependencies not only C / C ++ libraries, but also the necessary tools (CMake, GCC), projects from other ecosystems (npm, pip), services (redis);
• Ability to bind environment to commit. This means that, for example, the master branch can use boost 1.55, and devel - 1.60. When switching from branch to branch, Nix will automatically set up the environment for the required version, and this will take less than a second (if the assembly is already in the cache);
• Non-intrusiveness - the project does not depend on Nix, its use is a personal matter of everyone. You can collect all dependencies manually (or by your favorite package manager), specifying all the correct options for cmake.

What is Nix

Nix is ​​a functional programming language, tailored to the needs of the package manager (not surprisingly, it gained popularity in the Haskell community). Build a package is a function calculation in Nix. And as it should be for a functional programming language, repeated calls to a function with the same arguments produce the same result (binary package). This means that packages can be cached, which Nix does - all assemblies are stored in /nix/store/$HASH-$PKGNAME . In addition, you can check if someone else has a package with the same hash on the network, and if there is, download the binary package from it.

Thus, a “package” (here called derivation) in Nix is ​​a function, and “dependencies” are the arguments of this function. What is a repository ( NixPkgs )? This is also a function that has no arguments, which returns multiple packages. Is it possible that to use the repository you need to collect all 7344 packages? Not! Nix is ​​a lazy language, which means nothing will be calculated until it is explicitly required. A "request" package can be utilities.

Minimal environment

So, before you use Nix, you need to install it. To do this, you can either use the whole Linux distribution (NixOS), or install the package manager separately for your favorite OS (supported by Linux and MacOS). All Nix impacts will be limited to the /nix directory and files in the home directory ( ~/.nix-channel , .nix-defexpr , .nix-profile ).

~/.nix-profile stores symlinks for the packages requested by the user. We need to configure the environment not for the user, but for the project. To do this, use the nix-shell utility: it executes the given Nix input and launches the bash shell, in which the result is available (and only it). Checking:

 bash-3.2$ nix-shell -p stdenv [nix-shell:~]$ 

Here we use the ( -p ) stdenv package as an expression. stdenv is the minimal environment that contains the compiler, make, and other essential things.

Build Environment

If you run nix-shell with no arguments, the expression is read from the default.nix file. Create it:

 { pkgs ? import <nixpkgs> {} }: let stdenv = pkgs.stdenv; in rec { myProject = stdenv.mkDerivation { name = "my-project"; }; } 

Here we wrote a function that accepts a repository as an input (and if the parameter is not set, it imports the standard nixpkgs ) and returns the "package" of the environment of our project. Add to it fresh CMake, Boost and Google Test from the NixOS repository:

 # ... myProject = stdenv.mkDerivation { name = "my-project"; nativeBuildInputs = [ pkgs.cmake ]; buildInputs = [ pkgs.boost pkgs.gtest ]; }; 

Here, buildInputs are dependencies that are required for building. Why else nativeBuildInputs? The fact is that Nix supports cross-compiling. And here we say that the buildInputs packages should be built by the target toolbar, and the nativeBuildInputs should be compiled with the usual host toolbar. There is still propagatedBuildInputs - it adds dependency to all users of the package.

Now the next time you call nix-shell , Nix will extort the necessary binary packages and set the environment variables so that the libraries are found by standard means, for example, CMake:

 find_package(Boost 1.60 REQUIRED COMPONENTS system thread) find_path(GTEST_INCLUDE_DIRS NAMES gtest/gtest.h PATH_SUFFIXES gtest) 

The developer can only run cmake . && make cmake . && make , which we will inform him about when entering the nix-shell :

  myProject = stdenv.mkDerivation { # ... shellHook = ['' echo Welcome to myproject! echo Run \'mkdir build && cd build && cmake .. && make -j\' to build it. '']; }; 

We collect dependence which is not in nixpkgs

Now we want to add cppformat to our project. First we look for it in nixpkgs :

 $ nix-env -qaP | grep cppformat $ nix-env -qaP | grep cpp-format 

Is empty. You have to write your own expression. The benefit is only 10 lines. Add them to the "let":

 # ... let stdenv = pkgs.stdenv; fetchurl = pkgs.fetchurl; cppformat = stdenv.mkDerivation rec { version = "2.1.0"; name = "cppformat-${version}"; src = fetchurl { url = "https://github.com/cppformat/cppformat/archive/${version}.tar.gz"; sha256 = "0h8rydgwbm5gwwblx7jzpb43a9ap0dk2d9dbrswnbfmw50v5s7an"; }; buildInputs = [ pkgs.cmake ]; enableParallelBuilding = true; }; in rec { # ... buildInputs = [ # ... cppformat ]; # ... 

Now the next time you start nix-shell , Nix will download the cppformat sources, build them using cmake (he sees that the project uses cmake, so instead of the standard " ./configure && make install ", " cmake . && make install " is used) and caches the result builds in /nix/store . It is noteworthy that, unlike the utilities of most other package managers:

• If you fail to build the source will not be re-downloaded;
• If we change the expression, the package will be recompiled. If you later decide to roll back the expression, the old packet from the cache will be used automatically, even if the file modification date has changed (conveniently when changing the branch / commit).

Modifying the package from the repository

Sometimes the necessary package in the repository is, but not assembled the way we want. You need to build a specific version of it, apply a patch, use certain flags. Nix allows you to do this without having to copy-paste code from the repository:

  cpp-netlib = pkgs.cpp-netlib.overrideDerivation(oldAttrs: { postPatch = '' substituteInPlace CMakeLists.txt \ --replace "CPPNETLIB_VERSION_PATCH 1" "CPPNETLIB_VERSION_PATCH 3" ''; cmakeFlags = oldAttrs.cmakeFlags ++ [ "-DCMAKE_CXX_STANDARD=11" ]; src = fetchFromGitHub { owner = "cpp-netlib"; repo = "cpp-netlib"; rev = "9bcbde758952813bf87c2ff6cc16679509a40e06"; # 0.11-devel sha256 = "0abcb2x0wc992s5j99bjc01al49ax4jw7m9d0522nkd11nzmiacy"; }; }); 

Modifying the package in the repository

We can build a derived package X 'based on the original X from the repository and use it in our own. Moreover, if some package Y in the repository was dependent on X, then it will continue to use its old version. But what if you need to change the package inside the repository, i.e. so that 100,500 other packages start using it? And for this case in Nix there are tools. Rebuilding a boost from nixpkgs using GCC5 instead of the standard GCC 4.9:

 { nixpkgs ? import <nixpkgs> {} }: let overrideCC = nixpkgs.overrideCC; stdenv = if ! nixpkgs.stdenv.isLinux then nixpkgs.stdenv else overrideCC nixpkgs.stdenv nixpkgs.gcc5; pkgs = nixpkgs.overridePackages (self: super: { boost = super.boost.override { stdenv = stdenv; }; }); 

Here we changed the name of the argument from pkgs to nixpkgs and create a derivative repository pkgs , in which the booster is assembled as we want. Now all other boost-dependent packages need to be rebuilt to enable our build. Of course, only those packages that are used inside our expression will be (recursively) rebuilt - after all, Nix is ​​lazy.

Integration with third-party package managers and platforms

Everything is simple again - Nix has support for building packages for .NET, Emacs, Go, Haskell, Lua, Node, Perl, PHP, Python and Rust. For some of them, the integration is that Nix can use packages directly from the native package manager:

 nativeBuildInputs = [ pkgs.cmake pkgs.pkgconfig nodePackages.uglify-js ]; 

Integrating Nix into YouCompleteMe

YouCompleteMe is perhaps the most popular code completion engine for C ++, which is not part of the IDE. It came out of Vim, but there are already ports for Atom and, possibly, other editors. If earlier developers had to configure it independently for their system, then now we can do it universally:

 def ExportFromNix(): from subprocess import Popen, PIPE import shlex cmd = "nix-shell -Q --pure --readonly-mode --run 'echo $NIX_CFLAGS_COMPILE'"; proc = Popen(cmd, shell=True, stdout=PIPE) out = proc.stdout.read().decode("utf-8") return shlex.split(out) flags += ExportFromNix() 

Conclusion

Nix is ​​at the same time flexible, convenient and simple package manager, which is built on the principles of functional programming and claims to be a package manager for everything. Especially it can be convenient for C / C ++ programmers, since allows you to fill in the empty niche of the language. Using it, you can patch and add libraries to the project without causing pain and hating your colleagues. A newcomer who arrives in the team will not spend their first working days on building the project.