Geekly Articles each Day
📜 ⬆️ ⬇️

Use GYP to build a C / C ++ project.

Introduction


On Habré , the system for generating build scripts and CMake project files has been mentioned several times . The system is quite popular and well documented. Not everyone knows that there is a good alternative to this (certainly wonderful) system. Yes, this article is about the Generate Your Project (GYP) project.

Prehistory


GYP was developed to build a specific project - the Chrome / Chromium browser. The decision to develop their own tools caused a lot of controversy, many did not understand (perhaps still do not understand) why reinvent their own “bicycle”. Nevertheless, GYP has become quite a popular tool, and now it is used outside of the Chromium infrastructure, for example, for assembling V8 and native NodeJS modules.

Similarities with and differences from CMake


The creators of GYP pursued virtually the same goal as the authors of CMake: to implement a cross-platform system that allows describing the build process with high-level language suitable for generating various design files and scripts for build systems. GYP currently supports GNU make , SCons , Ninja , Eclipse (CDT), VisualStudio, and Xcode.

The main differences between GYP and CMake (in my opinion):

')
Those who want to learn about the differences between GYP and CMake first-hand can visit the corresponding page on the official Wiki . You can also find a link to the mailing list with a hot exchange of developers.

Installation


Installation will require Python, preferably version 2.6 or higher. Linux users will most likely not need to install it, Windows users can access ActivePython or download the distribution from the official Python website .

I prefer to use the latest version of GYP from the official repository . To install from source code, go to the source directory and execute the sudo python setup.py install command. If you do not want to install GYP globally with administrator rights, you can simply add a directory with GYP sources in the PATH environment variable.

Popular Linux distributions often have the gyp package in the standard repository (perhaps not the latest version).

I also like the approach of the V8 authors who checkout a specific tested version of GYP into the build subdirectory and use this version to generate Makefiles for various platforms.

For Emacs users, a plugin for editing GYP files may be useful.

It so happened that Linux is installed on my machine, so further narration will occur from the point of view of a Linux user who is tolerant of users of other systems and respecting their needs.

We assemble a self-contained module


So, it is time to show the tool in action. I think very few people are interested in demonstration using an example like “Hello World”, so first we will describe the configuration for building some useful module, for example, for the GTest library (for which the CMake script comes with, so GTest is twice as useful as a sample : a curious reader will be able to evaluate the readability of both specifications and choose the tool that he likes more).

Below is a somewhat simplified and trimmed version of the specification located in the Chromium repository .
 { 'targets': [ # <-   #   { 'target_name': 'gtest', # <-   'cflags': ['-pthread'], # <-   'link_settings': { # <-   'libraries': ['-lpthread'] # <-    }, 'type': 'static_library', # <-  ,   static_library, # shared_library, executable, none 'standalone_static_library': 1, # <-     thin archive #        GYP 'sources': [ # <-    'include/gtest/gtest-death-test.h', 'include/gtest/gtest-message.h', 'include/gtest/gtest-param-test.h', 'include/gtest/gtest-printers.h', 'include/gtest/gtest-spi.h', 'include/gtest/gtest-test-part.h', 'include/gtest/gtest-typed-test.h', 'include/gtest/gtest.h', 'include/gtest/gtest_pred_impl.h', 'include/gtest/internal/gtest-death-test-internal.h', 'include/gtest/internal/gtest-filepath.h', 'include/gtest/internal/gtest-internal.h', 'include/gtest/internal/gtest-linked_ptr.h', 'include/gtest/internal/gtest-param-util-generated.h', 'include/gtest/internal/gtest-param-util.h', 'include/gtest/internal/gtest-port.h', 'include/gtest/internal/gtest-string.h', 'include/gtest/internal/gtest-tuple.h', 'include/gtest/internal/gtest-type-util.h', 'src/gtest-all.cc', 'src/gtest-death-test.cc', 'src/gtest-filepath.cc', 'src/gtest-internal-inl.h', 'src/gtest-port.cc', 'src/gtest-printers.cc', 'src/gtest-test-part.cc', 'src/gtest-typed-test.cc', 'src/gtest.cc', ], 'sources!': [ # <-     , 'src/gtest-all.cc', #      ], #  conditions 'include_dirs': [ # <-      '.', './include', ], 'conditions': [ # <-   ,   ['OS == "linux"', { #     'defines': [ 'GTEST_HAS_RTTI=0', ], 'direct_dependent_settings': { 'defines': [ 'GTEST_HAS_RTTI=0', ], }, }], ['OS=="win" and (MSVS_VERSION=="2012" or MSVS_VERSION=="2012e")', { 'defines': [ '_VARIADIC_MAX=10', ], 'direct_dependent_settings': { 'defines': [ '_VARIADIC_MAX=10', ], }, }], ], 'direct_dependent_settings': { # <- ,     , #   gtest , . .   'defines': [ # <-   'UNIT_TEST', ], 'include_dirs': [ # <-     include  'include', #     , ], #     ,  #  </path/to/this/gypfile>/include 'msvs_disabled_warnings': [4800], }, }, #   { 'target_name': 'gtest_main', 'type': 'static_library', 'standalone_static_library': 1, 'dependencies': ['gtest'], # <-   ,    #      'sources': [ 'src/gtest_main.cc', ], }, ], }

I think this specification will seem quite transparent to many (with the possible exception of the conditional sections).

The need to list all the files with the source code is immediately apparent. This may seem tedious and overly verbose. GYP does not support the GLOB analog from CMake; moreover, this feature was not consciously realized. According to the developers, the lack of GLOB reduces the likelihood of errors and increases the "tightness" and reproducibility of assemblies.

Header files also need to be included in the source list, otherwise they simply will not be visible when generating Visual Studio projects.

To complete the build, just run the following commands:
 gyp --depth=. gtest.gyp #  Makefile make #  make

After modifying the gtest.gyp file , the Makefile will be automatically regenerated the next time make started.

The artifacts obtained as a result of the assembly can be found in the subdirectories of the out / BUILDTYPE directory , by default it is out / Default .

Note : The generator of project files is selected depending on the operating system (msvs for Windows, make for Linux, xcode for Mac) and the environment variable GYP GENERATORS , the latter taking precedence. To start a specific generator (or several at once), you need to execute a command similar to the following:
 GYP_GENERATORS=make,scons,eclipse gyp --depth=. gtest.gyp


Unfortunately, I have problems with the scons and msvs generators under Linux. I think it is advisable to use the default generator for your platform (nevertheless, the make generator for Mac should work without problems).

Variable substitution

Variable expansion in GYP occurs in two phases: at the “early” phase, the conditions inside the conditions section are calculated, and the variables declared with the quantifier < ; on “late” - calculation of conditions of the section target_conditions and variables with the quantifier > , as well as substitution of the output of external commands

For most tasks, variable "early" phase.

The variables of the early and late phases differ in the direction of the first character at the place of use: <(var) - the early phase (the arrow points to the left, ie, the calculation occurs earlier on the time scale), >(var) - the late phase (the arrow points to the right).

Variable values ​​can be calculated in two different contexts:


An example of using different contexts:
 { 'variables': { 'component_type': 'shared_library', 'public_api_headers': [ 'include/mylib.h', 'include/mylib_extra.h', ], 'private_headers': [ 'internals.h', ], }, 'targets': [ { 'target_name': 'mylib', 'type': '<(component_type)', # <-   'include_dirs': ['include'], 'sources': [ '<@(public_api_headers)', # <-  public_api_headers '<@(private_headers)', #  private_headers   'src/impl.cc', #   sources ], }, ], }

Variables whose value is a list can be calculated in string context. The result of the calculation will be a string consisting of list elements separated by spaces. Similarly, a variable whose value is a string can be calculated in list context. In this case a list will be constructed from it, a space will be used as a separator for the elements.

Sometimes it is required that the value of a variable be calculated with the help of an external command, for this purpose the constructions <!(cmd) and <!@(cmd) :
 'variables' : [ 'foo': '<!(echo Build Date <!(date))', ],

For variables, you can set a default value, it will be used, unless otherwise the variable is not defined at the place of use. The default setting syntax is not particularly intuitive:
 { 'variables': { 'component_type%': 'shared_library', # <-  %     #     } #... }

It is also possible to refer to variables defined in the external assembly system. In the case of make you can use the $ sign (for example, $(INCLUDES) ). Unfortunately, the use of such variables makes the assembly less portable.

Conditions

The conditions section allows you to declare parts of the configuration, depending on factors external to the module being assembled. For example, depending on the target operating system or the desired type of component to be assembled (a statically or dynamically linked library), it is required to change the compilation flags or add / exclude files with source code.

If a section condition is met, its declarations will be combined with the goal declarations in which the condition is defined (or with all goals declarations if the condition is declared in the target_conditions section).

A simple example:

 { 'target_name': 'mylib', 'type': 'static_library', # ... 'conditions': [ ['OS=="linux"', { 'sources': ['linux_extra.cc'], # <-    'defines': ['UNIX=1'], # <-   UNIX   1 }], ], }

Conditions are calculated by the Python interpreter using the eval() function with the __builtin__ dictionary disabled, therefore, they obey the syntax adopted in Python for evaluating Boolean expressions. For example, several conditions can be combined with the operators and and or . A list of predefined variables and more detailed examples can be found on the wiki .

Include files

In large projects, it is possible that a part of the declarations must be rewritten in each GYP file. This can be avoided by using a include file mechanism similar to the #include directive. In GYP, such a mechanism is implemented as a top-level list of includes :
 { 'includes': ['common.gypi', 'other.gypi'], # ... }

Included GYP files usually have the gypi extension and contain declarations of common variables, build configurations, header files, etc. All these declarations will be combined with the declarations of the GYP file, which will include the gypi file. Relative paths used inside the include file are calculated relative to the included file, not the including file.

Debug and Release : build configurations

Almost always, for debugging the application and for installing it, the customer requires various assembly parameters. In debug mode, you want to keep access to the characters for working with the debugger, the customer, it is desirable to deliver a compact, optimized version of the application. In GYP, this need is reflected in the configurations subsection.

Add the following lines to our gtest.gyp :
 { 'target_defaults': { 'configurations': { 'Release': { 'conditions': [ ['OS=="linux"', { 'cflags': ['-O2'], #   }], ], }, 'Debug': { 'conditions': [ ['OS=="linux"', { 'cflags': ['-g', '-O0'], #  ,  }], #    ], }, }, }, 'targets': [ # ... ], }

Pay attention to the fact that the configurations section should be nested in the target_defaults section. If you forget about this, then error messages will most likely not follow, but the configuration will not work.

To specify a configuration when building with make , it suffices to define the BUILDTYPE parameter. For debugging, it is also often useful to see the real commands executed by the build system. The V (verbose) flag is responsible for this:
 make BUILDTYPE=Release V=1

Of course, you can define an arbitrary number of configurations with arbitrary settings.

Now the library is great for use in many other projects, it is enough just to refer to it from the specification of a hierarchical project. The remaining boring work will take on the GYP. I think this composition should be considered in more detail, since it is very important in practice.

We assemble several modules


The convenience of assembling a project from independent modules is one of the main qualities that a good project management system should have, and GYP is excellent in this respect.

As the second module, I chose several functions to verify that the input string matches the simplified regular expressions described in the first chapter of the Beautiful Code book (ISBN-10: 0596510047) (an online version of this chapter is also available).

The source code and configuration repository is on GitHub .

The examples directory has two subdirectories: gtest-1.6 (component for writing unit tests, discussed above) and mini-regex , our micro-library, which needs independent development and testing. Here is a GYP file for building the libminiregex.a library, depending on the gtest component:
 { 'includes': ['../conf.gypi'], # <-   'targets': [ { 'target_name': 'miniregex', 'type': 'static_library', 'include_dirs': ['include'], 'sources': [ 'include/miniregex.hpp', # <-  'src/miniregex.cpp', # <-  ], 'direct_dependent_settings': { 'include_dirs': ['include'], }, }, { 'target_name': 'miniregex_test', 'type': 'executable', # <-   'dependencies': [ '../gtest-1.6/gtest.gyp:gtest', # <-      '../gtest-1.6/gtest.gyp:gtest_main', #   libminiregex,   'miniregex', ], 'sources': [ 'src/test/test_miniregex.cpp', # <-    ], }, ], }

The syntax for specifying dependencies declared in other GYP files deserves some explanation. To specify such a dependency, it is enough to specify the path to the GYP file and specify the name of the target (or asterisk, which means dependence on all the goals of the file) using a colon.

Note : it is very important to remember that when building a project consisting of several modules, all modules must define the appropriate configuration ( Debug , Release , etc.). In the absence of the required configuration, no warnings will be displayed at the design file layout stage, but when attempting to build, mysterious error messages will most likely be displayed.

For demonstration purposes, both modules share a common conf.gypi file containing configuration definitions. This makes sense if the modules need to be stored in one repository. Nevertheless, it seems to me a good idea to put independent modules suitable for reuse (our two libraries seem to be good for this) in separate repositories and use them through external links (like svn:externals or git submodule ).

To build the mini-regex module, go to the examples directory and execute the familiar commands:
 gyp --depth=. mini-regex/miniregex.gyp make

First, the gtest module libraries will be gtest , then the libminiregex.a library, then the libminiregex.a executable file, which can be found in the out / Debug directory, will be built. If everything is done correctly, when you run this executable file on the console should appear positive green output GTest.

So far, we have connected only two modules, but the approach is well scaled, allowing you to assemble module hierarchies without any problems, encapsulating all the details of assembling each of the modules in your own GYP file, and you can assemble each module independently of the others. A prominent example of such an architecture is the Chromium project.

Actions and Rules


Quite often during the assembly you need to perform some action or non-standard conversion.

The actions section is used to define one-time actions , and the rules section is used to define transformations. Rules can be used to build chains of transformations, in the same way as is implemented in GNU make. Rules can also be thought of as action patterns.

As an implementation, we implement the action for installing libraries and header files of the GTest module:
 { 'target_name': 'install', 'type': 'none', 'dependencies': ['gtest', 'gtest_main'], # <-    #   'actions': [ { 'inputs': [], 'outputs': ['$(LIBRARIES)/libgtest.a', '$(LIBRARIES)/libgtest_main.a'], 'action_name': 'copy_libs', 'action': ['cp', '<(PRODUCT_DIR)/libgtest.a', '<(PRODUCT_DIR)/libgtest_main.a', '$(LIBRARIES)'], 'message': 'Copying libraries', }, { 'inputs': [], 'outputs': ['$(INCLUDES)/gtest', '$(INCLUDES)/gtest/internal'], 'action_name': 'copy_headers', 'action': ['cp', '-R', 'include/gtest', '$(INCLUDES)'], 'message': 'Copying header files', } ], }

Here external variables are used, it is assumed that the invoked make will receive the INCLUDES and LIBRARIES via the environment or command line arguments:
 gyp --depth=. gtest.gyp #        sudo make install INCLUDES=/usr/include LIBRARIES=/lib64

In my project, I use a similar technique to build RPM packages.

An example of a rule is the rst2html rule, which I use to compile documentation from the RST format to the HTML format:
 { 'target_name': 'docs', 'type': 'none', 'sources': [ 'doc/Build.rst', 'doc/Dictionary.rst', 'doc/README.rst', ], 'rules': [{ 'rule_name': 'rst2html', 'extension': 'rst', 'inputs': ['doc/css/code.css'], 'action': ['rst2html.py', '--stylesheet-path=doc/css/code.css', '--embed-stylesheet', '<(RULE_INPUT_PATH)', '<(PRODUCT_DIR)/Doc/<(RULE_INPUT_ROOT).html'], 'outputs': ['<(PRODUCT_DIR)/Doc/<(RULE_INPUT_ROOT).html'], 'message': 'Compiling RST document <(RULE_INPUT_PATH)' \ 'to HTML <(PRODUCT_DIR)/Doc/<(RULE_INPUT_ROOT).html', }], },

The extension property specifies the extension of files that fall under the rule, and the list of inputs defines files that are additional dependencies (that is, if they change, you must reapply the rule). The variable RULE_INPUT_PATH bound to the absolute path of the input file of the action RULE_INPUT_ROOT - to the base of the path of the input file (that is, without an extension). The rest I think should not cause questions.

As you can see from the example, the syntax is quite simple, but the compactness leaves much to be desired. For comparison, this goal could be implemented like the following make code (of course, this is not the code that GYP generates):
 BUILDTYPE ?= Debug PRODUCT_DIR ?= out/$(BUILDTYPE) HTML_OUT := $(PRODUCT_DIR)/Doc RST_DOCS := doc/Build.rst doc/Dictionary.rst doc/README.rst HTML_DOCS := $(patsubst doc/%.rst,$(HTML_OUT)/%.html,$(RST_DOCS)) .PHONY: docs docs: $(HTML_DOCS) # ,   $(HTML_OUT)/%.html: doc/%.rst doc/css/code.css mkdir -p $(HTML_OUT) rst2html.py --stylesheet-path=../doc/css/code.css \ --embed-stylesheet $< $@

It can be seen that the syntax for defining GYP rules is somewhat more verbose than the syntax for make , but perhaps more readable for people not experienced in GNU make .

Actions and transformations are usually tried to be implemented using cross-platform tools, usually python scripts.

A description of the format of the action declaration and predefined variables can be found on the official wiki ( Actions , Rules ).

Out of source build


Those readers who tried running GYP under Linux noticed that, in addition to the desired artifacts neatly stacked in the out directory, GYP creates several makefiles (one or two for each target + one main Makefile) that litter the directories with the original code I would like these intermediate files to be created in the out directory too. An example of a solution to this problem can be found in the V8 source code. It is enough to set the gyp --generator-output option and run make from the directory specified by the option:
 gyp --depth=. --generator-output=./out gtest.gyp make -C out

Conclusion


GYP is a fairly convenient alternative to CMake with its own advantages and disadvantages. , GYP , , .

, . Chromium V8. wiki GYP-, , , . , .

Resources


Source: https://habr.com/ru/post/171697/

More articles:


All Articles