We roll the "resin ball" or create your own assembly rules using Qbs

Qbs (Qt Build System) is a build system that allows you to describe the process of building projects in a simple language QML (javascript-like declarative language), which accelerates the process of building products by building a detailed dependency graph. Although this system was created by Qt developers, it is not strictly tied to Qt and allows you to build any products for any programming languages ​​and even use them not for programming, for example, for administration. As stated in the official documentation:

A, a ball game

Today and consider the process of creating your products. Go…

Getting to know this build system was well described by mapron in this article , but we will not dwell on this and get down to business right away.

Unfortunately, there is too little information in the documentation and the network about the details of how Qbs works and the meaning of the properties of the elements. Therefore, I will try to fill in some of the gaps in this, based on my experience with this assembly system. And I will be grateful to readers for the addition or indication of inaccuracies in this material.

So Qbs converts one data to another using the Rule element. This element creates a conversion rule that converts input inputs into output outputs using a set of commands created in the prepare script. Each team can either transform itself or delegate this work to an external program.
')

Input artifacts

The input artifacts for the transformation rule are the files of the product being assembled with the tag specified in the inputs property, as well as artifacts obtained from the dependencies of the product being assembled with the tag specified in the inputsFromDependencies property (the implicit " installable " tag is assigned to all files that need to be installed, t. e. qbs.install: true ).

For example:

---product1.qbs--- Application { name: "simpleApplication"; targetName: "appSimple" files: ["main.cpp", "app.h", "app.cpp"] Depends {name: "cpp"} } ---product2.qbs--- Product { type: "exampleArtefact" name: "example" targetName: "myArtefact" Depends {name: "simpleApplication" } Depends {name: "exampleModule" } Group { files: ["description.txt", "readme.txt"] fileTags: ["txtFiles"] } Group { files: ["imgA.png", "imgB.png", "imgC.png"] fileTags: ["pngFiles"] } Group { files: ["fontA.ttf", "fontB.ttf"] fileTags: ["fontFiles"] } } ---exampleModule.qbs--- Module { Rule { inputs: ["pngFiles", "fontFiles"] inputsFromDependencies: ["application"] Artifact { filePath: product.targetName + ".out" fileTags: ["exampleArtefact"] } ... } ... } 

In this example, we have 2 products and 1 module:

1. A product named simpleApplication is of type application (the Application element is essentially Product {type: “application”} ) containing 3 files: main.cpp, app.h, app.cpp. This product depends on the cpp module, which indicates that this product will be compiled by the C ++ compiler and the output will be an artifact labeled “application” and the name specified in the targetName property, i.e. appSimple.exe (for windows or appSimple for unix platforms).
   
   
2. A product named example has type exampleArtefact containing 7 files tagged with three tags. This product depends on the simpleApplication product, which indicates that it will be processed after creating the simpleApplication product. And also from the module exampleModule, which indicates that the rules of transformation and properties from this module will be taken to create this product. And the output is expected artifact called myArtefact type (tagged) exampleArtefact.
   
   
3. The module exampleModule contains a rule for converting files with pngFiles and fontFiles tags, as well as artifacts with application tags, which are taken from the dependencies of the collected product.
   

When assembling a product, a list of modules will be defined on which the product and the modules themselves depend. They will search for rules for converting files marked with input tags to output artifacts that correspond to the type of product being collected. At first, the product is simpleApplication, because it only depends on the cpp module. It looks for rules for converting product files to the type of application. In the cpp module, there are FileTagger elements that use template tags for product files. Conversion of input files to output (s) can be performed both immediately and along the chain of conversions of files of one type to another, and then to the final one. At the output of processing the product simpleApplication, we get an appSimple application having the type (tag) of application.

Then build the example product will begin. For its files, a rule will be searched for, giving output artifacts of the type exampleArtefact. This rule requires for the input files of type (with tag) pngFiles, fontFiles and application. At the same time, files of type application are searched only in products on which the product being built depends. Since the product example already contains such files, the following files come into the rule: imgA.png, imgB.png, imgC.png, fontA.ttf, fontB.ttf and appSimple.exe. And at the output we get the file myArtefact.out of type exampleArtefact, which will be our final product.

Output Artifacts

As an output artifact for a rule, there can be one or several artifacts. The Artifact element is used to describe output artifacts:

 Artifact { filePath: input.fileName + ".out" fileTags: ["txt_output"] } 

This element describes what kind of artifact is produced at the output of the rule. Through the property filePath - specify the name of the output file. If you specify a relative path, Qbs will create this artifact relative to the build directory of the current product being built. The fileTags property specifies the list of tags that the artifact will have after it is created. This is necessary so that other build rules can use the output artifacts of this rule as their input artifacts. Also, if the product will have the same tag as its type, then these artifacts will be the result of the assembly of this product.

Each rule must have at least one output tag, otherwise the rule will not work. If there are several artifacts, then you can describe several Artifact elements, or you can use the outputArtifacts and outputFileTags properties for the Rule element.

The outputArtifacts property describes a list of JavaScript objects with properties similar to those of the Artifact element. This property is used for cases when the set of outputs is not fixed, but depends on the content of the input data. For example:

 outputArtifacts: [{ var artifactNames = inputs["pngFiles"].map(function(file){ return "pictures/"+file.fileName; }); artifactNames = artifactNames.concat(inputs["fontFiles"].map(function(file){ return "fonts/"+file.fileName; })); artifactNames = artifactNames.concat(inputs["application"].map(function(file){ return "app/"+file.fileName; })); var artifacts = artifactNames.map(function(art){ var a = { filePath: art, fileTags: ["exampleArtefact"] } return a; }); return artifacts; }] 

This example shows that for input files with the pngFiles tag, the rule will prepare an output artifact with the same name and place it in the pictures folder. For the fontFiles and application tags, also placing them respectively in the fonts and app folders. In this case, because If the paths are relative, then these folders will be created in the product assembly folder.

If we decide to use the outputArtifacts property, then we must also specify the outputFileTags property, which is the list of output tags that the rule potentially produces. For our example:

 outputFileTags: ["exampleArtefact"] 

All artifacts obtained during product assembly, marked with an output tag that matches the product type, during product installation, will be copied from the assembly directory to the installation directory.

Conversion rule

When the input and output artifacts are defined, it is necessary to prepare the sequence of commands to perform the conversion. The prepare property of the Rule element is used for this. This property is a JavaScript script that returns a list of commands for converting inputs to outputs. The code in this script is treated as a function with the function (project, product, inputs, outputs, input, output) signature.
The input and output parameters are undefined (undefined) if there are several input (and output) artifacts for this rule. They serve as syntactic sugar: input = inputs [0] and output = outputs [0] and are lists with one element. The project and product parameters are JavaScript objects through which the properties of the current project and product are accessible, respectively. Of particular interest are the inputs and outputs. Consider them in more detail.

Input and output objects

The inputs and outputs parameters are JavaScript objects, whose property keys are file tags, and property values ​​are lists of objects representing the artifacts corresponding to these tags. In our example, the inputs variable has 3 keys: pngFiles, fontFiles, application. And each input artifact is available through the inputs [“pngFiles”] (or inputs.pngFiles, which is equivalent). Each artifact in this list has the following properties:

• baseName - the base name of the file (for example, for the file c: \ 123 \ test.plugin.dll it will be a test)
• completeBaseName - the name of the file without the extension (for example, for the file c: \ 123 \ test.plugin.dll it will be test.plugin)
• fileName - the name of the file (for example, for the file c: \ 123 \ test.plugin.dll it will be test.plugin.dll)
• filePath - the full path to the file with its full name
• fileTags - list of tags assigned to the artifact

In addition, the artifacts contain all the properties for each module that is used in the product. This feature can be used to access module properties. For example, the inputs.application [0] .cpp.defines property returns for the simpleApplication artifact a list of definitions that will be transferred when the corresponding file is compiled. This is a very convenient and important point, allowing artifacts to set their values ​​through the properties of a module and group such artifacts or process them in a special way.

* It was noticed on Qbs version 1.7.2 that if the product replaces the properties of the module that contains the build rule, then these properties are not available in the artifact. Therefore, I carried these properties into a separate module.

* Also inputs.application [0] .cpp.defines does not always work, that's why I use the function inputs.application [0] .moduleProperty ("cpp", "defines") . If this function is applied to the input artifact, then the properties that the artifact uses in the specified module will be returned. If we apply it to the product (for example, product.moduleProperty ("cpp", "defines"), then the properties of the specified module will be returned, which the final product currently being assembled uses.

* A very convenient function is dumpObject (object) from the ModUtils module, which displays information about the properties of the parameter passed to it to the console. True, and it does not always show the properties of the modules used in the artifacts.

Teams

The result of the script is a list of commands. The command is what Qbs performs at build time. The command is always created in the rule preparation script. Teams are of two types:

1. Command that starts an external process
2. JavaScriptCommand , which executes arbitrary javascript code (although it can also run external processes)

Properties are available for both types of commands:

• description - the string displayed in the console when executing this command.
• extendedDescription - a string of detailed information displayed in the console with extended output
• highlight - type (tag) command. It affects how the description will be displayed in the console. Maybe: compiler, linker, codegen, filegen, etc.
• silent - whether to hide the description description when executing the command

To use Command , you need to create an object by passing in its constructor the full path to the executable program and the list of arguments.

 var myCommand = new Command("dir", ["/B", "/O", ">>", output.filePath]); 

For example, this command will write the output of the dir command to the output file of the build rule.
Useful properties that allow you to bypass the Windows restriction on the length of the command line are properties:

• responseFileThreshold contains a value. If this value is greater than zero and less than the length of the full command line, and if responseFileUsagePrefix is ​​not empty, then the contents of the command line are moved to a temporary file whose path becomes the full contents of the argument list. Then the program should read the full list of arguments from this file. This mechanism will only work with programs that explicitly support it.
• responseFileArgumentIndex — Indicates the index of the first argument to include in the response file.
• responseFileUsagePrefix - contains a prefix that tells the program that the rest of the argument is the path to the file containing the actual command line.
  

The following properties can be used to process the output of the command being executed:

• stderrFilterFunction is a function with an input parameter equal to the output of the program in stderr and which should return the converted string for further output.
  If not specified, no processing will be performed.
• stdoutFilterFunction - similar to stderrFilePath, only working with stdout
• stdoutFilePath - the name of the file; the code is sent to the previously filtered output of stdout. If not specified, output will be made to the console.
• stderrFilePath - similar to stdoutFilePath, only works with stderr

JavaScriptCommand command - is a JavaScript function that will be executed during assembly. This function is set in the sourceCode property. In the source code, functions are available project, product, inputs and outputs (giving access to the properties of the project, product, input and output artifacts, respectively). To pass arbitrary data to the function, you need to add arbitrary properties for the command and assign the desired values ​​to them. For example:

 var cmd = new JavaScriptCommand(); cmd.myFirstData = "This is 1 string"; cmd.mySecondData = "This is 2 string"; cmd.sourceCode = function() { console.info("String from source code"); // -->> "String from source code" console.info("Property 1: "+myFirstData); // -->> "Property 1: This is 1 string" console.info("Property 2: "+mySecondData); // -->> "Property 2: This is 2 string" }; 

In the function, you can also use various available services :

• Environment Service - to access the system and development environment
• File Service - for working with files (copying, moving, deleting files, checking for the existence of a file and the date of modification, creating paths and scanning the directory for contents)
• FileInfo Service - file path processing operations (getting the base file name, path to it, getting relative path, etc.)
• General Services - additional functions extending the functionality of processing strings, arrays, etc.
• Process Service - functions that allow you to run and manage external programs, as well as work with its input and output
• PropertyList Service - functions for working with property lists in JSON, XML, binaty and OpenStep formats for Darwin platforms (iOS, macOS, tvOS, and watchOS)
• TemporaryDir Service - functions for creating and managing a temporary directory
• TextFile Service - functions for working with text files and allowing to read / write from / to them.
• Utilities Service - functions for getting hashes

Other properties of the element Rule

The Rule element also has several additional properties:

The multiplex property, necessary to specify the order in which input artifacts are processed. With multiplex = true, one copy of the rule is created for all input artifacts and they are processed all in a crowd. Thus, the input property will contain all input artifacts. It is used in cases when it is necessary to produce a group processing of input artifacts. If multiplex = false, then for each input artifact a separate copy of the transformation rule will be created and its output artifact will be created. Thus, the inputs property will always contain one element. By default, this property is set to false.

The condition property indicates the condition for the execution of the rule. For example, specify that the conversion will be performed only in release mode and for the windows platform.

  qbs.buildVariant === "release" && qbs.targetOS.contains("windows") 

The explicitlyDependsOn property is a list of tags. Each artifact, the tag of which coincides with the tag listed in this property, is assigned to each output artifact. Thus, at the output, we can get output artifacts with already prepared dependencies for further processing.

The alwaysRun property indicates the conditions for executing commands in a rule. If alwaysRun = true, then commands will always be executed, even though the output artifacts are already updated. The default is false.

Example of preparing a transformation rule

As an example of the rule I will give the following task:

There is a project from several programs and libraries for which you need to copy all the libraries and plug-ins necessary for the operation of Qt programs to a given directory. To do this, we create a module with a transformation rule and an independent product that will depend on all project products for which it is necessary to prepare Qt libraries. Additionally, you need to prepare a test file with an indication of all Qt libraries prepared.

Links

• doc.qt.io/qbs/index.html - official documentation
• habrahabr.ru/post/144127 - acquaintance with Qbs
• www.opennet.ru/opennews/art.shtml?num=33102 - news about the Qbs presentation
• stackoverflow.com/questions/tagged/qbs - question and answer on emerging issues
• github.com/qt-labs/qbs - git repository