I think for most habrizhitel will not be a revelation that among the major software development technologies and languages are often far behind the advanced ones. Unfortunately, this is inevitable, since it is impossible at some point to take and rewrite several million lines of code into another language or using a more modern framework.
But at some point the backlog becomes so significant that the product ceases to be competitive and gradually disappears from the market. It was this situation of gradual fading that I was able to observe.
Here I want to talk about how, in a particular case, I tried to delay the inevitable and revitalize the development using the Maven project builder for this.
')
The main advantage of the system developed by me, perhaps, is that with a small amount of effort, with the help of Maven, you can teach a project implemented in any language, to be assembled from separate Packages that respect their versioning and automatically updated when the project is built. All that is needed for this is a command line utility that can compile the source, and in the case of scripting languages, the validation utility would be useful.
So given:
- A system whose “core” consists of a dll set and is given to application developers without source codes.
- An applied programming language that is similar to Pascal, but apart from the standard features of Pascal (of which not all are available) allows the use of objects implemented in the “core” dll. This application language is compiled into similarity of code bytes and executed by a separate dll of the “kernel”.
Thus, the main development is carried out in the above
magic application language, all the charms of which, perhaps, could be terrified by the respected community, and the “core” periodically (the new version every couple of months) comes from the head office and is not subject to modification.
It is very likely that developing and maintaining your own programming language similar to Pascal and made on the basis of Delphi was probably a bad idea, but 15-17 years ago, when it all started, nobody thought about it. Other goals were pursued that, in general, were successfully achieved.
The system itself is a product sold to a customer with a server, a client and a thin client. Naturally, most customers are not satisfied with the standard system functionality, so there are several departments of applied development that “finish” the system for their needs. Each project is engaged in a separate group of programmers. The interaction and exchange of experience between groups is catastrophically small. Thus, each group gets its chance to reinvent the bike and step on the same rake ten times.
After analyzing, it became clear that this lack of interaction between departments is primarily due to the lack of mechanisms for using common code. In other words, if after all, there was an exchange of experience and source codes between groups, then the library (source code) was simply copied to another project (product for a friend of the client) and began to live an independent life, retaining all the glitches and flaws that were present at the time copying.
To remedy the situation, it was decided to organize a repository (repository) and a project collection system using a set of common code libraries (
Packages ).
Maven was chosen as the tool for this, since it performs similar functions by default, has a flexible life cycle and great flexibility of the plug-in system.
But first things first
The main problem, which I didn’t suspect at the beginning of my research, was that Packages should contain files that are not compiled into bytes of code, namely, sources, since compilation should take place exactly under the version of the “kernel” for which they will be used. For simplicity, let's call our application sources bpas. This is a bit contrary to the standard life cycle in Maven.
After such revelations, it was decided to form a package from source, but when forming it is necessary to check their atomic compilability. In other words, use the almost standard Maven life cycle when .class is first compiled and then packed together with the resources in a jar. Only in my case, the Package, instead of the compiled files, was formed from the original bpas packed in a zip.
A little bit about how Maven works
The life cycle of Maven is quite complete, and a set of phases (phases) takes into account almost
all the steps involved in
assembling a project that may be required.
And trying to start some kind of life cycle, for example, “mvn compile”, I actually run the whole chain of phases from validate (project validation) to compile, without missing a single one. For some phases there are so-called default plugins, which will be called even though in the pom.xml (the main project Maven file) there is only a header and not a single indication of the launch of plug-ins.
Here it is worth mentioning the fact that Maven is a completely plug-in system. In other words, he doesn’t know anything but launch plug-ins, but they already know how to do a lot. It turns out that when we want to teach Maven some features of the project build, we need to add to pom.xml an indication of launching the right plug-in at the right phase and with the right parameters.
Here is such an absolutely valid empty pom.xml, despite its emptiness, when receiving the mvn deploy command, it will launch the Initialization plugin, compilation, packaging and deployment of Java source from the src / main folder.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.mycompany.projectgroup</groupId> <artifactId>projectname</artifactId> <version>01.03.03-00</version> <build> </build> </project>
The main use policy in Maven is that any action has its own default settings and additional settings are required only when these defaults are not enough or they are grossly violated. In my case, I had to abandon very many defaults, so pom.xml no longer looks so modest.
Building a new lifecycle in pom.xml
For the implementation of the package was chosen next life cycle.
initialize (intsialization) - Read the settings from the config (property or key = value) of the file and add them to the properties tag. We'll talk about the properties tag later.
generate-sources (source code generation) - Download and unpack all Packages that are dependencies of this package / project from zip into a separate directory for subsequent compilation, together with the sources of the current package / project.
compile (compilation) - We run our compilation plugin for our bpas, which determines the correct compilation order and runs the command line compiler for our language. Briefly, I will talk about my own plugin below, but I suggest taking the guide for writing it outside of this article.
assembly (assembly) - we pack a package consisting of bpas sources in zip with preserving the structure of the subdirectories of the source files.
deploy (in our case, uploading to the repository) - Assembly zip collected during the phase is sent to the local Maven repository with pom.xml and other information on the package added to it. This procedure is almost identical to the normal deploy file jar, but with special parameters.
clean - This phase is not part of the general life cycle of the assembly phases, but stands somewhat apart, but since it was also upgraded, it is also worth mentioning. In addition to the standard deletion of the directory in which the compiled files are located. (targetDirectory), it was necessary to remove the garbage that is generated during the process of downloading and unpacking package dependencies.
The general structure of pom.xml
I conditionally divide pom.xml into two parts: the header and the assembly.
The header is a package identification (groupId, artifactId, version), properties (properties that play the role of internal constants), an indication of the local repository (distributionManagement), an indication of the local repository of plug-ins (pluginRepositories), an indication of the local package repository (repositories) and an indication of dependencies this package (dependencies). In this case, all three repositories can point to the same repository, but in principle they are three different entities, each of which must be specified separately. So for example, we can decide to keep the plugins separate from the rest of the code, and to access the packages in the repository, use http access, whereas we will “deploy” there as in the file storage.
The build (build tag) is the second part of pom.xml, in which the processing features of a particular life cycle phase are configured by different plug-ins with non-default settings. In addition, there are configured directories and parameters that will be involved in the project build:
sourceDirectory is the directory where the sources for compilation are located.
finalName - the final name of the file after packing into the archive.
directory - the working directory in which the compiled files will be put.
In addition, once again I want to remind you that for all these parameters, there are default values, and their separate indication in our case is required only because they must differ from these very defaults.
Realization of life cycle in the build tag
Now let's go back to the life cycle we have defined and see how each of the phases of the life cycle is implemented by calling the right plug-in with the configuration we need.
initialize
Here again, let's digress a little and separately mention the properties tag. We will explain why it is needed and how it is used.
Speaking very roughly, this tag is similar to the declaration of constants that will be used in our program (pom.xml) in the future. But values can get there in three different ways. And each method has a priority that determines what the value will ultimately be at the moment when it is actually required.
The lowest priority is in direct writing properties to the properties tag, for example:
<properties> <helloText>Hello my friend</ helloText > </properties>
Higher priority for direct parameter setting when starting Maven, something like “mvn –DhelloText = Hi initialize”
When Maven is started with this parameter, the initial value of the helloText tag will be replaced with the one passed in the launch line for the current session, i.e. it will not be saved in xml. If such a constant did not exist at all, then for this session it will exist and can be used. The values of all nonexistent constants are an empty string.
The highest priority is to add values to the proprties tag in the current session. They also do not persist in pom.xml. It is this mechanism that we will use to make individual build settings in the properties file containing “name = value”
This is done using the properties-maven-plugin plugin.
<build> … <plugins> … <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>properties-maven-plugin</artifactId> <version>1.0-alpha-1</version> <executions> <execution> <id>1</id> <phase>initialize</phase> <goals> <goal>read-project-properties</goal> </goals> <configuration> <files> <file>..\build.conf</file> </files> </configuration> </execution> <execution> <id>2</id> <phase>pre-clean</phase> <goals> <goal>read-project-properties</goal> </goals> <configuration> <files> <file>..\build.conf</file> </files> </configuration> </execution> </executions> </plugin> … </plugins>
In addition to an interesting decision on making context-dependent settings in a separate file, there is a good demonstration of how the same plugin runs in completely different phases of the life cycle. This is achieved through the <executions /> tag, where a separate <execution /> tag is created for each desired phase. The <execution /> tag must include a unique id (if the execution is more than one), and it may also contain a separate configuration tag whose priority is higher than all others.
generate-sources
At the source generation stage, we recursively load from the repository and unpack all the packages we need, which are specified in dependencies. Again, practically nothing is needed. A plugin will do everything for us after giving it the correct settings.
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <version>2.2.1</version> <executions> <execution> <id>unpackSources</id> <phase>generate-sources</phase> <goals> <goal>unpack</goal> </goals> </execution> </executions> <configuration> <workDirectory>${packagesPath}</workDirectory> </configuration> </plugin>
The
$ {packagesPath} construction means that you need to take a value from the “/ project / properties / packagesPath” tag.
Separately, I want to note that the use of the
maven-assembly-plugin plugin for unpacking is considered deprecated and is not recommended for use in Maven 3. Instead, the
maven-dependency-plugin is used with settings similar to those specified above. I use the older version of the plugin to once again clearly show how the same plugin is configured to perform several tasks from its range.
compile
I had to tinker with the compilation stage, but the main difficulties arose with writing my own compilation plug-in. Step-by-step instructions on writing your own plug-in for Maven is a topic for a separate article, so now we will not focus on this. In the end, the material described here can also be used for scripting languages that are not required to be compiled at all.
One thing is certain, no matter how hard you try, you will not be able to disable the launch of the native
maven-compile-plugin plugin , the vocation of which is to compile Java sources (Without considering the possibility of editing superPom.xml). So the settings for my compilation plugin are as follows:
<plugin> <groupId>com.companyname.utils</groupId> <artifactId>BPASCompilerPlugin</artifactId> <version>0.1.0.2</version> <executions> <execution> <phase>compile</phase> <goals> <goal>bpascompile</goal> </goals> </execution> </executions> <configuration> <protectionServer>${protectionServerName}</protectionServer> <protectionAlias>${protectionServerAlias}</protectionAlias> <bpasccPath>${pathToBPASCC}</bpasccPath> <binaryVersion>${env.BINARY_VERSION}</binaryVersion> </configuration> </plugin>
used parameters:
protectionServer is a protection server, without which the command line compiler cannot be started.
protectionAlias is a section of the protection server license used.
bpasccPath - full or relative path to the command line compiler.
binaryVersion - The version that will be “embedded” in the compiled library.
This is not all the settings of my plugin, but as I said, this is a topic for a separate large article. In principle, the configuration section could not be specified at all, and then all the parameters that the plug-in needed would be initialized by the default values of the plug-in, which corresponds to the basic Maven concept.
assembly
When passing through the assembly phase, Maven defaults to launching the maven-assembly-plugin, explicitly specifying its launch in the assembly phase in the build tag, we can override its settings and make it work for us. We used the same plug-in to unpack packages before compiling, so now I will give the full version of the settings of this plug-in, including packing and unpacking.
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <version>2.2.1</version> <executions> <execution> <id>unpackSources</id> <phase>generate-sources</phase> <goals> <goal>unpack</goal> </goals> </execution> <execution> <id>packSources</id> <phase>assembly</phase> <goals> <goal>assembly</goal> </goals> </execution> </executions> <configuration> <descriptors> <descriptor>..\src.xml</descriptor> </descriptors> <workDirectory>${packagesPath}</workDirectory> </configuration> </plugin>
Here we see the second section of the execution with id = packSources and the necessary setting for this phase .. \ src.xml. Src.xml contains the settings for how to package the sources. In fact, all these settings can be placed directly in the descriptor tag, but this can be very cluttered with pom.xml.
Here is an example src.xml for completeness.
<?xml version="1.0" encoding="UTF-8"?> <assembly> <id>src</id> <formats> <format>zip</format> </formats> <fileSets> <fileSet> <includes> <include>pom.xml</include> </includes> </fileSet> <fileSet> <directory>SOURCE</directory> <excludes> <exclude>.svn</exclude> <exclude>**/*${packagesDirName}*/**</exclude> </excludes> </fileSet> </fileSets> </assembly>
packagesDirName is a constant from the / project / properties file pom.xml
Separately, I want to note that this removal of the packing settings into a separate file allowed me to create one configuration file for all Packages, which is extremely convenient.
deploy
The deploy phase is also launched by Maven regardless of whether we specified the settings for this plugin or not. Having redefined them, we and this plugin forced us to work for ourselves.
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-deploy-plugin</artifactId> <version>2.5</version> <configuration> <file>${project.build.directory}\${project.artifactId}-src.zip</file> <url>${project.distributionManagement.repository.url}</url> <repositoryId>${project.distributionManagement.repository.id}</repositoryId> <groupId>${project.groupId}</groupId> <artifactId>${project.artifactId}</artifactId> <version>${project.version}</version> <packaging>zip</packaging> <pomFile>pom.xml</pomFile> </configuration> </plugin>
With these manual settings, the
maven-deploy-plugin allows any file (or even a group of files) to be put into the Maven repository as a valid library (Package). Now we will analyze the settings in order.
file - the file that will be sent to the Maven repository as a Package.
url - the path to the Maven repository
repositoryId - the
repository identifier into which the depll will be produced.
groupId ,
artifactId ,
version - the standard package identification in the Maven repository. It is for these three parameters that you can uniquely identify the library. Packaging - deployment functionality also includes packing files that will be sent to the repository, so you need to point it to zip again so that it does not do anything with the package, otherwise it will unpack and pack as jar :-).
pomFile - if this parameter is specified, then the file that we specify as pom.xml will be added to the package, and its original name may be different. Maintaining the original pom.xml is beneficial for many reasons, so we will not disdain this opportunity.
clean
Phase clean, as I said, is not included in the standard life cycle. Its initial task is to ensure that after executing the mvn clean command there are no traces of vital activity left in the project. In our case, in addition to the standard targetSource folder (specified in the build tag), you also need to delete all the packages that were “merged” as dependencies for successful compilation of the package / project.
So, the settings:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-clean-plugin</artifactId> <version>2.4.1</version> <configuration> <filesets> <fileset> <directory>${packagesPath}</directory> </fileset> </filesets> </configuration> </plugin>
directory - actually specifying the directory to be deleted. It should be noted that here the creators of the plugin have moved away from the generally accepted concept, and explicitly specifying the settings of the plugin does not cancel its default actions. But in this case, it is very good, because it saves us from unnecessary settings.
Mindful of how hard it is to deal with individual configuration branches at first, I’ll give the full text of pom.xml of one of the packages below.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.companyname.packages</groupId> <artifactId>String</artifactId> <version>0.1.0.1</version> <name>String</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>windows-1251</project.build.sourceEncoding> <packagesPath>${basedir}/SOURCE/${packagesDirName}</packagesPath> </properties> <distributionManagement> <repository> <id>spb-maven-repo</id> <name>spb-maven-repo</name> <url>file://\\SPB-FS\maven-repo</url> </repository> </distributionManagement> <pluginRepositories> <pluginRepository> <id>spb-maven-repo</id> <name>spb-maven-repo</name> <releases> <enabled>true</enabled> <updatePolicy>always</updatePolicy> <checksumPolicy>fail</checksumPolicy> </releases> <snapshots> <enabled>true</enabled> <updatePolicy>always</updatePolicy> <checksumPolicy>fail</checksumPolicy> </snapshots> <url>http://spb-maven-repo</url> <layout>default</layout> </pluginRepository> </pluginRepositories> <repositories> <repository> <id>spb-maven-repo</id> <url>http://spb-maven-repo</url> </repository> </repositories> <dependencies> <dependency> <groupId>com.companyname.packages</groupId> <artifactId>Arithm</artifactId> <version>0.1.0.1</version> <type>zip</type> </dependency> <dependency> <groupId>com.companyname.packages</groupId> <artifactId>bUnit</artifactId> <version>0.9.0.0</version> <type>zip</type> </dependency> </dependencies> <build> <directory>USER</directory> <sourceDirectory>${basedir}/SOURCE</sourceDirectory> <finalName>${project.artifactId}</finalName> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>properties-maven-plugin</artifactId> <version>1.0-alpha-1</version> <executions> <execution> <id>1</id> <phase>initialize</phase> <goals> <goal>read-project-properties</goal> </goals> <configuration> <files> <file>..\build.conf</file> </files> </configuration> </execution> <execution> <id>2</id> <phase>pre-clean</phase> <goals> <goal>read-project-properties</goal> </goals> <configuration> <files> <file>..\build.conf</file> </files> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <version>2.2.1</version> <executions> <execution> <id>unpackSources</id> <phase>generate-sources</phase> <goals> <goal>unpack</goal> </goals> </execution> <execution> <id>packSources</id> <phase>assembly</phase> <goals> <goal>assembly</goal> </goals> </execution> </executions> <configuration> <descriptors> <descriptor>..\src.xml</descriptor> </descriptors> <workDirectory>${packagesPath}</workDirectory> </configuration> </plugin> <plugin> <groupId>com.companyname.utils</groupId> <artifactId>CompilerPlugin</artifactId> <version>0.1.0.2</version> <executions> <execution> <phase>compile</phase> <goals> <goal>compile</goal> </goals> </execution> </executions> <configuration> <protectionServer>${protectionServerName}</protectionServer> <protectionAlias>${protectionServerAlias}</protectionAlias> <bpasccPath>${pathToBPASCC}</bpasccPath> <binaryVersion>${env.BINARY_VERSION</binaryVersion> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-deploy-plugin</artifactId> <version>2.5</version> <configuration> <file>${project.build.directory}\${project.artifactId}-src.zip</file> <url>${project.distributionManagement.repository.url}</url> <repositoryId>${project.distributionManagement.repository.id}</repositoryId> <groupId>${project.groupId}</groupId> <artifactId>${project.artifactId}</artifactId> <version>${project.version}</version> <packaging>zip</packaging> <pomFile>pom.xml</pomFile> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-clean-plugin</artifactId> <version>2.4.1</version> <configuration> <filesets> <fileset> <directory>${packagesPath}</directory> </fileset> </filesets> </configuration> </plugin> </plugins> </build> </project>
Results
I tried to explain everything in as much detail as possible, but still, there are still a lot of questions that I plan to sanctify in subsequent articles.
- phase and goal their relationship between themselves and plugins.
- creating architects - a project template for quickly deploying your pom.xml configuration and the default set of files.
- Maven repositories and their management.
- how to write and debug your Maven plugin.
- how to NOT write your plugin for each "sneeze".
Despite the fact that I myself was pretty damn pleased with my work, I am ready to admit that in this form the system of the project package assembly is still far from perfect, however, the given description rather illustrates the implementation of the full package / project life cycle. In fact, this is a workable frame, on top of which it is possible to expand possibilities almost unlimitedly.
The main advantages of such a batch build are:
- ease of adding common code
- ease of adding dependencies and building a project
- transparency of the project (it is clear that what is used and what is the real amount of non-standard modifications)
- identifying circular dependencies between Packages, thanks to Maven mechanisms.
- automatic distribution of fixes in common code.
- ease of use by an ordinary developer / tester (it is enough to know 3-4 teams).
useful links
Versioning the dependencies usedThe most basic set of Maven plugins and documentation for them.Additional set of plugins included in the main Maven repository