📜 ⬆️ ⬇️

Configure GitLab CI to load java project in maven central


This article is designed for java developers who have a need to quickly publish their products in the sonatype and / or maven central repositories using GitLab. In this article I will talk about setting up gitlab-runner, gitlab-ci and maven-plugin to solve this problem.


Prerequisites:




Content




general information



To the content



Setting up a deploy project in GitLab



To the content



Gitlab runner


This section describes the configuration for running tasks on deploy using our own (Specific) and public (Shared) runner.



Specific Runner


I use my own runners, because first of all it is convenient, fast, cheap.
For runner, I recommend Linux VDS with 1 CPU, 2 GB RAM, 20 GB HDD. The price of the issue is ~ 3000â‚˝ per year


My runner

For the runner, I took the VDS 4 CPU, 4 GB RAM, 50 GB SSD. It cost about 11000â‚˝ and never regretted.
I have a total of 7 machines. 5 on aruba and 2 on ihor.


So, we have a runner. Now we will customize it.
Go to the machine on SSH and install java, git, maven, gnupg2.


To the content



Install gitlab runner



Screen



Process
 Runtime platform arch=amd64 os=linux pid=17594 revision=3001a600 version=11.10.0 Running in system-mode. Please enter the gitlab-ci coordinator URL (eg https://gitlab.com/): https://gitlab.com/ Please enter the gitlab-ci token for this runner: REGISTRATION_TOKEN Please enter the gitlab-ci description for this runner: [ih1174328.vds.myihor.ru]: Deploy Runner Please enter the gitlab-ci tags for this runner (comma separated): deploy Registering runner... succeeded runner=ZvKdjJhx Please enter the executor: docker-ssh, parallels, virtualbox, docker-ssh+machine, kubernetes, docker, ssh, docker+machine, shell: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! 


Screen



Example


To the content



GPG key generation



To the content



Maven Setup



Where,
GPG_SECRET_KEY_PASSPHRASE - GPG key password
SONATYPE_USERNAME - sonatype account login


At this runner setup is completed, you can go to the section GitLab CI


To the content



Shared runner



GPG key generation



To the content



Maven Setup



Where,
GPG_SECRET_KEY_PASSPHRASE - GPG key password
SONATYPE_USERNAME - sonatype account login


To the content



Deploy docker image



To the content



Gitlab ci



Deploy project


Add the .gitlab-ci.yml file to the root of the deploy-project.
The script presents two mutually exclusive tasks on the deploy. Specific Runner or Shared Runner respectively.


.gitlab-ci.yml
 stages: - deploy Specific Runner: extends: .java_deploy_template #      shell- tags: - deploy Shared Runner: extends: .java_deploy_template #      docker- tags: - docker #    GitLab Runner -> Shared Runner -> Docker image: registry.gitlab.com/group/deploy-project:latest before_script: #  GPG  - printf "${GPG_SECRET_KEY}" | gpg --batch --import #  maven  - printf "${SETTINGS_SECURITY_XML}" > ~/.m2/settings-security.xml - printf "${SETTINGS_XML}" > ~/.m2/settings.xml .java_deploy_template: stage: deploy #    ,    DEPLOY   java only: variables: - $DEPLOY == "java" variables: #     GIT_STRATEGY: none script: #        - git config --global credential.helper store #     gitlab-ci-token #       gitlab.com     - echo "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com" >> ~/.git-credentials #     - rm -rf .* * #   ,    Sonatype Nexus - git clone ${DEPLOY_CI_REPOSITORY_URL} . #     - git checkout ${DEPLOY_CI_COMMIT_SHA} -f #    pom.xml   autoReleaseAfterClose  . #          maven central - > for pom in $(find . -name pom.xml); do if [[ $(grep -q autoReleaseAfterClose "$pom" && echo $?) == 0 ]]; then echo "File $pom contains prohibited setting: <autoReleaseAfterClose>"; exit 1; fi; done #   DEPLOY_CI_COMMIT_TAG ,    SNAPSHOT- - > if [[ "${DEPLOY_CI_COMMIT_TAG}" != "" ]]; then mvn versions:set -DnewVersion=${DEPLOY_CI_COMMIT_TAG} else VERSION=$(mvn -q -Dexec.executable=echo -Dexec.args='${project.version}' --non-recursive exec:exec) if [[ "${VERSION}" == *-SNAPSHOT ]]; then mvn versions:set -DnewVersion=${VERSION} else mvn versions:set -DnewVersion=${VERSION}-SNAPSHOT fi fi #        - mvn clean deploy -DskipTests=true 

To the content



Java project


In java projects that are supposed to be uploaded to public repositories, you need to add 2 steps to download Release and Snapshot versions.


.gitlab-ci.yml
 stages: - build - test - verify - deploy <...> Release: extends: .trigger_deploy #    o . only: - tags Snapshot: extends: .trigger_deploy #     SNAPSHOT   when: manual #   ,   . except: - tags .trigger_deploy: stage: deploy variables: #     GIT_STRATEGY: none #    deploy- URL: "https://gitlab.com/api/v4/projects/<deploy project ID>/trigger/pipeline" #  deploy- POST_DATA: "\ token=${DEPLOY_TOKEN}&\ ref=master&\ variables[DEPLOY]=${DEPLOY}&\ variables[DEPLOY_CI_REPOSITORY_URL]=${CI_REPOSITORY_URL}&\ variables[DEPLOY_CI_PROJECT_NAME]=${CI_PROJECT_NAME}&\ variables[DEPLOY_CI_COMMIT_SHA]=${CI_COMMIT_SHA}&\ variables[DEPLOY_CI_COMMIT_TAG]=${CI_COMMIT_TAG} " script: #   cURL,     --fail --show-error #     ,  HTTP  400   - wget --content-on-error -qO- ${URL} --post-data ${POST_DATA} 

In this solution, I went a little further and decided to use one CI template for java projects.


In details

I created a separate gitlab-ci project in which I placed the CI template for java projects in common.yml .


common.yml
 stages: - build - test - verify - deploy variables: SONAR_ARGS: "\ -Dsonar.gitlab.commit_sha=${CI_COMMIT_SHA} \ -Dsonar.gitlab.ref_name=${CI_COMMIT_REF_NAME} \ " .build_java_project: stage: build tags: - touchbit-shell variables: SKIP_TEST: "false" script: - mvn clean - mvn package -DskipTests=${SKIP_TEST} artifacts: when: always expire_in: 30 day paths: - "*/target/reports" .build_sphinx_doc: stage: build tags: - touchbit-shell variables: DOCKERFILE: .indirect/docs/Dockerfile script: - docker build --no-cache -t ${CI_PROJECT_NAME}/doc -f ${DOCKERFILE} . .junit_module_test_run: stage: test tags: - touchbit-shell variables: MODULE: "" script: - cd ${MODULE} - mvn test artifacts: when: always expire_in: 30 day paths: - "*/target/reports" .junit_test_run: stage: test tags: - touchbit-shell script: - mvn test artifacts: when: always expire_in: 30 day paths: - "*/target/reports" .sonar_review: stage: verify tags: - touchbit-shell dependencies: [] script: - > if [ "$CI_BUILD_REF_NAME" == "master" ]; then mvn compile sonar:sonar -Dsonar.login=$SONAR_LOGIN $SONAR_ARGS else mvn compile sonar:sonar -Dsonar.login=$SONAR_LOGIN $SONAR_ARGS -Dsonar.analysis.mode=preview fi .trigger_deploy: stage: deploy tags: - touchbit-shell variables: URL: "https://gitlab.com/api/v4/projects/10345765/trigger/pipeline" POST_DATA: "\ token=${DEPLOY_TOKEN}&\ ref=master&\ variables[DEPLOY]=${DEPLOY}&\ variables[DEPLOY_CI_REPOSITORY_URL]=${CI_REPOSITORY_URL}&\ variables[DEPLOY_CI_PROJECT_NAME]=${CI_PROJECT_NAME}&\ variables[DEPLOY_CI_COMMIT_SHA]=${CI_COMMIT_SHA}&\ variables[DEPLOY_CI_COMMIT_TAG]=${CI_COMMIT_TAG} " script: - wget --content-on-error -qO- ${URL} --post-data ${POST_DATA} .trigger_release_deploy: extends: .trigger_deploy only: - tags .trigger_snapshot_deploy: extends: .trigger_deploy when: manual except: - tags 

As a result, in the java projects themselves .gitlab-ci.yml looks very compact and not verbose


.gitlab-ci.yml
 include: https://gitlab.com/TouchBIT/gitlab-ci/raw/master/common.yml Shields4J: extends: .build_java_project Sphinx doc: extends: .build_sphinx_doc variables: DOCKERFILE: .docs/Dockerfile Sonar review: extends: .sonar_review dependencies: - Shields4J Release: extends: .trigger_release_deploy Snapshot: extends: .trigger_snapshot_deploy 

To the content



Pom.xml configuration


This topic is described in great detail in Googolplex in Setting Up Mavena to automatically sign and download artifacts in the snapshot and staging repositories , so I will describe some of the nuances of using plugins. I will also describe how easily and naturally you can use the nexus-staging-maven-plugin if you do not want or cannot use org.sonatype.oss: oss-parent as the parent for your project.



maven-install-plugin


Installs modules into a local repository.
Very useful for local verification of solutions in other projects, as well as a checksum.


 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-install-plugin</artifactId> <executions> <execution> <id>install-project</id> <!--          --> <phase>install</phase> <!--       --> <configuration> <file>target/${project.artifactId}-${project.version}.jar</file> ```target/${project.artifactId}-${project.version}-sources.jar</sources> <pomFile>dependency-reduced-pom.xml</pomFile> <!--     --> <updateReleaseInfo>true</updateReleaseInfo> <!--      --> <createChecksum>true</createChecksum> </configuration> </execution> </executions> </plugin> 

To the content



maven-javadoc-plugin


Generate javadoc for project.


 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-javadoc-plugin</artifactId> <executions> <execution> <goals> <goal>jar</goal> </goals> <!--  javadoc       --> <phase>prepare-package</phase> <configuration> <!--      --> <failOnError>true</failOnError> <failOnWarnings>true</failOnWarnings> <!--      target  --> <detectOfflineLinks>false</detectOfflineLinks> </configuration> </execution> </executions> </plugin> 

If you have a module that does not contain java (for example, only resources)
Or you don’t want to generate javadoc, then to help maven-jar-plugin


 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <executions> <execution> <id>empty-javadoc-jar</id> <phase>generate-resources</phase> <goals> <goal>jar</goal> </goals> <configuration> <classifier>javadoc</classifier> <classesDirectory>${basedir}/javadoc</classesDirectory> </configuration> </execution> </executions> </plugin> 

To the content



maven-gpg-plugin


 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-gpg-plugin</artifactId> <executions> <execution> <id>sign-artifacts</id> <!--   ,   GPG  --> <!--      deploy --> <phase>deploy</phase> <goals> <goal>sign</goal> </goals> </execution> </executions> </plugin> 

To the content



nexus-staging-maven-plugin


Configuration:


 <project> <!-- ... --> <build> <plugins> <!-- ... --> <plugin> <groupId>org.sonatype.plugins</groupId> <artifactId>nexus-staging-maven-plugin</artifactId> </plugin> </plugins> <pluginManagement> <plugins> <plugin> <groupId>org.sonatype.plugins</groupId> <artifactId>nexus-staging-maven-plugin</artifactId> <extensions>true</extensions> <configuration> <serverId>sonatype</serverId> <nexusUrl>https://oss.sonatype.org/</nexusUrl> <!--  ,     release --> <!--    snapshot  --> <updateReleaseInfo>true</updateReleaseInfo> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-deploy-plugin</artifactId> <configuration> <!--   --> <skip>true</skip> </configuration> </plugin> </plugins> </pluginManagement> </build> <distributionManagement> <snapshotRepository> <id>sonatype</id> <name>Nexus Snapshot Repository</name> <url>https://oss.sonatype.org/content/repositories/snapshots/</url> </snapshotRepository> <repository> <id>sonatype</id> <name>Nexus Release Repository</name> <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url> </repository> </distributionManagement> </project> 

If you have a multi-module project, and you do not need to load a specific module into the repository, then in the pom.xml of this module you need to add a nexus-staging-maven-plugin with the skipNexusStagingDeployMojo flag


 <build> <plugins> <plugin> <groupId>org.sonatype.plugins</groupId> <artifactId>nexus-staging-maven-plugin</artifactId> <configuration> <skipNexusStagingDeployMojo>true</skipNexusStagingDeployMojo> </configuration> </plugin> </plugins> </build> 

After downloading, snapshot / release versions are available in the staging repository.


 <repositories> <repository> <id>SonatypeNexus</id> <url>https://oss.sonatype.org/content/groups/staging/</url> <!--     snapshot/release   --> </repository> </repositories> 

More pluses



To the content



Result



SNAPSHOT version publication


When building a project, it is possible to manually launch the task to load the SNAPSHOT version in nexus



When you run this task, the corresponding task in the deploy project ( example ) is triggered.


Clipped log
 Running with gitlab-runner 11.10.0 (3001a600) on Deploy runner JSKWyxUw Using Shell executor... Running on ih1174328.vds.myihor.ru... Skipping Git repository setup Skipping Git checkout Skipping Git submodules setup $ rm -rf .* * $ git config --global credential.helper store $ echo "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com" >> ~/.git-credentials $ git clone ${DEPLOY_CI_REPOSITORY_URL} . Cloning into 'shields4j'... $ git checkout ${DEPLOY_CI_COMMIT_SHA} Note: checking out '850f86aa317194395c5387790da1350e437125a7'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b new_branch_name HEAD is now at 850f86a... skip deploy test-core $ for pom in $(find . -name pom.xml); do # collapsed multi-line command $ if [[ "${DEPLOY_CI_COMMIT_TAG}" != "" ]]; then # collapsed multi-line command [INFO] Scanning for projects... [INFO] Inspecting build with total of 4 modules... [INFO] Installing Nexus Staging features: [INFO] ... total of 4 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] Shields4J [pom] [INFO] test-core [jar] [INFO] Shields4J client [jar] [INFO] TestNG listener [jar] [INFO] [INFO] --------------< org.touchbit.shields4j:shields4j-parent >--------------- [INFO] Building Shields4J 1.0.0 [1/4] [INFO] --------------------------------[ pom ]--------------------------------- [INFO] [INFO] --- versions-maven-plugin:2.5:set (default-cli) @ shields4j-parent --- [INFO] Searching for local aggregator root... [INFO] Local aggregation root: /home/gitlab-deployer/JSKWyxUw/0/TouchBIT/deploy/shields4j [INFO] Processing change of org.touchbit.shields4j:shields4j-parent:1.0.0 -> 1.0.0-SNAPSHOT [INFO] Processing org.touchbit.shields4j:shields4j-parent [INFO] Updating project org.touchbit.shields4j:shields4j-parent [INFO] from version 1.0.0 to 1.0.0-SNAPSHOT [INFO] [INFO] Processing org.touchbit.shields4j:client [INFO] Updating parent org.touchbit.shields4j:shields4j-parent [INFO] from version 1.0.0 to 1.0.0-SNAPSHOT [INFO] Updating dependency org.touchbit.shields4j:test-core [INFO] from version 1.0.0 to 1.0.0-SNAPSHOT [INFO] [INFO] Processing org.touchbit.shields4j:test-core [INFO] Updating parent org.touchbit.shields4j:shields4j-parent [INFO] from version 1.0.0 to 1.0.0-SNAPSHOT [INFO] [INFO] Processing org.touchbit.shields4j:testng [INFO] Updating parent org.touchbit.shields4j:shields4j-parent [INFO] from version 1.0.0 to 1.0.0-SNAPSHOT [INFO] Updating dependency org.touchbit.shields4j:client [INFO] from version 1.0.0 to 1.0.0-SNAPSHOT [INFO] Updating dependency org.touchbit.shields4j:test-core [INFO] from version 1.0.0 to 1.0.0-SNAPSHOT [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Shields4J 1.0.0 .................................... SUCCESS [ 0.992 s] [INFO] test-core .......................................... SKIPPED [INFO] Shields4J client ................................... SKIPPED [INFO] TestNG listener 1.0.0 .............................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.483 s [INFO] Finished at: 2019-04-21T02:40:42+03:00 [INFO] ------------------------------------------------------------------------ $ mvn clean deploy -DskipTests=${SKIP_TESTS} [INFO] Scanning for projects... [INFO] Inspecting build with total of 4 modules... [INFO] Installing Nexus Staging features: [INFO] ... total of 4 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] Shields4J [pom] [INFO] test-core [jar] [INFO] Shields4J client [jar] [INFO] TestNG listener [jar] [INFO] [INFO] --------------< org.touchbit.shields4j:shields4j-parent >--------------- [INFO] Building Shields4J 1.0.0-SNAPSHOT [1/4] [INFO] --------------------------------[ pom ]--------------------------------- ... DELETED ... [INFO] * Bulk deploy of locally gathered snapshot artifacts finished. [INFO] Remote deploy finished with success. [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Shields4J 1.0.0-SNAPSHOT ........................... SUCCESS [ 2.375 s] [INFO] test-core .......................................... SUCCESS [ 3.929 s] [INFO] Shields4J client ................................... SUCCESS [ 3.815 s] [INFO] TestNG listener 1.0.0-SNAPSHOT ..................... SUCCESS [ 36.134 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 47.629 s [INFO] Finished at: 2019-04-21T02:41:32+03:00 [INFO] ------------------------------------------------------------------------ 

As a result, version 1.0.0-SNAPSHOT is loaded into nexus.


All snapshot versions can be removed from the repository on oss.sonatype.org under your account.



To the content



Release release version


When the tag is installed, the corresponding task in the deploy project will automatically trigger to load the release version in nexus ( example ).



The best part is that the close release in nexus automatically works.


 [INFO] Performing remote staging... [INFO] [INFO] * Remote staging into staging profile ID "9043b43f77dcc9" [INFO] * Created staging repository with ID "orgtouchbit-1037". [INFO] * Staging repository at https://oss.sonatype.org:443/service/local/staging/deployByRepositoryId/orgtouchbit-1037 [INFO] * Uploading locally staged artifacts to profile org.touchbit [INFO] * Upload of locally staged artifacts finished. [INFO] * Closing staging repository with ID "orgtouchbit-1037". Waiting for operation to complete... ......... [INFO] Remote staged 1 repositories, finished with success. [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Shields4J 1.0.0 .................................... SUCCESS [ 9.603 s] [INFO] test-core .......................................... SUCCESS [ 3.419 s] [INFO] Shields4J client ................................... SUCCESS [ 9.793 s] [INFO] TestNG listener 1.0.0 .............................. SUCCESS [01:23 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:47 min [INFO] Finished at: 2019-04-21T04:05:46+03:00 [INFO] ------------------------------------------------------------------------ 

And if something went wrong, the task will surely collapse.
 [INFO] Performing remote staging... [INFO] [INFO] * Remote staging into staging profile ID "9043b43f77dcc9" [INFO] * Created staging repository with ID "orgtouchbit-1038". [INFO] * Staging repository at https://oss.sonatype.org:443/service/local/staging/deployByRepositoryId/orgtouchbit-1038 [INFO] * Uploading locally staged artifacts to profile org.touchbit [INFO] * Upload of locally staged artifacts finished. [INFO] * Closing staging repository with ID "orgtouchbit-1038". Waiting for operation to complete... ....... [ERROR] Rule failure while trying to close staging repository with ID "orgtouchbit-1039". [ERROR] [ERROR] Nexus Staging Rules Failure Report [ERROR] ================================== [ERROR] [ERROR] Repository "orgtouchbit-1039" failures [ERROR] Rule "signature-staging" failures [ERROR] * No public key: Key with id: (1f42b618d1cbe1b5) was not able to be located on &lt;a href=http://keys.gnupg.net:11371/&gt;http://keys.gnupg.net:11371/&lt;/a&gt;. Upload your public key and try the operation again. ... [ERROR] Cleaning up local stage directory after a Rule failure during close of staging repositories: [orgtouchbit-1039] [ERROR] * Deleting context 9043b43f77dcc9.properties [ERROR] Cleaning up remote stage repositories after a Rule failure during close of staging repositories: [orgtouchbit-1039] [ERROR] * Dropping failed staging repository with ID "orgtouchbit-1039" (Rule failure during close of staging repositories: [orgtouchbit-1039]). [ERROR] Remote staging finished with a failure: Staging rules failure! [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Shields4J 1.0.0 .................................... SUCCESS [ 4.073 s] [INFO] test-core .......................................... SUCCESS [ 2.788 s] [INFO] Shields4J client ................................... SUCCESS [ 3.962 s] [INFO] TestNG listener 1.0.0 .............................. FAILURE [01:07 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ 

. .



, Maven central


, maven .
robots.txt, .




Conclusion




GitLab CI . CI " " , . GitLab . . ( :) ).


.


, GitLab CI ( docker-compose), shell .


Source: https://habr.com/ru/post/449664/


All Articles