📜 ⬆️ ⬇️

Development Environment: Redmine + Git + ownCloud

This article appeared to summarize rather lengthy attempts to build a comfortable environment for working on projects. Undoubtedly, there are many services ready to provide similar functionality, but their use is not always convenient and, for various reasons, may be unacceptable. If such a situation arises, I hope the configuration presented in the article will be useful.



The scenario of using this bundle can be briefly described as follows:
')


The system deployment plan includes the configuration of the following services:

  1. OpenLDAP - single account for all services;
  2. Redmine - launch in Docker container, create and bind Git repository, LDAP authentication;
  3. NGINX - access to the Git repository via HTTPS and LDAP authentication;
  4. ownCloud - LDAP authentication and folder mounting via davfs2.


Hard binding to the distribution and operating system, there is not. However, I have a Gentoo distribution on my Linux server, Kubuntu on my client, so some examples will have specific features.

For ease of reproduction, some of the examples contain predefined passwords. Therefore, I strongly recommend that instead of the [EXTERNAL_IP] label, use a local IP that is not accessible from the outside, or immediately change passwords to strong ones.

Openldap



LDAP is a data access protocol organized in the form of directories. Its implementation will be discussed here - OpenLDAP. This service is essentially an optimized database for storing tree structures such as directories. In this case, it is important that LDAP is well suited for storing user data and is a standard that somehow supports most of the services that work with user accounts.

Minimum required configuration:

# /etc/openldap/slapd.conf include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/nis.schema #     cn=Subschema  . access to dn.base="" by * read access to dn.base="cn=Subschema" by * read #     cn=manager,dc=example,dc=com. # ,    rootdn    . access to dn.regex=".+,dc=example,dc=com$" by self write by dn.exact="cn=manager,dc=example,dc=com" read by anonymous auth #         . access to * by self write by anonymous auth by * none pidfile /var/run/openldap/slapd.pid argsfile /var/run/openldap/slapd.args #        . #   1  2,    - 256 ( 2048). loglevel 0 modulepath /usr/lib64/openldap/openldap #    LDAPS. TLSCertificateFile /etc/ssl/openldap/server.pem TLSCertificateKeyFile /etc/ssl/openldap/server.key ###    database hdb #       700   ldap:ldap. directory /var/lib/openldap-data #  ,   . suffix "dc=example,dc=com" # DN (Distinguished Name)       . #        . rootdn "cn=admin,dc=example,dc=com" #   rootdn,    slappasswd -s [] #     : passwd rootpw {SSHA}70m8+2axDu++Adp6EOLPVpISPxbMVPFv #        memberOf  DN ,    . moduleload memberof.la overlay memberof memberof-group-oc groupOfUniqueNames memberof-member-ad uniqueMember memberof-refint true #     . #   ,     ,  . moduleload refint.la overlay refint refint_attributes uniqueMember #    .  ,  . refint_nothing "cn=admin,dc=example,dc=com" #    . index objectClass eq index uid,uidNumber,gidNumber,memberUid eq 


For clarity, the configuration here is written in the slapd.conf file, but since OpenLDAP version 2.4, this file has been declared obsolete. The new format is called OLC (on-line configuration) and is presented as a config database. It is convenient because changing the settings no longer requires rebooting the service.

To migrate to OLC, add at the end of the slapd.conf file:

 ###     #      (root)  . # Ex: ldapadd/ldapmodify -Q -Y EXTERNAL -H ldapi:/// -f [config_update].ldif database config access to * by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage by * none 


And execute commands:

 mkdir -p /etc/openldap/slapd.d #  slapd   ,        . slaptest -f /etc/openldap/slapd.conf -F /etc/openldap/slapd.d chown ldap:ldap -R /etc/openldap/slapd.d /var/lib/openldap-data /etc/openldap/slapd.conf chmod 750 /etc/openldap/slapd.d && chmod 640 /etc/openldap/slapd.conf 


Settings will be recorded in the specified folder, still in text form, but you should not edit them manually anymore.

Next, you need to change the startup parameters of the daemon so that it reads the settings from the database instead of the file. In Gentoo, for this, you need to change the /etc/conf.d/slapd file:

 # /etc/conf.d/slapd #    . #OPTS_CONF="-f /etc/openldap/slapd.conf" #     . OPTS_CONF="-F /etc/openldap/slapd.d" #      ldaps;  - ldap    (%2f -  ). OPTS="${OPTS_CONF} -h 'ldaps://[_IP]:636 ldap://127.0.0.1:389 ldapi://%2fvar%2frun%2fopenldap%2fslapd.sock'" 


Now you can start the service and add data:

Contents of the backup.ldif file
dn: dc = example, dc = com
objectClass: top
objectClass: dcObject
objectClass: organization
dc: example
o: Example.com

dn: ou = people, dc = example, dc = com
objectClass: organizationalUnit
ou: People

dn: ou = groups, dc = example, dc = com
objectClass: organizationalUnit
ou: Groups

dn: uid = example, ou = people, dc = example, dc = com
cn: Example User
givenName: User
sn: Example
uid: example
uidNumber: 1001
gidNumber: 1000
homeDirectory: /var/www/example.com
mail: example@example.com
objectClass: top
objectClass: posixAccount
objectClass: shadowAccount
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
loginShell: / sbin / nologin
userPassword: {SSHA} 1zGVasPHdQ7LXhBJxzAOseflZiqlecKT

dn: cn = manager, dc = example, dc = com
cn: Manager
objectClass: top
objectClass: simpleSecurityObject
objectClass: organizationalRole
userPassword: {SSHA} KihawPs8IchHTS / Lc7aqKGd1rfkpEKyi

dn: cn = developers, ou = groups, dc = example, dc = com
objectClass: groupOfUniqueNames
cn: developers
description: Developers
uniqueMember: uid = example, ou = people, dc = example, dc = com


 ldapadd -x -D 'cn=admin,dc=example,dc=com' -w 'passwd' -f backup.ldif 


In order to use your passwords, you must replace the value of the userPassword fields. New can be obtained with the command:

 slappasswd -s [] 


When restoring from a backup, it is important to make sure that the records for adding groups are located after the records for adding users , and preferably at the very end of the script, otherwise the memberof overlay will not be able to correctly process them.

Verify that OpenLDAP is correctly configured and works as intended, by typing:

 ldapsearch -D 'cn=manager,dc=example,dc=com' -w 'passwd' -b 'ou=people,dc=example,dc=com' '(uid=example)' memberOf 


The answer should be as follows:

 … dn: uid=example,ou=people,dc=example,dc=com memberOf: cn=developers,ou=groups,dc=example,dc=com # search result search: 2 result: 0 Success … 


In addition to the test user, cn = manager, dc = example, dc = com was also added here, which will be used later to access the database. Its rights must be the minimum necessary, and the password must be different from the password cn = admin, dc = example, dc = com , since you will have to specify it in plain text in the settings files. The password passwd is used here, which is certainly a bad example.

Working with entries in OpenLDAP is far from the most convenient and obvious, but Apache Directory Studio can help with this.

Redmine



The Redmine web application is very well suited for project management. However, installing it requires a large number of Ruby dependencies. My activities are not related to Ruby or Rails, so installing the additional 76 packages that ebuild required on the server was not desirable. An acceptable option was found very quickly - the Docker container is the sameersbn / docker-redmine .

Containers are easy to configure and run using docker-compose . This utility allows you to combine Docker command line parameters into a single configuration file and manage running containers.

The options for setting up docker-redmine are very broad and it’s better to start working with it right away with official documentation. The following example is only one of the options that turned out to be convenient in my case:

Redmine container settings
 # /var/www/redmine.example.com/docker-compose.yml redmine: image: quay.io/sameersbn/redmine:3.2.0-2 #           restart: always environment: - TZ=Europe/Moscow #    uid:gid  ,  #  ,     ,  . - USERMAP_UID=[NGINX_UID] - USERMAP_GID=[NGINX_GID] - DB_TYPE=postgres # Docker-   ,   bridge IP (172.17.0.1),   #         .   , #   ,   .      IP, #          . - DB_HOST=10.0.10.10 - DB_USER=redmine - DB_PASS=[] - DB_NAME=redmine_production - REDMINE_HTTPS=true - REDMINE_PORT=10083 - SMTP_ENABLED=true - SMTP_OPENSSL_VERIFY_MODE=none #  MTA   Docker- 172.17.0.0/16,     . - SMTP_HOST=[_IP] - SMTP_PORT=25 - IMAP_ENABLED=false ports: - "127.0.0.1:10083:80" volumes: - /var/www/redmine.example.com/data:/home/redmine/data #      . - /var/www/redmine.example.com/logs:/var/log/redmine 



The container is started by the command:

 docker-compose -f /var/www/redmine.example.com/docker-compose.yml up -d 


In order for Redmine to automatically create accounts based on accounts from OpenLDAP and use its authentication mechanism, you must add the appropriate method to Administration → LDAP authentication → New authentication mode :

 Name *: Example.com LDAP Host *: [_IP] Port *: 636; LDAPS: x Account: cn=manager,dc=example,dc=com Password: passwd Base DN *: ou=people,dc=example,dc=com LDAP filter: (&(objectClass=person)(memberOf=cn=developers,ou=groups,dc=example,dc=com)) Timeout (in seconds): On-the-fly user creation: x Login attribute *: uid Firstname attribute: givenName Lastname attribute: sn Email attribute: mail 


Only existing Git repositories can be added to the Redmine project, so the repository folder should be accessible from the container. Since in this case you only need to track locally located repositories, you can limit yourself to mounting repository folders in the container, but this is not convenient. If you need to connect an external repository, then you will have to place its clone in a directory with your own repositories, besides mounting the system directory in a container is not beautiful.

As a result, I got the following set of scripts for working with local repositories:

Script to create a new Git repository
 #!/bin/bash set -e # /var/git/create_repo.sh LOCAL_GIT_DIR="/var/git" SCRIPT_NAME=`basename $0` E_OPTERROR=65 function usage() { echo "USAGE:" echo " $SCRIPT_NAME [repo_name]" echo "OPTIONS:" echo " repo_name - name of the new Git repository." exit $E_OPTERROR } function fatal_error() { echo "$1" > /dev/stderr exit 1 } #   . if [ $# -ne 1 ]; then echo "Wrong number of arguments specified." usage fi REPO_NAME=$1 cd ${LOCAL_GIT_DIR} if [ -d "${REPO_NAME}" ]; then fatal_error "Error: the repository already exists!" fi #   . mkdir ${REPO_NAME} cd ${REPO_NAME} git --bare init git update-server-info -f #    Redmine    . cd .. cp ./post-update "${REPO_NAME}/hooks/" chmod 755 "${REPO_NAME}/hooks/post-update" chown -R nginx:nginx ${REPO_NAME} echo "Git repository successfully created." exit 0 



Script to migrate existing Git repository
 #!/bin/bash set -e # /var/git/migrate_repo.sh LOCAL_GIT_DIR="/var/git" REDMINE_GIT_DIR="/var/www/redmine.example.com/data/git" SCRIPT_NAME=`basename $0` E_OPTERROR=65 function usage() { echo "USAGE:" echo " $SCRIPT_NAME [repo_name]" echo "OPTIONS:" echo " repo_name - name of the existing Git repository." exit $E_OPTERROR } function fatal_error() { echo "$1" > /dev/stderr exit 1 } #   . if [ $# -ne 1 ]; then echo "Wrong number of arguments specified." usage fi REPO_NAME=$1 if [ ! -d "${LOCAL_GIT_DIR}/${REPO_NAME}" ]; then fatal_error "Error: the repository does not exists!" fi # ,     . if [ -f "${LOCAL_GIT_DIR}/${REPO_NAME}/hooks/post-update" ]; then fatal_error "Error: post-update hook already exists! The repository already migrated or should be migrated manually." fi if [ -d "${REDMINE_GIT_DIR}/${REPO_NAME}" ]; then fatal_error "Error: redmine already contains the repository with the same name!" fi #    Redmine    . cp "${LOCAL_GIT_DIR}/post-update" "${LOCAL_GIT_DIR}/${REPO_NAME}/hooks/" chown nginx:nginx "${LOCAL_GIT_DIR}/${REPO_NAME}/hooks/post-update" chmod 755 "${LOCAL_GIT_DIR}/${REPO_NAME}/hooks/post-update" #  Redmine  . cd "${REDMINE_GIT_DIR}" git clone --mirror "${LOCAL_GIT_DIR}/${REPO_NAME}" ${REPO_NAME} echo "Git repository successfully migrated." exit 0 



Hook to synchronize Redmine repository copies with local
 #!/bin/bash # /var/git/post-update #     ,  git push     . REDMINE_GIT_DIR="/var/www/redmine.example.com/data/git" REPO_PATH=${PWD} REPO_NAME=$(basename "${REPO_PATH}") LOG_FILE="/var/log/nginx/git_hooks_log" function log_message() { echo `date '+%d-%m-%y %H:%M:%S'` "$1" >>"${LOG_FILE}" } if [ -d "${REDMINE_GIT_DIR}" ]; then cd "${REDMINE_GIT_DIR}" if [ -d "${REPO_NAME}" ]; then #     ,  . cd "${REPO_NAME}" log_message "UPDATED: ${PWD}" exec git fetch -q --all -p &>>"${LOG_FILE}" else #    . log_message "NEW: ${REPO_PATH} : ${REPO_NAME} : ${PWD}" exec git clone -q --mirror ${REPO_PATH} ${REPO_NAME} &>>"${LOG_FILE}" fi fi 



If it is necessary to connect an external repository, for example GitHub, it will also need to be cloned into the /var/www/redmine.example.com/data/git folder, then install the task in cron to periodically synchronize Redmine copies with the remote repository.

Nginx



Next, you need to open access to the repositories via HTTPS. Unfortunately, out of the box, NGINX does not support LDAP authorization, for this it needs to be built with a third-party kvspb / nginx-auth-ldap module.

In Gentoo, all you need to do is copy the ebuild to a local portage (PORTDIR_OVERLAY) and add this module to it. The official ebuild connects a lot of modules, add another one is not difficult. For the current version of NGINX, I got the following patch:

Patch adding module http_auth_ldap for nginx-1.8.0.ebuild
 --- nginx-1.8.0.ebuild 2015-08-05 14:31:19.000000000 +0300 +++ nginx-1.8.0.ebuild.new 2015-08-07 08:19:35.899578187 +0300 @@ -126,6 +126,12 @@ HTTP_MOGILEFS_MODULE_URI="http://www.grid.net.ru/nginx/download/nginx_mogilefs_module-${HTTP_MOGILEFS_MODULE_PV}.tar.gz" HTTP_MOGILEFS_MODULE_WD="${WORKDIR}/nginx_mogilefs_module-${HTTP_MOGILEFS_MODULE_PV}" +# http_auth_ldap (https://github.com/kvspb/nginx-auth-ldap, ??? license) +HTTP_AUTH_LDAP_MODULE_PV="master" +HTTP_AUTH_LDAP_MODULE_P="ngx_http_auth_ldap-${HTTP_AUTH_LDAP_MODULE_PV}" +HTTP_AUTH_LDAP_MODULE_URI="https://github.com/kvspb/nginx-auth-ldap/archive/${HTTP_AUTH_LDAP_MODULE_PV}.tar.gz" +HTTP_AUTH_LDAP_MODULE_WD="${WORKDIR}/nginx-auth-ldap-${HTTP_AUTH_LDAP_MODULE_PV}" + inherit eutils ssl-cert toolchain-funcs perl-module flag-o-matic user systemd versionator multilib DESCRIPTION="Robust, small and high performance http and reverse proxy server" @@ -148,7 +154,8 @@ nginx_modules_http_security? ( ${HTTP_SECURITY_MODULE_URI} -> ${HTTP_SECURITY_MODULE_P}.tar.gz ) nginx_modules_http_push_stream? ( ${HTTP_PUSH_STREAM_MODULE_URI} -> ${HTTP_PUSH_STREAM_MODULE_P}.tar.gz ) nginx_modules_http_sticky? ( ${HTTP_STICKY_MODULE_URI} -> ${HTTP_STICKY_MODULE_P}.tar.bz2 ) - nginx_modules_http_mogilefs? ( ${HTTP_MOGILEFS_MODULE_URI} -> ${HTTP_MOGILEFS_MODULE_P}.tar.gz )" + nginx_modules_http_mogilefs? ( ${HTTP_MOGILEFS_MODULE_URI} -> ${HTTP_MOGILEFS_MODULE_P}.tar.gz ) + nginx_modules_http_auth_ldap? ( ${HTTP_AUTH_LDAP_MODULE_URI} -> ${HTTP_AUTH_LDAP_MODULE_P}.tar.gz )" LICENSE="BSD-2 BSD SSLeay MIT GPL-2 GPL-2+ nginx_modules_http_security? ( Apache-2.0 ) @@ -180,7 +187,8 @@ http_push_stream http_sticky http_ajp - http_mogilefs" + http_mogilefs + http_auth_ldap" IUSE="aio debug +http +http-cache ipv6 libatomic luajit +pcre pcre-jit rtmp selinux ssl userland_GNU vim-syntax" @@ -220,7 +228,8 @@ nginx_modules_http_auth_pam? ( virtual/pam ) nginx_modules_http_metrics? ( dev-libs/yajl ) nginx_modules_http_dav_ext? ( dev-libs/expat ) - nginx_modules_http_security? ( >=dev-libs/libxml2-2.7.8 dev-libs/apr-util www-servers/apache )" + nginx_modules_http_security? ( >=dev-libs/libxml2-2.7.8 dev-libs/apr-util www-servers/apache ) + nginx_modules_http_auth_ldap? ( net-nds/openldap )" RDEPEND="${CDEPEND} selinux? ( sec-policy/selinux-nginx ) " @@ -440,6 +449,11 @@ myconf+=" --add-module=${HTTP_MOGILEFS_MODULE_WD}" fi + if use nginx_modules_http_auth_ldap; then + http_enabled=1 + myconf+=" --add-module=${HTTP_AUTH_LDAP_MODULE_WD}" + fi + if use http || use http-cache; then http_enabled=1 fi 



NGINX configuration option for git.example.com host:

 # /etc/nginx/conf.d/git.example.com.ssl.conf #   fcgiwrap,       nginx:nginx. upstream fastcgi-server { server unix:/run/fcgiwrap.sock-1; } ldap_server ldap_git_users { # ldap[s]://hostname:port/base_dn?attributes?scope?filter url "ldap://127.0.0.1:389/ou=people,dc=example,dc=com?uid?sub?(objectClass=person)"; # DN   . binddn "cn=manager,dc=example,dc=com"; binddn_passwd passwd; #        memberOf . group_attribute uniqueMember; #     DN  . group_attribute_is_dn on; # satisfy any; require group "cn=developers,ou=groups,dc=example,dc=com"; } #      LDAP. # $repo -    URL  location; $repo_login -  . map $repo $repo_login { default ""; #        . "example.com" "example"; } #   HTTPS server { listen [_IP]:80; server_name git.example.com; return 301 https://git.example.com$request_uri; } # HTTPS server { listen [_IP]:443 ssl; add_header Strict-Transport-Security max-age=2592000; server_name git.example.com; charset utf-8; ssl_certificate /etc/ssl/nginx/git.example.com.pem; ssl_certificate_key /etc/ssl/nginx/git.example.com.key; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; access_log /var/log/nginx/nginx_git.example.com-ssl_access_log main; error_log /var/log/nginx/nginx_git.example.com-ssl_error_log info; #      . #        ,      . root /var/git/empty; #   Git . location ~ "^/(?<repo>[^/]+)/objects/([0-9a-f]{2}/[0-9a-f]{38}|pack/pack-[0-9a-f]{40}\.(pack|idx))$" { auth_ldap "git::repository"; auth_ldap_servers ldap_git_users; #   $repo_login    map  $repo if ($remote_user = $repo_login) { root /var/git; } } #   ,     git-http-backend. location ~ "^/(?<repo>[^/]+)/(HEAD|info/refs|objects/info/[^/]+|git-(upload|receive)-pack)$" { auth_ldap "git::repository"; auth_ldap_servers ldap_git_users; fastcgi_param SCRIPT_FILENAME /usr/libexec/git-core/git-http-backend; fastcgi_param PATH_INFO $uri; fastcgi_param GIT_HTTP_EXPORT_ALL ""; fastcgi_param GIT_PROJECT_ROOT /var/git; fastcgi_param REMOTE_USER $remote_user; include fastcgi_params; if ($remote_user = $repo_login) { fastcgi_pass fastcgi-server; } } #      404. location / { return 404; } } 


ownCloud



Cloud storage ownCloud already contains everything you need to work with LDAP. Simply go to Apps → Not enabled to find the “LDAP user and group backend” and click Enable. As a result, an LDAP item will appear in the settings. You need to go to it and, in the Server tab, specify the already familiar settings:

 SERVER: 127.0.0.1; Port: 389 DN: cn=manager,dc=example,dc=com PASS: passwd BASEDN: ou=ou=people,dc=example,dc=com 


To make the group selection on the next tab, you need to go to Advanced → Directory Settings and change the following settings:

 Group Display Name Field: cn Base Group Tree: ou=groups,dc=example,dc=com Group-Member association: uniqueMember 


However, even after that, I only had one posixGroup, which was clearly not enough. Since the settings allow you to specify all the filters manually, it is easier and more reliable to use this feature. To do this, it is desirable to enable the “Manually enter LDAP filters” setting on the Server tab, then set all filters manually:

 Users: (&(objectClass=person)(memberOf=cn=developers,ou=groups,dc=example,dc=com)) Login Attributes: (&(objectClass=person)(memberOf=cn=developers,ou=groups,dc=example,dc=com)(uid=%uid)) Groups:     ,     ownCloud   . 


Having finished with this, you can go to Users and make sure that all the necessary users are in the list.

To mount your ownCloud folder using davfs2, you need to add the following lines to the files on the client side:

/etc/davfs2/davfs2.conf

 use_locks 0 

/ etc / davfs2 / secrets

 /home/<user>/workspace/example/cloud example "passwd" 

/ etc / fstab

 https://cloud.example.com/remote.php/webdav /home/<user>/workspace/example/cloud davfs user,noauto 0 0 


And finally, an example of the alias wo (work on) to activate the working environment using the example of the Django project:

 # ~/.bashrc alias wo=" \ mkdir -p ~/workspace/example/cloud && \ (mountpoint -q ~/workspace/example/cloud || mount ~/workspace/example/cloud) && \ cp -u ~/workspace/example/cloud/deploy/settings_local.py ~/workspace/example/src/project/settings_local.py && \ source ~/VEnvs/example/bin/activate && \ cd ~/workspace/example/src \ " 


All configuration files and scripts used in the article can be found on GitHub

Source: https://habr.com/ru/post/274187/


All Articles