⬆️ ⬇️

Mercurial + Nginx configuration for managing a large number of repositories

An example of the configuration of the bundle mercurial + nginx is described under the cat and the automation script for all of the above is given.

Tasks:

Creating a repository repository based on the Mercurial version control system with the ability to securely transfer data and separate access levels.

Automate the above actions.

Decision:

Proxy built-in http server (hg serve) with proxy level access sharing.

To avoid data interception, the transmission is carried out via the HTTPS protocol.

To minimize the resources consumed, Nginx acts as a proxy.

Access control is made at the location directive level.

Agreements:

Linux distribution in which everything will happen - Gentoo

https://hg.expample.com/reponame - link to access the repository

hg.example.com - domain name of the repository repository server

reponame - the name of the required repository

/ home / repos - root folder for repositories

/ etc / hg - root folder for configuration files

This reader has minimal Linux administration skills.

Install the necessary packages:

hg ~ # emerge mercurial <br> hg ~ # emerge nginx
The built-in http server will be launched with the command
hg ~ # /usr/bin/hg serve -d -A /var/log/hg_access.log -p 8080 -a 127.0.0.1 --pid-file /var/run/hgserver.pid --encoding utf8 --webdir-conf /etc/hg/web.config
Launch directives:

-d - start the server as a daemon

-A /var/log/hg_access.log - server access log

-p 8080 - the port number on which the server is waiting for requests

-a 127.0.0.1 - ip address where the server is started

--pid-file /var/run/hgserver.pid - file with server process ID

--encoding utf8 - encoding

--webdir-conf /etc/hg/web.config - server configuration file
hg ~ # cat /etc/hg/web.config <br> [web] // <br> allow_push = * // “” ( ) <br> push_ssl = false // ssl ( ) <br> [paths] // “” <br> rep1=/home/repos/rep1 // : - rep1 <br> rep2=/home/repos/rep2 // : - rep2 <br>
Writing the Include directive to the main Nginx configuration file
hg ~ # cat /etc/nginx/nginx.conf |grep -i include <br> include "/etc/hg/nginx.conf";
Example configuration file /etc/hg/nginx.conf
hg ~ # cat /etc/hg/nginx.conf <br> server <br> {<br> listen 443;<br> server_name hg.example.com;<br> client_max_body_size 128M;<br> ssl on;<br> ssl_certificate /etc/ssl/nginx/nginx.pem;<br> ssl_certificate_key /etc/ssl/nginx/nginx.key;<br>location /repo1<br> {<br> proxy_pass http://127.0.0.1:8080;<br> auth_basic "Restricted";<br> auth_basic_user_file /etc/hg/nginx/repo1.pass;<br> access_log /var/log/nginx/repo1.hg.example.com.ssl_access_log main;<br> error_log /var/log/nginx/repo1.hg.example.com.ssl_error_log info;<br> }<br>location /repo2<br> {<br> proxy_pass http://127.0.0.1:8080;<br> auth_basic "Restricted";<br> auth_basic_user_file /etc/hg/nginx/repo2.pass;<br> access_log /var/log/nginx/repo2.hg.example.com.ssl_access_log main;<br> error_log /var/log/nginx/repo2.hg.example.com.ssl_error_log info;<br> }<br>}<br>
Consider the section: location / repo1

proxy_pass - specify where to send requests with successful authorization

auth_basic - use HTTP Basic Authentication

auth_basic_user_file - specify in which file the password database is located

access_log and error_log - repository access logging directives

Create authorization files for each of the repositories with the htpasswd command

(The htpasswd program is included in the apache package, so you'll have to install it with the emerge apache command

hg ~ # htpasswd -bc /etc/hg/nginx/repo2.pass test2 testpass2 <br> hg ~ # htpasswd -bc /etc/hg/nginx/repo1.pass test1 testpass1
-b - use the password specified on the command line.

-c - create a new password database file

/etc/hg/nginx/repo1.pass - the name of the password database file

test1 - user login added

testpass1 - user password added

It remains to initialize the repositories with the hg init command.

hg ~ # hg init /home/repos/repo1 <br> hg ~ # hg init /home/repos/repo2
We start Nginx team
hg ~ # /etc/init.d/nginx start
Now repo1 and repo2 repositories

available at

https://hg.example.com/repo1 and https://hg.example.com/repo2 respectively.

With configured separation of access levels.

This was the end of the article, for a simple configuration that is updated every 3-5 months is enough. But what to do when the configuration has to be changed often? The correct system administrator will immediately begin to look for a way to make life easier for you.

At least three solutions to the problem:

1. Use the finished product management repositories.

2. Delegate an honorary duty to his subordinate.

3. Create a management system yourself.

Unfortunately, neither the first nor the second option suited me , so I had to overcome my laziness and do it myself.

The result was a script that fully automates the process of managing the repository server.

In which it was implemented:

1. Create (if necessary) and provide access to repositories.

2. Prohibition of access to inactive repositories. (When removed from the configuration file, the repository is not physically deleted).

3. Regeneration of user passwords for authorization of Nginx.

4. Distribution of access to repositories at the user level.

The syntax of the configuration file allows:

1. use blank lines to visually separate sections

2. use the “#” symbol to highlight comments

The configuration file consists of three sections.

[users] - : -. “=”<br>user1=pass1<br>user2=pass2<br>user3=pass3<br> [repos] - <br>repo1<br>repo2<br>repo3<br> [access] - , “,”<br>repo1 = user1 , user2,user3 - user1,user2,user3<br>repo2 = user1,user2 - user1 user2<br>repo3 = user3 - user3<br>
Listing script /usr/local/sbin/hgmkrep.sh
  1. #! / bin / bash
  2. tmphtpass = "/ var / tmp / htpass" # define a temporary password database file
  3. repohome = "/ home / repos /" # define the root folder for repositories
  4. hgservepid = "/var/run/hgserver.pid" #pid hg serve
  5. hgaccesslog = "/var/log/hg_access.log" #access log file for hg server
  6. domain = "exapmple.com" #tld server name
  7. confdir = "/ etc / hg /" # define the root folder for configs
  8. confile = $ {confdir} "repo.cfg" # main config file /etc/hg/repo.cfg
  9. webconfig = $ {confdir} "web.config" # config for hg server /etc/hg/web.config
  10. nginxconfig = $ {confdir} "nginx.conf" # config for nginx /etc/hg/nginx.conf
  11. nginxauthdir = $ {confdir} "nginx /" # folder for password bases for access to repositories
  12. [ -s $ {confile} ] || echo "where is config file?" # check the presence of the main config
  13. [ -s $ {confile} ] || exit 0 # grieve if there is no main config
  14. # parsim section [repos] for repositories
  15. repos = ` cat $ {confile} | sed '/ ^ $ / d' | sed '/ ^ # / d' | sed 's / \ // g' | awk '/ \ [repos \] / {
  16. is_repos = 1;
  17. while (is_repos == 1)
  18. {if (getline <= 0 || index ($ 0, "[") == 1)
  19. {is_repos = 0;}
  20. else
  21. {print $ 0;}}} ' `
  22. # Check the availability of folders with repositories and, if necessary, create
  23. for i in $ {repos}
  24. do ; [ -d $ {repohome} $ {i} ] || / usr / bin / hg init $ {repohome} $ {i} ; done
  25. # generate config for hg server
  26. echo "[web]
  27. allow_push = *
  28. push_ssl = false
  29. [paths] " > $ {webconfig}
  30. # allow access only to active repositories
  31. for i in $ {repos}
  32. do ; echo $ {i} = $ {repohome} $ {i} >> $ {webconfig} ; done
  33. # reboot hg serve
  34. [ -a $ {hgservepid} ] && / bin / kill `/ bin / cat $ {hgservepid} ` && rm $ {hgservepid}
  35. / usr / bin / hg serve -d -A $ {hgaccesslog} -p 8080 -a 127.0.0.1 --pid-file $ {hgservepid} --encoding utf8 --webdir-conf $ {webconfig}
  36. # create config for nginx
  37. echo "server
  38. {
  39. listen 443;
  40. server_name hg. " $ {domain} ";
  41. client_max_body_size 128M;
  42. ssl on;
  43. ssl_certificate /etc/ssl/nginx/nginx.pem;
  44. ssl_certificate_key /etc/ssl/nginx/nginx.key; " > $ {nginxconfig}
  45. # create lacation for active repositories
  46. for i in $ {repos}
  47. do
  48. echo "location /" $ {i} "
  49. {
  50. proxy_pass http://127.0.0.1:8080;
  51. auth_basic \ " Restricted \" ;
  52. auth_basic_user_file " $ {nginxauthdir} $ {i} " .pass;
  53. access_log / var / log / nginx / " $ {i} " .hg. " $ {domain} " .ssl_access_log main;
  54. error_log / var / log / nginx / " $ {i} " .hg. " $ {domain} " .ssl_error_log info;
  55. } " >> $ {nginxconfig}
  56. done
  57. echo "}" >> $ {nginxconfig}
  58. # create (reset, just in case) temporary password database
  59. cat / dev / null > $ {tmphtpass}
  60. # parsim section of the main config [users]
  61. # generate passwords for all active users
  62. cat $ {confile} | sed '/ ^ $ / d' | sed '/ ^ # / d' | sed 's / \ // g' | awk -v passfile = $ tmphtpass ' / \ [users \] / {
  63. is_users = 1;
  64. while (is_users == 1)
  65. {if (getline <= 0 || index ($ 0, "[") == 1)
  66. {is_users = 0;}
  67. else
  68. {split ($ 0, userpass, "="); system ("htpasswd -b" passfile "" userpass [1] "" userpass [2]);}}} '
  69. # parsim section [access] of the main config
  70. # and get the list of privileges of the form repo = user1, user2
  71. access = ` cat $ {confile} | sed '/ ^ $ / d' | sed '/ ^ # / d' | sed 's / \ // g' | awk '/ \ [access \] / {
  72. is_access = 1; while (is_access == 1)
  73. {if (getline <= 0 || index ($ 0, "[") == 1)
  74. {is_access = 0;}
  75. else
  76. {print $ 0;}}} ' `
  77. # check if there is a folder for storing password databases
  78. [ -d $ {nginxauthdir} ] || mkdir -p $ {nginxauthdir}
  79. # delete old password database files
  80. find $ {nginxauthdir} -type f -name * .pass -delete
  81. # for each repository from the [access] section we generate a personal password database
  82. for i in $ {access}
  83. do ; echo $ {i} | sed 's /, / \ | / g' | awk -v tmphtpass = $ {tmphtpass} -v nginxauthdir = $ {nginxauthdir} \
  84. 'BEGIN {FS = "="} {system ("cat" tmphtpass "| egrep \" "$ 2" \ ">" nginxauthdir "" $ 1 ".pass")}' done
  85. # restart nginx
  86. / etc / init.d / nginx restart
  87. # delete temporary password database file
  88. rm $ {tmphtpass}
I also post a link to the working script and the config example: hgmkrep.tar.gz

Existing disadvantages:

1. There is no distinction for reading and writing; each user allowed to the repository body has rw rights by default.

2. Not implemented using groups to restrict access

3. Configuration rollback is not implemented with incorrect syntax of the config (the system will collapse)

4. With an increase in the number of users and repositories, the config abundantly loses its visibility



')

Source: https://habr.com/ru/post/115007/



All Articles