Mercurial + Nginx configuration for managing a large number of repositories

An example of the configuration of the bundle mercurial + nginx is described under the cat and the automation script for all of the above is given.

Tasks:

Creating a repository repository based on the Mercurial version control system with the ability to securely transfer data and separate access levels.
Automate the above actions.

Decision:

Proxy built-in http server (hg serve) with proxy level access sharing.
To avoid data interception, the transmission is carried out via the HTTPS protocol.
To minimize the resources consumed, Nginx acts as a proxy.
Access control is made at the location directive level.

Agreements:

Linux distribution in which everything will happen - Gentoo
https://hg.expample.com/reponame - link to access the repository
hg.example.com - domain name of the repository repository server
reponame - the name of the required repository
/ home / repos - root folder for repositories
/ etc / hg - root folder for configuration files
This reader has minimal Linux administration skills.

Install the necessary packages:

hg ~ # emerge mercurial <br> hg ~ # emerge nginx

The built-in http server will be launched with the command

hg ~ # /usr/bin/hg serve -d -A /var/log/hg_access.log -p 8080 -a 127.0.0.1 --pid-file /var/run/hgserver.pid --encoding utf8 --webdir-conf /etc/hg/web.config

Launch directives:
-d - start the server as a daemon
-A /var/log/hg_access.log - server access log
-p 8080 - the port number on which the server is waiting for requests
-a 127.0.0.1 - ip address where the server is started
--pid-file /var/run/hgserver.pid - file with server process ID
--encoding utf8 - encoding
--webdir-conf /etc/hg/web.config - server configuration file

hg ~ # cat /etc/hg/web.config <br> [web] // <br> allow_push = * // “” ( ) <br> push_ssl = false // ssl ( ) <br> [paths] // “” <br> rep1=/home/repos/rep1 // : - rep1 <br> rep2=/home/repos/rep2 // : - rep2 <br>

Writing the Include directive to the main Nginx configuration file

hg ~ # cat /etc/nginx/nginx.conf |grep -i include <br> include "/etc/hg/nginx.conf";

Example configuration file /etc/hg/nginx.conf

hg ~ # cat /etc/hg/nginx.conf <br> server <br> {<br> listen 443;<br> server_name hg.example.com;<br> client_max_body_size 128M;<br> ssl on;<br> ssl_certificate /etc/ssl/nginx/nginx.pem;<br> ssl_certificate_key /etc/ssl/nginx/nginx.key;<br>location /repo1<br> {<br> proxy_pass http://127.0.0.1:8080;<br> auth_basic "Restricted";<br> auth_basic_user_file /etc/hg/nginx/repo1.pass;<br> access_log /var/log/nginx/repo1.hg.example.com.ssl_access_log main;<br> error_log /var/log/nginx/repo1.hg.example.com.ssl_error_log info;<br> }<br>location /repo2<br> {<br> proxy_pass http://127.0.0.1:8080;<br> auth_basic "Restricted";<br> auth_basic_user_file /etc/hg/nginx/repo2.pass;<br> access_log /var/log/nginx/repo2.hg.example.com.ssl_access_log main;<br> error_log /var/log/nginx/repo2.hg.example.com.ssl_error_log info;<br> }<br>}<br>

Consider the section: location / repo1
proxy_pass - specify where to send requests with successful authorization
auth_basic - use HTTP Basic Authentication
auth_basic_user_file - specify in which file the password database is located
access_log and error_log - repository access logging directives
Create authorization files for each of the repositories with the htpasswd command
(The htpasswd program is included in the apache package, so you'll have to install it with the emerge apache command

hg ~ # htpasswd -bc /etc/hg/nginx/repo2.pass test2 testpass2 <br> hg ~ # htpasswd -bc /etc/hg/nginx/repo1.pass test1 testpass1

-b - use the password specified on the command line.
-c - create a new password database file
/etc/hg/nginx/repo1.pass - the name of the password database file
test1 - user login added
testpass1 - user password added
It remains to initialize the repositories with the hg init command.

hg ~ # hg init /home/repos/repo1 <br> hg ~ # hg init /home/repos/repo2

We start Nginx team

hg ~ # /etc/init.d/nginx start

Now repo1 and repo2 repositories
available at
https://hg.example.com/repo1 and https://hg.example.com/repo2 respectively.
With configured separation of access levels.
This was the end of the article, for a simple configuration that is updated every 3-5 months is enough. But what to do when the configuration has to be changed often? The correct system administrator will immediately begin to look for a way to make life easier for you.

At least three solutions to the problem:

1. Use the finished product management repositories.
2. Delegate an honorary duty to his subordinate.
3. Create a management system yourself.
Unfortunately, neither the first nor the second option suited me , so I had to overcome my laziness and do it myself.
The result was a script that fully automates the process of managing the repository server.

In which it was implemented:

1. Create (if necessary) and provide access to repositories.
2. Prohibition of access to inactive repositories. (When removed from the configuration file, the repository is not physically deleted).
3. Regeneration of user passwords for authorization of Nginx.
4. Distribution of access to repositories at the user level.

The syntax of the configuration file allows:

1. use blank lines to visually separate sections
2. use the “#” symbol to highlight comments

The configuration file consists of three sections.

[users] - : -. “=”<br>user1=pass1<br>user2=pass2<br>user3=pass3<br> [repos] - <br>repo1<br>repo2<br>repo3<br> [access] - , “,”<br>repo1 = user1 , user2,user3 - user1,user2,user3<br>repo2 = user1,user2 - user1 user2<br>repo3 = user3 - user3<br>

Listing script /usr/local/sbin/hgmkrep.sh

1. #! / bin / bash
2. tmphtpass = "/ var / tmp / htpass" # define a temporary password database file
3. repohome = "/ home / repos /" # define the root folder for repositories
4. hgservepid = "/var/run/hgserver.pid" #pid hg serve
5. hgaccesslog = "/var/log/hg_access.log" #access log file for hg server
6. domain = "exapmple.com" #tld server name
7. confdir = "/ etc / hg /" # define the root folder for configs
8. confile = $ {confdir} "repo.cfg" # main config file /etc/hg/repo.cfg
9. webconfig = $ {confdir} "web.config" # config for hg server /etc/hg/web.config
10. nginxconfig = $ {confdir} "nginx.conf" # config for nginx /etc/hg/nginx.conf
11. nginxauthdir = $ {confdir} "nginx /" # folder for password bases for access to repositories
12. [ -s $ {confile} ] || echo "where is config file?" # check the presence of the main config
13. [ -s $ {confile} ] || exit 0 # grieve if there is no main config
14. # parsim section [repos] for repositories
15. repos = ` cat $ {confile} | sed '/ ^ $ / d' | sed '/ ^ # / d' | sed 's / \ // g' | awk '/ \ [repos \] / {
16. is_repos = 1;
17. while (is_repos == 1)
18. {if (getline <= 0 || index ($ 0, "[") == 1)
19. {is_repos = 0;}
20. else
21. {print $ 0;}}} ' `
22. # Check the availability of folders with repositories and, if necessary, create
23. for i in $ {repos}
24. do ; [ -d $ {repohome} $ {i} ] || / usr / bin / hg init $ {repohome} $ {i} ; done
25. # generate config for hg server
26. echo "[web]
27. allow_push = *
28. push_ssl = false
29. [paths] " > $ {webconfig}
30. # allow access only to active repositories
31. for i in $ {repos}
32. do ; echo $ {i} = $ {repohome} $ {i} >> $ {webconfig} ; done
33. # reboot hg serve
34. [ -a $ {hgservepid} ] && / bin / kill `/ bin / cat $ {hgservepid} ` && rm $ {hgservepid}
35. / usr / bin / hg serve -d -A $ {hgaccesslog} -p 8080 -a 127.0.0.1 --pid-file $ {hgservepid} --encoding utf8 --webdir-conf $ {webconfig}
36. # create config for nginx
37. echo "server
38. {
39. listen 443;
40. server_name hg. " $ {domain} ";
41. client_max_body_size 128M;
42. ssl on;
43. ssl_certificate /etc/ssl/nginx/nginx.pem;
44. ssl_certificate_key /etc/ssl/nginx/nginx.key; " > $ {nginxconfig}
45. # create lacation for active repositories
46. for i in $ {repos}
47. do
48. echo "location /" $ {i} "
49. {
50. proxy_pass http://127.0.0.1:8080;
51. auth_basic \ " Restricted \" ;
52. auth_basic_user_file " $ {nginxauthdir} $ {i} " .pass;
53. access_log / var / log / nginx / " $ {i} " .hg. " $ {domain} " .ssl_access_log main;
54. error_log / var / log / nginx / " $ {i} " .hg. " $ {domain} " .ssl_error_log info;
55. } " >> $ {nginxconfig}
56. done
57. echo "}" >> $ {nginxconfig}
58. # create (reset, just in case) temporary password database
59. cat / dev / null > $ {tmphtpass}
60. # parsim section of the main config [users]
61. # generate passwords for all active users
62. cat $ {confile} | sed '/ ^ $ / d' | sed '/ ^ # / d' | sed 's / \ // g' | awk -v passfile = $ tmphtpass ' / \ [users \] / {
63. is_users = 1;
64. while (is_users == 1)
65. {if (getline <= 0 || index ($ 0, "[") == 1)
66. {is_users = 0;}
67. else
68. {split ($ 0, userpass, "="); system ("htpasswd -b" passfile "" userpass [1] "" userpass [2]);}}} '
69. # parsim section [access] of the main config
70. # and get the list of privileges of the form repo = user1, user2
71. access = ` cat $ {confile} | sed '/ ^ $ / d' | sed '/ ^ # / d' | sed 's / \ // g' | awk '/ \ [access \] / {
72. is_access = 1; while (is_access == 1)
73. {if (getline <= 0 || index ($ 0, "[") == 1)
74. {is_access = 0;}
75. else
76. {print $ 0;}}} ' `
77. # check if there is a folder for storing password databases
78. [ -d $ {nginxauthdir} ] || mkdir -p $ {nginxauthdir}
79. # delete old password database files
80. find $ {nginxauthdir} -type f -name * .pass -delete
81. # for each repository from the [access] section we generate a personal password database
82. for i in $ {access}
83. do ; echo $ {i} | sed 's /, / \ | / g' | awk -v tmphtpass = $ {tmphtpass} -v nginxauthdir = $ {nginxauthdir} \
84. 'BEGIN {FS = "="} {system ("cat" tmphtpass "| egrep \" "$ 2" \ ">" nginxauthdir "" $ 1 ".pass")}' done
85. # restart nginx
86. / etc / init.d / nginx restart
87. # delete temporary password database file
88. rm $ {tmphtpass}

I also post a link to the working script and the config example: hgmkrep.tar.gz

Existing disadvantages:

1. There is no distinction for reading and writing; each user allowed to the repository body has rw rights by default.
2. Not implemented using groups to restrict access
3. Configuration rollback is not implemented with incorrect syntax of the config (the system will collapse)
4. With an increase in the number of users and repositories, the config abundantly loses its visibility