Deploying Mercurial Repositories via FastCGI using Nginx on FreeBSD
I succumbed to the influence of fashion and exciting prospects for DVCS recently. This pushed me out of Subversion + Trac’s track gauge and made me look for new schemes for how to store the source code in different companies. And provide them with easy access for developers, customers and other interested individuals.
It so happens that I specialize in FreeBSD and I don’t understand Linux so well. And I also prefer where you can use Nginx instead of Apache httpd. Therefore, I decided to make for myself a unified architecture that will allow you to store an unlimited number of repositories and delimit access for them to different groups of people on this platform.
Of course, Bitbucket is our everything. But any developer has closed projects that I would not like to publish in public. You can, of course, pay $ 50 per month for the opportunity to host 25 projects on bitbucket. I personally think that it is better to spend this money on a dedicated server and raise yourself as many projects as you like. It will not be so convenient, but its own and with the possibility of tuning, backup and other goodies.
')
You must have a Mercurial repository repository that can be accessed via the web. Access rights must be read / write for specific users.
Details on how to deploy multiple repositories can be found on the official page . To do this, use the cgi script hgwebdir.cgi. We, of course, will use its FastCGI version of hgwebdir.fcgi.
Mercurial does not include a self-contained FastCGI server that can be run on the correct port. It is assumed that hgwebdir.fcgi will be launched using apache httpd or lighttpd. To solve this problem, use fastcgi wrapper spawn-fcgi.
So, first of all we install the following ports.
Copy the default script to our settings directory
We edit our hgwebdir.fcgi. I have it like this
Next, in the configuration file /usr/home/www/cgi/hgweb.config, we write the paths to the collections and the address setting
The baseurl parameter says that all links that will be generated by hgwebdir will contain / hg at the beginning of the path (remember, we have an address where repository dev.example.com/hg/ should be accessible)
In the FreeBSD configuration file /etc/rc.conf we add the following lines
As spawn_fcgi_app, you can specify the script itself, and not the path to the python interpreter. But then the service will not stop correctly, because system scripts will search for a process named hgwebdir.fcgi to stop, but the process is actually called python, since hgwebdir.fcgi is a program that is run using an interpreter.
Now you can run the daemon
In the configuration file /usr/local/etc/nginx/nginx.conf, create a new server section similar to the following:
At this point, be extremely careful. Nginx by default does not set the variables that are needed for correct work with mercurial. Therefore, I will try to explain the meaning of each line. They were given to me by studying the source code of mercurial :)
set_real_ip_from , real_ip_header - as I mentioned above, external https access serves a separate nginx, which is located on a server that looks into the world. So that on our local server the real addresses of clients were logged, and not the address of this external server that proxies requests, and these directives were added.
if block and variable PATH_INFO - hgwebdir uses the variable PATH_INFO to determine the repository that is being accessed. We need to cut off the beginning of the path / hg, since otherwise, hgwebdir will think that you are trying to access the repository under the name "hg". Here you need to be careful and use the nginx variable $ uri to calculate PATH_INFO, and not $ request_uri, as some manuals on the Internet mention. Because $ request_uri also contains $ args besides the path. And if you pass it to PATH_INFO, then you will get 404 Not Found when you try to work with the repository.
client_max_body_size — by default, the maximum length of the request to nginx is 1M. Naturally, if you try to commit changes with more than 1M, or make an initial push of a large repository, the request will not work. Therefore, this size is increased to 100M. I think in most cases this is enough.
include fastcgi_params - include other standard variables of type QUERY_STRING, etc.
REMOTE_USER - in this variable hgwebdir expects to get the username.
Well, the last two lines - setting http auth authorization.
All, now you can fix other non-hg parameters in nginx.conf (for example, where to add logs) and start the web server.
Add
Now we are ready to create the first repository. Everything is done from the user www, because it is under it that nginx and hgwebdir work and it needs write access to this directory. Let's make a test repository. We give read access to user1 and read / write permissions to user2.
We edit test / .hg / hgrc so that it looks like this
Done, you can check access.
In addition to the general migration from subversion to mercurial, I now look intently at redmine instead of trac. I liked the idea that users can register themselves and create projects. Unfortunately, I did not find a ready-made solution for how to automatically create the hg repository for the created project. And with the cloning of access rights as described in this article scheme. If someone knows - I will be grateful for the reference. In any case, I'm going to solve this problem in the near future, either by writing a plug-in to redmine, or in a different / correct way.
If you liked this article and you would like to see a sequel, leave feedback. I will then share the new knowledge of integrating our new environment with redmine.
It so happens that I specialize in FreeBSD and I don’t understand Linux so well. And I also prefer where you can use Nginx instead of Apache httpd. Therefore, I decided to make for myself a unified architecture that will allow you to store an unlimited number of repositories and delimit access for them to different groups of people on this platform.
Of course, Bitbucket is our everything. But any developer has closed projects that I would not like to publish in public. You can, of course, pay $ 50 per month for the opportunity to host 25 projects on bitbucket. I personally think that it is better to spend this money on a dedicated server and raise yourself as many projects as you like. It will not be so convenient, but its own and with the possibility of tuning, backup and other goodies.
')
Formulation of the problem
You must have a Mercurial repository repository that can be accessed via the web. Access rights must be read / write for specific users.
Specifically point by point
- Repositories should be stored in the / usr / home / www / repos / directory
- Mercurial settings are stored in / usr / home / www / repos / cgi /
- Hosting takes place on a local server. Therefore, access is only via the http protocol. Outside, access will be over https served by another server, also based on nginx, and transferred to the local port 80.
- Let's say the server will be called dev.example.com. The list of repositories should be available at
http://dev.example.com/hg/
Installing the necessary components
Details on how to deploy multiple repositories can be found on the official page . To do this, use the cgi script hgwebdir.cgi. We, of course, will use its FastCGI version of hgwebdir.fcgi.
Mercurial does not include a self-contained FastCGI server that can be run on the correct port. It is assumed that hgwebdir.fcgi will be launched using apache httpd or lighttpd. To solve this problem, use fastcgi wrapper spawn-fcgi.
So, first of all we install the following ports.
- www / spawn-fcgi
- devel / mercurial
- www / nginx
- www / py-flup
Customize hgwebdir
Copy the default script to our settings directory
sudo -u www mkdir /usr/home/www/cgi/
cp /usr/local/share/mercurial/www/hgwebdir.fcgi /usr/home/www/cgi/
We edit our hgwebdir.fcgi. I have it like this
#!/usr/bin/env python
from mercurial import demandimport; demandimport.enable()
from mercurial.hgweb.hgwebdir_mod import hgwebdir
from flup.server.fcgi import WSGIServer
WSGIServer(hgwebdir('/usr/home/www/cgi/hgweb.config')).run()
Next, in the configuration file /usr/home/www/cgi/hgweb.config, we write the paths to the collections and the address setting
[collections]
/usr/home/www/repos = /usr/home/www/repos
[web]
baseurl = /hg
The baseurl parameter says that all links that will be generated by hgwebdir will contain / hg at the beginning of the path (remember, we have an address where repository dev.example.com/hg/ should be accessible)
Configuring spawn-fcgi
In the FreeBSD configuration file /etc/rc.conf we add the following lines
# Mercurial
spawn_fcgi_enable="YES"
spawn_fcgi_app="/usr/local/bin/python"
spawn_fcgi_app_args="/usr/home/www/cgi/hgwebdir.fcgi"
spawn_fcgi_pidfile="/var/run/hg.pid"
spawn_fcgi_bindsocket="/var/run/hg.socket"
As spawn_fcgi_app, you can specify the script itself, and not the path to the python interpreter. But then the service will not stop correctly, because system scripts will search for a process named hgwebdir.fcgi to stop, but the process is actually called python, since hgwebdir.fcgi is a program that is run using an interpreter.
Now you can run the daemon
sudo /usr/local/etc/rc.d/spawn-fcgi start
We configure nginx
In the configuration file /usr/local/etc/nginx/nginx.conf, create a new server section similar to the following:
server {
listen 80;
server_name dev.example.com;
set_real_ip_from 172.16.224.1;
real_ip_header X-Forwarded-For;
location / {
root / usr / local / www / nginx;
index index.html index.htm;
}
location / hg {
if ($ uri ~ /hg(/.*)$) {
set $ path $ 1;
}
client_max_body_size 100M;
fastcgi_pass unix: /var/run/hg.socket;
include fastcgi_params;
fastcgi_param PATH_INFO $ path;
fastcgi_param REMOTE_USER $ remote_user;
auth_basic "Mercurial repositories";
auth_basic_user_file hg_htpasswd;
}
}
At this point, be extremely careful. Nginx by default does not set the variables that are needed for correct work with mercurial. Therefore, I will try to explain the meaning of each line. They were given to me by studying the source code of mercurial :)
set_real_ip_from , real_ip_header - as I mentioned above, external https access serves a separate nginx, which is located on a server that looks into the world. So that on our local server the real addresses of clients were logged, and not the address of this external server that proxies requests, and these directives were added.
if block and variable PATH_INFO - hgwebdir uses the variable PATH_INFO to determine the repository that is being accessed. We need to cut off the beginning of the path / hg, since otherwise, hgwebdir will think that you are trying to access the repository under the name "hg". Here you need to be careful and use the nginx variable $ uri to calculate PATH_INFO, and not $ request_uri, as some manuals on the Internet mention. Because $ request_uri also contains $ args besides the path. And if you pass it to PATH_INFO, then you will get 404 Not Found when you try to work with the repository.
client_max_body_size — by default, the maximum length of the request to nginx is 1M. Naturally, if you try to commit changes with more than 1M, or make an initial push of a large repository, the request will not work. Therefore, this size is increased to 100M. I think in most cases this is enough.
include fastcgi_params - include other standard variables of type QUERY_STRING, etc.
REMOTE_USER - in this variable hgwebdir expects to get the username.
Well, the last two lines - setting http auth authorization.
All, now you can fix other non-hg parameters in nginx.conf (for example, where to add logs) and start the web server.
Add
nginx_enable="YES" to /etc/rc.conf and do sudo /usr/local/etc/rc.d/nginx startRepository creation example
Now we are ready to create the first repository. Everything is done from the user www, because it is under it that nginx and hgwebdir work and it needs write access to this directory. Let's make a test repository. We give read access to user1 and read / write permissions to user2.
#
cd /usr/home/www/repos
#
sudo -u www hg init test
# nginx
# -c -
# htpasswd - www/apache22
sudo htpasswd -b -c /usr/local/etc/nginx/hg_htpasswd user1 pass1
sudo htpasswd -b /usr/local/etc/nginx/hg_htpasswd user2 pass2
We edit test / .hg / hgrc so that it looks like this
[web]
# ssl, nginx
push_ssl = False
allow_push = user2
allow_read = user1,user2
Done, you can check access.
Instead of conclusion
In addition to the general migration from subversion to mercurial, I now look intently at redmine instead of trac. I liked the idea that users can register themselves and create projects. Unfortunately, I did not find a ready-made solution for how to automatically create the hg repository for the created project. And with the cloning of access rights as described in this article scheme. If someone knows - I will be grateful for the reference. In any case, I'm going to solve this problem in the near future, either by writing a plug-in to redmine, or in a different / correct way.
If you liked this article and you would like to see a sequel, leave feedback. I will then share the new knowledge of integrating our new environment with redmine.
Source: https://habr.com/ru/post/90066/
All Articles