Assembling and installing Calamari monitoring system packages for CEPH 0.87 distributed storage on Ubuntu 04/14/1 (Trusty Tahr)
Given: existing CEPH 0.87 cluster. Objective: to ensure monitoring. Problem number 2: Write instructions (easier).
I googled and found Calamari. Dug further and found several articles in English on the assembly and installation. I tried to put, stepped on a few "rake". As a result, came to writing this article.
I note that the packages compiled as a result are unlikely to fully comply with debian-policy for building deb packages. Provide monitoring is more important.
Calamari is a CEPH distributed file storage cluster monitoring and management system. It includes:
Of course, other software will also be required (for example, PostgresQL). But it does not need to be further configured, unlike those mentioned. Only required to install.
1. Install fresh Ubuntu 04/14/1 (Trusty Tahr) .
In my version the minimum standard server configuration is set, i.e. the following line is present in /var/log/apt/history.log
Thus, some packages (for example: software-properties-common ) are already present in the system.
Also, although some packages necessary for building (for example: libssl-dev ) will come depending on the packages being installed, I will still specify them for installation, as they are indicated in the documentation and articles reviewed.
')
For convenience, Midnight Commander is installed.
Created user ceph with root rights via sudo .
By default, aptitude is configured to automatically install recommended packages. Leave unchanged.
I assume that you can do everything written earlier without anyone’s help. And then I will try to describe my actions as detailed as possible.
2. Update:
3. If something important has been updated. For example, in my case, this is a linux-image kernel package, then reboot
4. Change the time zone from Samara to Moscow (optional):
5. Install the tools to build packages:
Important: In the next three points, I will build packages. Each of the packages can be assembled on a separate machine, but the initial settings of the distribution should be carried out necessarily.
Install dependencies to build calamari-server . They are listed in Build-Depends or found by me as important to the build process. As mentioned at the beginning of the article - some of them have already been installed.
Create a directory in which we will work and clone repositories with calamari-server .
We try to collect
We see an error
and three lines that have eyes hurt:
The Calamari version is no longer the same. The distribution should be trusty . Yes, and I collect the package, not Gary. I also do not like the error that caused the build to stop building.
Therefore, I will collect the package in my own way.
BugFix dpkg-buildpackage error occurs because in the version from the developer the debian package format is specified
version '3.0 (quilt)'. Change the format.
Putting the package together : Changing the debian / changelog file according to our name, email address, version, revision, release, and importance of the package.
Check
After that, the package is assembled without errors, with the correct version, revision, release and importance. For obvious reasons, I do not provide the output of dpkg-buildpackage.
Check
Go back to the parent directory
Important: If we build a package on a separate machine, do not forget about the implementation of the 1st paragraph of the article.
Install dependencies to build a diamond
Download the diamond repository.
Check
We collect package
Check
Go back to the parent directory
Important: If we build a package on a separate machine, do not forget about the implementation of the 1st paragraph of the article.
Add repositories with node.js :
Updating package lists:
Install dependencies to build calamari-clients:
Install dependencies using NPM and Gem:
In some articles it is recommended to put a gem without sudo , but I swore for access rights to system directories. Stepping on another “rake”, set via sudo and the next line corrected the consequences of the root user
Warning: Installing dependencies via NPM and Gem , and the build of calamari-clients coincided with the blocking time of github. Be carefull. Without access to this resource you will catch a lot of problems. Perhaps, not solved without a full-fledged (not anonymous) proxy or vpn.
Download calamari-clients repository
Download the patch (in the parent directory)
In case the githab fall off again I will leave the diff file here. When copying from the screen, the diff file is crooked.
Apply patch:
We collect calamari-clients deb-package.
Check
Go back to the parent directory
I install the server on a separate machine. I carry out all subparagraphs of point 1, except the last (5).
UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Set the saltstack as follows
Installing dependencies for calamari-server
Install calamari-server and calamari-clients
Calamari setup
UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Set the saltstack as follows.
On all nodes
Install gdebi
Diamond and its settings
Salt-minion and its settings
Restart
After completing the previous item, salt-minion clients from Ceph machines will attempt to upgrade from the wizard.
On the machine with calamari-server and salt-master we perform
We see the list of cluster machines. We allow them access
Login using the browser on http: // CALAMARI_SERVER_IP with the login and password specified during setup (calamari-ctl initialize).
The synchronization of the diamond settings on the Ceph nodes occurs automatically. However, there is a command to update the settings manually.
The following command will check the connectivity of Ceph nodes with calamari-server
Check on the salt-master of all connected salt-minions
Note: The article does not include some intermediate results. For example, a patch that eliminated an unpleasant error was excluded. But, checking the article, I once again cloned calamari from the github, applied my patch and the patch did not work. In general, as it turned out, while I was writing an article - calamari'vtsy fixed a bug in the main branch.
Compiling Calamari for Ceph on Ubuntu 14.04
Ceph Calamari: Step-by-Step
Build package and Install Calamari on Debian Weezy
Ceph Calamari: The Survival Guide
Calamari Documentation
Precise Build Instructions
I googled and found Calamari. Dug further and found several articles in English on the assembly and installation. I tried to put, stepped on a few "rake". As a result, came to writing this article.
I note that the packages compiled as a result are unlikely to fully comply with debian-policy for building deb packages. Provide monitoring is more important.
0. Brief Description of Calamari
Calamari is a CEPH distributed file storage cluster monitoring and management system. It includes:
- calamari-server - the server itself, the backend
- calamari-clients - front-end (web interface), installed with the server
- salt-master - configuration file management server, installed with calamari-server
- salt-minon - client receiving configuration files, installed on all CEPH cluster machines
- diamond - statistics collector, installed on all machines of the CEPH cluster
Of course, other software will also be required (for example, PostgresQL). But it does not need to be further configured, unlike those mentioned. Only required to install.
1. Initial Distribution Settings
1. Install fresh Ubuntu 04/14/1 (Trusty Tahr) .
In my version the minimum standard server configuration is set, i.e. the following line is present in /var/log/apt/history.log
Commandline: apt-get -o APT::Status-Fd=4 -o APT::Keep-Fds::=5 -o APT::Keep-Fds::=6 -q -y install minimal^ openssh-server^ server^ standard^ Thus, some packages (for example: software-properties-common ) are already present in the system.
Also, although some packages necessary for building (for example: libssl-dev ) will come depending on the packages being installed, I will still specify them for installation, as they are indicated in the documentation and articles reviewed.
')
For convenience, Midnight Commander is installed.
Created user ceph with root rights via sudo .
By default, aptitude is configured to automatically install recommended packages. Leave unchanged.
I assume that you can do everything written earlier without anyone’s help. And then I will try to describe my actions as detailed as possible.
2. Update:
sudo aptitude update && sudo aptitude -y upgrade 3. If something important has been updated. For example, in my case, this is a linux-image kernel package, then reboot
sudo reboot 4. Change the time zone from Samara to Moscow (optional):
sudo dpkg-reconfigure tzdata 5. Install the tools to build packages:
sudo aptitude -y install build-essential debhelper devscripts git make g++ Important: In the next three points, I will build packages. Each of the packages can be assembled on a separate machine, but the initial settings of the distribution should be carried out necessarily.
2. Installing the environment for calamari-server and building the package
Install dependencies to build calamari-server . They are listed in Build-Depends or found by me as important to the build process. As mentioned at the beginning of the article - some of them have already been installed.
sudo aptitude -y install libcairo2-dev libpq-dev python-dev python-pip python-virtualenv python-crypto python-m2crypto python-mako \ python-msgpack python-zmq cython libssl-dev lsb-release openssl curl software-properties-common swig libzmq-dev python-cairo \ python-sphinx reprepro Create a directory in which we will work and clone repositories with calamari-server .
ceph@calamari:~$ mkdir -p ~/dev; cd ~/dev ceph@calamari:~/dev$ git clone https://github.com/ceph/calamari.git ceph@calamari:~/dev$ cd calamari We try to collect
ceph@calamari:~/dev/calamari$ dpkg-buildpackage Get
dpkg-buildpackage: source package calamari dpkg-buildpackage: source version 1.0.0-1 dpkg-buildpackage: source distribution precise dpkg-buildpackage: source changed by Gary Lowell <glowell@pudgy.ops.newdream.net> dpkg-buildpackage: host architecture amd64 dpkg-source --before-build calamari fakeroot debian/rules clean dh clean --with python2 dh_testdir dh_auto_clean make -j1 clean make[1]: Entering directory `/home/ceph/dev/calamari' target: clean rm -rf venv rest-api/calamari_rest/version.py make[1]: Leaving directory `/home/ceph/dev/calamari' dh_clean rm -f debian/calamari-server.substvars rm -f debian/calamari-server.*.debhelper rm -rf debian/calamari-server/ rm -f debian/*.debhelper.log rm -f debian/files find . \( \( -type f -a \ \( -name '#*#' -o -name '.*~' -o -name '*~' -o -name DEADJOE \ -o -name '*.orig' -o -name '*.rej' -o -name '*.bak' \ -o -name '.*.orig' -o -name .*.rej -o -name '.SUMS' \ -o -name TAGS -o \( -path '*/.deps/*' -a -name '*.P' \) \ \) -exec rm -f {} + \) -o \ \( -type d -a -name autom4te.cache -prune -exec rm -rf {} + \) \) rm -f *-stamp dpkg-source -b calamari dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../calamari_1.0.0.orig.tar.{bz2,gz,lzma,xz} dpkg-buildpackage: error: dpkg-source -b calamari gave error exit status 255 We see an error
dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../calamari_1.0.0.orig.tar.{bz2,gz,lzma,xz} dpkg-buildpackage: error: dpkg-source -b calamari gave error exit status 255 and three lines that have eyes hurt:
dpkg-buildpackage: source version 1.0.0-1 dpkg-buildpackage: source distribution precise dpkg-buildpackage: source changed by Gary Lowell <glowell@pudgy.ops.newdream.net> The Calamari version is no longer the same. The distribution should be trusty . Yes, and I collect the package, not Gary. I also do not like the error that caused the build to stop building.
Therefore, I will collect the package in my own way.
BugFix dpkg-buildpackage error occurs because in the version from the developer the debian package format is specified
version '3.0 (quilt)'. Change the format.
ceph@calamari:~/dev/calamari$ echo '3.0 (native)' > debian/source/format Putting the package together : Changing the debian / changelog file according to our name, email address, version, revision, release, and importance of the package.
ceph@calamari:~/dev/calamari$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" dch \ -v `./get-versions.sh VERSION`-`./get-versions.sh REVISION`-1 -D trusty -u low 'Switch to dpkg-source 3.0 (native) format' Check
ceph@calamari:~/dev/calamari$ head -5 debian/changelog calamari (1.2.1-100-ge0b9b21-1) trusty; urgency=low * Switch to dpkg-source 3.0 (native) format -- Your Name <your@email.com> Wed, 10 Dec 2014 17:05:10 +0300 After that, the package is assembled without errors, with the correct version, revision, release and importance. For obvious reasons, I do not provide the output of dpkg-buildpackage.
ceph@calamari:~/dev/calamari$ dpkg-buildpackage Check
ceph@calamari:~/dev/calamari$ ls -la .. total 12016 drwxrwxr-x 3 ceph ceph 4096 Dec 10 17:20 . drwxr-xr-x 5 ceph ceph 4096 Dec 10 17:11 .. drwxrwxr-x 20 ceph ceph 4096 Dec 10 17:11 calamari -rw-rw-r-- 1 ceph ceph 1396 Dec 10 17:20 calamari_1.2.1-100-ge0b9b21-1_amd64.changes -rw-rw-r-- 1 ceph ceph 750 Dec 10 17:11 calamari_1.2.1-100-ge0b9b21-1.dsc -rw-rw-r-- 1 ceph ceph 1278706 Dec 10 17:11 calamari_1.2.1-100-ge0b9b21-1.tar.gz -rw-r--r-- 1 ceph ceph 10998618 Dec 10 17:20 calamari-server_1.2.1-100-ge0b9b21-1_amd64.deb Go back to the parent directory
ceph@calamari:~/dev/calamari$ cd .. 3. Setting the environment and assembling the diamond
Important: If we build a package on a separate machine, do not forget about the implementation of the 1st paragraph of the article.
Install dependencies to build a diamond
sudo aptitude -y install python-mock cdbs python-support python-configobj Download the diamond repository.
mkdir -p ~/dev; cd ~/dev ceph@calamari:~/dev$ git clone https://github.com/ceph/Diamond.git --branch=calamari ceph@calamari:~/dev$ cd Diamond ceph@calamari:~/dev/Diamond$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" dch -v `./version.sh`-1 -D trusty \ -u low `/bin/echo -n "built on "; date` Check
ceph@calamari:~/dev/Diamond$ head -5 debian/changelog diamond (3.4.67-1) trusty; urgency=low * built on Wed Dec 10 17:57:02 MSK 2014 -- Your Name <your@email.com> Wed, 10 Dec 2014 17:57:02 +0300 We collect package
ceph@calamari:~/dev/Diamond$ dpkg-buildpackage Check
ceph@calamari:~/dev/Diamond$ ls -la .. total 4572 drwxrwxr-x 3 ceph ceph 4096 Dec 10 18:00 . drwxr-xr-x 4 ceph ceph 4096 Dec 10 17:54 .. drwxrwxr-x 11 ceph ceph 4096 Dec 10 18:00 Diamond -rw-r--r-- 1 ceph ceph 232292 Dec 10 18:00 diamond_3.4.67-1_all.deb -rw-rw-r-- 1 ceph ceph 1182 Dec 10 18:00 diamond_3.4.67-1_amd64.changes -rw-rw-r-- 1 ceph ceph 723 Dec 10 18:00 diamond_3.4.67-1.dsc -rw-rw-r-- 1 ceph ceph 4427329 Dec 10 18:00 diamond_3.4.67-1.tar.gz Go back to the parent directory
ceph@calamari:~/dev/Diamond$ cd .. 4. Setting up the environment for calamari-clients and building it
Important: If we build a package on a separate machine, do not forget about the implementation of the 1st paragraph of the article.
Add repositories with node.js :
echo "deb http://ppa.launchpad.net/chris-lea/node.js/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/nodejs.list sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B9316A7BC7917B12 Updating package lists:
sudo aptitude update Install dependencies to build calamari-clients:
sudo aptitude -y install ruby1.9.1 ruby1.9.1-dev python-software-properties nodejs Install dependencies using NPM and Gem:
sudo npm install -g bower@1.3.8 sudo npm install -g grunt-cli sudo gem install compass In some articles it is recommended to put a gem without sudo , but I swore for access rights to system directories. Stepping on another “rake”, set via sudo and the next line corrected the consequences of the root user
sudo chown -R ceph ~/.npm Warning: Installing dependencies via NPM and Gem , and the build of calamari-clients coincided with the blocking time of github. Be carefull. Without access to this resource you will catch a lot of problems. Perhaps, not solved without a full-fledged (not anonymous) proxy or vpn.
Download calamari-clients repository
mkdir -p ~/dev; cd ~/dev ceph@calamari:~/dev$ git clone https://github.com/ceph/calamari-clients.git ceph@calamari:~/dev$ cd calamari-clients Download the patch (in the parent directory)
ceph@calamari:~/dev/calamari-clients$ wget https://raw.githubusercontent.com/avssav/patches/master/calamari-clients/makefile.patch -qO \ ../makefile.patch In case the githab fall off again I will leave the diff file here. When copying from the screen, the diff file is crooked.
../makefile.patch
ceph@calamari:~/dev/calamari-clients$ echo 'diff -ruN a/Makefile b/Makefile --- a/Makefile 2014-12-03 10:13:32.486463458 +0300 +++ b/Makefile 2014-12-03 10:14:10.994462934 +0300 @@ -45,7 +45,7 @@ DATESTR=$(shell /bin/echo -n "built on "; date) set_deb_version: DEBEMAIL=$(DEBEMAIL) dch \ - --newversion $(VERSION)-$(REVISION)$(BPTAG) \ + --newversion $(VERSION)-$(REVISION)$(BPTAG) -u low \ -D $(DIST) --force-bad-version --force-distribution "$(DATESTR)" build:' > ../makefile.patch Apply patch:
ceph@calamari:~/dev/calamari-clients$ patch -ZEfsp1 < ../makefile.patch We collect calamari-clients deb-package.
ceph@calamari:~/dev/calamari-clients$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" DIST=trusty BPTAG=-1 REAL_BUILD=y make dpkg Check
ceph@calamari:~/dev/calamari-clients$ ls -la .. total 936 drwxrwxr-x 3 ceph ceph 4096 Dec 10 18:54 . drwxr-xr-x 8 ceph ceph 4096 Dec 10 18:45 .. drwxrwxr-x 11 ceph ceph 4096 Dec 10 18:41 calamari-clients -rw-r--r-- 1 ceph ceph 838592 Dec 10 18:54 calamari-clients_1.2.1.1-53-gddd7187-1_all.deb -rw-rw-r-- 1 ceph ceph 810 Dec 10 18:54 calamari-clients_1.2.1.1-53-gddd7187-1_amd64.changes -rw-rw-r-- 1 ceph ceph 92902 Dec 10 18:54 calamari-clients-make-dpkg.txt -rw-rw-r-- 1 ceph ceph 431 Dec 10 18:41 makefile.patch Go back to the parent directory
ceph@calamari:~/dev/calamari-clients$ cd .. 5. Installing calamari-server
I install the server on a separate machine. I carry out all subparagraphs of point 1, except the last (5).
UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Set the saltstack as follows
sudo aptitude -y install python-pip sudo pip install 'salt==2014.7.0' Setting salt-master until 2015.07 (leave for information)
Add repositories with saltstack :
Install the configuration management system:
echo "deb http://ppa.launchpad.net/saltstack/salt/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/saltstack.list sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4759FA960E27C0A6 sudo aptitude update Install the configuration management system:
sudo aptitude -y install salt-master salt-minion salt-syndic Installing dependencies for calamari-server
sudo aptitude install -y apache2 libapache2-mod-wsgi libcairo2 supervisor python-cairo libpq5 postgresql python-txamqp python-gevent \ python-sqlalchemy Install calamari-server and calamari-clients
sudo dpkg -i ./calamari-server_1.2.1-100-ge0b9b21-1_amd64.deb sudo dpkg -i calamari-clients_1.2.1.1-53-gddd7187-1_all.deb Calamari setup
ceph@calamari:~$ sudo calamari-ctl initialize [INFO] Loading configuration.. [INFO] Starting/enabling salt... [INFO] Starting/enabling postgres... [INFO] Initializing database... [INFO] Initializing web interface... [INFO] You will now be prompted for login details for the administrative user account. This is the account you will use to log into the web interface once setup is complete. Username (leave blank to use 'root'): Email address: your@email.com Password: Password (again): Superuser created successfully. [INFO] Starting/enabling services... [INFO] Restarting services... [INFO] Complete. 6. Add Ceph nodes to calamari monitoring
UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Set the saltstack as follows.
On all nodes
Install gdebi
sudo aptitude -y install gdebi-core Diamond and its settings
sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n diamond_3.4.67-1_all.deb sudo cp /etc/diamond/diamond.conf.example /etc/diamond/diamond.conf Salt-minion and its settings
wget https://github.com/avssav/packages/raw/master/saltstack/salt-common_2014.7.0%2Bds-2trusty1_all.deb wget https://github.com/avssav/packages/raw/master/saltstack/salt-minion_2014.7.0%2Bds-2trusty1_all.deb sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n salt-common_2014.7.0+ds-2trusty1_all.deb sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n salt-minion_2014.7.0+ds-2trusty1_all.deb echo "master: CALAMARI_SERVER_IP" | sudo tee /etc/salt/minion.d/calamari.conf sudo update-rc.d salt-minion defaults Setting salt-minion to 2015.07 (leave for information)
On all nodes
Add a salt-minion repository
Install dependencies to install diamond
Diamond and its settings
Salt-minion and its settings
Add a salt-minion repository
echo "deb http://ppa.launchpad.net/saltstack/salt/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/saltstack.list wget -q -O - "http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x4759FA960E27C0A6" | sudo apt-key add - sudo aptitude update Install dependencies to install diamond
sudo aptitude -y install python-support python-configobj Diamond and its settings
sudo dpkg -i diamond_3.4.67-1_all.deb sudo cp /etc/diamond/diamond.conf.example /etc/diamond/diamond.conf Salt-minion and its settings
sudo aptitude -y install salt-minion echo "master: CALAMARI_SERVER_IP" | sudo tee /etc/salt/minion.d/calamari.conf sudo update-rc.d salt-minion defaults Restart
sudo service salt-minion restart sudo service diamond restart 7. Authorization of the Ceph cluster in Calamari
After completing the previous item, salt-minion clients from Ceph machines will attempt to upgrade from the wizard.
On the machine with calamari-server and salt-master we perform
ceph@calamari:~$ sudo salt-key -L Accepted Keys: Unaccepted Keys: ceph1 ceph2 ceph3 ceph4 ceph5 Rejected Keys: We see the list of cluster machines. We allow them access
ceph@calamari:~$ sudo salt-key -A The following keys are going to be accepted: Unaccepted Keys: ceph1 ceph2 ceph3 ceph4 ceph5 Proceed? [n/Y] y Key for minion ceph1 accepted. Key for minion ceph2 accepted. Key for minion ceph3 accepted. Key for minion ceph4 accepted. Key for minion ceph5 accepted. 9. Everything should work
Login using the browser on http: // CALAMARI_SERVER_IP with the login and password specified during setup (calamari-ctl initialize).
10. Useful stuff
The synchronization of the diamond settings on the Ceph nodes occurs automatically. However, there is a command to update the settings manually.
sudo salt-call state.highstate The following command will check the connectivity of Ceph nodes with calamari-server
sudo salt-call ceph.get_heartbeats Check on the salt-master of all connected salt-minions
sudo salt '*' ceph.get_heartbeats Note: The article does not include some intermediate results. For example, a patch that eliminated an unpleasant error was excluded. But, checking the article, I once again cloned calamari from the github, applied my patch and the patch did not work. In general, as it turned out, while I was writing an article - calamari'vtsy fixed a bug in the main branch.
9. Materials
Compiling Calamari for Ceph on Ubuntu 14.04
Ceph Calamari: Step-by-Step
Build package and Install Calamari on Debian Weezy
Ceph Calamari: The Survival Guide
Calamari Documentation
Precise Build Instructions
Source: https://habr.com/ru/post/246277/
All Articles