Greetings to all the farmers!
As you might guess, it will be about backups.
Timely backup is an extremely important part of the work of the system administrator. A timely backup makes your sleep calm, and your nerves become steel, gives strength and protects your health.
I think it would be quite reasonable to assume that this topic has already turned sore, but still I would venture to share my experience. The client-server implementation of the backup scheme will be presented to the reader. As a tool, I chose the open source Bacula project. For more than half a year of experience in using it, I remain satisfied with my choice.
')
Bacula consists of several demons, each of which carries its own functional load. The figure below schematically shows the relationship of these demons.
Under habrakat I will describe all demons in detail
In my case, the backups are:- Configuration files of various daemons from all servers.
- MySQL database.
- Document management from a Windows file server.
- Various important data from nix servers (site / forum engines, etc ..)
1.Description of Bacula Demons
The system is built on client-server technology, and uses TCP for data transfer. Backups are created in their own, fully open format.
The Bacula data backup system consists of four main elements: Director Daemon, Storage Daemon, File Daemon and Bacula Console. All these elements are implemented as standalone applications.
Director Daemon (DD) is the central element of the system that manages its remaining components. Its tasks include managing the backup / restore process, providing a management interface for administrators, and more. Simply put, it is a dispatcher who initiates all processes and monitors their progress.
Storage Daemon (SD) is an application that is responsible for reading / writing data directly to storage devices. Accepts control commands from DD, as well as backup data from / to File Daemon.
File Daemon (FD) - this element can still be called the Agent. After all, it works within the operating system, the data of which must be backed up. File Daemon performs the entire routine by accessing the backed up files and their further transfer to SD. Also on the FD side, backup encryption is performed, if defined by the configuration.
Bacula Console (BC) - system administrator interface. At its core, it is a command interpreter for managing Bacula. Strictly speaking, the Bacula Console can be expanded with the help of graphical control systems, which, as a rule, are just a superstructure above BC. Such systems include Tray Monitor and Bat. The first one is installed on the system administrator’s computer and monitors the backup system operation, while the second provides the ability to control via a graphical interface.
Bacula Catalog is a database that stores information about all the reserved files and their location in backup copies. The directory is necessary to ensure effective addressing to the required files. MySql, PostgreSql and SqLite are supported.
This structural division allows you to organize a very flexible backup system when the Storage Daemon is deployed on a dedicated server with multiple storage devices. Also, Bacula Director can manage multiple instances of SD, providing backup of part of data to one storage device, and parts - on the other.
2. OS and iron
Now, when the reader has formed an idea of the work of the demons of Bacula, I will go on to describe how I realized all this beauty in myself.
As a hardware for DD, SD and Bacula Catalog I have a PC with the following characteristics
Device | Model | quantity | Capacity / Frequency |
HDD | Hitachi HDS723020BLA642 | 3 | 2Tb |
CPU | AMD Phenom (tm) II X4 970 Processor | one | 3500 Mhz |
Motherboard | Gigabyte GA-880GA-UD3H | one | - |
Ram | | | 3541 Mb |
About OS and Software Versions Used on the Server
Data storage is handled by several software (mdadm) RAID arrays.
Under the system, three partitions on three disks, you can boot from any of them, under the backups of one array of two partitions.
Array name | from which partitions | mount point | File system | Array level |
md0 | / dev / sda1, / dev / sdb1, / dev / sdc1 | boot | ext2 | one |
md1 | / dev / sda2, / dev / sdb2, / dev / sdc2 | / | ext3 | one |
md2 | / dev / sda3, / dev / sdb3 | / backup | ext4 | one |
3. Description of the backup scheme and settings of the Bacula daemons
I have a total of 19 Bacula clients configured, but I’ll discuss in detail the description of the backup of the billing server and documents from the Windows file server. The focus on these two servers is due to the fact that other clients are configured in a similar way, and you can build your configurations using the example of these server clients.
Backup server billing is, in fact, backup mysql database and configuration files of demons.
BD allows you to run a local script on the client both before and after the job.
Every night, at the start of the task on the backup server, a local script is launched (on the billing server itself), which creates an archive of the billing database, then this archive picks up the BD and places volumes on the corresponding pool (in fact, SD controls read / write operations, but it is not important now). Immediately after the task is completed, another script is launched, which in turn moves the created archive to a separate folder on the billing server, for greater reliability. Thus, the database archive will be both in Bacula and locally on the billing server (yes, I am paranoid). These mechanisms and scripts will be described in more detail below.
From the Windows file server, we save all the necessary workflow. On Sunday, a full backup is created, every next day, from Monday to Saturday, Diff backups.
Now about the configuration files of the Bacula daemons. Let's start with the most voluminous - bacula-dir.conf.
The configuration files of all Bacula daemons consist of descriptions of the so-called resources. Each of the resources describes a specific functional demon.
I will comment on every line in the config, so the Bacula file resource blocks (bacula-dir.conf, bacula-sd.conf, bacula-fd.conf) will follow, if something needs to be explained in more detail, indicate this in the comments.
Dirtector Resource
Director {
Resource catalog, describe the connection to the database
Catalog { Name = MyCatalog dbname = "bacula"; dbuser = "bacula"; dbpassword = "edsfweo8vhwpe" }
Resource Messages
Messages {
For each client in the tasks indicated Pool and Storage.
Pool, sorry for the tautology, this is a pool of volumes (volume) on which backup copies of customer data are placed. My volumes are bacula files located on the software raid array. Different pools of volumes can be defined for different clients. For example, I have created 6 pools for different types of clients. In the example below, only one of them is described, for billing data.
Storage describes which physical devices will be used as volumes.
(Storage BGB-ST is described in the SD config)
Pool resource
Pool {
Resource Storage
Storage {
Task for example backup database billing.
Resource Client
Client {
The task itself.
Resource Job
Job {
I promised to elaborate on the scripts running before and after the task.
Script to set
$ sudo cat /root/sh/before_bg_db_backup.sh
Script after the job
$ sudo cat /root/sh/after_bg_db_backup.sh
Resource FileSet (what we backup and what not)
FileSet { Name = "bgbilling-set" Include { Options {
Schedule launch tasks.
Resource Schedule
Schedule {
I will not comment on resources for backing up documents from a Windows server in detail, I’ll give the full part of the bacula-dir.conf config
Storage { Name = WINDOWS-F Address = backupsrv.domain.ru
The BD configuration file is now complete. Go to the SD configuration - the description of the file bacula-sd.conf
Resource Storage
Storage {
Resource Director (connection to BD described in the bacula-dir.conf config)
Director {
The description of various devices begins, in total 4 different devices are used by me. I will give as an example two, for billing and for Windows.
Resource Device for billing.
Device {
Resource Device for Windows File Server
Device { Name = WINDOWS-ST Media Type = File Archive Device = /backup/windows LabelMedia = yes; Random Access = Yes; AutomaticMount = yes; RemovableMedia = no; AlwaysOpen = no; }
Resource Messeges.
Messages {
Configuration file bconsole.conf, access to the console Bacula.
Director { Name = backupsrv.ray-com.ru-dir DIRport = 9101 address = 10.1.19.2 Password = "bacula_paS$w0rD10*" }
Do not forget to create the appropriate Storage folders and assign bacula the owner of these folders.Advice from the comments
@ / usr / local / etc / bacula / client.conf
@ / usr / local / etc / bacula / job.conf
@ / usr / local / etc / bacula / pool.conf
@ / usr / local / etc / bacula / fileset.conf
Configs can be divided into different files.
Options {signature = MD5 compression = GZIP}
and enable compression.
Configuring the server side is complete.
Config client
It is important to note that each of the clients must rezolvit fqdn server name in its ip address! Provide it with dns or write to the hosts!
Resource Director.
Director {
I mentioned in the comments of the configuration files about the correspondence scheme of passwords and daemon names in various configuration files, so if you get confused somewhere, use the image below.

4. Sample recovery procedure
To monitor and restore your backups, it is convenient to use the bat utility.
In ubuntu it is put so
I didn't find it in Gentoo Portage, so I collected it from sources.
The configuration file bat.conf is completely analogous to bconsole.conf given earlier.So, for example, I want to restore the archive of the billing database for a certain number. The algorithm that I use is as follows:
1. Open the bat and find the desired task.

2. enter the command list files jobid = 3059 to make sure that the task contains the necessary files

3. Now go to the console (it's easier for me just to =)). In the console, I will restore the billing archive to another client
$ sudo bconsole [sudo] password for onotole: Connecting to Director 10.1.19.2:9101 1000 OK: backupsrv.domain.ru-dir Version: 5.0.3 (30 August 2010) Enter a period to cancel a command. *restore Automatically selected Catalog: MyCatalog Using Catalog "MyCatalog" First you select one or more JobIds that contain files to be restored. You will be presented several methods of specifying the JobIds. Then you will be allowed to select which files from those JobIds are to be restored. To select the JobIds, you have the following choices: 1: List last 20 Jobs run 2: List Jobs where a given File is saved 3: Enter list of comma separated JobIds to select 4: Enter SQL list command 5: Select the most recent backup for a client 6: Select backup for a client before a specified time 7: Enter a list of files to restore 8: Enter a list of files to restore before a specified time 9: Find the JobIds of the most recent backup for a client 10: Find the JobIds for a backup for a client before a specified time 11: Enter a list of directories to restore for found JobIds 12: Select full restore to a specified Job date 13: Cancel Select item: (1-13): 9 Defined Clients: 1: 192.168.15.12-fd 2: 1.1.1.1-fd 3: 1.1.1.75-fd 4: ASTERISK-configs-fd 5: DHCPD-configs-fd 6: GW1-configs-fd 7: GW2-configs-fd 8: NAS-20-configs-fd 9: NAS-21-configs-fd 10: NAS-6-configs-fd 11: NAS-ololo-configs-fd 12: NS_AND_MAIL-configs-fd 13: RADIUS-ololo-configs-fd 14: VIRTSRV1-configs-fd 15: bgbilling-fd 16: configs-fd 17: domain.ru-fd 18: mydomain.ru-fd 19: tv.domain.ru-fd 20: zabbix.domain.ru-fd Select the Client (1-20): 15 Automatically selected FileSet: bgbilling-set +-------+-------+----------+----------------+---------------------+------------+ | JobId | Level | JobFiles | JobBytes | StartTime | VolumeName | +-------+-------+----------+----------------+---------------------+------------+ | 3,292 | F | 1,666 | 10,874,552,420 | 2011-12-19 02:31:08 | Vol0014 | +-------+-------+----------+----------------+---------------------+------------+ To select the JobIds, you have the following choices: 1: List last 20 Jobs run 2: List Jobs where a given File is saved 3: Enter list of comma separated JobIds to select 4: Enter SQL list command 5: Select the most recent backup for a client 6: Select backup for a client before a specified time 7: Enter a list of files to restore 8: Enter a list of files to restore before a specified time 9: Find the JobIds of the most recent backup for a client 10: Find the JobIds for a backup for a client before a specified time 11: Enter a list of directories to restore for found JobIds 12: Select full restore to a specified Job date 13: Cancel Select item: (1-13): 12 Enter JobId to get the state to restore: 3059 Selecting jobs to build the Full state at 2011-12-06 02:28:47 You have selected the following JobId: 3059 Building directory tree for JobId(s) 3059 ... +++++++++++++++++++++++++++++++++++++++++++++ 1,535 files inserted into the tree. You are now entering file selection mode where you add (mark) and remove (unmark) files to be restored. No files are initially added, unless you used the "all" keyword on the command line. Enter "done" to leave this mode. cwd is: / $ ls etc/ root/ usr/ $ ls usr usr/ $ mark usr 1,667 files marked. $ done Bootstrap records written to /backup/bacula-work//backupsrv.domain.ru-dir.restore.8.bsr The job will require the following Volume(s) Storage(s) SD Device(s) =========================================================================== Vol0010 BGB-F BGB-ST Volumes marked with "*" are online. 1,667 files selected to be restored. Run Restore job JobName: restore Bootstrap: /backup/bacula-work//backupsrv.domain.ru-dir.restore.8.bsr Where: /usr/restore Replace: always FileSet: bgbilling-set Backup Client: bgbilling-fd Restore Client: bgbilling-fd Storage: BGB-F When: 2011-12-26 15:01:38 Catalog: MyCatalog Priority: 10 Plugin Options: *None* OK to run? (yes/mod/no): mod Parameters to modify: 1: Level 2: Storage 3: Job 4: FileSet 5: Restore Client 6: When 7: Priority 8: Bootstrap 9: Where 10: File Relocation 11: Replace 12: JobId 13: Plugin Options Select parameter to modify (1-13): 5 The defined Client resources are: 1: bgbilling-fd 2: GW1-configs-fd 3: GW2-configs-fd 4: NAS-6-configs-fd 5: NAS-20-configs-fd 6: NAS-21-configs-fd 7: NAS-ololo-configs-fd 8: DHCPD-configs-fd 9: ASTERISK-configs-fd 10: NS_AND_MAIL-configs-fd 11: VIRTSRV1-configs-fd 12: mydomain.ru-fd 13: tv.domain.ru-fd 14: domain.ru-fd 15: 1.1.1.1-fd 16: 1.1.1.75-fd 17: zabbix.domain.ru-fd 18: 192.168.15.12-fd Select Client (File daemon) resource (1-18): 2 Run Restore job JobName: restore Bootstrap: /backup/bacula-work//backupsrv.ray-com.ru-dir.restore.8.bsr Where: /usr/restore Replace: always FileSet: bgbilling-set Backup Client: bgbilling-fd Restore Client: GW1-configs-fd Storage: BGB-F When: 2011-12-26 15:01:38 Catalog: MyCatalog Priority: 10 Plugin Options: *None* OK to run? (yes/mod/no): yes Job queued. JobId=3453 You have messages. *
4. We are waiting for the successful completion of the task, the status can be monitored in the same bat.

Some more screenshots



I thank everyone who read my opus to the end.
By way of completion I will allow myself some more tips.
It is important not only to make backups and keep track of what they have completed without errors, but also deploy and check them regularly. This practice gives another +100 to those indicated at the beginning of calmness, nerve endurance and health. It is also a very good practice to regularly back up the bacula database and bsr files.
Happy New Year to you !!!
Sources used:
1.
www.ibm.com/developerworks/ru/library/l-Backup_42.
www.bacula.org/en/?page=documentation