Customize and understand Bacula

Sometimes when you wake up in the morning you clearly understand that something is wrong. Although you shaved and never even cut yourself, the coffee did not boil away, it’s sunny morning outside, got to work quickly and without incident, everything seems to be good, but all the same something is wrong. But when you enter the office, you see a general panic, hysterical cries, that everything is lost and "the entire office plankton" will die, and you are at the head of those who will die.
It turns out at night file and mail servers failed. And then you realize that it is not so simple that the morning began so well. There is enough work to be done, but the data is securely saved, because you took care of their backup.

In principle, the usual working situation, if there are backups of data. We will talk about the “on time made backup” today.
There are a lot of data backup systems, with open source - much less, and enterprise level, and even with open source, can be counted on the fingers, hands, or feet, as it is more convenient.
After a detailed study of the possibilities, a system with a discreet name Bacula was chosen.
Minimum list of requirements:
1.client-server architecture
2. the possibility of backup. installation of nix, win systems
3. Various backup options (full, differential, incremental)
4. opportunity to vary the type of backup from runtime
5.rotation backups
6. “sufficient” documentation
7. high reliability (the level of "you can not kill the stick")
Whoever hears for the first time about Bacula, I advise you to visit Wikipedia in order to have a minimal idea of ​​the "articles" topic.
For those who "mastered a lot of letters," but do not want to climb on Wikipedia cite a list of basic modules.
Bacula Director - the process controlling the system as a whole (management, planning, recovery of backups).
Storage Director - runs on the server responsible for the "physical" data storage.
File Director is a service launched on each of the clients.
Bconsole - management console.

The task: to set up a backup and data recovery system in such a way that: on server1 and server3 on Monday, a “full” backup was made, from Tuesday to Sunday, inclusive, diff. backups, and for server2 “full” backups were made all week.
Data must be stored for at least 3 weeks.
For example, take the configuration for backup and restore data from server3.

Start by setting up the Bacula Director
(config file: /etc/bacula/bacula-dir.conf. Location: server running bacula-director)
Director {
Name = backup-dir
Dirport = 9101
QueryFile = "/etc/bacula/scripts/query.sql" # sql
WorkingDirectory = "/var/lib/bacula"
PidDirectory = "/var/run/bacula"
Password = "some_password"
Messages = Daemon
DirAddress = 10.10.0.1
}

')
Due to the fact that the system intensively stores metadata in the database, we set up access to the mysql database.
Catalog {
Name = MyCatalog
dbname = bacula; DB Address = "10.10.0.1"; user = bacula; password = "some_password"
}

Configuring access to the management console
Console {
Name = backup-mon
Password = "some_password"
CommandACL = status, .status
}

Sending reports on the work done by mail to the administrator
Messages {
Name = Daemon
mailcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula daemon message\" %r"
mail = admin@domain.com = all, !skipped
console = all, !skipped, !saved
append = "/var/lib/bacula/log" = all, !skipped
}

Determine the server "storage"
Storage {
Name = stor_server1
Address = 10.10.0.2
SDPort = 9103
Password = "storage_pass"
Device = FileStorage
Media Type = File
}

We create schedules according to which we will perform copy or restore tasks.
According to the schedule on Monday, a “full” backup will be created, on the other days differential backups will be created. For example, in order to restore data for Friday, you need to deploy the backup task executed on Monday, and then “roll over” the Friday differential. backup
Schedule {
Name = "WeeklyDiff"
Run = Level=Full on mon at 05:01
Run = Level=Differential on tue-sun at 02:02
}

Schedule to create "full" backups. It is used for data, the loss of which will lead to a “complete” disappointment of the authorities in “your qualification” with entering into a personal matter, or “in the chest”, as you are lucky.
Schedule {
Name = "WeeklyFull"
Run = Level=Full on mon-sun at 03:03
}

Create a task for server3 backup
Job {
Name = "server3" #
Type = Backup # ( )
Level = Differential #
Client=server3-fd #
FileSet="server3" #
Storage = stor_server1 # «»
Pool = mainpool # «»()
Messages = Standard #
Schedule = "WeeklyDiff" #
}


Specify what exactly and how we will save from server3
FileSet {
Name = "server3"
Include {
Options {
signature = MD5 # MD5
Compression=GZIP # GZIP
}
File = /etc #
File = /home/
File = /var/www

}

Exclude { # ,
File = /home/logs
File = /var/www/logs
}
}

Description of client options for server3
Client {
Name = server3-fd
Address = 10.10.0.3
FDPort = 9102
Catalog = MyCatalog
Password = "fd_password
File Retention = 28 days #
#
Job Retention = 28 days # #
AutoPrune = yes #
}

In this section, we determine the rotation parameters.
Based on the configuration: we use 4 volumes, no more than 7 buildings are stored in the volume (weekly backup), the storage time of the volume is 3 weeks (21 days).
Thus, we store each task for 21 days, on day 22 we clear the volume with these tasks and use it again.
So, if we want to store data not 3 weeks, but 4, then Volume Retention should be not 21, but 28 and Maximum Volumes should be 5. And do not forget to create an additional volume.
Pool {
Name = mainpool
Pool Type = Backup
Recycle = yes #
AutoPrune = yes #
Volume Retention = 21 days # ""
Maximum Volume Jobs = 7 #
Maximum Volumes = 4 # #
}

,
Job {
Name = "server3-resotre"
Type = Restore
Client=server3
FileSet="server3"
Storage = stor_server1
Pool = mainpool
Messages = Standard
Where = /var/lib/bacula-restores
}


For the bacula-director, backup and server3 installation of this config is enough.
What we have done: on Monday, there is a “full” backup server, from Tuesday to Monday go diff. backups.
In the north, the “repository” for this task creates 4 volumes, each volume stores 7 tasks. (The volume is linked (created) with the label command)
By filling all 4 volumes, the oldest volume is cleared.
Next comes the storage server and client directories for server3

Description of settings for Bacula Storage Director
(config file: /etc/bacula/bacula-sd.conf. Location: server running bacula-sd (storage director))
Storage {
Name = stor_server1
SDPort = 9103
WorkingDirectory = "/var/lib/bacula"
Pid Directory = "/var/run/bacula"
SDAddress = 10.10.0.2
}
Director {
Name = backup-dir
Password = "storage_pass"
}

Device {
Name = FileStorage
Media Type = File
Archive Device = /var/bacula
LabelMedia = yes;
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
}

Messages {
Name = Standard
director = backup-dir = all
}


Description of settings for file-director on server3

(config file: /etc/bacula/bacula-fd.conf. Location: server running bacula-fd (server3))
Director {
Name = backup-dir
Password = "server3-fd"
}

FileDaemon {
Name = server3-fd
FDport = 9102
WorkingDirectory = /var/lib/bacula
Pid Directory = /var/run/bacula
FDAddress = 10.10.0.3
}

Messages {
Name = Standard
director = server3-fd = all, !skipped, !restored
}

I hope the comments in the configuration files of the server “storage” and the client on server3 are superfluous.

The result of this article may be an understanding of how to set up a flexible, reliable backup and recovery system with the discreet name Bacula.

P.S. For those who read the introduction, ask why I woke up in the morning I didn’t know that 2 servers had fallen at night, because no one had canceled the monitoring systems, and probably such important servers can be deployed in raid arrays.
The situation with the failure of two servers at once is fictional and is given solely for example.