Configuring Jail on FreeBSD 11.1

Introduction

I was pushed to write this publication by the fact that there is very little information on Jail administration in FreBSD on the Internet. You can certainly find excellent publications on this topic, but they are mostly mostly written many years ago, and do not affect the new features of Jail and the FreeBSD operating system itself.
The publication is divided into two parts. The first part will cover the preparation and configuration of FreeBSD, and the second part will deal directly with the creation of Jail.

Part 1. Preparation and configuration of FreeBSD.

In order for everything that is written here to work correctly, you must use FreeBSD version 11.1, since starting with this version, support for limiting disk I / O, etc., has been included in the system. If this is not necessary, then version 10.X will do.
Add several parameters to rc.conf:

sysrc jail_enable="YES" sysrc rctl_enable="YES" sysrc rctl_rules="/etc/rctl.conf" sysrc zfs_enable="YES" sysrc ifconfig_em0_alias="192.168.1.105/24" 

The first line indicates Jail to start automatically with the system, the second line indicates the inclusion of restrictions for Jail, the third line indicates the rule file with restrictions. The fourth line activates the ability to use the ZFS file system (all Jail will be saved on ZFS partitions), this parameter is necessary if the system uses the native UFS file system. You can start ZFS with the command:

 /etc/rc.d/zfs start 

The fifth line creates an alias for Jail, if several Jails are necessary, then add the necessary number of ip aliases in that case (since I use VMware ESXI the name of the network card is em0 in me, you need to use the name of your card).
In FreeBSD, the kernel is compiled with the ability to limit resources disabled by default, but fortunately this restriction is easy to eliminate, just add one line to the loader.conf file with the command:
')

 echo 'kern.racct.enable="1"' >> /boot/loader.conf 

Changes will take effect after the system is rebooted. It is also necessary to enable support for the iscsi protocol, since the backup will be done through this protocol, you can add support by the following command:

 echo 'iscsi_initiator_load="YES"' >> /boot/loader.conf 

Changes will take effect after the system is rebooted.
The last thing you need to configure on the system (except for Jail itself) is the firewall on ipfw. The following command will create a file with ipfw rules:

 ee /etc/firewall.sc 

The following lines must be entered into this file:

 ipfw -q -f flush c="ipfw -q add " $c 00105 allow tcp from any to 192.168.1.105 80 setup keep-state $c 00110 allow tcp from any to me 22 setup keep-state $c 00140 allow tcp from me to any 443,80,21,53,3260 setup keep-state $c 00143 allow icmp from me to any keep-state $c 00144 allow udp from me to any 53 keep-state $c 40533 deny all from any to any frag $c 40534 deny all from any to any established $c 40535 deny all from any to any 

These rules will allow all Jail to make outgoing connections through ports 443, 80, 21, 53, 3260 (iscsi), and it will also be possible to connect to all Jail via SSH. Line:

 $c 00105 allow tcp from any to 192.168.1.105 80 setup keep-state 

Zack is responsible for connecting to the future Jail, and in particular for the web server, if you need to add other ports, enter them separated by commas (80,21,443,68, etc.). If you need a udp connection, then you need to copy the line and replace tcp with udp, and change the line number, and remove the setup, since the udp protocol does not have the SYN flag:

 $c 00105 allow tcp from any to 192.168.1.105 80,21,22,443 setup keep-state $c 00106 allow udp from any to 192.168.1.105 53 keep-state 

Execute the following commands in sequence:

 sysrc firewall_enable="YES" sysrc firewall_script="/etc/firewall.sc" service ipfw start 

After executing these commands, you will most likely need to reconnect over SSH. This completes the initial setup, let's move on to the settings of the iscsi target server.

Iscsi target setup
To configure, you need another server on the network, or a virtual machine (as in my case).
To configure the iscsi target, we will use ctld (included in FreeBSD), we will add an entry in rc.conf:

 sysrc ctld_enable="YES" 

In the next step, you need to create a configuration file for ctld:

 ee /etc/ctl.conf 

Add lines to the created file:

 auth-group group1 { chap "user" "password1234" } portal-group pg0 { discovery-auth-group group1 listen 192.168.1.106:3260 } target iqn.iscsi:target1 { alias "Example target" auth-group group1 portal-group pg0 lun 0 { path /dev/md0 size 10G } } 

In the chap line, specify the required name and password (minimum 12 characters). Title
in the target string must start at iqn. In the listen line, specify the ip address of the current server. In the path, specify the path to the disk.
Here is a virtual hard disk, if you use a physical one, then indicate it, and if you want to use a virtual one, then read on. Navigate to the directory in which you want to create the file for the virtual hard disk and execute the command:

 dd if=/dev/zero of=disk bs=1k count=10m 

The count parameter is responsible for the number of gigabytes, in this case a file of 10 gigabytes in size will be created, if you specify a different number, in this case you need to change the LUN 0 parameter in ctl.conf. Data process takes relatively little time. After the process is completed, a disk file will be created in the current folder, all that remains is to create a virtual hard disk with the command:

 mdconfig -a -t vnode -f disk 

After executing this command, the name of the virtual disk is displayed (in my case - md0), if the name is different, then you also need to change the LUN 0 parameter in ctl.conf. In order for this disk to not disappear after a reboot, you must run the command:

 sysrc mdconfig_md0="-a -t vnode -f disk" 

Or specify the path to the file:

 sysrc mdconfig_md0="-a -t vnode -f /home/user/disk" 

There is only one touch left - the firewall. As on the main system, create the file:

 ee /etc/firewall.sc 

Add lines:

 ipfw -q -f flush c="ipfw -q add " $c 00110 allow tcp from any to me 22,3260 setup keep-state $c 00140 allow tcp from me to any 443,80,21,53,3260 setup keep-state $c 00143 allow icmp from me to any keep-state $c 00144 allow udp from me to any 53 keep-state $c 40533 deny all from any to any frag $c 40534 deny all from any to any established $c 40535 deny all from any to any 

After saving the changes, enter:

 sysrc firewall_enable="YES" sysrc firewall_script="/etc/firewall.sc" service ipfw start 

This completes the iscsi server setup, now proceed to configure it directly Jail.

Part 2. Configuring Jail

Beginning with FreeBSD 9, the Jail configuration is moved to a separate file, /etc/jail.conf. Let's create this file and make the necessary changes, enter the command:

 ee /etc/jail.conf 

The following lines must be entered into this file:

  allow.raw_sockets = 1; exec.clean; exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; mount.devfs; allow.set_hostname = 1; allow.sysvipc = 1; jail1 { host.hostname = "jail"; path = "/jails/1/"; interface = "em0"; ip4.addr = 192.168.1.105; } 

After you save the changes, you can start building the Jail environment. According to this file, one Jail will be used with the name jail1; if it is necessary to use additional Jail, it is enough to change the file view as follows:

 allow.raw_sockets = 1; exec.clean; exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; mount.devfs; allow.set_hostname = 1; allow.sysvipc = 1; jail1 { host.hostname = "jail"; path = "/jails/1/"; interface = "em0"; ip4.addr = 192.168.1.105; } jail2 { host.hostname = "jail"; path = "/jails/2/"; interface = "em0"; ip4.addr = 192.168.1.107; } 

In this publication will be considered the creation of a single Jail. Create a directory for the future Jail team:

 mkdir -p /jails/1 

To create the environment, you need the “sources”, the easiest way is to install them during the installation process, or use subversion (a rather unpleasant process). To create an environment, go to the / usr / src directory, with the command:

 cd /usr/src 

To create an environment, enter the command:

 make -j4 world DESTDIR=/jails/1 

The process is quite long, on my system an Intel Core i5 3550 processor is installed, the creation of the environment took about an hour. In the environment creation command, the -j4 parameter is used, the number indicates the number of cores in the processor; the larger, the faster. After the environment is created, you need to add configuration files to jail with the command:

 make distribution DESTDIR=/jails/1 

At the creation of the environment is complete. Enter the command:

 /etc/rc.d/jail start 

There is no possibility to connect to the newly created Jail via ssh, since ssh is disabled. In order to enter jail, run the command:

 jexec jail1 

The first thing to do is add a DNS server, run:

 ee /etc/resolv.conf 

Add a line to the created file:

 nameserver 8.8.8.8 

It remains to add an account (add it to the wheel group), create a password for root and run ssh, all this can be done with the commands:

 adduser sysrc sshd_enable="YES" service sshd start passwd root 

I can recommend installing midnight commander:

 pkg install mc 

During the mc installation, many common dependencies will catch up, such as python, perl. After performing these manipulations, you must exit this jail with the exit command. Next, it stops jail:

 /etc/rc.d/jail stop 

Imagine the situation when you need to create 5 jails, such a task will take a lot of time, but fortunately you can create an archive with the contents of this jail, as well as retain all rights for the files. The tar archiver will help in this situation. Go to the directory with jail:

 cd /jails/1 

Run the command:

 tar -zcvpf jail.tar * 

After the archive is created, it must be moved to another directory (this directory will be deleted):

 mv jail.tar /jail.tar 

Removing the directory / jails will not succeed until the “flag” of “immutability” is removed from all files:

 chflags -R noschg /jails rm -rf /jails/ 

If something does not go away, then you just need to reboot the system and execute the rm -rf command again. Let's start creating a virtual hard disk, create a file for the disk:

 dd if=/dev/zero of=disk bs=1k count=10m 

And the virtual disk itself:

 mdconfig -a -t vnode -f disk 

Add automatic disk creation:

 sysrc mdconfig_md0="-a -t vnode -f disk" 

Specify the name of the disk you created (if the name is different), as well as the file path for the virtual disk. The next step is to automatically connect the drive through iscsi. For a correct connection to the disk, you must create a configuration file:

 ee /etc/iscsi.conf 

Make the following changes to this file:

 iscsi_disk{ authmethod=CHAP chapIName=user1 chapSecret=password1234 initiatorname=nxl TargetName=iqn.iscsi:target1 TargetAddress=192.168.1.106:3260,1 LoginTimeout=10 AuthTimeout=10 IdleTimeout=10 ConnFailTimeout=10 AbortTimeout=10 ResetTimeout=10 } 

If the settings in the ctl.conf file are set in accordance with this publication, then the connection will occur correctly. The only way to automatically connect the drive through iscsi I found only the way the script is placed in rc.d. Create this script:

 ee /etc/rc.d/iscsi.sc 

Add the following lines:

 iscontrol -c /etc/iscsi.conf -n iscsi_disk zfs mount jails/1 /etc/rc.d/jail start 

It is necessary to take into account the fact that if you have created another jail, then it must be added to this script (zfs mount jails / 2 for example). The first line connects the disk via iscsi, the second line mounts the file system (if the local hard disk “falls off”), the third line starts jail. It remains only to make the file executable:

 chmod +x /etc/rc.d/iscsi.sc 

After executing this script, or restarting the system, the remote hard disk will be available for manipulation. In my cases, the name of the disk is da1, you also need to use the name that you use. Create a zfs pool of these two disks:

 zpool create jails mirror md0 da1 

zfs pool will be a mirror, as you might guess from the team.
Create a section for jail:

 zfs create jails/1 

Let's set a limit for this directory to 5 gigabytes:

 zfs set quota=5g jails/1 

Copy the archive from jail to the directory / jails / 1 and move to this directory:

 cp /jail.tar /jails/1 cd /jails/1 

Unpack this archive and delete it:

 tar -zxvpf jail.tar rm jail.tar 

Run jail:

 /etc/rc.d/jail start 

After these manipulations, you can connect to the jail via SSH, as well as install the necessary roles for the server. It remains only to configure the restrictions for Jail. In order to configure rctl, you only need to add a configuration file:

 ee /etc/rctl.conf 

Add the following lines to this file:

 jail:jail1:memoryuse:deny=1073741824 jail:jail1:readbps:throttle=4097152 jail:jail1:writebps:throttle=4097152 jail:jail1:pcpu:deny=50 

The 1st line will limit the use of 1 gigabyte of memory, the 2nd and 3rd lines will limit the use of reading and writing to 4 megabytes of disk, the 4th line will limit the use of each core to 50 percent. This is not the entire list of restrictions; in the list of sources I will indicate a link to the FreeBSD website where this is described in detail. After saving this file, you must restart rctl:

 service rctl restart 

The restrictions will take effect in just a few seconds.

Additionally

It would be possible to finish this, but a situation may arise such that not the hard disk will fail, but the server itself (for example, it will burn). In such cases, you can use a disk that is located on a remote server, but you just won't be able to use it, first of all you need to stop the ctld service and enable zfs:

 service ctld stop sysrc zfs_enable="YES" /etc/rc.d/zfs start 

After that, enter the command:

 zpool import 

After executing this command, all possible pools that can be imported are displayed on the screen, in this case the jails pool, and the drive name will be displayed as md0. In order to mount this pool, you must run the command:

 zpool import -f jails jails 

Be sure to specify -f, otherwise zpool will swear that the pool belongs to another server. If necessary, you can also configure jail on this server using this pool, which in turn will keep idle time to a minimum. In order to use this disk for iscsi again, you need to disable this pool:

 zpool export jails 

And also run ctld:

 service ctld start 

At this point you can finish.

The list of sources that have greatly helped in writing this publication:
Michael Lucas FreeBSD. Detailed guide.
www.freebsd.org/cgi/man.cgi?query=rctl&sektion=8
www.freebsd.org/doc/ru_RU.KOI8-R/books/handbook/disks-adding.html
docs.oracle.com/cd/E19253-01/820-0836/gavwn/index.html
www.freebsd.org/cgi/man.cgi?query=ctl.conf&sektion=5&apropos=0&manpath=FreeBSD+11.1-RELEASE+and+Ports