Configuring iSCSI initiator on linux
Abstract: how open-iscsi works (iSCSI initiator in linux), how to configure it and a little bit about the iSCSI protocol itself.
Lyrics: There are many articles on the Internet that explain quite well how to set up an ISCSI target, however, for some reason, there are practically no articles about working with the initiator. Despite the fact that the target is technically more complicated, the administrative fuss with initiator is more - there are more complicated concepts and not very obvious principles of work.
Before you talk about iSCSI - a few words about the different types of remote access to information in modern networks.
')
Explorer . At its core, it is 'scsi over tcp'. The SCSI protocol itself (without the letter 'i') is a very complex structure, since it can work through different physical environments (for example, UWSCSI - parallel bus, SAS - serial - but the protocol is the same). This protocol allows you to do much more than just “stick disks to a computer” (as invented in SATA), for example, it supports device names, the presence of several links between the block device and the consumer, switching support (yeah, SAS switch, such in nature), connecting several consumers to one block device, etc. In other words, this protocol was simply requested as the basis for a network block device.
target is the one who provides the block device. The closest analogue from the ordinary computer world is the server.
initiator - the client, the one who uses the block device. Analog client.
WWID - unique identifier of the device, its name. Analog DNS name.
LUN - the number of "piece" of the disk, which is accessed. The closest analogue is the partition on the hard disk.
ISCSI brings the following changes: WWID disappears, the concept of IQN (iSCSI Qualified Name) comes into its place - that is, a pure name that is similar to the extent of mixing with the DNS (with slight differences). Here is an example of IQN: iqn.2011-09.test: name.
IETD and open-iscsi (server and client under Linux) bring another very important concept, which is not often written about in the iscsi tutorials - portal. Portal is, to put it bluntly, several target'ov, which are announced by one server. There are no analogies with www, but if the web server could be asked to list all of its virtualhosts, that would be it. portal indicates the list of target'and available IP, which can be accessed (yes, iscsi supports multiple routes from initiator to target).
The article is not about target, so I give a very brief description of what target does. He takes the block device, sends the name and the LUN to it and publishes it on his portal, after which he allows everyone (authorization to taste) to access it.
Here is an example of a simple configuration file, I think it will be clear from it what the target is doing (configuration file using the IET example):
(complex from simple differs only in export options). Thus, if we have a target, then we want to connect it. And here begins the difficult, because initiator has its own logic, it is not at all like the trivial mount for nfs.
Open-iscsi is used as an initiator. So, the most important thing is that it has modes of operation and state . If we give the command in the wrong mode or if we do not take into account the state, the result will be extremely discouraging.
So, the modes of operation:
From this list, the life cycle is quite understandable - first find, then connect, then disconnect, then reconnect. Open-iscsi keeps the session open, even if the block device is not in use. Moreover, it keeps the session open (up to certain limits, of course), even if the server went into reboot. An iscsi session is not the same as an open TCP connection, iSCSI can transparently reconnect to the target. Disconnect / connect - operations that are controlled "outside" (either from other software or by hand).
A little about the state. After discovery, open-iscsi remembers all found target's (they are stored in / etc / iscsi /), in other words, discovery is a constant operation, NOT at all appropriate, for example, dns resolving). Found targets can be removed by hand (by the way, a common mistake is when in open-iscsi, as a result of experiments and settings, a bunch of target'es found, when trying to login to which many errors creep out due to the fact that half of target'ov are a config that no longer exists on the server, but I remember open-iscsi). Moreover, open-iscsi allows you to change the settings of the stored target, and this “memory” influences further work with the targets even after the daemon is rebooted / restarted.
The second question that torments many in the beginning is - where does it go after connecting? open-iscsi creates a network device, albeit a SCSI class (it’s not for nothing that I’m telling me), that is, it gets a letter in the / dev / sd family, for example, / dev / sdc. The first free letter is used, since For the rest of the system, this block device is a typical hard disk, no different from that connected via usb-sata or simply directly to sata.
This often causes a panic "how can I find out the name of the block device?". It is output in the detailed output of iscsiadm (# iscsiadm -m session -P 3).
Unlike SAS / UWSCSI, iSCSI is available for connection to just anyone. To protect against these, there is a login and password (chap), and their transfer to iscsiadm is another headache for novice users. It can be done in two ways - by changing the properties of the target that has already been found and writing the login / password in the open-iscsi configuration file.
The reason for such difficulties is that the password and the login process are attributes of the system, not the user. ISCSI is a cheap version of the FC infrastructure, and the concept of "user" in the context of the person behind the keyboard is not applicable here. If you have a sql-base on an iscsi block device, then of course, you would like the sql-server to be started by itself, and not after a minute of personal attention of the operator.
This is a very important file, because in addition to the login / password, it also describes the behavior of open-iscsi when finding errors. It may give the error "back" not immediately, but with a pause (for example, about five minutes, which is enough to reload the server with data). Also, the login process is controlled there (how many times to try, how long to wait between attempts) and any fine tuning of the work process itself. Note, these parameters are quite important for work and you need to understand how your iscsi will lead if you remove the power cord for 10-20s, for example.
I don’t like to quote easily accessible mana and lines, so I’ll give a typical iscsi usage scenario:
first we find the target we need, for this we need to know the IP / dns-name of the initiator:
Another important question in serious decisions is the support of several routes to the source. The beauty of iscsi is in using the usual ip, which can be processed in the usual way, like any other traffic (although in practice it is usually not routed, but only switched - the load is too great there). So, iscsi supports multipath in the “do not resist” mode. By itself, open-iscsi does not know how to connect to several IP of the same target. If it is connected to several IPs of the same target, this will lead to the appearance of several block devices.
However, the solution is - it is multipathd, which finds disks with the same identifier and processes them as expected in multipath, with customizable policies. This article is not about multipath, so I will not explain in detail the mystery of the process, however, here are some important points:
Lyrics: There are many articles on the Internet that explain quite well how to set up an ISCSI target, however, for some reason, there are practically no articles about working with the initiator. Despite the fact that the target is technically more complicated, the administrative fuss with initiator is more - there are more complicated concepts and not very obvious principles of work.
ISCSI
Before you talk about iSCSI - a few words about the different types of remote access to information in modern networks.
')
NAS vs SAN
There are two methods of accessing data located on another computer: file (when a file is requested from a remote computer, and which file systems do this - no one cares), characteristic representatives of NFS, CIFS (SMB); and block - when blocks are requested from a remote computer from disk media (in the same way as they are read from a hard disk). In this case, the requesting party itself does the file system on the block device, and the server giving the block device does not know about the file systems on it. The first method is called NAS (network attached storage), and the second is SAN (storage area network). The names generally point to other signs (SAN implies a dedicated network to the storage), but it so happens that the NAS is the file, and the SAN is the block device over the network. And although everyone (?) Understands that these are wrong names, the further, the more they are fixed.scsi over tcp
One of the protocols to access block devices is iscsi. The letter 'i' in the title does not refer to the production of Apple, but to InternetTerminology
The following terms are used in the SCSI world:target is the one who provides the block device. The closest analogue from the ordinary computer world is the server.
initiator - the client, the one who uses the block device. Analog client.
WWID - unique identifier of the device, its name. Analog DNS name.
LUN - the number of "piece" of the disk, which is accessed. The closest analogue is the partition on the hard disk.
ISCSI brings the following changes: WWID disappears, the concept of IQN (iSCSI Qualified Name) comes into its place - that is, a pure name that is similar to the extent of mixing with the DNS (with slight differences). Here is an example of IQN: iqn.2011-09.test: name.
IETD and open-iscsi (server and client under Linux) bring another very important concept, which is not often written about in the iscsi tutorials - portal. Portal is, to put it bluntly, several target'ov, which are announced by one server. There are no analogies with www, but if the web server could be asked to list all of its virtualhosts, that would be it. portal indicates the list of target'and available IP, which can be accessed (yes, iscsi supports multiple routes from initiator to target).
target
The article is not about target, so I give a very brief description of what target does. He takes the block device, sends the name and the LUN to it and publishes it on his portal, after which he allows everyone (authorization to taste) to access it.
Here is an example of a simple configuration file, I think it will be clear from it what the target is doing (configuration file using the IET example):
Target iqn.2011-09.example: data
IncomingUser username Pa $$ w0rd
Lun 0 Path = / dev / md1
(complex from simple differs only in export options). Thus, if we have a target, then we want to connect it. And here begins the difficult, because initiator has its own logic, it is not at all like the trivial mount for nfs.
Initiator
Open-iscsi is used as an initiator. So, the most important thing is that it has modes of operation and state . If we give the command in the wrong mode or if we do not take into account the state, the result will be extremely discouraging.
So, the modes of operation:
- Search target'ov (discovery)
- Connect to target
- Work with connected target
From this list, the life cycle is quite understandable - first find, then connect, then disconnect, then reconnect. Open-iscsi keeps the session open, even if the block device is not in use. Moreover, it keeps the session open (up to certain limits, of course), even if the server went into reboot. An iscsi session is not the same as an open TCP connection, iSCSI can transparently reconnect to the target. Disconnect / connect - operations that are controlled "outside" (either from other software or by hand).
A little about the state. After discovery, open-iscsi remembers all found target's (they are stored in / etc / iscsi /), in other words, discovery is a constant operation, NOT at all appropriate, for example, dns resolving). Found targets can be removed by hand (by the way, a common mistake is when in open-iscsi, as a result of experiments and settings, a bunch of target'es found, when trying to login to which many errors creep out due to the fact that half of target'ov are a config that no longer exists on the server, but I remember open-iscsi). Moreover, open-iscsi allows you to change the settings of the stored target, and this “memory” influences further work with the targets even after the daemon is rebooted / restarted.
Block device
The second question that torments many in the beginning is - where does it go after connecting? open-iscsi creates a network device, albeit a SCSI class (it’s not for nothing that I’m telling me), that is, it gets a letter in the / dev / sd family, for example, / dev / sdc. The first free letter is used, since For the rest of the system, this block device is a typical hard disk, no different from that connected via usb-sata or simply directly to sata.
This often causes a panic "how can I find out the name of the block device?". It is output in the detailed output of iscsiadm (# iscsiadm -m session -P 3).
Authorization
Unlike SAS / UWSCSI, iSCSI is available for connection to just anyone. To protect against these, there is a login and password (chap), and their transfer to iscsiadm is another headache for novice users. It can be done in two ways - by changing the properties of the target that has already been found and writing the login / password in the open-iscsi configuration file.
The reason for such difficulties is that the password and the login process are attributes of the system, not the user. ISCSI is a cheap version of the FC infrastructure, and the concept of "user" in the context of the person behind the keyboard is not applicable here. If you have a sql-base on an iscsi block device, then of course, you would like the sql-server to be started by itself, and not after a minute of personal attention of the operator.
configuration file
This is a very important file, because in addition to the login / password, it also describes the behavior of open-iscsi when finding errors. It may give the error "back" not immediately, but with a pause (for example, about five minutes, which is enough to reload the server with data). Also, the login process is controlled there (how many times to try, how long to wait between attempts) and any fine tuning of the work process itself. Note, these parameters are quite important for work and you need to understand how your iscsi will lead if you remove the power cord for 10-20s, for example.
Quick Reference
I don’t like to quote easily accessible mana and lines, so I’ll give a typical iscsi usage scenario:
first we find the target we need, for this we need to know the IP / dns-name of the initiator:
iscsiadm -m discovery -t st -p 192.168.0.1 -t st is the send targets command.iscsiadm -m node (list found for login)iscsiadm -m node -l -T iqn.2011-09.example:data (login, that is, connect and create a block device).iscsiadm -m session (display a list of what you are connected to)iscsiadm -m session -P3 (output it, but in more detail - at the very end of the output there will be an indication of which block device to which target belongs).iscsiadm - m session -u -T iqn.2011-09.example:data (log out of a particular)iscsiadm -m node -l (login to all detected target)iscsiadm -m node -u (log out of all target)iscsiadm -m node --op delete -T iqn.2011-09.example:data delete the target from the detected ones.mulitpath
Another important question in serious decisions is the support of several routes to the source. The beauty of iscsi is in using the usual ip, which can be processed in the usual way, like any other traffic (although in practice it is usually not routed, but only switched - the load is too great there). So, iscsi supports multipath in the “do not resist” mode. By itself, open-iscsi does not know how to connect to several IP of the same target. If it is connected to several IPs of the same target, this will lead to the appearance of several block devices.
However, the solution is - it is multipathd, which finds disks with the same identifier and processes them as expected in multipath, with customizable policies. This article is not about multipath, so I will not explain in detail the mystery of the process, however, here are some important points:
- When using multipath, you should set small timeouts - switching between bad paths should happen quickly enough
- Under more or less fast channel conditions (10G and higher, in many cases gigabit), parallelism of load should be avoided, since the opportunity to use bio coalesing is lost, which in some types of load can unpleasantly hit the target.
Source: https://habr.com/ru/post/97529/
All Articles