ISCSI
iSCSI is a protocol standard for the consolidation of storage data. iSCSI allows FreeNAS® to act like a storage area network (SAN) over an existing Ethernet network. Specifically, it exports disk devices over an Ethernet network that iSCSI clients (called initiators) can attach to and mount. Traditional SANs operate over fibre channel networks which require a fibre channel infrastructure such as fibre channel HBAs, fibre channel switches, and discrete cabling. iSCSI can be used over an existing Ethernet network, although dedicated networks can be built for iSCSI traffic in an effort to boost performance. iSCSI also provides an advantage in an environment that uses Windows shell programs; these programs tend to filter "Network Location" but iSCSI mounts are not filtered. FreeNAS® uses istgt to provide iSCSI.
FreeNAS® supports multiple iSCSI drives. When configuring multiple iSCSI LUNs, create a new target for each LUN. Portal groups and initiator groups can be reused without any issue. Since istgt multiplexes a target with multiple LUNs over the same TCP connection, you will experience contention from TCP if there is more than one target per LUN.
Before configuring the iSCSI service, you should be familiar with the following iSCSI terminology:
CHAP: an authentication method which uses a shared secret and three-way authentication to determine if a system is authorized to access the storage device and to periodically confirm that the session has not been hijacked by another system. In iSCSI, the initiator (client) performs the CHAP authentication.
Mutual CHAP: a superset of CHAP in that both ends of the communication authenticate to each other.
Initiator: a client which has authorized access to the storage data on the FreeNAS® system. The client requires initiator software to connect to the iSCSI share.
Target: a storage resource on the FreeNAS® system.
Extent: the storage unit to be shared. It can either be a file or a device.
LUN: stands for logical unit number and represents a logical SCSI device. An initiator negotiates with a target to establish connectivity to a LUN; the result is an iSCSI connection that emulates a connection to a SCSI hard disk. Initiators treat iSCSI LUNs the same way as they would a raw SCSI or IDE hard drive; rather than mounting remote directories, initiators format and directly manage filesystems on iSCSI LUNs.
In order to configure iSCSI:
1. Decide if you will use authentication, and if so, whether it will be CHAP or mutual CHAP. If using authentication, create an authorized access.
2. Create either a device extent or a file extent to be used as storage.
3. Determine which hosts are allowed to connect using iSCSI and create an initiator.
4. Create at least one portal.
5. Review the target global configuration parameters.
6. Create a target.
7. Associate a target with an extent.
8. Start the iSCSI service in Services → Control Services.
The rest of this section describes these steps in more detail.
Contents |
Authorized Accesses
If you will be using CHAP or mutual CHAP to provide authentication, you must create an authorized access in Services → ISCSI → Authorized Accesses → Add Authorized Access. This screen is shown in Figure 8.7a.
NOTE: this screen sets login authentication. This is different from discovery authentication which is set in Target Global Configuration.
Figure 8.7a: Adding an iSCSI Authorized Access
Table 8.7a summarizes the settings that can be configured when adding an authorized access:
Table 8.7a: Authorized Access Configuration Settings
| Setting | Value | Description |
| Group ID | integer | allows different groups to be configured with different authentication profiles; for instance, all users with a Group ID of 1 will inherit the authentication profile associated with Group 1 |
| User | string | name of user account that will be created on the FreeNAS® device for CHAP authentication with the user on the remote system; many initiators default to using the initiator name as the user |
| Secret / Secret (Confirm) | string | password to be associated with User; the iSCSI standard requires that this be at least 12 characters long |
| Peer User | string | only input when configuring mutual CHAP; in most cases it will need to be the same value as User |
| Initiator Secret / Initiator Secret (Confirm) | string | the mutual secret password which must be different than the Secret; required if the Peer User is set |
NOTE: CHAP does not work with GlobalSAN initiators on Mac OS X.
As authorized accesses are added, they will be listed under View Authorized Accesses. In the example shown in Figure 8.7b, three users (test1, test2, and test3) and two groups (1 and 2) have been created, with group 1 consisting of one CHAP user and group 2 consisting of one mutual CHAP user and one CHAP user. Each authorized access entry provides an Edit and a Delete button.
Figure 8.7b: Viewing Authorized Accesses
Extents
In iSCSI, the target virtualizes something and presents it as a device to the iSCSI client. That something can be a device extent or a file extent:
Device extent: virtualizes an unformatted physical disk, RAID controller, zvol, zvol snapshot, or an existing HAST device.
Virtualizing a single disk is slow as there is no caching but virtualizing a hardware RAID controller has higher performance due to its cache. This type of virtualization does a pass-through to the disk or hardware RAID controller. None of the benefits of ZFS are provided and performance is limited to the capabilities of the disk or controller.
Virtualizing a zvol adds the benefits of ZFS such as its read cache and write cache. Even if the client formats the device extent with a different filesystem, as far as FreeNAS® is concerned, the data benefits from ZFS features such as block checksums and snapshots.
File extent: allows you to export a portion of a ZFS volume. When creating a file extent, you can specify either a non-existing file name or an existing ZFS dataset. The advantage of a file extent is that you can create multiple exports per volume.
In theory, a zvol and a file extent should have identical performance. In practice, a file extent outperforms in reads/writes but this is only noticeable at 10 GB Ethernet speeds or higher. For high performance, file extents are recommended at this time. Future changes to FreeBSD's zvol code will increase its performance.
Adding a Device Extent
To add a device extent, go to Services → ISCSI → Device Extents → Add Device Extent. In the example shown in Figure 8.7c, the device extent is using the export zvol that was previously created from the /mnt/volume1 volume.
NOTE: in FreeNAS® versions prior to 8.3.1, if a physical disk was used instead of a zvol to create a device extent, a bug wiped the partition table on the disk, resulting in data loss. This bug was fixed in 8.3.1.
Figure 8.7c: Adding an iSCSI Device Extent
Table 8.7b summarizes the settings that can be configured when creating a device extent:
Table 8.7b: Device Extent Configuration Settings
| Setting | Value | Description |
| Extent Name | string | required |
| Comment | string | optional |
| Disk device | drop-down menu | select the unformatted disk, controller, zvol, zvol snapshot, or HAST device |
Adding a File Extent
File extents are created in Services → ISCSI → File Extents → Add File Extent. In the example shown in Figure 8.7d, a file extent named data with a maximum size of 20 GB will be created on the ZFS dataset /mnt/volume1. Note that file extent creation will fail if you do not append the name of the file to be created to the volume/dataset name.
Figure 8.7d: Adding an iSCSI File Extent
Table 8.7c summarizes the settings that can be configured when creating a file extent:
Table 8.7c: File Extent Configuration Settings
| Setting | Value | Description |
| Extent Name | string | name of file extent; if the Extent size is not 0, it can not be an existing file within the volume/dataset |
| Path to the extent | browse button | either browse to an existing file and use 0 as the Extent size, or browse to the volume or dataset, click the Close button, append the Extent name to the path, and specify a value in Extent size |
| Extent size | integer | if the size is specified as 0, the file must already exist and the actual file size will be used; otherwise specifies the size of the file to create |
| Comment | string | optional |
Initiators
The next step is to configure authorized initiators, or the systems which are allowed to connect to the iSCSI targets on the FreeNAS® system. To configure which systems can connect, use Services → ISCSI → Initiators → Add Initiator, shown in Figure 8.7e:
Figure 8.7e: Adding an iSCSI Initiator
NOTE: beginning with 8.2.0, FreeNAS® contains iscontrol(8). This utility allows the FreeNAS® system to act as an initiator (rather than a target) and must be run from the command line. If you create a custom configuration for iscontrol, back it up as it will not survive a reboot of the system.
Table 8.7d summarizes the settings that can be configured when adding an initiator:
Table 8.7d: Initiator Configuration Settings
| Setting | Value | Description |
| Initiators | string | use ALL keyword or a list of initiator hostnames separated by commas with no space |
| Authorized network | string | use ALL keyword or a network address with CIDR mask such as 192.168.2.0/24 |
| Comment | string | optional description |
In the example shown in Figure 8.7f, two groups have been created. Group 1 allows connections from any initiator on any network; Group 2 allows connections from any initiator on the 10.10.1.0/24 network.
Figure 8.7f: Sample iSCSI Initiator Configuration
NOTE: if you delete an initiator, a warning will indicate if any targets or target/extent mappings depend upon the initiator. If you confirm the delete, these will be deleted as well.
Portals
A portal specifies the IP address and port number to be used for iSCSI connections. Services → ISCSI → Portals → Add Portal will bring up the screen shown in Figure 8.7g:
Figure 8.7g: Adding an iSCSI Portal
Table 8.7e summarizes the settings that can be configured when adding a portal. If you need to assign additional IP addresses to the portal, click the link “Add extra Portal IP”.
Table 8.7e: Portal Configuration Settings
| Setting | Value | Description |
| Comment | string | optional description; portals are automatically assigned a numeric group ID |
| Portal IP address | drop-down menu | select the IP address associated with an interface or the wildcard address of 0.0.0.0 (any interface) |
| Port | integer | TCP port used to access the iSCSI target; default is 3260 |
FreeNAS® systems with multiple IP addresses or interfaces can use a portal to provide services on different interfaces or subnets. This can be used to configure multi-path I/O (MPIO). MPIO is more efficient than a link aggregation.
If the FreeNAS® system has multiple configured interfaces, portals can also be used to provide network access control. For example, consider a system with four interfaces configured with the following addresses:
192.168.1.1/24
192.168.2.1/24
192.168.3.1/24
192.168.4.1/24
You could create a portal containing the first two IP addresses (group ID 1) and a portal containing the remaining two IP addresses (group ID 2). You could then create a target named A with a Portal Group ID of 1 and a second target named B with a Portal Group ID of 2. In this scenario, istgt would listen on all four interfaces, but connections to target A would be limited to the first two networks and connections to target B would be limited to the last two networks.
Another scenario would be to create a portal which includes every IP address except for the one used by a management interface. This would prevent iSCSI connections to the management interface.
Target Global Configuration
Services → iSCSI → Target Global Configuration, shown in Figures 8.7h, contains settings that apply to all iSCSI shares.
Figure 8.7h: iSCSI Target Global Configuration Variables
Table 8.7f summarizes the settings that can be configured in the Target Global Configuration screen. The integer values in the table are used to tune network performance; most of these values are described in RFC 3720.
LUC (Logical Unit Controller) is an API provided by istgt to control removable media by providing functions to list targets, load or unload a media to a unit, change media file, or reset a LUN.
In order to dynamically add or remove targets without restarting the iSCSI service, which can disrupt iSCSI initiators, set the following options:
- check the Enable LUC box
- leave the Controller IP address and Control Authorized Network at their default values
- change the Controller Auth Method to None
NOTE: the following operations will not take affect until you restart the iSCSI service: changing a target on the fly, adding or deleting LUNs, or changing the size of an existing extent.
Table 8.7f: Target Global Configuration Settings
| Setting | Value | Description |
| Base Name | string | see the “Constructing iSCSI names using the iqn. format” section of RFC 3721 if you are unfamiliar with this format |
| Discovery Auth Method | drop-down menu | configures the authentication level required by the target for discovery of valid devices, where None will allow anonymous discovery, CHAP and Mutual CHAP require authentication, and Auto lets the initiator decide the authentication scheme |
| Discovery Auth Group | drop-down menu | depends on Discovery Auth Method setting: required if set to CHAP or Mutual CHAP, optional if set to Auto, and not needed if set to None |
| I/O Timeout | integer representing seconds | sets the limit on how long an I/O can be outstanding before an error condition is returned; values range from 0-300 with a default of 30 |
| NOPIN Interval | integer representing seconds | how often the target sends a NOP-IN packet to keep a discovered session alive; values range from 0-300 with a default of 20 |
| Max. Sessions | integer | limits the number of sessions the target portal will create/accept from initiator portals; values range from 1-64 with a default of 16 |
| Max. Connections | integer | the number of connections a single initiator can make to a single target; values range from 1-64 with a default of 8 |
| Max. pre-send R2T | integer | values range from 1-255 with a default of 32 |
| MaxOutstandingR2T | integer | the maximum number of ready to receive packets (R2Ts) the target can have outstanding for a single iSCSI command, where larger values should yield performance increases until MaxOutstandingR2T exceeds the size of the largest Write I/O divided by MaxBurstLength; values range from 1-255 with a default of 16 |
| First burst length | integer | maximum amount in bytes of unsolicited data an iSCSI initiator may send to the target during the execution of a single SCSI command; values range from 1- 2^32 with a default of 65,536 |
| Max burst length | integer | maximum write size in bytes the target is willing to receive between R2Ts; values range from 1-2^32 with a default of 262,144 |
| Max receive data segment length | integer | in bytes; values range from 1-2^32 with a default of 262,144 |
| DefaultTime2Wait | integer | minimum time in seconds to wait before attempting a logout or an active task reassignment after an unexpected connection termination or reset; values range from 1-300 with a default of 2 |
| DefaultTime2Retain | integer | maximum time in seconds after Time2Wait before which an active task reassignment is still possible after an unexpected connection termination or reset; values range from 1-300 with a default of 60 |
| Enable LUC | checkbox | check if you need to dynamically add and remove targets; if checked, the next three fields are activated and required |
| Controller IP address | IP address | keep the default value of 127.0.0.1 |
| Controller TCP port | integer | possible values range from 1024-65535 with a default value of 3261 |
| Controller Authorized netmask | subnet mask | keep the default value of 127.0.0.0/8 |
| Controller Auth Method | drop-down menu | choices are None, Auto, CHAP, or Mutual CHAP; use None if Enable LUC is checked |
| Controller Auth Group | drop-down menu | required if Controller Auth Method is set to CHAP or Mutual CHAP, optional if set to Auto, and not needed if set to None |
If the settings in this screen differ from the settings on the initiator, set them to be the same. When making changes, always match the larger setting.
If you are changing integer values to optimize the connection, refer to the iSCSI initiator's documentation. For example, the following modifications are recommended if the iSCSI initiator is running on Xenserver:
- Max. pre-send R2T: 255
- MaxOutstandingR2T: 64
- First burst length: 262,144
- Max burst length: 2,097,152
Targets
Next, create a Target using Services → ISCSI → Targets → Add Target, as shown in Figure 8.7i. A target combines a portal ID, allowed initiator ID, and an authentication method.
NOTE: an iSCSI target creates a block device that may be accessible to multiple initiators. A clustered filesystem is required on the block device, such as VMFS used by VMWare ESX/ESXi, in order for multiple initiators to mount the block device read/write. If a traditional filesystem such as EXT, XFS, FAT, NTFS, UFS, or ZFS is placed on the block device, care must be taken that only one initiator at a time has read/write access or the result will be filesystem corruption. If you need to support multiple clients to the same data on a non-clustered filesystem, use CIFS or NFS instead of iSCSI or create multiple iSCSI targets (one per client).
Figure 8.7i: Adding an iSCSI Target
Table 8.7g summarizes the settings that can be configured when creating a Target:
Table 8.7g: Target Settings
| Setting | Value | Description |
| Target Name | string | required value; base name will be appended automatically if it does not start with iqn |
| Target Alias | string | optional user-friendly name |
| Serial | string | unique ID for target to allow for multiple LUNs; the default is generated from the system's MAC address |
| Target Flags | drop-down menu | choices are read-write or read-only |
| Portal Group ID | drop-down menu | leave empty or select number of existing portal to use |
| Initiator Group ID | drop-down menu | select which existing initiator group has access to the target |
| Auth Method | drop-down menu | choices are None, Auto, CHAP, or Mutual CHAP |
| Authentication Group number | drop-down menu | None or integer representing number of existing authorized access |
| Queue Depth | integer | see this post for an explanation of the math involved; values are 0-255 where 0 is disabled and default is 32 |
| Logical Block Size | integer | should only be changed to emulate a physical disk's size or to increase the block size to allow for larger filesystems on an operating system limited by block count; default is 512 |
Targets/Extents
The last step is associating an extent to a target within Services → iSCSI → Targets/Extents → Add Target/Extent. This screen is shown in Figure 8.7j. Use the drop-down menus to select the existing target and extent.
Figure 8.7j: Associating iSCSI Targets/Extents
Table 8.7h summarizes the settings that can be configured when associating targets and extents:
Table 8.7h: Target/Extents Configuration Settings
| Setting | Value | Description |
| Target | drop-down menu | select the pre-created target |
| Extent | drop-down menu | select the pre-created extent |
It is recommended to always associate extents to targets in a 1:1 manner, even though the software will allow multiple extents to be associated with the same target.
Once iSCSI has been configured, don't forget to start it in Services → Control Services. Click the red OFF button next to iSCSI. After a second or so, it will change to a blue ON, indicating that the service has started.
In order to access the iSCSI target, clients will need to use iSCSI initiator software.
An iSCSI Initiator client is pre-installed with Windows 7. A detailed how-to for this client can be found here. A client for Windows 2000, XP, and 2003 can be found here. This how-to shows how to create an iSCSI target for a Windows 7 system.
Mac OS X does not include an initiator. globalSAN is a commercial, easy-to-use Mac initiator.
BSD systems provide command line initiators: iscontrol(8) comes with FreeBSD, iscsi-initiator(8) comes with NetBSD, and iscsid(8) comes with OpenBSD.
Some Linux distros provide the command line utility iscsiadm from Open-iSCSI. Use a web search to see if a package exists for your distribution should the command not exist on your Linux system.
Instructions for connecting from a VMware ESXi Server can be found at How to configure FreeNAS® 8 for iSCSI and connect to ESX(i). Note that the requirements for booting vSphere 4.x off iSCSI differ between ESX and ESXi. ESX requires a hardware iSCSI adapter while ESXi requires specific iSCSI boot firmware support. The magic is on the booting host side, meaning that there is no difference to the FreeNAS® configuration. See the iSCSI SAN Configuration Guide for details.
If you can see the target but not connect to it, check the discovery authentication settings in Target Global Configuration.
If the LUN is not discovered by ESXi, make sure that promiscuous mode is set to Accept in the vswitch.
To determine which initiators are connected, type istgtcontrol info within Shell.
Growing LUNs
The method used to grow the size of an existing iSCSI LUN depends on whether the LUN is backed by a file extent or a zvol. Both methods are described in this section.
After the LUN is expanded using one of the methods below, use the tools from the initiator software to grow the partitions and the filesystems it contains.
Zvol Based LUN
Before growing a zvol based LUN, make sure that all initiators are disconnected. Stop the iSCSI service in Control Services.
Open Shell and identify the zvol to be grown:
zfs list -t volume NAME USED AVAIL REFER MOUNTPOINT tank/iscsi_zvol 4G 17.5G 33.9M -
Then, grow the zvol. This example grows tank/iscsi_zvol from 4G to 6G:
zfs set volsize=6G tank/iscsi_zvol zfs set refreservation=6G tank/iscsi_zvol
Verify that the changes have taken effect:
zfs list -t volume NAME USED AVAIL REFER MOUNTPOINT tank/iscsi_zvol 6G 17.5G 33.9M -
You can now start the iSCSI service and allow initiators to connect.
File Extent Based LUN
Before growing a file extent based LUN, make sure that all initiators are disconnected. Stop the iSCSI service in Control Services.
Then, go to Services → iSCSI → File Extents → View File Extents to determine the path of the file extent to grow. Open Shell to grow the extent. This example grows /mnt/volume1/data by 2G:
truncate -s +2g /mnt/volume1/data
Go back to Services → iSCSI → File Extents → View File Extents and click the Edit button for the file extent. Set the size to 0 as this causes the iSCSI target to use the existing size of the file.
You can now start the iSCSI service and allow initiators to connect.
