FreeNAS® is © 2011-2017 iXsystems

FreeNAS® and the FreeNAS® logo are registered trademarks of iXsystems

FreeBSD® is a registered trademark of the FreeBSD Foundation

Written by users of the FreeNAS® network-attached storage operating system.

Version 11.1

Copyright © 2011-2017 iXsystems

This Guide covers the installation and use of FreeNAS® 11.1.

The FreeNAS® User Guide is a work in progress and relies on the contributions of many individuals. If you are interested in helping us to improve the Guide, read the instructions in the README. IRC Freenode users are welcome to join the #freenas channel where you will find other FreeNAS® users.

The FreeNAS® User Guide is freely available for sharing and redistribution under the terms of the Creative Commons Attribution License. This means that you have permission to copy, distribute, translate, and adapt the work as long as you attribute iXsystems as the original source of the Guide.

FreeNAS® and the FreeNAS® logo are registered trademarks of iXsystems.

Active Directory® is a registered trademark or trademark of Microsoft Corporation in the United States and/or other countries.

Apple, Mac and Mac OS are trademarks of Apple Inc., registered in the U.S. and other countries.

Broadcom is a trademark of Broadcom Corporation.

Chelsio® is a registered trademark of Chelsio Communications.

Cisco® is a registered trademark or trademark of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

Django® is a registered trademark of Django Software Foundation.

Facebook® is a registered trademark of Facebook Inc.

FreeBSD® and the FreeBSD® logo are registered trademarks of the FreeBSD Foundation®.

Intel, the Intel logo, Pentium Inside, and Pentium are trademarks of Intel Corporation in the U.S. and/or other countries.

LinkedIn® is a registered trademark of LinkedIn Corporation.

Linux® is a registered trademark of Linus Torvalds.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates.

Twitter is a trademark of Twitter, Inc. in the United States and other countries.

UNIX® is a registered trademark of The Open Group.

VirtualBox® is a registered trademark of Oracle.

VMware® is a registered trademark of VMware, Inc.

Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

Windows® is a registered trademark of Microsoft Corporation in the United States and other countries.

Typographic Conventions

The FreeNAS® 11.1 User Guide uses these typographic conventions:

Table 1 Text Format Examples
Item Visual Example
Graphical elements: buttons, icons, fields, columns, and boxes Click the Import CA button.
Menu selections Select System Information.
Commands Use the scp command.
File names and volume and dataset names Locate the /etc/rc.conf file.
Keyboard keys Press the Enter key.
Important points This is important.
Values entered into fields, or device names Enter 127.0.0.1 in the address field.

1. Introduction

FreeNAS® is an embedded open source network-attached storage (NAS) operating system based on FreeBSD and released under a 2-clause BSD license. A NAS has an operating system optimized for file storage and sharing.

FreeNAS® provides a browser-based, graphical configuration interface. The built-in networking protocols provide storage access to multiple operating systems. A plugin system is provided for extending the built-in features by installing additional software.

1.1. New Features in 11.1

FreeNAS® 11.1 is a feature release, which includes several new significant features, many improvements and bug fixes to existing features, and version updates to the operating system, base applications, and drivers. Users are encouraged to Update to this release in order to take advantage of these improvements and bug fixes.

These base applications and drivers have been updated or added:

  • The base operating system has been updated to FreeBSD 11.1-STABLE. This brings in many new features and drivers. Improvements have been made to the em(4), ixl(4), ixgbe(4), and mps(4) drivers. Additionally, the netmap(4) kernel module has been added to the build as some NIC drivers depend upon it.
  • Support has been added for the HBA 9400-81, Intel Skylake and Kaby Lake processors, and Ryzen processors.
  • There have been many improvements to OpenZFS. Users should notice a significant speed difference when listing a large number of snapshots or when deleting multiple snapshots and large files.
  • The algorithm used for scrubs and resilvers has received many improvements which will be most noticeable on defragmented pools.
  • Samba has been patched to address these security vulnerabilities.
  • The Dojo Toolkit has been updated to version 1.12.2.
  • OpenVPN has been updated to version 2.4.3.
  • Iperf version 3.2 has been added. To use this version, specify iperf3 instead of iperf.
  • Iocage has been updated to version 0.9.10.
  • The new middleware now uses Python asyncio which simplifies asynchronous code and makes it more readable.
  • The SNMP MIB has many improvements, including the ability to send SNMP traps for new alerts.
  • The system now sends an email when a scrub finishes.
  • mmv has been added. It can be used from the command line to safely move or copy multiple files using patterns, without any unexpected deletion of files due to target name collisions.
  • s3cmd has been added back as a CLI alternative to S3.
  • The CLI zfs-stats utility has been added. Type zfs-stats to see its command usage.
  • The hardware watchdog has been reenabled for recent firmware versions of AsrockRack C2750D4I. The BMC bug which required the watchdog to be disabled is resolved with the 00.30.00 or newer BMC firmware version.
  • The system will issue an alert if the system reboots itself.

These major features are new in this version:

  • It is now possible to pause and resume scrubs from the command line. Since scrub pause state and progress are periodically synced to disk, if the system is restarted or pool is exported during a paused scrub, the scrub will remain paused until it is resumed. Once resumed the scrub will pick up from the place where it was last checkpointed to disk. To resume a paused scrub, type zpool scrub. To manually pause a scrub, use zpool scrub -p. A future version of FreeNAS® will add a button to the UI to resume or pause a scrub.
  • Cloud Credentials has been added to System. This can be used to provide a secure connection to a cloud services providers. Supported services include Amazon S3, Azure Blob Storage, Backblaze B2, and Google Cloud Storage.
  • Cloud Sync has been added to Tasks and can be used to synchronize files or directories to remote cloud storage providers using the specified transfer mode.
  • The Server Side Encryption drop-down menu has been added to Tasks Cloud Sync Add Cloud Sync, when an S3 provider is selected.
  • Resilver Priority has been added to Storage. This provides the ability to run resilvering at a higher priority at configurable times and days of the week.
  • The Netdata real-time performance and monitoring system has been added to Services.
  • VMs have received significant improvements, including:
    • support for non-US keyboards.
    • the ability to specify the NIC used by the VM as well as the MAC address for the VM NIC. These options can be set with VMs Devices Network Interface.
    • the ability to specify the sector size used by the emulated disk has been added to VMs Devices Disk.
    • the ability to edit the VNC screen resolution, select the IP address to bind to, set the VNC password, and select the option to use the Web version of VNC. These options can be set with VMs Devices VNC.

These screens have changed:

  • Each device in a mirrored boot pool now displays a Detach button in System Boot Status. This can be used to remove a device from the boot pool.
  • The Enable Console Menu in System Advanced has been renamed to Show Text Console Without Password Prompt.
  • The Report CPU usage in percentage checkbox has been added to System Advanced.
  • The FreeNAS-11-Nightlies-SDK train has been added and the FreeNAS-9.3-STABLE train has been removed from System Update.
  • The Send Test Alert button has been added to System Alert Services Edit.
  • The Subject Alternate Names field has been added to System CAs Create Internal CA, System CAs Create Intermediate CA, System Certificates Create Internal Certificate, and System Certificates Create Certificate Signing Request screens.
  • The Sign CSR button has been added to System CAs.
  • The ability to edit an existing certificate’s Name, Certificate, and Private Key fields has been added to System Certificates View.
  • An Enabled checkbox has been added to Tasks Init/Shutdown Scripts.
  • The Additional domains field has been added to Network Global Configuration. This allows up to six additional DNS search domains with the caveat that adding more domains may negatively impact DNS lookup time.
  • The Identify Light button has been added to Network IPMI to make it easier to identify a system in a rack by flashing its IPMI LED light.
  • The Priority Code Point (CoS) field has been added to Network VLANs Add VLAN. This can be useful in datacenter environments to classify storage traffic on a given VLAN interface using IEEE 802.1p Class of Service (COS).
  • The Read-Only drop-down menu has been added to Storage Datasets Add Dataset Advanced Mode.
  • The Promote Dataset button has been added to Storage Volumes.
  • The Replication column has been removed from Storage Snapshots.
  • The Time Machine Quota checkbox has been added to Sharing Apple (AFP) Shares Add Apple (AFP) Share.
  • The Access Based Share Enumeration checkbox has been added to Sharing SMB (Windows) Shares Add SMB (Windows) Share.
  • The Home Share Time Machine checkbox has been added to Services AFP.
  • The CheckIP Server SSL, CheckIP Server, CheckIP Path, and Use SSL fields have been added to Services DDNS. The Forced update period and Auxiliary parameters fields have been removed. In addition, several dozen DDNS providers have been added to the Provider drop-down menu.
  • The Certificate drop-down menu has been added to Services S3 in order to configure encrypted S3 connections.
  • The Server minimum protocol and Server maximum protocol fields have been removed from Services SMB.
  • The Log Level drop-down menu has been added to Services SNMP. It defaults to the Error log level.
  • The No Communication Warning Time field has been added to Services UPS. This can be used to configure the frequency of email notifications during the loss of UPS communications.
  • The No Authentication choice has been added to the Services WebDAV HTTP Authentication drop-down menu.

1.2. Path and Name Lengths

Names of files, directories, and devices are subject to some limits imposed by the FreeBSD operating system. The limits shown here are for names using plain-text characters that each occupy one byte of space. Some UTF-8 characters take more than a single byte of space, and using those characters reduces these limits proportionally. System overhead can also reduce the length of these limits by one or more bytes.

Table 1.2.1 Path and Name Lengths
Type Maximum Length Description
File Paths 1024 bytes

Total file path length (PATH_MAX). The full path includes directory separator slash characters, subdirectory names, and the name of the file itself. For example, the path /mnt/tank/mydataset/mydirectory/myfile.txt is 42 bytes long.

Using very long file or directory names can be problematic. A complete path with long directory and file names can exceed the 1024-byte limit, preventing direct access to that file until the directory names or filename are shortened or the file is moved into a directory with a shorter total path length.

File and Directory Names 255 bytes Individual directory or file name length (NAME_MAX).
Mounted Filesystem Paths 88 bytes Mounted filesystem path length (MNAMELEN). Longer paths can prevent a device from being mounted.
Device Filesystem Paths 63 bytes devfs(8) device path lengths (SPECNAMELEN). Longer paths can prevent a device from being created.

1.3. Hardware Recommendations

FreeNAS® 11.1 is based on FreeBSD 11.1 and supports the same hardware found in the FreeBSD Hardware Compatibility List. Supported processors are listed in section 2.1 amd64. FreeNAS® is only available for 64-bit processors. This architecture is called amd64 by AMD and Intel 64 by Intel.

Note

FreeNAS® boots from a GPT partition. This means that the system BIOS must be able to boot using either the legacy BIOS firmware interface or EFI.

Actual hardware requirements vary depending on the usage of the FreeNAS® system. This section provides some starter guidelines. The FreeNAS® Hardware Forum has performance tips from FreeNAS® users and is a place to post questions regarding the hardware best suited to meet specific requirements. Hardware Recommendations gives detailed recommendations for system components, with the FreeNAS® Quick Hardware Guide providing short lists of components for various configurations. Building, Burn-In, and Testing your FreeNAS® system has detailed instructions on testing new hardware.

1.3.1. RAM

The best way to get the most out of a FreeNAS® system is to install as much RAM as possible. The recommended minimum is 8 GB of RAM. The more RAM, the better the performance, and the FreeNAS® Forums provide anecdotal evidence from users on how much performance is gained by adding more RAM.

Depending upon the use case, your system may require more RAM. Here are some general rules of thumb:

  • To use Active Directory with many users, add an additional 2 GB of RAM for winbind’s internal cache.
  • For iSCSI, install at least 16 GB of RAM if performance is not critical, or at least 32 GB of RAM if good performance is a requirement.
  • When installing FreeNAS® on a headless system, disable the shared memory settings for the video card in the BIOS.
  • To use ZFS deduplication, ensure the system has at least 5 GB of RAM per TB of storage to be deduplicated.

If the hardware supports it and the budget allows for it, install ECC RAM. While more expensive, ECC RAM is highly recommended as it prevents in-flight corruption of data before the error-correcting properties of ZFS come into play, thus providing consistency for the checksumming and parity calculations performed by ZFS. If you consider your data important, use ECC RAM. This Case Study describes the risks associated with memory corruption.

Unless the system has at least 8 GB of RAM, consider adding RAM before using FreeNAS® to store data. Many users expect FreeNAS® to function with less memory, just at reduced performance. The bottom line is that these minimums are based on feedback from many users. Requests for help in the forums or IRC are sometimes ignored when the installed system does not have at least 8 GB of RAM because of the abundance of information that FreeNAS® may not behave properly with less memory.

1.3.2. The Operating System Device

The FreeNAS® operating system is installed to at least one device that is separate from the storage disks. The device can be a USB stick, SSD, compact flash, or DOM (Disk on Module). Installation to a hard drive is discouraged as that drive is then not available for data storage.

Note

To write the installation file to a USB stick, two USB ports are needed, each with an inserted USB device. One USB stick contains the installer. The other USB stick is the destination for the FreeNAS® installation. Take care to select the correct USB device for the FreeNAS® installation. It is not possible to install FreeNAS® onto the same USB stick containing the installer. After installation, remove the installer USB stick. It might also be necessary to adjust the BIOS configuration to boot from the new FreeNAS® USB stick.

When determining the type and size of the target device where FreeNAS® will be installed, keep these points in mind:

  • the bare minimum size is 8 GB. This provides room for the operating system and several boot environments. Since each update creates a boot environment, this is the recommended minimum. 32 GB provides room for more boot environments.
  • if you plan to make your own boot environments, budget about 1 GB of storage per boot environment. Consider deleting older boot environments after making sure they are no longer needed. Boot environments can be created and deleted using System Boot.
  • use quality, name-brand USB sticks, as ZFS will quickly reveal errors on cheap, poorly-made sticks.
  • for a more reliable boot disk, use two identical devices and select them both during the installation. This will create a mirrored boot device.

1.3.3. Storage Disks and Controllers

The Disk section of the FreeBSD Hardware List lists the supported disk controllers. In addition, support for 3ware 6 Gbps RAID controllers has been added along with the CLI utility tw_cli for managing 3ware RAID controllers.

FreeNAS® supports hot pluggable drives. Using this feature requires enabling AHCI in the BIOS.

Reliable disk alerting and immediate reporting of a failed drive can be obtained by using an HBA such as an Broadcom MegaRAID controller or a 3Ware twa-compatible controller.

Note

Upgrading the firmware of Broadcom SAS HBAs to the latest version is recommended.

Some Highpoint RAID controllers do not support pass-through of S.M.A.R.T. data or other disk information, potentially including disk serial numbers. It is best to use a different disk controller with FreeNAS®.

Note

The system is configured to prefer the mrsas(4) driver for controller cards like the Dell PERC H330 and H730 which are supported by several drivers. Although not recommended, the mfi(4) driver can be used instead by removing the loader Tunable: hw.mfi.mrsas_enable or setting the Value to 0.

Suggestions for testing disks before adding them to a RAID array can be found in this forum post. Additionally, badblocks is installed with FreeNAS® for testing disks.

If the budget allows optimization of the disk subsystem, consider the read/write needs and RAID requirements:

  • For steady, non-contiguous writes, use disks with low seek times. Examples are 10K or 15K SAS drives which cost about $1/GB. An example configuration would be six 600 GB 15K SAS drives in a RAID 10 which would yield 1.8 TB of usable space, or eight 600 GB 15K SAS drives in a RAID 10 which would yield 2.4 TB of usable space.

For ZFS, Disk Space Requirements for ZFS Storage Pools recommends a minimum of 16 GB of disk space. Due to the way that ZFS creates swap, it is not possible to format less than 3 GB of space with ZFS. However, on a drive that is below the minimum recommended size, a fair amount of storage space is lost to swap: for example, on a 4 GB drive, 2 GB will be reserved for swap.

Users new to ZFS who are purchasing hardware should read through ZFS Storage Pools Recommendations first.

ZFS vdevs, groups of disks that act like a single device, can be created using disks of different sizes. However, the capacity available on each disk is limited to the same capacity as the smallest disk in the group. For example, a vdev with one 2 TB and two 4 TB disks will only be able to use 2 TB of space on each disk. In general, use disks that are the same size for the best space usage and performance.

The ZFS Drive Size and Cost Comparison spreadsheet is available to compare usable space provided by different quantities and sizes of disks.

1.3.4. Network Interfaces

The Ethernet section of the FreeBSD Hardware Notes indicates which interfaces are supported by each driver. While many interfaces are supported, FreeNAS® users have seen the best performance from Intel and Chelsio interfaces, so consider these brands when purchasing a new NIC. Realtek cards often perform poorly under CPU load as interfaces with these chipsets do not provide their own processors.

At a minimum, a GigE interface is recommended. While GigE interfaces and switches are affordable for home use, modern disks can easily saturate their 110 MB/s throughput. For higher network throughput, multiple GigE cards can be bonded together using the LACP type of Link Aggregations. The Ethernet switch must support LACP, which means a more expensive managed switch is required.

When network performance is a requirement and there is some money to spend, use 10 GigE interfaces and a managed switch. Managed switches with support for LACP and jumbo frames are preferred, as both can be used to increase network throughput. Refer to the 10 Gig Networking Primer for more information.

Note

At present, these are not supported: InfiniBand, FibreChannel over Ethernet, or wireless interfaces.

Both hardware and the type of shares can affect network performance. On the same hardware, SMB is slower than FTP or NFS because Samba is single-threaded. So a fast CPU can help with SMB performance.

Wake on LAN (WOL) support depends on the FreeBSD driver for the interface. If the driver supports WOL, it can be enabled using ifconfig(8). To determine if WOL is supported on a particular interface, use the interface name with the following command. In this example, the capabilities line indicates that WOL is supported for the re0 interface:

ifconfig -m re0
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=42098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
        capabilities=5399b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_UCAST,WOL_MCAST, WOL_MAGIC,VLAN_HWFILTER,VLAN_H WTSO>

If WOL support is shown but not working for a particular interface, create a bug report using the instructions in Support.

1.4. Getting Started with ZFS

Readers new to ZFS should take a moment to read the ZFS Primer.