Rsync Tasks
Contents |
Rsync is a utility that automatically copies specified data from one system to another over a network. Once the initial data is copied, rsync reduces the amount of data sent over the network by sending only the differences between the source and destination files. Rsync can be used for backups, mirroring data on multiple systems, or for copying files between systems.
To configure rsync, you need to configure both ends of the connection:
- the rsync server: this system pulls (receives) the data. This system is referred to as PULL in the configuration examples.
- the rsync client: this system pushes (sends) the data. This system is referred to as PUSH in the configuration examples.
FreeNAS® can be configured as either an rsync client or an rsync server. The opposite end of the connection can be another FreeNAS® system or any other system running rsync. In FreeNAS® terminology, an rysnc task defines which data is synchronized between the two systems. If you are synchronizing data between two FreeNAS® systems, create the rsync task on the rsync client.
FreeNAS® supports two modes of rsync operation:
- rsync module mode: exports a directory tree, and its configured settings, as a symbolic name over an unencrypted connection. This mode requires that at least one module be defined on the rsync server. It can be defined in the FreeNAS® GUI under Services → Rsync → Rsync Modules. In other operating systems, the module is defined in rsyncd.conf(5).
- rsync over SSH: synchronizes over an encrypted connection. Requires the configuration of SSH user and host public keys.
This section summarizes the options when creating an Rsync Task. It then provides a configuration example between two FreeNAS® systems for each mode of rsync operation.
Creating an Rsync Task
Figure 4.4a shows the screen that appears when you click System → Rsync Tasks → Add Rsync Task.
Figure 4.4a: Adding an Rsync Task
Table 4.4a summarizes the options that can be configured when creating an rsync task:
Table 4.4a: Rsync Configuration Options
| Setting | Value | Description |
| Path | browse button | select the volume/dataset/directory that you wish to copy; note that a path length greater than 255 characters will fail |
| Remote Host | string | IP address or hostname of the remote system that will store the copy |
| Remote SSH Port | integer | only available in Rsync module mode; allows you to specify an alternate SSH port other than the default of 22 |
| Rsync mode | drop-down menu | choices are Rsync module or Rsync over SSH |
| Remote Module Name/Remote Path | string | when using Rsync module mode, at least one module must be defined in rsyncd.conf(5) of rsync server or in Services → Rsync → Rsync Modules of another FreeNAS® system; when using Rsync over SSH mode, input the path on the remote host to push or pull (e.g. /mnt/volume) |
| Direction | drop-down menu | choices are Push or Pull; default is to push from the FreeNAS® system to a remote host |
| Short Description | string | optional |
| Minute | slider or minute selections | if use the slider, sync occurs every N minutes; if use minute selections, sync occurs at the highlighted minutes |
| Hour | slider or hour selections | if use the slider, sync occurs every N hours; if use hour selections, sync occurs at the highlighted hours |
| Day of month | slider or day selections | if use the slider, sync occurs every N days; if use day selections, sync occurs on the highlighted days |
| Month | checkboxes | task occurs on the selected months |
| Day of week | checkboxes | task occurs on the selected days of the week |
| User | drop-down menu | specified user must have permission to write to the specified directory on the remote system; due to a limitation in FreeBSD, the user name can not contain spaces or exceed 17 characters |
| Recursive | checkbox | if checked, copy will include all subdirectories of the specified volume |
| Times | checkbox | preserve modification times of files |
| Compress | checkbox | recommended on slow connections as reduces size of data to be transmitted |
| Archive | checkbox | equivalent to -rlptgoD (recursive, copy symlinks as symlinks, preserve permissions, preserve modification times, preserve group, preserve owner (super-user only), and preserve device files (super-user only) and special files) |
| Delete | checkbox | delete files in destination directory that don't exist in sending directory |
| Quiet | checkbox | suppresses information messages from the remote server |
| Preserve permissions | checkbox | preserves original file permissions; useful if User is set to root |
| Preserve extended attributes | checkbox | both systems must support [extended attributes] |
| Extra options | string | [rsync(1)] options not covered by the GUI |
| Enabled | checkbox | uncheck if you would like to disable the rsync task without deleting it |
If the rysnc server requires password authentication, input --password-file=/PATHTO/FILENAME in the "Extra options" box, replacing /PATHTO/FILENAME with the appropriate path to the file containing the value of the password.
Configuring Rsync Module Mode Between Two FreeNAS® Systems
This configuration example will configure rsync module mode between the two following FreeNAS® systems:
- 192.168.2.2 has existing data in /mnt/local/images. It will be the rsync client, meaning that an rsync task needs to be defined. It will be referred to as PUSH.
- 192.168.2.6 has an existing volume named /mnt/remote. It will be the rsync server, meaning that it will receive the contents of /mnt/local/images. An rsync module needs to be defined on this system and the rsyncd service needs to be started. It will be referred to as PULL.
On PUSH, an rsync task is defined in System → Rsync Tasks → Add Rsync Task as shown in Figure 4.4b. In this example:
- the Path points to /usr/local/images, the directory to be copied
- the Remote Host points to 192.168.2.6, the IP address of the rsync server
- the Rsync Mode is Rsync module
- the Remote Module Name is backups; this will need to be defined on the rsync server
- the Direction is Push
- the rsync is scheduled to occur every 15 minutes
- the User is set to root so it has permission to write anywhere
- the Preserve Permissions checkbox is checked so that the original permissions are not overwritten by the root user
Figure 4.4b: Configuring the Rsync Client
On PULL, an rsync module is defined in Services → Rsync Modules → Add Rsync Module as shown in Figure 4.4c. In this example:
- the Module Name is backups; this needs to match the Remote Module Name on the rsync client
- the Path is /mnt/remote; rsync will automatically create a subdirectory called images to hold the contents of /usr/local/images
- the User is set to root so it has permission to write anywhere
- Hosts allow is set to 192.168.2.2, the IP address of the rsync client
Descriptions of the configurable options can be found in Rsync Modules.
Figure 4.4c: Configuring the Rsync Server
To finish the configuration, start the rsync service on PULL in Services → Control Services. If the rsync is successful, the contents of /mnt/local/images/ will be mirrored to /mnt/remote/images/.
4.4.3 Configuring Rsync over SSH Mode Between Two FreeNAS® Systems
SSH replication mode does not require the creation of an rsync module or for the rsync service to be running on the rsync server. It does require SSH to be configured before creating the rsync task:
- a public/private key pair for the rsync user account (typically root) must be generated on PUSH and the public key copied to the same user account on PULL
- to mitigate the risk of man-in-the-middle attacks, the public host key of PULL must be copied to PUSH
- the SSH service must be running on PULL
To create the public/private key pair on PUSH, open Shell. The / filesystem must first be mounted as read-write. In the following example, the root user is generating an RSA type of public/private key pair. When creating the key pair, do not enter the passphrase as the key is meant to be used for an automated task.
mount -o rw / ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: f5:b0:06:d1:33:e4:95:cf:04:aa:bb:6e:a4:b7:2b:df root@freenas.local The key's randomart image is: +--[ RSA 2048]----+ | .o. oo | | o+o. . | | . =o + | | + + o | | S o . | | .o | | o. | | o oo | | **oE | +-----------------+
NOTE: FreeNAS® supports the following types of SSH keys: DSA, and RSA. When creating the key, specify the type you wish to use or, if you are generating the key on another operating system, select a type of key the key generation software supports.
Next, view and copy the contents of the generated public key:
more .ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1lBEXRgw1W8y8k+lXPlVR3xsmVSjtsoyIzV/PlQPo SrWotUQzqILq0SmUpViAAv4Ik3T8NtxXyohKmFNbBczU6tEsVGHo/2BLjvKiSHRPHc/1DX9hofcFti4h dcD7Y5mvU3MAEeDClt02/xoi5xS/RLxgP0R5dNrakw958Yn001sJS9VMf528fknUmasti00qmDDcp/kO xT+S6DFNDBy6IYQN4heqmhTPRXqPhXqcD1G+rWr/nZK4H8Ckzy+l9RaEXMRuTyQgqJB/rsRcmJX5fApd DmNfwrRSxLjDvUzfywnjFHlKk/+TQIT1gg1QQaj21PJD9pnDVF0AiJrWyWnR root@freenas.local
Go to PULL and paste (or append) the copied key into the SSH Public Key field of Account → Users → View Users → root → Modify User. The paste for the above example is shown in Figure 5.5d. When pasting the key, ensure that it is pasted as one long line and, if necessary, remove any extra spaces representing line breaks.
Figure 4.4d: Pasting the User's SSH Public Key
While on PULL, verify that the SSH service is running in Services → Control Services and start it if it is not.
Next, copy the host key of PULL using Shell on PUSH. The following command copies the RSA host key of the PULL server used in our previous example. Be sure to include the double bracket >> to prevent overwriting any existing entries in the known_hosts file.
ssh-keyscan -t rsa 192.168.2.6 >> /root/.ssh/known_hosts
You are now ready to create the rsync task on PUSH. To configure rsync SSH mode using the systems in our previous example, the configuration would be as follows:
- the Path points to /mnt/local/images, the directory to be copied
- the Remote Host points to 192.168.2.6, the IP address of the rsync server
- the Rsync Mode is Rsync over SSH
- the rsync is scheduled to occur every 15 minutes
- the User is set to root so it has permission to write anywhere; the public key for this user must be generated on PUSH and copied to PULL
- the Preserve Permissions checkbox is checked so that the original permissions are not overwritten by the root user
Once you save the rsync task, the rsync will automatically occur according to your schedule. In this example, the contents of /mnt/local/images/ will automatically appear in /mnt/remote/images/ after 15 minutes. If the content does not appear, use Shell on PULL to read /var/log/messages. If the message indicates a \n (newline character) in the key, remove the space in your pasted key--it will be after the character that appears just before the \n in the error message.
