In our first Application Note Easy, efficient, and reliable data backups for CDRouter we presented an approach for CDRouter data backups using local USB disks. In this Application Note we’ll look at an alternative approach involving backups over a network to a server or NAS device. This has many advantages including the ability to easily coordinate data backups for multiple CDRouter systems using a single script and storage device.
This Application Note assumes that a single server will be used to communicate with multiple CDRouter systems. The actual backups will be performed by a small program called Rsnapshot. Rsnapshot is “… a filesystem snapshot utility for making backups of local and remote systems.” Rsnapshot utilizes Rsync and essentially sets up and automates the backup system outlined in Mike Rubel’s article Easy Automated Snapshot-Style Backups with Linux and Rsync. The steps required to configure Rsnapshot for remote backups, as well as setting up each CDRouter host machine to accept connections without requiring a password will be outlined.
There are many different ways to set up a reliable and easy to use backup scheme. This Application Note presents only one technique that is aplicable to both BuddyWeb and buddy users and is scalable for multiple CDRouter systems.
The first step is to download and install Rsnapshot. Once installed you will find the main configuration file for Rsnasphot in the /etc/ directory:
# /etc/rsnapshot.conf
Please refer to Section 3 of the Rsnapshot HOWTO for information on installing Rsnapshot.
Note: SSH must be installed and running on both the server and all CDRouter host machines that you wish to backup.
Before configuring rsnapshot each CDRouter host machine to be backed up must be configured to allow SSH connections for a specific user without requesting a password. Without this step, the rsnapshot utility will request a password each time it is invoked, making automation difficult.
Fortunately public key authentication makes it easy to accept SSH connections without requiring a password. Furthermore, setting up public key authentication is also quite simple. The following steps are based on waelchatila.com’s article "Automatic SSH/SCP Login without Password” and will guide you through the process of setting up basic public key authentication.
Note: setting up password-less public key authentication presents a security risk. Please be sure that this is a risk you are willing to take before proceeding.
Select an existing user account or create a new user account and log in to the server as that user. The server is the machine that will run Rsnapshot and that will store all of the CDRouter system backups. We recommend not using the ‘root’ account for security reasons. In the following steps we’ll assume that a new unprivileged user ‘rsnapshot’ has been created on the server.
Create a new user named ‘rsnapshot’ on the first CDRouter system to be backed up.
While logged in to the server as rsnapshot, generate a password-less SSH key-pair using the following command (hit enter when prompted for a password):
# ssh-keygen –t rsa
This command will generate two files: a private key (typically called id_rsa) and a public key (id_rsa.pub). These files should be located in the /home/rsnapshot/.ssh/ directory and are usually installed in this directory by default when you generate the key-pair.
The next step is to copy the public key to the first CDRouter system. An easy way to do this is to use the ‘scp’ command from the server:
# cd /home/rsnapshot/.ssh/
# scp id_rsa.pub rsnapshot@CDRouter_host_IP:/home/rsnapshot/
Now use SSH to log in to the first CDRouter host system as the user rsnapshot:
# ssh rsnapshot@CDRouter_host_IP
If the directory /home/rsnapshot/.ssh/ on the CDRouter system does not already exist, create it and move the public key to this directory:
# mkdir /home/rsnapshot/.ssh
# mv /home/rsnapshot/id_rsa.pub /home/rsnapshot/.ssh/id_rsa.pub
Within the /home/rsnapshot/.ssh/ directory there should be a file called ‘authorized_keys’. If this file does not exist, create it and append rsnapshot’s public key to it:
# cd /home/rsnapshot/.ssh/
# touch authorized_keys
# cat id_rsa.pub >> authorized_keys
The final step is to ensure that the only the user rsnapshot has read/write permissions for the /home/rsnapshot/.ssh/ directory and authorizied_keys file:
# chmod 700 /home/rsnapshot/.ssh/
# chmod 700 /home/rsnapshot/.ssh/authorized_keys
At this point, you should be able to log in to the server as rsnapshot and use SSH to access the CDRouter system you just configured. If public key authentication is working properly, you won't be prompted to enter rsnapshot's password.
Repeat the above steps for each CDRouter system that you’re interested in backing up.
The Rsnapshot configuration file is located in /etc/rsnapshot.confM. Section 4 of the Rsnapshot HOWTO provides an excellent overview of the basic configuration of Rsnapshot. Note that all options are presented in a tab separated format. To enable an option, simply remove the 'pound' (#) character preceding it. Relevant options include:
This is the location on the server where all backups will be stored.
To enable Rsnapshot/Rsync over SSH this option must be enabled. This option should point to the SSH executable (typically /usr/bin/ssh).
You will have to enable the -i (identity_file) option and specify the private key for the user rsnapshot:
ssh_args -i /home/rsnapshot/.ssh/id_rsa
Note: an alternative (and more secure approach) to this can be found in Troy Johnson's article Using Rsnapshot and SSH.
This is the location of the Rsnapshot logfile. You will want to review the logfile periodically to ensure that Rsnapshot is running properly without errors.
This is what Rsnapshot uses to set up the backup directory structure. This value does not determine how often Rsnapshot is executed (this is determined by cron). Please see the Rsnapshot HOWTO for more information. To perform daily backups spanning seven days, configure the interval 'daily' as follows:
interval daily 7
This will create seven folders in the 'rsnapshot_root' directory labeled daily.0 through daily.6.
Each remote directory on each remote machine that you're interested in backup up must have a separate backup entry (collectively referred to as backup points. To backup the /usr/buddyweb/ directory on three different CDRouter systems, you would have three separate entries in the Rsnapshot config file:
backup rsnapshot@CDRouter_host1_IP:/usr/buddyweb/ CDRouter_host1_backup/
backup rsnapshot@CDRouter_host2_IP:/usr/buddyweb/ CDRouter_host2_backup/
backup rsnapshot@CDRouter_host3_IP:/usr/buddyweb/ CDRouter_host3_backup/
These entries tell Rsnapshot to remotely access each CDRouter system via SSH user rsnapshot and backup the /usr/buddyweb/ directory to the snapshot_root/CDRouter_hostX_backup/ directory.
Once you've configured finished editing the Rsnapshot config file, you can verify that the syntax is valid by executing the following command:
# rsnapshot configtest
After resolving all syntactical errors, you should test your Rsnapshot configuration. To invoke Rsnapshot, execute the following command as root on the backup server:
# /usr/local/bin/rsnapshot daily
If you've configured everything properly, Rsnapshot should start backing data up from each of the backup points identified in the config file. You may want to review the logfile for detailed information about the Rsnapshot process. All backups will be made to the backup_root directory. When Rsnapshot finishes you should be able to navigate to the backup_root directory on the server and view the backup points for each CDRouter system that you've specified in the Rsnapshot configuration file.
Note: it may take a significant amount of time for the Rsnapshot routine to complete the first time it is executed. This is because the first backup performed by Rsnapshot is a full backup, whereas subsequent backups are incremental - only files that are new or have been modified are copied to the backup server.
The final step is to automate Rsnapshot using cron. We recommend running Rsnapshot using root's crontab. To edit root's crontab, log in to the server as root and execute the following command:
# crontab -e
In the Rsnapshot configuration example above, we've assumed that we will be running Rsnapshot daily for a span of seven days. We configured the interval daily option in the configuration file and therefore should execute Rsnapshot once daily. To set the Rsnapshot daily routine to execute once per day at 10:00pm, make the following entry in root's crontab:
# m h dom mon dow command
00 22 * * * /usr/local/bin/rsnapshot daily
Save root's crontab and the Rsnapshot will not be executed daily. Note that Rsnapshot is very powerful and allows you to easily set up weekly, daily, and hourly incremental backups. In the example above we've only set up basic daily backups. Please read the Rsnapshot HOWTO for more information on setting additional weekly or hourly backups.
You may want to write a simple shell script that controls the Rsnapshot backup routine. Scripting give you more control over the backup process and allows you to determine the status of the backup process and calculate and monitor disk usage on the backup server. In addition, with a shell script you could use a lightweight mail program like nail to send periodic email notifications containing relevant information and statistics.
It may take a few attempts to get Rsnapshot to run without errors. While debugging Rsnapshot, you will want to review the contents of the logfile closely and perhaps increase the level of logging. One common issue is that the user rsnapshot may not have adequate permissions to read/modify certain files or directories that you would like to backup. In this cases the Rsnapshot routine will copy all other files and produce an error. The error message will not provide information as to which file or directories were not backed up, although you can determine this relatively quickly by comparing the backup on the server with the remote backup points on each CDRouter system. One workaround is to modify the permissions of the offending files or directories so that the user rsnapshot can access them.
Please contact QA Cafe Support: support@qacafe.com
www.qacafe.com
© 2008 QA Cafe