A safe backup-server with rsync, chroot and forced commands 
I recently needed to create a backup-server, and since my final solution was not described anywhere so I decided to show you how I did this. While creating solutions like this, I like to think about the worst possible situations I needed to prevent. So lets assume:
We will create this chroot in /srv/rsync/ using a simple script which uses ldd to copy binaries and their libraries to our chroot:
Put the following in /srv/rsync/bin/chroot_update and chmod it executable, then execute it too:
bash:
Test if the chroot is functional by executing chroot /srv/rsync rsync. Now we'll setup our sshd, add the following lines to /etc/ssh/sshd_config and restart your sshd
code:
With our chroot we have shielded the backup environment from the rest of the server; but we still need to make sure clients can only access their own backups. We'll use ForcedCommands for this, combined with (passwordless) ssh keys. Also, in this example I will restrict the source IP, to allow only a single host from using this key. This line should be put in /home/backup/.ssh/authorized_keys, and you should repeat this for each client you add (obviously you need to use your own public key instead of our RANDOM version):
code:
An additional advantage of this forced command is that we can define most of the rsync options in our script, making it easy to configure clients. For example, we don't need our clients to specify the target-directory on the server. Our wrapper will also take care of locking, so a single client cannot connect multiple times simultaneously.
Put this script in /srv/rsync/bin/rsync_wrapper and chmod it executable:
code:
Testing this setup from our client is pretty easy; just execute rsync using your preferred source directories and possible excludes:
code:
Notice how we don't define a target directory, since our forced-command will ignore this. The --delete is necessary for our client to understand the output from our server (which also uses --delete). Extending this setup with hard links is trivial.
You know you've got to Back the fuck up
- Compromised clients shouldn't affect backups from other clients
- Compromised clients shouldn't be able to access parts of the server they shouldn't
- Adding new clients shouldn't cause any downtime for existing clients
- The 'management' overhead of the solution should be minimal
We will create this chroot in /srv/rsync/ using a simple script which uses ldd to copy binaries and their libraries to our chroot:
Put the following in /srv/rsync/bin/chroot_update and chmod it executable, then execute it too:
bash:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| #!/bin/bash
cd `dirname $0`/../
mkdir -p ./bin ./usr/bin ./.lock/
function mcp {
[[ ! -d .`dirname $1` ]] && mkdir -p .`dirname $1`
cp -v $1 .$1
}
mcp `ldd /bin/bash | grep ld-linux | cut -d' ' -f1`
for b in bash rsync ln rm date mkdir
do
b=`which $b`
cp -v $b .$b
ldd $b | cut -d' ' -f3 | while read l
do
[[ -z $l || -f .$l ]] && continue
mcp $l
done
done |
Test if the chroot is functional by executing chroot /srv/rsync rsync. Now we'll setup our sshd, add the following lines to /etc/ssh/sshd_config and restart your sshd
code:
1
2
3
| Match user backup
ChrootDirectory /srv/rsync
AllowTcpForwarding no |
code:
1
| from="192.168.7.*",command="/bin/rsync_wrapper client.example.com" ssh-rsa ssh-rsa AAARANDOM== root@client.example.com |
An additional advantage of this forced command is that we can define most of the rsync options in our script, making it easy to configure clients. For example, we don't need our clients to specify the target-directory on the server. Our wrapper will also take care of locking, so a single client cannot connect multiple times simultaneously.
Put this script in /srv/rsync/bin/rsync_wrapper and chmod it executable:
code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| #!/bin/bash
# this will be a forced command which passes the hostname as first argument
set -e
HOST=$1
if [[ -f /.lock/$HOST ]]
then
echo ERROR: Target $HOST is locked >&2
exit 1
else
trap "{ c=$? ; rm -f /.lock/$HOST ; exit $c ; }" EXIT
>/.lock/$HOST
fi
# echo DEBUG: $SSH_ORIGINAL_COMMAND >&2
[[ ! -d /backups/$HOST/ ]] && mkdir /backups/$HOST/
rsync --server -logDtprxze.iLsf --delete-during --delete-excluded . /backups/$HOST/ |
Testing this setup from our client is pretty easy; just execute rsync using your preferred source directories and possible excludes:
code:
1
| rsync -xaz --delete --exclude=*.mp3 /home/ /etc backup@server.example.com:/ |
Notice how we don't define a target directory, since our forced-command will ignore this. The --delete is necessary for our client to understand the output from our server (which also uses --delete). Extending this setup with hard links is trivial.
You know you've got to Back the fuck up
03-'12 KPN Glasvezel zonder Experia box
Comments
Very interesting. Might use this knowledge somewhere in the future 
Nice, keep us posted!
maybe you should give your backup user a scponly shell, cause this way somebody could ssh into the jailed environment, and maybe use a bug to escalate to root, that way the attacker could also affect/infect other backups!
Also you might want to switch to one-user-per-client , cause now everything is the jail is owned by the user backup, which would allow to alter the file for any backup in the jail.
So you have done a excellent job in shielding the OS from a potential attacker, but the backups can not be trusted.
Also you might want to switch to one-user-per-client , cause now everything is the jail is owned by the user backup, which would allow to alter the file for any backup in the jail.
So you have done a excellent job in shielding the OS from a potential attacker, but the backups can not be trusted.
No they cannot, the forced-command option shields the server from such attacks. You cannot 'ssh into the jailed enviroment' because you will not be offered a shell to abuse.ameesters wrote on Wednesday 14 November 2012 @ 12:00:
maybe you should give your backup user a scponly shell, cause this way somebody could ssh into the jailed environment, and maybe use a bug to escalate to root, that way the attacker could also affect/infect other backups!
A user per backup (client) would have been preferable but offers significantly less flexibility.Also you might want to switch to one-user-per-client , cause now everything is the jail is owned by the user backup, which would allow to alter the file for any backup in the jail.
Comments are closed