A safe backup-server with rsync, chroot and forced commands en

By Spider.007 on Saturday 10 November 2012 14:59 - Comments (4)
Category: code, Views: 5.155

I recently needed to create a backup-server, and since my final solution was not described anywhere so I decided to show you how I did this. While creating solutions like this, I like to think about the worst possible situations I needed to prevent. So lets assume:
  • Compromised clients shouldn't affect backups from other clients
  • Compromised clients shouldn't be able to access parts of the server they shouldn't
  • Adding new clients shouldn't cause any downtime for existing clients
  • The 'management' overhead of the solution should be minimal
This is what I came up with; first of all we are going to create a chroot for a user called 'backup', to prevent this user from accessing anything except backups. Start with useradd -m backup (all these command should be executed as root).

We will create this chroot in /srv/rsync/ using a simple script which uses ldd to copy binaries and their libraries to our chroot:

Put the following in /srv/rsync/bin/chroot_update and chmod it executable, then execute it too:
bash:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#!/bin/bash
cd `dirname $0`/../
mkdir -p ./bin ./usr/bin ./.lock/

function mcp {
    [[ ! -d .`dirname $1` ]] && mkdir -p .`dirname $1`
    cp -v $1 .$1
}

mcp `ldd /bin/bash | grep ld-linux | cut -d' ' -f1`

for b in bash rsync ln rm date mkdir
do
    b=`which $b`
    cp -v $b .$b
    ldd $b | cut -d' ' -f3 | while read l
    do
        [[ -z $l || -f .$l ]] && continue
        mcp $l
    done
done

Test if the chroot is functional by executing chroot /srv/rsync rsync. Now we'll setup our sshd, add the following lines to /etc/ssh/sshd_config and restart your sshd
code:
1
2
3
Match user backup
        ChrootDirectory /srv/rsync
        AllowTcpForwarding no
With our chroot we have shielded the backup environment from the rest of the server; but we still need to make sure clients can only access their own backups. We'll use ForcedCommands for this, combined with (passwordless) ssh keys. Also, in this example I will restrict the source IP, to allow only a single host from using this key. This line should be put in /home/backup/.ssh/authorized_keys, and you should repeat this for each client you add (obviously you need to use your own public key instead of our RANDOM version):

code:
1
from="192.168.7.*",command="/bin/rsync_wrapper client.example.com" ssh-rsa ssh-rsa AAARANDOM== root@client.example.com


An additional advantage of this forced command is that we can define most of the rsync options in our script, making it easy to configure clients. For example, we don't need our clients to specify the target-directory on the server. Our wrapper will also take care of locking, so a single client cannot connect multiple times simultaneously.

Put this script in /srv/rsync/bin/rsync_wrapper and chmod it executable:
code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#!/bin/bash
# this will be a forced command which passes the hostname as first argument
set -e
HOST=$1

if [[ -f /.lock/$HOST ]]
then
    echo ERROR: Target $HOST is locked >&2
    exit 1
else
    trap "{ c=$? ; rm -f /.lock/$HOST ; exit $c ; }" EXIT
    >/.lock/$HOST
fi
# echo DEBUG: $SSH_ORIGINAL_COMMAND >&2

[[ ! -d /backups/$HOST/ ]] && mkdir /backups/$HOST/
rsync --server -logDtprxze.iLsf --delete-during --delete-excluded . /backups/$HOST/

Testing this setup from our client is pretty easy; just execute rsync using your preferred source directories and possible excludes:
code:
1
rsync -xaz --delete --exclude=*.mp3 /home/ /etc backup@server.example.com:/

Notice how we don't define a target directory, since our forced-command will ignore this. The --delete is necessary for our client to understand the output from our server (which also uses --delete). Extending this setup with hard links is trivial.

You know you've got to Back the fuck up

Volgende: GO - Create a dependency graph 12-'13 GO - Create a dependency graph
Volgende: KPN Glasvezel zonder Experia box 03-'12 KPN Glasvezel zonder Experia box

Comments


By Tweakers user Gtoniser, Saturday 10 November 2012 15:54

Very interesting. Might use this knowledge somewhere in the future :)

By Tweakers user Spider.007, Saturday 10 November 2012 16:02

Nice, keep us posted!

By Tweakers user ameesters, Wednesday 14 November 2012 12:00

maybe you should give your backup user a scponly shell, cause this way somebody could ssh into the jailed environment, and maybe use a bug to escalate to root, that way the attacker could also affect/infect other backups!

Also you might want to switch to one-user-per-client , cause now everything is the jail is owned by the user backup, which would allow to alter the file for any backup in the jail.

So you have done a excellent job in shielding the OS from a potential attacker, but the backups can not be trusted.

By Tweakers user Spider.007, Wednesday 14 November 2012 12:35

ameesters wrote on Wednesday 14 November 2012 @ 12:00:
maybe you should give your backup user a scponly shell, cause this way somebody could ssh into the jailed environment, and maybe use a bug to escalate to root, that way the attacker could also affect/infect other backups!
No they cannot, the forced-command option shields the server from such attacks. You cannot 'ssh into the jailed enviroment' because you will not be offered a shell to abuse.
Also you might want to switch to one-user-per-client , cause now everything is the jail is owned by the user backup, which would allow to alter the file for any backup in the jail.
A user per backup (client) would have been preferable but offers significantly less flexibility.

Comments are closed