bigsync can only be used in order to backup single files. However, virtual server environments often mix a number of large files (disk images) with other valuable information. It would be nice if one could just name a directory and say "backup everything, use bigsync for the large files". Luckily, there is such a way. The script below is very simple and might need some refinement, but it is a good starting point for your own backup strategy.

This example script can be used in order to backup multiple running (!!!) virtual machines on your host safely and very efficiently.

Setting

For this example, we assume the following:

• Your data is stored in a directory /volumes/data (change this suiting your needs).
• An lvm volume group vg0 with a logical volume lv0 exists and is mounted in /volumes/data
• Your kernel has full support for lvm features (including snapshots)
• The volume group has about 20G space left unused for the snapshot
• Call the script with two parameters. Parameter 1 is the name of an existing backup, Parameter 2 is the name of the new backup (must not exist yet). The new backup directory will be created and filled with data, bigsync will try to reuse as much of the existing backup data as possible.

#! /usr/bin/perl
# (C) 2010 by Stefan Hoefer (
  This e-mail address is being protected from spambots. You need JavaScript enabled to view it
 )
# Permission is hereby granted to use this script in whatever way you want, free of charge, 
# including altering and redistributing it. Note however that whatever you do, you do it at 
# your own risk! The author does not take any responsibility for any harm that might come 
# from using this script or any software derived from it.

use Proc::PID::File;

if (@ARGV != 2) {
 die "Syntax: backup.pl reference newdir\n";
}

my $backupdir = "/volumes/backup/";
my $sourcedir = "/volumes/data/";

!Proc::PID::File->running() or die "Already running!\n";

my $reference = $backupdir.$ARGV[0];
my $newdir = $backupdir.$ARGV[1];

if (-e $newdir) {
 system("/bin/rm -rf $newdir");
}
system("/bin/mkdir $newdir");

if (!-e $reference) {
 system("/bin/mkdir $reference");
}

# perform the backup

# create the snapshot

system("/sbin/lvcreate -n lvbackup -L 20G -p r -s /dev/vg0/lv0");

system("/bin/mkdir -p /mnt/backup");
system("/bin/mount /dev/vg0/lvbackup /mnt/backup");

# perform the data copy

# first the small files and the directory structure

$subdir = "/mnt/backup/";

system("/usr/bin/rsync -a --max-size=10M --link-dest=${reference} $subdir ${newdir}"); 

# now all the large files

if (open FILELIST, "/usr/bin/find $subdir -size +10M |") {
 while (my $file = readline *FILELIST) {
   chomp $file;
 
   my $subfile = substr($file, length($subdir));

   if (-e "${reference}/$subfile") {
     system("/usr/bin/bigsplit --reference ${reference}/$subfile $file ${newdir}/$subfile");
   }
   else {
     system("/usr/bin/bigsplit $file ${newdir}/$subfile");
   }
 }

 close FILELIST;
}

# remove the snapshot again

system("/bin/sync");
system("/bin/umount /mnt/backup");
while (system("/sbin/lvremove -f /dev/vg0/lvbackup")) {
 sleep 1;
}