I have been backing up my EC2 cloud servers to S3 using a simple tar script to get the main data from the EBS mounts on each cloud server. I use cron to run the backup each night, saving the data first to the EBS mounts in an /apps/backup directory, then I copy the backup to the S3 storage. After the script runs successfully, I remove any old backups locally on EBS and S3 that are older than 3 days. This way I always have a 3 day rotation of the saved data.
In order to transfer to S3 storage, you will need the s3cmd, which can be downloaded here. The s3cmd will need to be stored inside the /root/bin directory unless you alter the cloud_backup script. Once you have the s3cmd files in place, run the s3cmd –configure command to configure it to your S3 instance. You will need your S3 access key and your secret key to complete the configuration.
I created the script in /root/bin and called it cloud_backup:
# Create the backup of certain /apps directories with the output as <HOSTNAME_DATE>.tgz
/bin/tar -czvf /apps/backup/`hostname`_`date +%m%d%y`.tgz /apps/apache /apps/data /apps/keys /apps/tomcat /apps/virtual /root && s3cmd put /apps/backup/`hostname`_`date +%m%d%y`.tgz s3://<bucket.domain.com>/`hostname`_`date +%m%d%y`.tgzfor i in `find /apps/backup -name *.tgz -mtime +3| cut -d ‘/’ -f 4`; do echo s3cmd del s3://<bucket.domain.com>/$i; echo rm -f /apps/backup/$i; done
# Move the backup file to S3 storage
/bin/tar -czvf /apps/backup/`hostname`_`date +%m%d%y`.tgz /apps/apache /apps/data /apps/keys /apps/tomcat /apps/virtual /root && s3cmd put /apps/backup/`hostname`_`date +%m%d%y`.tgz s3://<bucket.domain.com>/`hostname`_`date +%m%d%y`.tgz
# Look for backups older than 3 days and delete the files from the /apps/backup dir and from S3
for i in `find /apps/backup -name *.tgz -mtime +3| cut -d ‘/’ -f 4`; do echo s3cmd del s3://<bucket.domain.com>/$i; echo rm -f /apps/backup/$i; done
After the script is in place, make it executable with chmod +x /root/bin/cloud_backup. Once I have the script saved, I can add the job to cron so that it runs every night. Open your cron entries with crontab -e.
# Cloud Daily Backup
0 4 * * * /root/bin/cloud_backup
I have been backing up my EC2 cloud servers to S3 using a simple tar script to get the main data from the EBS mounts on each cloud server. I use cron to run the backup each night, saving the data first to the EBS mounts in an /apps/backup directory, then I copy the backup to the S3 storage. After the script runs successfully, I remove any old backups locally on EBS and S3 that are older than 3 days. This way I always have a 3 day rotation of the saved data.
In order to transfer to S3 storage, you will need the s3cmd, which can be downloaded here. The s3cmd will need to be stored inside the /root/bin directory unless you alter the cloud_backup script. Once you have the s3cmd files in place, run the s3cmd –configure command to configure it to your S3 instance. You will need your S3 access key and your secret key to complete the configuration.
I created the script in /root/bin and called it cloud_backup:
# Create the backup of certain /apps directories with the output as <HOSTNAME_DATE>.tgz
/bin/tar -czvf /apps/backup/`hostname`_`date +%m%d%y`.tgz /apps/apache /apps/data /apps/keys /apps/tomcat /apps/virtual /root && s3cmd put /apps/backup/`hostname`_`date +%m%d%y`.tgz s3://<bucket.domain.com>/`hostname`_`date +%m%d%y`.tgzfor i in `find /apps/backup -name *.tgz -mtime +3| cut -d ‘/’ -f 4`; do echo s3cmd del s3://<bucket.domain.com>/$i; echo rm -f /apps/backup/$i; done
# Move the backup file to S3 storage
/bin/tar -czvf /apps/backup/`hostname`_`date +%m%d%y`.tgz /apps/apache /apps/data /apps/keys /apps/tomcat /apps/virtual /root && s3cmd put /apps/backup/`hostname`_`date +%m%d%y`.tgz s3://<bucket.domain.com>/`hostname`_`date +%m%d%y`.tgz
# Look for backups older than 3 days and delete the files from the /apps/backup dir and from S3
for i in `find /apps/backup -name *.tgz -mtime +3| cut -d ‘/’ -f 4`; do echo s3cmd del s3://<bucket.domain.com>/$i; echo rm -f /apps/backup/$i; done
After the script is in place, make it executable with chmod +x /root/bin/cloud_backup. Once I have the script saved, I can add the job to cron so that it runs every night. Open your cron entries with crontab -e.
# Cloud Daily Backup
0 4 * * * /root/bin/cloud_backup
This entry was posted on November 4, 2009 at 9:46 am and is filed under Cloud (Tags: Amazon, backup, Cloud, cron, crontab, EBS, EC2, linux, S3, s3cmd, tar, UNIX).
You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.