Tag: essay

You heard about CodeSpaces didn’t you?

On June 17th they got hit with a DDoS. It happens. On June 18th, the attacker deleted their data. And the backups. Because the backups were on the same server… You can read the story here and make up your own mind.

But that brings us to this. Are you making your own, personal, backups?

My server makes entire server backups every day and collocates them, but I also have my own backups of my own, personal, data. Not my email. If that blew up today I would lose nothing I can’t get back. That’s right, I don’t keep much email. If it’s important, I store it on my laptop, on iCloud or Dropbox, and I backup my laptop to TimeMachine. Oh and I check that backup regularly.

So how do I backup my sites? It’s three fold.

On the Server

I use a DB script from Daniel D Vork. He backs up files to DropBox, which is cool, but for me, I have this script on my server and it stores to a non-web-accessible folder called ‘backups’:

#!/bin/bash
 
USER="your_user"
PASSWORD="your_password"
OUTPUT="/Users/YOURUSERNAME/backups"
 
rm "$OUTPUT/*gz" > /dev/null 2>&1
 
databases=`mysql --user=$USER --password=$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database`
 
for db in $databases; do
    if [[ "$db" != "information_schema" ]] && [[ "$db" != _* ]] ; then
        echo "Dumping database: $db"
        mysqldump --force --opt --user=$USER --password=$PASSWORD --databases $db > $OUTPUT/`date +%Y%m%d`.$db.sql
        gzip $OUTPUT/`date +%Y%m%d`.$db.sql -f
    fi
done

That script is called every day at midnight via a cron job.

Bring it local

On my laptop, under the ~/Sites/ folder, I have a folder for each domain. So there’s one for ipstenu.org (which is where this site lives), and in there are the following:

backup-exclude.txt       backup.sh          log.txt
public_html/

The public_html folder is a full backup of my site files. It’s not that crazy, don’t panic.

The backup.sh file does an rsync:

#!/bin/sh

cd $(dirname $0)

TODAY=$(date)
echo "
-----------------------------------------------------
Date: $TODAY
Host: ipstenu.org hosted sites
-----------------------------------------------------\n" > log.txt

echo "Backup files..." >> log.txt
rsync -aCv --delete --exclude-from 'backup-exclude.txt' -e ssh backups@ipstenu.org:/home/ipstenu/public_html/ public_html > log.txt

echo "\nBackup databases..." >> log.txt
rsync -aCv --delete --exclude-from 'backup-exclude.txt' -e ssh backups@ipstenu.org:/home/ipstenu/backups/ databases >> log.txt

echo "\nEnd Backup. Have a nice day." >> log.txt

Backups is not the name of the account but I do have a backup only account for this. The backup-exclude.txt file it calls lists folders like ‘cache’ or ‘mutex’ so I don’t accidentally back them up! It’s simply just each file or folder name that I don’t want to backup on it’s own line. And yes, I like pretty output in my logs so I can read them when I’m having a brainless moment.

The cd $(dirname $0) at the beginning is so that I can call this from other folders. Remember! If your script uses relative paths to access local resources, then your script will break if you call if from another folder. This has a reason why in the next section.

Automate that shit!

I’m on a Mac. I decided I wanted that backup to run every time I logged in to my computer. Not rebooted, logged in. And waking from sleep. That became problematic, but let’s get into this code.

Writing the scripts

First I made a new folder called ~/Development/backups and I’ll be stashing my code there. In there I have a couple files. First is website-backup.sh:

#!/bin/sh

/Users/ipstenu/Sites/ipstenu.org/backup.sh
/Users/ipstenu/Sites/othersite.net/backup.sh

Basically for every site I want to run backup for, it’s in there. This is why I have the change-directory comment on the backup scripts.

The other file is my launchd file, called com.ipstenu.website-backups.plist and I got this code from stackexchange:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>Label</key>
   <string>com.ipstenu.website-backups</string>
   <key>Program</key>
   <string>/Users/ipstenu/Development/backups/website-backup.sh</string>
   <key>RunAtLoad</key>
   <true/>
</dict>
</plist>

Instead of copying the file, though, I did a symlink:

ln -sfv /Users/ipstenu/Development/backups/com.ipstenu.website-backups.plist ~/Library/LaunchAgents

This lets me change it if I need to, which I doubt I will. I’ll just edit that .sh script. The filename of the plist is intentional to tell me what the heck it is.

But wait, what about waking from sleep? Logging in from a sleeping computer is not the same as a log in to a Mac, and there’s no built in tool to monitor sleep and wake for some reason. There are apps that can do it, but there’s also SleepWatcher, which can be installed via Brew! Since I’m running an rsync, it’s not a big deal to run multiple times a day. Heck it may actually be faster.

First we install Sleepwatcher:

brew install sleepwatcher

Now Sleepwatcher looks for user scripts named ~/.sleep and ~/.wakeup which sure makes my life easier. My ~/.wakeup file calls website-backup.sh, and while I could have it repeat the code, I chose not to for a reason. I know my backup scripts will live in ~/Development/backups/ so I can add a new one for something else without messing around with more than one file.

Do you remember launchd a moment ago? We want to use that again to tell Sleepwatcher it’s okay to run on startup or login. This time, since we’re only using Sleepwatcher for sleep and wake, we can symlink the sample files to the proper LauchAgents directories. In my case, it’s only running for me, so it’s all local:

ln -sfv /usr/local/Cellar/sleepwatcher/2.2/de.bernhard-baehr.sleepwatcher-20compatibility-localuser.plist ~/Library/LaunchAgents

If you’re interested in doing more with sleepwatcher, read Mac OS X: Automating Tasks on Sleep by Kodiak.

Finally we’re going to load both of these commands into launchctl:

launchctl load ~/Library/LaunchAgents/com.ipstenu.website-backups.plist
launchctl load ~/Library/LaunchAgents/de.bernhard-baehr.sleepwatcher-20compatibility-localuser.plist

Now every time I log in on my laptop, it runs a backup, be that a real login, or a wake-from-sleep one.

And remember, this is on top of my full server backups and my personal git repository for my code, so I have my data backed up in the important places. Everything on my laptop is backed up to the TimeMachine, so really I can just look back a year or three and find that html file I used once.

The other thing I do is check these backups pretty regularly. I scheduled a day every month to check that everything’s working right, that the files are restorable, and that I feel secure. Thus far, the most I’ve lost has been 16 hours of work on a Wiki.