Tag: website

Git The Master Race
I was working on a project, a side project for fun with a friend, and she expressed a slight worry. We weren’t using any versioning on our theme or mu-plugins code. Or anything else for that matter. There were three of us. We thought about using GitHub until I pointed out I had git on my server, where I was hosting it all.

In many ways I don’t use git like a ‘normal’ person, or so I’ve been told. I don’t have public git repo of everything because some of it is just private. I don’t even have a public GUI interface for people to file issues on my private code. Public code? Sure, and I use GitHub for it. Personal stuff is not up there, and I don’t care to make it.

What I do instead is, since I do have Git installed on this server, is allow each user account to make a git repository folder off their home folder. Everyone has ~/repositories/ and in there they can store their code!

Here are the ones for mine:
```
~/repositories/ipstenu-code.git/
~/repositories/ipstenu-mu-plugins.git/
~/repositories/mothra.git/
~/repositories/scripts.git/
~/repositories/slides.git/
```
The two that start ‘ipstenu’ are for the website ipstenu.org and all it’s children. Mothra is some server scripts I want to keep backed up, scripts are user scripts, and slides are my slides.

But how do I get there, and how do I keep everything synced up? First is the basic setup for git:
```
cd ~/repositories/
mkdir projectname.git
cd projectname.git
git --bare init
```
From here it’s a simple git clone, too:
```
git clone user@example.com:/home/user/repositories/projectname.git
```
But… I have that project named ipstenu-mu-plugins. When I’ve checked in code, do I manually copy the files over? Of course not.

When you go into that git folder where I ran the init, you see this:
```
[~/repositories/ipstenu-mu-plugins.git]# ls
./  ../  branches/  config  description  HEAD  hooks/  index  info/  objects/  refs/
```
If you go into the hooks folder, you get a list of files:
```
ipstenu@ipstenu.org [~/repositories/ipstenu-mu-plugins.git/hooks]# ls
./                      commit-msg.sample   pre-applypatch.sample      pre-push.sample
../                     post-update         pre-commit.sample          pre-rebase.sample
applypatch-msg.sample  post-update.sample  prepare-commit-msg.sample  update.sample
```
You’ll notice I only have one file that isn’t a .sample: post-update

That’s my secret sauce.
```
#!/bin/sh
export GIT_WORK_TREE=~/public_html/wp-content/mu-plugins
git checkout -f master
```
This just uses git checkout, forces the master branch, and dumps the files where I want them. I set it up for each repo when I’m ready to start deploying, and then my dev process is this.
1. Pull master and all remote branches to make sure I’m up to date
2. Make a new branch to develop
3. Push the new branch to origin when I’m sure it’s right
4. Merge the new branch into master
5. Push master
And in that moment, the push will copy my files live.

Since I don’t ever do dev work in master, this is safe for me. It’s five steps, but I actually have a script to run that first one every single day, so that’s almost step zero. When I work with other people, I always make a backup local branch of master as ‘last known good’ just so I can roll back fast if I’ve been phenomenally stupid. This is usually a date like “GOOD_20141117”, which I know is different from my actual release labels, that are always “REL_1.2.3” style. If it’s a weird thing, like when I was folding Ninja Forms in my site, it was “NINJAFORMS_1.0” because I wanted to be clear what that dev branch was for.

By the way, if you need to set up a shared repository, you should read Kovshenin’s “How To Create a Remote Shared Git Repository”. I don’t, since everyone logs in using SSH keys to the account (so on my shared project, everyone uses ssh projectaccount@projectdomain.com) since they actually need that access anyway.
17 November, 2014
Just Push Publish

“Real artists ship.” — Steve Jobs, 1983

I don’t write good all the time. I’m a little lazy and spell poorly, I don’t proofread enough, and if I had a genie to grant me a wish, one would be for an editor who I wasn’t related or married to. When I post a new article, I often see typos and while I do go back and fix them, I still push publish (or schedule), knowing things aren’t perfect.

This is a major departure from the “traditional” way of writing, when you write, have it reviewed, edit, re-write, re-edit, and so on. To many people, this is seen as ‘lazy’ writing, where we toss out things that are ‘good enough’ and call it a day, but the reality is that publishing promptly, be it writing or code, is what keeps up with the fast changing pace of news, information, and needs. But when it comes to writing, it falls a little bit under the aegis of “If you build it, they will come.” Or rather, if you don’t build it, they won’t come at all.

Publish or Perish

Well known to academia is the concept that if you don’t constantly publish works to sustain your career, you won’t have a career. The added pressure is that you have to publish fast so your information isn’t out of date before it hits the ground. The idea is that if you’re not publishing something then you’re not producing something, and you’re thereby sitting on your laurels. In software and blogging, this is actually important too! If you’re not producing code, or writing about it, you’re not demonstrating what you’ve learned. If you release code or write on your blog once a year, people will forget about you.

Release and Iterate

Also known as “Release early, release often,” this model of development makes important the concept of early and frequent releases. This necessitates people test, though, and developers respond quickly to issues reported by users. WordPress works by this model. Reid Hoffman, the founder of LinkedIn, said “If you are not embarrassed by the first version of your product, you’ve launched too late.” And if you look at many of the recent technological innovations (including the iPhone), version one was okay, but not great, and had a lot of bugs and annoyances.

Fear, Uncertainty, Doubt

The biggest hold up to most of us pushing that publish button is FUD. What if we’re wrong? What if we’re saying things no one cares about? What if… We don’t want to be horribly embarrassed by that typo where we get their/they’re/there wrong, or worse, where we get all that technical information wrong. And it’s that place of fear, that home of uncertainty, that realm of doubt, that we stop. We don’t share what we know, we don’t explain what we think, and we turtle up.

Democratizing Publishing

The mission of the WordPress open source project is to democratize publishing through Open Source, GPL, software. By letting any of us write what we want, we’re able to publish at will. That anyone can upload a book to Kindle or Apple and sell their works has changed the world. In many ways, it’s lowered the bar so anyone can sell anything which causes a dearth of quality. And yet, the stamp of quality products has never rested solely in the hands of ‘official’ publishers. Some of the best music we heard was from underground tapes made in basements. Some of the best stories we read were mimeographed in purple ink and handed out on the QT at fan conventions. All we’ve done here is take the barriers away and given you the freedom to say what’s on your mind.

Write the Change You Want to See

It takes bravery to post your thoughts, technical or personal, out there. You should only put out work you can stand behind, you should put the best work you can do out there, but you should be willing to post. You should be willing to release that code. You won’t grow, you can’t grow, if you don’t step up and put yourself out there.

Don’t worry. We know you’ll fix it.

30 July, 2014
La Vitesse

A little bit ago I talked about Varnish, how to install and configure it, and why I’m not using it at the moment. The actual goal of all this stuff is to speed up a website. Site speed is an insanely fickle beast, and measuring it without going insane is nigh impossible.

When we talk about site speed we don’t just mean how fast the site loads. We mean how well it performs on the front and back. Does it load everything it needs to be a page in a non-jumpy way? Does it load and then magically change to another format because you’re on an iPad? Does it hang and then load? We mean all those aspects that go into a site and make it zippy.

Which brings us to caching. The goal of caching is blindingly simple: Don’t put extra load on the server while serving up webpages and make it faster. The how of caching is crazy.

When I talked about Multisite Caching, I brought up the different types and why and where I saw each one being used. But I didn’t really explain why very well. In order to understand it, you need to understand why we need to cache.

If your website was all plain, static, HTML files, it would be really fast. The web was (initially) built for that stuff and it was all basic. “I want this page.” And the server would reply “Okay, here it is and some images.” When we start adding in stuff like dynamic PHP (WordPress), we put more load on the server because for every time someone visits your site, it has to ask WordPress “Do you have this page?” and WordPress has to check, see if it does, generate the output, and then it loads. The more complex the site is, the more big images and javascript fancy stuff, the slower the site gets.

Logical stuff, right? You’re also going to be limited by how fast your server is and how much of it you can use. If you’re on a dedicated server, the limit is your hardware and bandwidth pipe. If you’re on shared, though, the limit is lower, and really varied and complicated. While I mention a ‘bandwidth pipe’ and we techs always joke about the sturdy internet tubes, it’s not a fully accurate analogy, and even with all the bandwidth available in the world, the speed of your server is going to limit you far more.

There’s a phenomena called the “noisy neighbor” which impacts people on shared hosts a lot and is a lot of why people get confused about the bandwidth thing. You see, if you’re on shared servers, you share services. If one of your neighbors uses a lot of memory, there’s less available for you. This makes perfect sense, and hosts combat this by limiting how much you can do. I know a lot of companies say that you have ‘unlimited’ space and bandwidth, and while that’s true, it doesn’t mean you get to use all the power available to the server. Basically on shared servers, when you see ‘unlimited’ you should read it as ‘unlimited until you start making other people’s sites run worse.’

What does this have to do with caching? It’s the reason why we cache! WordPress does not make static HTML pages at all. If you look on your server for a file named ‘about’ you won’t find one. Instead, WordPress uses the .htaccess file to magically run your request for example.com/about/ through the index.php file, which then checks the database and pulls the content for that page. It’s entirely dynamic, and every single page request is run through the database. And yeah, that gets slow over time. The dynamism is fantastic though, and that’s why things like comments magically update the rendered page right away.

Thus, in order to make our super dynamic websites run by WordPress run faster, we turn to methods to generate static file caches. Converting a WordPress page from the PHP queries to a static file is complicated, and in essence every single tool has to generate that dynamic page, copy the output, and save it to a location where it can be pulled from directly. At the same time, it has to alter the server in some way to say “If I have a static file, use that instead.” When you use a plugin, generally it does this via your .htaccess file.

The obvious problem with this is that while the page may be faster for visitors, you’re still putting load on your server by having it generate these html files and serve them. And you, the logged in user, won’t get the cached page, generally, not even with something as cool as Varnish, so we have to still consider the rest of the server.

Speaking of Varnish … the simplest explanation I can give you about it is this: Instead of having WordPress use a plugin to generate the page, Varnish lets WordPress load, takes a snapshot of the resulting page, and saves it somewhere else. That means that in-between your visitor and the WordPress install is the Varnish cached page, which means the load is off your server more! No more loading the html page, Varnish is going to do it and make it a little faster. You’ll still want a plugin to allow WordPress to tell Varnish to delete pages, but it can significantly run faster.

But … what about the server speed itself? Is there a way to cache that and speed it up to? There is! But that’s a longer post, all it’s own.

21 July, 2014
Local Backups
You heard about CodeSpaces didn’t you?

On June 17th they got hit with a DDoS. It happens. On June 18th, the attacker deleted their data. And the backups. Because the backups were on the same server… You can read the story here and make up your own mind.

But that brings us to this. Are you making your own, personal, backups?

My server makes entire server backups every day and collocates them, but I also have my own backups of my own, personal, data. Not my email. If that blew up today I would lose nothing I can’t get back. That’s right, I don’t keep much email. If it’s important, I store it on my laptop, on iCloud or Dropbox, and I backup my laptop to TimeMachine. Oh and I check that backup regularly.

So how do I backup my sites? It’s three fold.

On the Server

I use a DB script from Daniel D Vork. He backs up files to DropBox, which is cool, but for me, I have this script on my server and it stores to a non-web-accessible folder called ‘backups’:
```
#!/bin/bash
 
USER="your_user"
PASSWORD="your_password"
OUTPUT="/Users/YOURUSERNAME/backups"
 
rm "$OUTPUT/*gz" > /dev/null 2>&amp;1
 
databases=`mysql --user=$USER --password=$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database`
 
for db in $databases; do
    if [[ "$db" != "information_schema" ]] &amp;&amp; [[ "$db" != _* ]] ; then
        echo "Dumping database: $db"
        mysqldump --force --opt --user=$USER --password=$PASSWORD --databases $db > $OUTPUT/`date +%Y%m%d`.$db.sql
        gzip $OUTPUT/`date +%Y%m%d`.$db.sql -f
    fi
done
```
That script is called every day at midnight via a cron job.

Bring it local

On my laptop, under the ~/Sites/ folder, I have a folder for each domain. So there’s one for ipstenu.org (which is where this site lives), and in there are the following:
```
backup-exclude.txt       backup.sh          log.txt
public_html/
```
The public_html folder is a full backup of my site files. It’s not that crazy, don’t panic.

The backup.sh file does an rsync:
```
#!/bin/sh

cd $(dirname $0)

TODAY=$(date)
echo "
-----------------------------------------------------
Date: $TODAY
Host: ipstenu.org hosted sites
-----------------------------------------------------\n" > log.txt

echo "Backup files..." >> log.txt
rsync -aCv --delete --exclude-from 'backup-exclude.txt' -e ssh backups@ipstenu.org:/home/ipstenu/public_html/ public_html > log.txt

echo "\nBackup databases..." >> log.txt
rsync -aCv --delete --exclude-from 'backup-exclude.txt' -e ssh backups@ipstenu.org:/home/ipstenu/backups/ databases >> log.txt

echo "\nEnd Backup. Have a nice day." >> log.txt
```
Backups is not the name of the account but I do have a backup only account for this. The backup-exclude.txt file it calls lists folders like ‘cache’ or ‘mutex’ so I don’t accidentally back them up! It’s simply just each file or folder name that I don’t want to backup on it’s own line. And yes, I like pretty output in my logs so I can read them when I’m having a brainless moment.

The cd $(dirname $0) at the beginning is so that I can call this from other folders. Remember! If your script uses relative paths to access local resources, then your script will break if you call if from another folder. This has a reason why in the next section.

Automate that shit!

I’m on a Mac. I decided I wanted that backup to run every time I logged in to my computer. Not rebooted, logged in. And waking from sleep. That became problematic, but let’s get into this code.

Writing the scripts

First I made a new folder called ~/Development/backups and I’ll be stashing my code there. In there I have a couple files. First is website-backup.sh:
```
#!/bin/sh

/Users/ipstenu/Sites/ipstenu.org/backup.sh
/Users/ipstenu/Sites/othersite.net/backup.sh
```
Basically for every site I want to run backup for, it’s in there. This is why I have the change-directory comment on the backup scripts.

The other file is my launchd file, called com.ipstenu.website-backups.plist and I got this code from stackexchange:
```
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>Label</key>
   <string>com.ipstenu.website-backups</string>
   <key>Program</key>
   <string>/Users/ipstenu/Development/backups/website-backup.sh</string>
   <key>RunAtLoad</key>
   <true/>
</dict>
</plist>
```
Instead of copying the file, though, I did a symlink:
```
ln -sfv /Users/ipstenu/Development/backups/com.ipstenu.website-backups.plist ~/Library/LaunchAgents
```
This lets me change it if I need to, which I doubt I will. I’ll just edit that .sh script. The filename of the plist is intentional to tell me what the heck it is.

But wait, what about waking from sleep? Logging in from a sleeping computer is not the same as a log in to a Mac, and there’s no built in tool to monitor sleep and wake for some reason. There are apps that can do it, but there’s also SleepWatcher, which can be installed via Brew! Since I’m running an rsync, it’s not a big deal to run multiple times a day. Heck it may actually be faster.

First we install Sleepwatcher:
```
brew install sleepwatcher
```
Now Sleepwatcher looks for user scripts named ~/.sleep and ~/.wakeup which sure makes my life easier. My ~/.wakeup file calls website-backup.sh, and while I could have it repeat the code, I chose not to for a reason. I know my backup scripts will live in ~/Development/backups/ so I can add a new one for something else without messing around with more than one file.

Do you remember launchd a moment ago? We want to use that again to tell Sleepwatcher it’s okay to run on startup or login. This time, since we’re only using Sleepwatcher for sleep and wake, we can symlink the sample files to the proper LauchAgents directories. In my case, it’s only running for me, so it’s all local:
```
ln -sfv /usr/local/Cellar/sleepwatcher/2.2/de.bernhard-baehr.sleepwatcher-20compatibility-localuser.plist ~/Library/LaunchAgents
```
If you’re interested in doing more with sleepwatcher, read Mac OS X: Automating Tasks on Sleep by Kodiak.

Finally we’re going to load both of these commands into launchctl:
```
launchctl load ~/Library/LaunchAgents/com.ipstenu.website-backups.plist
launchctl load ~/Library/LaunchAgents/de.bernhard-baehr.sleepwatcher-20compatibility-localuser.plist
```
Now every time I log in on my laptop, it runs a backup, be that a real login, or a wake-from-sleep one.

And remember, this is on top of my full server backups and my personal git repository for my code, so I have my data backed up in the important places. Everything on my laptop is backed up to the TimeMachine, so really I can just look back a year or three and find that html file I used once.

The other thing I do is check these backups pretty regularly. I scheduled a day every month to check that everything’s working right, that the files are restorable, and that I feel secure. Thus far, the most I’ve lost has been 16 hours of work on a Wiki.
20 June, 2014
HSTS and Chromium
I use Chromium, which is like the super most beta beta of Chrome, for one reason: It lets me run Chrome ‘twice’ in separate environments. I have a reason for it, but it doesn’t matter for the purpose of this post. Neither does the reason why I was editing a subdomain I rarely use. What matters is this.

I went to http://test.ipstenu.org and I got this:

My bp must have dropped because my first thought was “Oh shit, I’ve been hacked!” and I got the cold sweats. After a rant on Twitter and a plea for test help, I determined this was only happening on Chromium, which was interested, and my friend Benny pointed me to this Chromium Issue: Issue 350912: Chrome fetches https URL even when http URL explicitly specified which sure sounded right.

Now the rest of the error was telling me that test.ipstenu.org was asking I use HTTPS (which it was not) and that the SSL Cert didn’t match that domain (which was correct, but again, there is no SSL cert for that subdomain). Weird, right? Why would it force https when I didn’t ask it to! But what if I was? As I read through the ticket, I saw that you could see the headers better at chrome://net-internals/#hsts so I ran that and got this:
```
mode: STRICT sts_include_subdomains:true pkp_include_subdomains:false sts_observed:1402609861.497659 pkp_observed:1402608721.515596 domain:test.ipstenu.org pubkey_hashes: [...]
```
That was odd. I knew I’d set this in my .htaccess file:
```
Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains"
```
But from what I’d read that was getting set only if you accessed the site via HTTPS (or anything on the page was HTTPS). After all, no other browser was acting like this, not even Beta Chrome. Still I turned it off, flushed my server cache, flushed my browser cache, and deleted the TransportSecurity files from Chrome. Now it worked. Turned it back on? Error came back.

It was here I realized “Chromium has decided that if you set HSTS, you mean that everything should be HTTPS regardless of the domain…” Well that’s just peachy! I pulled the HSTS line from .htaccess and instead tossed it into a PHP file for the subdomain(s) where I needed it:
```
header("Strict-Transport-Security:max-age=15768000; includeSubDomains");
```
Now. There are two situations particular to me that make this an epic pain in the ass:

First, I’m using WordPress multisite and I have SSL turned on for one domain. Second, I’m using WMH, which means editing my vhosts file to make this work properly is more of a hassle than it’s worth.

Really wouldn’t it make more sense to check “Is this an HTTPS page? No? Okay, skip!” But this is a (to me) departure from the logic train. As my friend James (and I) understood, the logic of HSTS was “If any aspect of the page is HTTPS, force HTTPS” Instead of that happening, if you went to, say, ‘fake.ipstenu.org’ instead of going to my about page (yes, that’s on purpose), it gave me the error. If you go to ‘https://fake.ipstenu.org’ you should get a specific warning page and then a ‘shucky darn!’ page. And yet here I saw that no matter if I picked HTTPS or HTTP, I was forced to HTTPS and that’s just silly.

Does my fix work? Of course. Am I happy? Nope! Oh, and yes, I filed a bug report with Chromium. You can read it at Issue 384069: HSTS in .htaccess forces https even when not specified
13 June, 2014
Bad Browser Complacency

Back in the day, you may have seen notices on browsers like “This site best viewed in Internet Explorer v4.” Since then, we’ve moved into the belief that a website should work on as many browsers as possible, and degrade nicely when it can’t. So imagine my surprise when I’m looking at photos online and I get a message saying I’m using an unsupported browser.

I happened to be using Chrome on my iPad, so I clicked the link which took me to http://www.corbisimages.com/BadBrowser and I saw this:

I was taken aback. Not that they say ‘IE 7 and up’ as it’s something I try to support as well. Frankly making sites look awesome on IE is about as easy and fun as a dental student doing your root canal. You’ll get there in the end, but it may hurt like hell.

Now, Corbis has certainly one of the more out of date designs for a photo sites I regularly visit (there are 10), but it’s not the most egregious. Yes, there are worse ones. Still for a site to have an alert like this in 2014 and to omit Chrome is rather shocking. Someone remarked it looked like that site was designed circa 2003 and Chrome, if you didn’t know, only came out in 2008. So while I remember this new design for Corbis being rather recent (2012 or 2013, but that may have just been some tweaks), it’s clear they’ve not visited that page in at least six years.

I talked to my family about web design at a recent brunch, stressing that I do not do website design per se, but I am happy to help them find people and upload their content. At that time, I pointed out that the trick to a website was to frequently update it and make it more modern. “It’s like the runway shows,” I replied. “What’s in this season was weird last season and may be out by next season. So making a site and never changing it is as smart as never updating your wardrobe.”

People judge by how things look. If someone only wears a black turtleneck and jeans (Steve Jobs), we create a specific mindview of them and it rarely changes. Someone who always wears avant-garde clothes that are nearly unwearable (Katy Perry), we create another. If that person always wears a suit jacket (Tim Gunn), we have yet another view. Neither is right or wrong, of course, and they all have their places.

We update our wardrobes when we gain and lose weight, when we decide we want a change, when we feel different, when we have to change, when we want to. While I tease that Brian Gardner is never satisfied with a web design and is always changing, I’m often just as guilty of this as I don’t feel things fit forever. If I’m not afraid of changing my wardrobe, why would I be afraid of changing my website?

And yet. We worry a lot more about the change of design, to the point that sites like Corbis haven’t significantly changed or adapted since 2002, when the site was born. Since then, 12 years have gone by, browsers have changed, security changed, and the viewing experience is wildly different. Corbis on a phone? Yeah not a great experience.

When your site never changes with the times, never grows to adapt to it’s new audience, you lose respect in the world of the Internet. We have to keep up with the times, test and retest on as many browsers as humanly possible, and make sure that it all works. We can’t just say “Yes, this one design is good” and more so, we can’t say “This site works best on…” anymore.

Unless you’re the sort to say “This jeans only work with turtlenecks.” Then, by all means, never change.

2 May, 2014