Author: Ipstenu (Mika Epstein)

La Vitesse 2: Cruise Control

Now you know all about caching and how it works. But what about speeding up the server itself?

Near the end of the previous post, I mentioned that all the caching in the world didn’t really speed up the server. This is somewhat of a lie, since if, say, you’re using Varnish to cache your site, then your visitors won’t be hitting your WordPress install, speeding it up for you to do work. But it’s not the full picture.

WordPress is big and it’s getting bigger and more complex and more impressive. So is Drupal and … well pretty much everything else. In order to make your site do more, like all that super fancy layout transformations, we have to upgrade and innovate. But then you start getting into extending these apps, like using custom fields and extra meta values to store more information so you can change search results in more impressive ways! Your site scrolls and changes backgrounds! Your site dynamically changes what products are available based on check boxes, without reloading!

What did that have to do with caching? Well … how do you cache things that aren’t static?

My coworker, Mike, likes to talk about things that should be cached and things that should never be cached. Things that have to be dynamic and run without a page refresh, like ajax and javascript, can be cached to an extent, since those plugins and Varnish will just keep the code in-line, which means it’ll still run. But when you start looking at dynamic things like shopping carts, we hit a new world and a new wall. But I’m not even talking about that level of caching. I’m talking about going back a layer into the part where WordPress (or any app) has PHP query the database. If we speed that up, caching safe content, can’t we speed things up? You bet we can!

A few years ago I talked about APC and how I was using it to speed up PHP by having it cache things. Then less than a year later, I switched to Zend and memcached. I did those things because I decided that it would be better to have my server, a workhorse, do the hard work instead of asking WordPress to do it. And this was, in general, a pretty good idea.

Memcached is an add-on for your server, and acts as “an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.” In English? It remembers what you did and can retrieve results faster because of it. Of course, you have to tell your apps to play well with it, but once you do, the backend of your site starts to speed up because, hey, caching! The Illustrated Memcache’d story is kind of an awesome explanation (the images on the archive page are broken, but the links work). And yes, I do use memcached and ZendOptomizer+ on my server, because it really does make things faster, even when two of the ten domains are having a 10k pageviews in a day.

I keep telling everyone my server isn’t overkill….

The point of that, though is the other end of speed is totally separate from your application code. When you install WordPress, you know it runs SQL and PHP, so if you can make those faster, WordPress will be faster. The same applies to speeding up Apache, maybe by putting Nginx in front of it, or maybe by tuning the hard to understand settings in httpd.conf to make sure that it elegantly handles the 300 people who come running to your site at once.

But unlike plugins, this aspect of server speed is the hard stuff. When you look at WP Super Cache, you think it’s complicated until you see W3 Total Cache. Both are amazing and awesome, but they’re giving you a ton of options and asking you to make decisions. The same is true of the servers, but now you have even more options. The world is your oyster, and we haven’t even looked at hardware.

For me, it boils down to how can I make my server do it’s job fast and efficiently. If that means I hire someone to install software to cache better, or I pay for more memory, then that’s what I do. I also trim the fat and get ride of everything I’m not using and don’t need, so my server doesn’t have to do more than it needs to. And one day, one day, I’ll be looking at nginx for everything.

23 July, 2014
La Vitesse

A little bit ago I talked about Varnish, how to install and configure it, and why I’m not using it at the moment. The actual goal of all this stuff is to speed up a website. Site speed is an insanely fickle beast, and measuring it without going insane is nigh impossible.

When we talk about site speed we don’t just mean how fast the site loads. We mean how well it performs on the front and back. Does it load everything it needs to be a page in a non-jumpy way? Does it load and then magically change to another format because you’re on an iPad? Does it hang and then load? We mean all those aspects that go into a site and make it zippy.

Which brings us to caching. The goal of caching is blindingly simple: Don’t put extra load on the server while serving up webpages and make it faster. The how of caching is crazy.

When I talked about Multisite Caching, I brought up the different types and why and where I saw each one being used. But I didn’t really explain why very well. In order to understand it, you need to understand why we need to cache.

If your website was all plain, static, HTML files, it would be really fast. The web was (initially) built for that stuff and it was all basic. “I want this page.” And the server would reply “Okay, here it is and some images.” When we start adding in stuff like dynamic PHP (WordPress), we put more load on the server because for every time someone visits your site, it has to ask WordPress “Do you have this page?” and WordPress has to check, see if it does, generate the output, and then it loads. The more complex the site is, the more big images and javascript fancy stuff, the slower the site gets.

Logical stuff, right? You’re also going to be limited by how fast your server is and how much of it you can use. If you’re on a dedicated server, the limit is your hardware and bandwidth pipe. If you’re on shared, though, the limit is lower, and really varied and complicated. While I mention a ‘bandwidth pipe’ and we techs always joke about the sturdy internet tubes, it’s not a fully accurate analogy, and even with all the bandwidth available in the world, the speed of your server is going to limit you far more.

There’s a phenomena called the “noisy neighbor” which impacts people on shared hosts a lot and is a lot of why people get confused about the bandwidth thing. You see, if you’re on shared servers, you share services. If one of your neighbors uses a lot of memory, there’s less available for you. This makes perfect sense, and hosts combat this by limiting how much you can do. I know a lot of companies say that you have ‘unlimited’ space and bandwidth, and while that’s true, it doesn’t mean you get to use all the power available to the server. Basically on shared servers, when you see ‘unlimited’ you should read it as ‘unlimited until you start making other people’s sites run worse.’

What does this have to do with caching? It’s the reason why we cache! WordPress does not make static HTML pages at all. If you look on your server for a file named ‘about’ you won’t find one. Instead, WordPress uses the .htaccess file to magically run your request for example.com/about/ through the index.php file, which then checks the database and pulls the content for that page. It’s entirely dynamic, and every single page request is run through the database. And yeah, that gets slow over time. The dynamism is fantastic though, and that’s why things like comments magically update the rendered page right away.

Thus, in order to make our super dynamic websites run by WordPress run faster, we turn to methods to generate static file caches. Converting a WordPress page from the PHP queries to a static file is complicated, and in essence every single tool has to generate that dynamic page, copy the output, and save it to a location where it can be pulled from directly. At the same time, it has to alter the server in some way to say “If I have a static file, use that instead.” When you use a plugin, generally it does this via your .htaccess file.

The obvious problem with this is that while the page may be faster for visitors, you’re still putting load on your server by having it generate these html files and serve them. And you, the logged in user, won’t get the cached page, generally, not even with something as cool as Varnish, so we have to still consider the rest of the server.

Speaking of Varnish … the simplest explanation I can give you about it is this: Instead of having WordPress use a plugin to generate the page, Varnish lets WordPress load, takes a snapshot of the resulting page, and saves it somewhere else. That means that in-between your visitor and the WordPress install is the Varnish cached page, which means the load is off your server more! No more loading the html page, Varnish is going to do it and make it a little faster. You’ll still want a plugin to allow WordPress to tell Varnish to delete pages, but it can significantly run faster.

But … what about the server speed itself? Is there a way to cache that and speed it up to? There is! But that’s a longer post, all it’s own.

21 July, 2014
Mailbag: Playing the Middle

This is from Ben in Minnesota and … It’s not about WordPress as much as learning and support, but here is the meat of his issue:

I’m just learning things. I’m really familiar with Drupal and okay with a vps, but I took over a WP install on a dedicated server and I’m way out of my league! I don’t understand half the questions. They treat me like I should know everything already because I’m experienced and tell me to just ask the vendor. But the hardware scares me and I don’t know how to get the information I need to solve things!

Do you have any advice, besides learning faster?

My least favorite role is when I have to play the middle man between two tech groups. Group A has a problem, so they ask me to ask it of Group B, and I have no familiarity with what the subject is. Happens a hell of a lot, and it exposes the lack of depth of knowledge in specific areas.

I hate it. It makes me feel like I’m stupid, and then when I ask for clarification, I get vague, top-level answers and what I need are examples. Much of this has to do with how I learn best, but the other problem is people have a tacit assumption that I know what the hell they’re talking about, when I clearly do not.

Basically? They’re giving me shitty support based on their preconceived notions about how “everyone” thinks. And yes, it pisses me off and I have been at the point of tears of anger me frustration over this before. I’ve been there, man, and recently too. It’s worse probably because I am clever and can pick things up quickly. They assume I know, or will figure it out, so I get half-assed help.

So. What do I do? Well first I quote them. “My DB guy said this [quote]. Do you need any specific information? I’ll ask him, but I’m not familiar with this topic.” Sadly that tends to net me a pretty generic reply like “Just filter it.” It does make me want to scream, you’re not alone there.

Lately I’ve been stopping them before it gets that far, though. When I’m told “Can you ask Group B about this?” I say “Can you explain like I’m 5, real fast, so I can make sure I ask them the right things and make sure that I don’t have to go back and forth really a million times and bug the hell out of you?” If I already understand a little about it, I may say “I thought that ModSecurity could hook into IP Tables and auto-block people who hammered my login files?” to set the tone of what I did know.

Basically the only path out of ignorance is to explain that you are uneducated in this topic, and while you will learn as fast as you can, you need a little more help than that. If they still won’t help you out, take them aside and ask if you’re doing something wrong, because you need their help in a different way than you’re getting. Be firm. Be up front. Be honest.

Good luck, Ben! And just for some fun, here’s a scene from Office Space:

People skills!

18 July, 2014
DreamUp Security
Not that long ago, I did a ‘DreamUp’ for my company, where we held a Google Hangout and I talked about WordPress security and how to be smarter about things. You can catch the video here:

http://www.youtube.com/watch?v=Uu-3-o80rEE

One thing I tried not to do was to list too many plugins and too much code because a lot of security talks are about how we’re all dooooomed, learn all this code. The concept of security is to be smarter about things, so to simplify it, I wanted to talk about the silly things we all do that make us LESS secure, and how to start thinking about what we do to know it’s smarter.

Here are some of the links from my talk:
16 July, 2014
Diving Into Varnish
We use it at DreamPress a lot, and I’m still learning its ways, but with me, the best way to learn a thing is to do a thing. So when I had a random server crash with nginxcp, I decided to play around and see about using Varnish on my server instead.

Varnish is an HTTP accelerator designed for content-heavy dynamic web sites (like WordPress). Unlike nginx, there’s no support for SPDY or SSL, which I can’t use anyway unless I spring for another server in front of my Apache box to be a true nginx box. Since I wasn’t getting any benefits out of nginx for those, I’m not too worried about it here yet. Should the world go to SSL, then my POV will change. The Varnish gurus aren’t fans of SPDY as it happens, which I find fascinating.

Back on point. I’m going to use Varnish as a proxy, which means when someone comes to my server to ask for a file, Varnish will first check itself for a cache and then if it’s found, serve it without touching Apache. Apache is slow. This is good! While nginx can handle static files rather well, I found that where I ht slowness people told me to use a CDN. That’s nice, but I don’t want to right now, so it makes nginx less of a draw. On the other hand, Varnish will fill in the gap where Apache + mod_php == poor static-file performance. And yes, I’m using mod_php.

Installing Varnish

First change Apache non-SSL port to 8080. I’m on WHM for this particular box, so I go to WHM -> Server Configurarion -> Tweak Settings and set value of field Apache non-SSL IP/port to 8080

Next I install the Varnish RPM for RedHat REL6. This can be either Varnish 3x or 4x, but I picked the latest version.
```
rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.0.el6.rpm
yum install varnish
```
Edit the config file – /etc/sysconfig/varnish – and set the VARNISH_LISTEN_PORT to 80.

Now we edit /etc/varnish/default.vcl with what we want.

Deep breath. A whole heckuvalot changed from 3.x to 4.x and it took me a couple hours to bang out, since my examples were all from Varnish 3.x. In the end, I made my own fork of DreamHost’s Varnish VCL. Grab my Varnish VCL Collection and I use the wordpress-example.vcl as my default. It’s a whole ‘nother post on how I did that one. A lot of trial and error.

The default VCL is skewed to WordPress in a specific way: If you’re logged in or have a cookie that isn’t the default WP cookie, or are on SSL, you do not get cached pages. This means my site will be slower for me.

Configuring Your CMS

Speaking of WordPress… Here’s the major difference between it an nginx: I need a plugin for WordPress. I took over Varnish HTTP Purge last year in order to fix it (instead of fork it) for DreamPress, and in doing so I’ve added a lot of little tweaks, like a ‘purge all’ feature and a button on the toolbar.

Oddly, this is the reason I didn’t want to use Varnish. Where nginx just works, needing a plugin means I have to either install and activate for everyone using WordPress or any other CMS on my system, or I have to figure out a way to not need a plugin? Oh, and I don’t just used WordPress. Ugh.

This is moderately trivial to do with Mediawiki but I came up full short when I looked at Zenphoto. While I don’t post often to it (once a week generally), I do post a lot of data and I need the purge to be done. Certainly I could code in a system for it, like I did with WordPress, using a CURL call.

But it’s the need to do that for Varnish that made me make faces.

Not using Varnish

At the end of the day, while I did get Varnish up and running, I chose not to use it. Yet. I have to overcome some hurdles with other apps not knowing how to play well with purging, and figure out how to command purges like I do with WordPress. You can see I have my work cut out for me porting a WordPress plugin to Zenphoto.

In addition, I’m not really sure I like the fact that I have to do that. Certainly I can just let the cache expire on it’s own, but that seems to somewhat defeat the purpose of having it be able to handle dynamism as well as it does if it can’t magically detect when my content changes, and the cache needs a bump.
14 July, 2014
Mailbag: Multisite Theme Activation
Briany from Ireland has a cool idea (put the guilt away, Briany, I think this is a pretty wild concept and I like it). This is the sort of ‘support’ email I love because it’s not a yes/no answer, but a theoretical concept to think about and parse!

Here’s the meat of the email:

SO MY FASCINATING QUESTION IS THIS; In a multi site network is there any way to use 2 separate themes for each or any sub-site based on the URL used (or any other method) Example 1 Visit www.sitename.platform.com and view the whole site using theme A (Standard) Visit www.sitename.com and view the whole site using theme B (custom)

At first I thought “Briany can’t possibly be asking ‘How do I activate a separate theme per site.’” and then I realized the question was for the network! So to phrase it in WordPress terms, if I visit the site via foo.example.com, I get theme foo on every site on the network.

(If you just want Site A to have one theme, and Site B another, that’s easy. Either network activate the theme and select it on the site, or go to WP Admin -> Network -> Sites, click on edit, click on the themes tab, and activate the theme you want for the sites. Then go back to the site and select that theme. A couple steps, yes, but relatively easy.)

Theoretically yes, yes you can. There’s a plugin called Theme Switcher which lets users pick, and based on some code on StackExchange, you can change the theme based on users, but the issue here is that you want to only change it per domain for that user.

It’s certainly possible to change settings by detecting the domain, I do that with SSL.
```
if ( $_SERVER["HTTP_HOST"] == "foo.example.com" ) {
    // My Code Here
}
```
That’s the easy part. The hard part is keeping that setting when I go from foo.example.com to example.com…

I sat and kicked this idea around for a while. It would be much easier on a single install of WordPress, since I could make everything relative and then just use the host name. But you have to have a way to track the starter domain, and have it be per-visitor, which means you have to use cookies, and read from that, using setcookie() (which is a PHP thing, not WP specific).

At that point, I think I would close the book and say “No, it’s possible, but not a good idea.”

Why, you ask? Caching. How the hairy hell could I possibly cache that if the theme changes every time for every user? Maybe, maybe, I would do it with multi-networks, and define a theme per network, but not per-domain. Obviating any caching would pretty much kill my sites, and even a good opcode cache (I use memcache) will be usless in that scenario.

By the way, there are a lot of neato plugins to change themes based on weird things, like Domain Theme, which is great for single installs of WordPress.
11 July, 2014