Tag: cache

Diving Into Varnish
We use it at DreamPress a lot, and I’m still learning its ways, but with me, the best way to learn a thing is to do a thing. So when I had a random server crash with nginxcp, I decided to play around and see about using Varnish on my server instead.

Varnish is an HTTP accelerator designed for content-heavy dynamic web sites (like WordPress). Unlike nginx, there’s no support for SPDY or SSL, which I can’t use anyway unless I spring for another server in front of my Apache box to be a true nginx box. Since I wasn’t getting any benefits out of nginx for those, I’m not too worried about it here yet. Should the world go to SSL, then my POV will change. The Varnish gurus aren’t fans of SPDY as it happens, which I find fascinating.

Back on point. I’m going to use Varnish as a proxy, which means when someone comes to my server to ask for a file, Varnish will first check itself for a cache and then if it’s found, serve it without touching Apache. Apache is slow. This is good! While nginx can handle static files rather well, I found that where I ht slowness people told me to use a CDN. That’s nice, but I don’t want to right now, so it makes nginx less of a draw. On the other hand, Varnish will fill in the gap where Apache + mod_php == poor static-file performance. And yes, I’m using mod_php.

Installing Varnish

First change Apache non-SSL port to 8080. I’m on WHM for this particular box, so I go to WHM -> Server Configurarion -> Tweak Settings and set value of field Apache non-SSL IP/port to 8080

Next I install the Varnish RPM for RedHat REL6. This can be either Varnish 3x or 4x, but I picked the latest version.
```
rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.0.el6.rpm
yum install varnish
```
Edit the config file – /etc/sysconfig/varnish – and set the VARNISH_LISTEN_PORT to 80.

Now we edit /etc/varnish/default.vcl with what we want.

Deep breath. A whole heckuvalot changed from 3.x to 4.x and it took me a couple hours to bang out, since my examples were all from Varnish 3.x. In the end, I made my own fork of DreamHost’s Varnish VCL. Grab my Varnish VCL Collection and I use the wordpress-example.vcl as my default. It’s a whole ‘nother post on how I did that one. A lot of trial and error.

The default VCL is skewed to WordPress in a specific way: If you’re logged in or have a cookie that isn’t the default WP cookie, or are on SSL, you do not get cached pages. This means my site will be slower for me.

Configuring Your CMS

Speaking of WordPress… Here’s the major difference between it an nginx: I need a plugin for WordPress. I took over Varnish HTTP Purge last year in order to fix it (instead of fork it) for DreamPress, and in doing so I’ve added a lot of little tweaks, like a ‘purge all’ feature and a button on the toolbar.

Oddly, this is the reason I didn’t want to use Varnish. Where nginx just works, needing a plugin means I have to either install and activate for everyone using WordPress or any other CMS on my system, or I have to figure out a way to not need a plugin? Oh, and I don’t just used WordPress. Ugh.

This is moderately trivial to do with Mediawiki but I came up full short when I looked at Zenphoto. While I don’t post often to it (once a week generally), I do post a lot of data and I need the purge to be done. Certainly I could code in a system for it, like I did with WordPress, using a CURL call.

But it’s the need to do that for Varnish that made me make faces.

Not using Varnish

At the end of the day, while I did get Varnish up and running, I chose not to use it. Yet. I have to overcome some hurdles with other apps not knowing how to play well with purging, and figure out how to command purges like I do with WordPress. You can see I have my work cut out for me porting a WordPress plugin to Zenphoto.

In addition, I’m not really sure I like the fact that I have to do that. Certainly I can just let the cache expire on it’s own, but that seems to somewhat defeat the purpose of having it be able to handle dynamism as well as it does if it can’t magically detect when my content changes, and the cache needs a bump.
14 July, 2014
Mailbag: Multisite Caching

Dave has a question abtou Multisite and caching!

I asked about caching on multisite setups. I’ve got 10 of these, with between 10-50 sites added to each- and the content is static- never going to change it much. I have a feeling I could benefit from caching, you recommended server side. I could use some help here. I know your site says you’re maxed out time wise- I’m thinking I just need someone to help me with one site, and show me what to do. Or, if they want to set things up on all of them. Do you have any recommendations?

Thanks again for the help.

Dave

Caching! There are three main types: plugin, server, and proxy.

Plugin caching is WP Super Cache, or W3TC, and it’s letting WP make a static cache file on demand. Server caching is APC/Memecached and such, which lets the server make a dynamic cache. Proxy caching is Varnish and the like, which puts a cache before the WordPress server, kind of like a static file on another server.

Which one is best for Multisite with static content? Probably a proxy cache there. Though I’m going to go out on a limb and say if you’re not using WordPress for all it’s useful for, maybe you don’t need it at all. If you have a pretty static site, where you’re not changing content all that much, there’s nothing wrong with static HTML. But if you want to use WordPress (like you’re a WordPress only shop) then you have to consider the options. What is your site doing?

This is actually not a Multisite Question so much as a cache management question. The part that’s multisite specific is who to you let manage what aspect of the cache. I maintain a plugin called Varnish HTTP Purge, which is a pretty basic plugin that checks for common actions, like posting or commenting, and flushes your cache selectively on Varnish when those things happen. One of the Multisite specific issues with that code came up when I added in a new feature: Purge Cache.

That feature lets you press a button to flush the whole cache. Pretty cool, right? The problem was Multisite with Subfolders. See, the way the cache flushes is to say “Dump everything under domain.com!” which works fine with subdomain sites, as that will only flush domain.com and not foo.domain.com. On a subfolder, though, it’ll flush domain.com, domain.com/foo/ and domain.com/bar/ and everything else. That would suck! So the best thought I had was I should lock the flush down to Super Admins only on the main site on a subdomain setup. Normally I allow the site admin to flush their own cache, but this way only the Super Admin could make a full flush.

And that’s the crux of the issue with caching on Multisite. Do you want each admin to maintain their own cache? Do you want it to be something you control for everyone? W3 Total Cache can do that, by the way. You can do it both ways with that plugin, and only control what you want to control for the network. But what about if even I don’t want to do that? What if I want to be lazy and trust my tools? That’s when you look at the server itself and try to make it as brainless as possible. Decisions, not options, remember?

And that brings up another question. Why do I want to manage who flushes at all? In my Varnish example, it’s because careless flushing clears the whole cache, which means my server and site have to recache everything and it slows everyone down. Obviously I want to limit that as much as possible. The other reason is work. People who have a managed hosting site generally aren’t the most technical savvy users in the world. Some are, of course, but many are more likely the ones who don’t want to know about the crazy server stuff. This is great, but it means I want to make their life easier so I have to provide less support, then I want things that do their job and do it well, and that means taking options away and making decisions for them.

How would I cache a static Multisite network and fully control it as a network admin? Probably server side caching like Varnish or ZendAccelerator Memcache. If I can’t do that, I’d use W3TC and lock it down so only I, the super admin, in my fancy cape, can edit settings and purge the cache. And always, always, remember to only catch what should be caught in your cache. If you have a site that’s a store, account for that.

27 June, 2014
Trading APC for Zend
The last thing I did before Passover was a totally unannounced, not telling anyone, surprise flip from my old standby, APC, to the new hotness, Zend.

As of PHP 5.5, Zend Optimizer+ will be included. Back in 2009, I decided to use APC for a couple reasons: It was made by the PHP blokes and it worked well with mod_php and MediaWiki. Since PHP decided to change, I joined in. Why the change? Zend went open source.

Installing Zend

I grabbed the latest tagged version from the official GitHub repository, since I don’t cotton to using bleeding edge all the time.
```
$ wget https://github.com/zend-dev/ZendOptimizerPlus/archive/v7.0.1.zip
$ unzip v7.0.1
$ cd ZendOptimizerPlus-7.0.1/
$ phpize
$ ./configure
$ make
$ make install
```
That gave me the final output of: Installing shared extensions: /usr/local/lib/php/extensions/no-debug-non-zts-20100525/ You want to hang on to that path, because we’re going to edit php.ini and put this in below the line for IonCube, if you have that installed:
```
 zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20100525/opcache.so
```
Replace the path with your own.

I also followed their advice on the recommended config and put this in my php.ini as well:
```
[Zend]
opcache.memory_consumption=128
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=4000
opcache.revalidate_freq=60
opcache.fast_shutdown=1
opcache.enable_cli=1
```
Since I had APC there before, I removed all those lines while I was at it. Restarted PHP (well, httpd in my case, I like a clean slate) and I was done.

Cleanup

As I mentioned before, I removed the APC lines while I was in the php.ini and, since I installed it via a downloaded pecl, I just removed the apc.so file from the same folder where opcache.so was installed.

On WordPress, I also removed the APC Object Cache Backend plugin from the two sites that had it, since I’m not using APC. There doesn’t appear to be an equivilent for Zend, but I may or may not need that.

For MediaWiki, I had to edit LocalSettings.php and remove $wgMainCacheType = CACHE_ACCEL; as that’s no longer being used. Then I went into my maintenance folder and ran php rebuildLocalisationCache.php --force to flush everything and my errors went away. MediaWiki has a pretty different cache method than other apps, and for yonks flat out didn’t work with Zend. It does now, and there seemed to be no speed loss without an OpCode cache.

None of my other apps seemed to care, so I moved on to see how this all played out.

Results

Much like the ManageWP guys, I saw an immediate drop in memory and CPU. And I’m still on PHP 5.4! There was an initial spike while I was mucking around, which I expected, and then everything dropped. Some things did not so much drop as level out and change my ‘spike’ range. Normally my load average for 1 minute would spike at 0.8 or so every hour. Now it’s spiking at 0.5, give or take, which is a success in my book.

Can you guess when Zend went on?

Before someone points out that, hey, this is hardly a real check, I want to note that I was having one of those 200% traffic days. Actually I was having a few of them in a row, and I know it’s going to go up from there. Here’s a quick look at my traffic:

The real test will be tonight and tomorrow, though, as I push some large data (videos) on a popular topic, but frankly, this is looking good.

The benchmarks used in the ticket to include Zend in PHP (why are they using WP 2? I have no idea) were startling compared to normal PHP. Compared to APC it’s a minimal kick in the pants, but the assurance of bundling to come speaks for itself. This is where PHP is going.

Credit: Dimitry

I’m not yet on PHP 5.5, but I’ve already found that 5.4 is a notable kick to my speed. Zend took me from a B to an A in PageSpeed and YSlow (when you tell YSlow that I’m a small blog and not Amazon). Also it ‘feels’ faster, which is totally subjective, but still a valid remark. The site feels fast, it doesn’t hang (yet) and it didn’t crash over Passover!

Former bbPress regular, _ck_, went and wrote a neat little Control Panel for Zend Optimizer+ which I snagged and tossed into my secret bin for poking at later.

Since I’m the Zend Rookie, anyone have any tips and tricks for configuring it to make it sing? Remember, I don’t just use WordPress on this box!
2 April, 2013
CentOS and PHP 5.4
I finally got around to PHP 5.4

Alas this meant reinstalling certain things, like ImageMagick and APC.

This also brought up the question of pagespeed, which I keep toying with. I use it at work, but since this server’s on CentOS with EasyApache, there’s no ‘easy’ way to install PageSpeed yet (not even a yum install will work), so it’s all manual work plus fiddling. I don’t mind installing ImageMagick and APC, but Google’s own ‘install from source’ aren’t really optimized for CentOS, even though they say they are, and I’m nervous about the matter. Well… I did it anyway. It’s at the bottom.

The only reason I had to do this all over is that I moved to a new major version of PHP. If I’d stayed on 5.3 and up’d to 5.3.21, that wouldn’t have mattered. But this changed a lot of things, and thus, a reinstall.

ImageMagick

I started using ImageMagick shortly after starting with DreamHost, since my co-worker Shredder was working on the ‘Have WP support ImageMagick’ project. It was weird, since I remembered using it before, and then everyone moved to GD. I used to run a photo gallery with Gallery2, and it had a way to point your install to ImageMagick. Naturally I assumed I still had it on my server, since I used to (in 2008). Well since 2008, I’ve moved servers. Twice. And now it’s no longer default.

Well. Let’s do one of the weirder installs.

First you install these to get your dependancies:
```
yum install ImageMagick
yum install ImageMagick-devel
```
Then you remove them, because nine times out of ten, the yum packages are old:
```
yum remove ImageMagick
yum remove ImageMagick-devel
```
This also cleans out any old copies you may have, so it’s okay.

Now we install ImageMagick latest and greatest from ImageMagick:
```
cd ~/tmp/
wget http://imagemagick.mirrorcatalogs.com/ImageMagick-6.8.1-10.tar.gz
tar zxf ImageMagick-6.8.1-10.tar.gz
cd ImageMagick-6.8.1-10
./configure --with-perl=/usr/bin/perl
make
make install
```
Next we install the -devel again, but this time we tell it where from:
```
rpm -i --nodeps http://www.imagemagick.org/download/linux/CentOS/x86_64/ImageMagick-devel-6.8.1-10.x86_64.rpm
```
Finally we can install the PHP stuff. Since I’m on PHP 5.4, I have to use imagick-3.1.0RC2 – Normally I’m not up for RCs on my live server, but this is a case where if I want PHP 5.4, I have to. By the way, next time you complain that your webhost is behind on PHP, this is probably why. If they told you ‘To get PHP 5.4, I have to install Release Candidate products, so your website will run on stuff that’s still being tested,’ a lot of you would rethink the prospect.
```
cd ~/tmp/
wget http://pecl.php.net/get/imagick-3.1.0RC2.tgz
tar zxf imagick-3.1.0RC2.tgz
cd imagick-3.1.0RC2
phpize
./configure
make
make install
```
Next, edit your php.ini to add this:
```
extension=imagick.so
```
Restart httpd (service httpd restart) and make sure PHP is okay (php -v), and you should be done! I had to totally uninstall and start over to make it work, since I wasn’t starting from clean.

Speaking of clean, cleanup is:
```
yum remove ImageMagick-devel
rm -rf ~/tmp/ImageMagick-6.8.1-10*
rm -rf ~/tmp/imagick-3.1.0RC2*
```
APC

I love APC. I can use it for so many things, and I’m just more comfortable with it than xcache. Part of it stems from a feeling that if PHP built it, it’s more likely to work. Also it’s friendly with my brand of PHP, and after 15 years, I’m uninclined to change. I like DSO, even if it makes WP a bit odd.

Get the latest version and install:
```
cd ~/tmp/
wget http://pecl.php.net/get/APC-3.1.14.tgz
tar -xzf APC-3.1.14.tgz
cd APC-3.1.14
phpize
./configure
make
make install
```
Add this to your php.ini:
```
extension = "apc.so"
```
Restart httpd again, clean up that folder, and then one more…

mod_pagespeed

I hate Google. Well, no I don’t, but I don’t trust them any more than I do Microsoft, and it’s really nothing personal, but I have issues with them. Now, I use PageSpeed at work, so I’m more comfortable than I was, and first I tried Google’s installer. The RPM won’t work, so I tried to install from source, but it got shirty with me, fast, and I thought “Why isn’t this as easy as the other two were!?” I mean, APC was stupid easy, and even easier than that would be yum install pagespeed right?

Thankfully for my sanity, someone else did already figure this out for me, Jordan Cooks, and I’m reproducing his Installing mod_pagespeed on a cPanel/WHM server notes for myself.(By the way, I keep a copy of this article saved to DropBox since invariably I will half-ass this and break my site.) Prerequisite was to have mod_deflate, which I do.

The commands are crazy simple:
```
cd /usr/local/src
mkdir mod_pagespeed
cd mod_pagespeed
wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-beta_current_x86_64.rpm
rpm2cpio mod-pagespeed-beta_current_x86_64.rpm | cpio -idmv
cp usr/lib64/httpd/modules/mod_pagespeed.so /usr/local/apache/modules/
chmod 755 /usr/local/apache/modules/mod_pagespeed.so
mkdir -p /var/mod_pagespeed/cache
chown nobody:nobody /var/mod_pagespeed/*
```
Once you do this, you have to edit the file, and this is where I differ from Jordan’s direction. He just copied this over /usr/local/apache/conf/pagespeed.conf but I had an older version from a ‘Let’s try Google’s way….’ attempt and someone else’s directions, so I made a backup and then took out the ModPagespeedGeneratedFilePrefix line since I know that’s deprecated. I also added in a line to tell it to ignore wp-admin.

Here’s my pagespeed.conf (edited):
```
LoadModule pagespeed_module modules/mod_pagespeed.so

	# Only attempt to load mod_deflate if it hasn't been loaded already.
<IfModule !mod_deflate.c>
	LoadModule deflate_module modules/mod_deflate.so
</IfModule>

<IfModule pagespeed_module>
	ModPagespeed on

	AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html

	ModPagespeedFileCachePath "/var/mod_pagespeed/cache/"

    ModPagespeedEnableFilters rewrite_javascript,rewrite_css
    ModPagespeedEnableFilters collapse_whitespace,elide_attributes
    ModPagespeedEnableFilters rewrite_images
    ModPagespeedEnableFilters remove_comments

	ModPagespeedFileCacheSizeKb 102400
	ModPagespeedFileCacheCleanIntervalMs 3600000
	
	# Bound the number of images that can be rewritten at any one time; this
	# avoids overloading the CPU. Set this to 0 to remove the bound.
	#
	# ModPagespeedImageMaxRewritesAtOnce 8

	<Location /mod_pagespeed_beacon>
		SetHandler mod_pagespeed_beacon
	</Location>

	<Location /mod_pagespeed_statistics>
		Order allow,deny
		# You may insert other "Allow from" lines to add hosts you want to
		# allow to look at generated statistics. Another possibility is
		# to comment out the "Order" and "Allow" options from the config
		# file, to allow any client that can reach your server to examine
		# statistics. This might be appropriate in an experimental setup or
		# if the Apache server is protected by a reverse proxy that will
		# filter URLs in some fashion.
		Allow from localhost
		Allow from 127.0.0.1
		SetHandler mod_pagespeed_statistics
	</Location>

	ModPagespeedMessageBufferSize 100000	
	ModPagespeedDisallow */wp-admin/*
	ModPagespeedXHeaderValue "Powered By mod_pagespeed"

	<Location /mod_pagespeed_message>
		Allow from localhost
		Allow from 127.0.0.1
		SetHandler mod_pagespeed_message
	</Location>
	<Location /mod_pagespeed_referer_statistics>
		Allow from localhost
		Allow from 127.0.0.1
		SetHandler mod_pagespeed_referer_statistics
	</Location>
</IfModule>
```
To tell Apache to run this, edit /usr/local/apache/conf/includes/pre_main_global.conf and add:
```
Include conf/pagespeed.conf
```
Note: We put this code here because EasyApache and httpd.conf will eat your changes.

Finally you rebuild Apache config and restart apache and test your headers to see goodness! My test was a success.
```
HTTP/1.1 200 OK
Date: Mon, 21 Jan 2013 03:12:13 GMT
Server: Apache
X-Powered-By: PHP/5.4.10
Set-Cookie: PHPSESSID=f4bcdae48a1e5d5c5e8868cfef35593a; path=/
Cache-Control: max-age=0, no-cache
Pragma: no-cache
X-Pingback: https://ipstenu.org/xmlrpc.php
X-Mod-Pagespeed: Powered By mod_pagespeed
Vary: Accept-Encoding
Content-Length: 30864
Content-Type: text/html; charset=UTF-8
```
For those wondering why I’m ignoring wp-admin, well … sometimes, on some servers, in some setups, if you don’t do this, you can’t use the new media uploader. It appears that PageSpeed is compressing the already compressed JS files, and changing their names, which makes things go stupid. By adding in the following, I can avoid that:
```
	ModPagespeedDisallow */wp-admin/*
```
Besides, why do I need to cache admin things anyway, I ask you?

So there you are! Welcome to PHP 5.4!
20 January, 2013