Half-Elf on Tech

Thoughts From a Professional Lesbian

Tag: wordpress

  • But If, Baby, I’m The Bottom, You’re The TOP

    Earlier this month I talked about how my server was acting wonky and how I fixed it using, among other tools, TOP.

    This week I was chatting with a fellow about CPU usage and his site. He runs a rather large WordPress blog and the database is about 500 megs. As a comparison, this site, with about 500 posts, is under 5 megs, and my big site, with thousands of posts, comments, and a forum, is 10 megs. The biggest site I run on my server is 850 megs (just down from 910 after some clean up). The difference between his site and mine is that his is slow and he knows it. As we discussed ways to speed it up, I had some thoughts on WordPress and how, at a certain point, you’re going to need to dig into the guts of your server and learn TOP.

    The ‘problem’ with most ‘How do I make my WordPress site run faster?’ tutorials, as I’ve seen it, is they address surviving the digg effect. That is, they talk about how to deal with having a high volume of traffic on your site and, for the most part, you can make it with just adding caching plugins.

    Once your site gets ‘big’ or ‘popular’ you’re going to have to move off shared/cloud hosting and over to your own server. For most of us, the first step is a VPS (Virtual Private Server). Shared Hosting means ‘You have an account on a server with a hundred other people.’ It’s great for small sites, inexpensive and easy to use. The problem is you could have terrible neighbors, who use up all the CPU. Think of it like those old New York apartments where someone’s a jerk at 5am and uses up the hot water so you, at 7am, have none. Yeah, it’s kind of like that. That’s the day you think ‘I want a house!’

    Only, well, we’re not all up for houses just yet. A house would be a dedicated server, where it’s just you. Cloud hosting, which I touched on earlier, would be the college dorms of webhosting. It has a lot of benefits for the really small sites, and actually some for large sites, but I’m not sold of their overall usefullness yet, so I’ll talk about them some other time. What I want to talk about are Virtual Private Servers, the condo-sub-leasing (or rent-to-own maybe) of website hosting, and how the new VPS user should really get on TOP of things (sorry, bad pun) to make their lives easier.

    TOP. Well ‘top’ really. Unix commands are generally all lower case like that.

    The top command is a system monitor tool that outputs a list of processes. Have you ever seen Task Manager in Windows? It’s kind of like that tab for ‘Processes’ that you look at and run away from. The default view of top is by percentage of CPU usage and the “top” CPU users are listed. See? The name made sense. You can also see how much processing power is being used, memory hogs and other cool things. Most modern Unix-systems let you sort the list, colorize it, etc, though you have to be command line savvy.

    Here’s what top looked like for me about an hour ago.

    top - 12:44:44 up 126 days, 23:13,  1 user,  load average: 0.12, 0.17, 0.17
Tasks:  91 total,   1 running,  90 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0% us,  0.0% sy,  0.0% ni, 100.0% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:    524288k total,   358248k used,   166040k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
15616 nobody    15   0 94540  65m  20m S  1.0 12.8   0:00.27 httpd
12261 ipstenu   16   0  1908 1012  780 R  0.4  0.2   0:00.30 top
[...]
28630 root      16   0  107m  86m 1096 S  0.0 16.8   1:06.30 /usr/sbin/clamd

    I wanted to point out clamd, which has been the bane of my existance. Thing won’t DIE. I ended up going in to /etc/exim.conf and manually commended out the clamd line (and restarted the service) to finally get it gone.

    But top, as you can see, has a freakishly large amount of information. My server is doing fine, at this point, so I don’t have a whole lot to show you. What you can see right away is that I can tell, with a glance, what’s going on. I could see, though and at this point I have a ‘nobody’ process. That just means someone’s accessing my website. No, really! That’s good! The CPU and memory usage seem high, but they vanish in a second. Basically, someone rang my doorbell and for that brief moment, electricity was used. The next thing I see is the top command, which is run by me (hi!) and down the line is that idiot, clamd.

    I actually scan top a lot at work these days, trying to understand what’s causing issues. It’s good for ‘right now!’ things, but not so much if I want to see what started a strange spike a couple hours (weeks) ago. For that you need a whole mess of tools.

  • I take it back. WP-Super-Cache is a Super Hero

    cupspikes I’m going to be upfront and admit that I’ve never actually liked this plugin. A very large part of me wants to side with Matt Mullenweg in that if you have a good server, configured properly, with a decent host, you should be just fine. Also, it doesn’t really work well with my favorite anti-spam plugin, Bad Behavior, which stops 99.999% of my spam cold. But. Over the years of running a vaguely popular fan site, I’ve been nailed by service spikes that killed me and everyone else on my shared hosting setup (multiple websites, not all connected, sharing a virtual server). At one point, I had to offload ‘news’ to LiveJournal, but since then, I’ve pulled it all back to WordPress, moved to a virtual private server (VPS, just me and my sites on a virtual server) due to the need for better support, and I was kind of complacent. Things were trucking along just fine, we had some major news that were handled without a blip, and I thought I was cool.

    Yesterday I had to cycle the HTTP service three times to clear things up. The first time, someone was using a really old URL (for a part of the site gone 2 years now) and, when it didn’t give them what they wanted, they kept hitting it. I blocked the IP address and we were fine. But. Then the news that I had some new, cool, information hit, and suddenly I was spiking like mad. I checked my stats, trying to see what was the culprit. The gallery is pretty tolerant of these things (though I have turned on the Static HTML cache right now) and while I did have some hefty images (1 to 2 MB, I usually try to keep ’em to .75 megs), it wasn’t ZenPhoto borking.

    No, my poor, poor WordPress was having a heart attack because I’d gotten myself crosslinked from a couple high traffic sites. How bad? Well that spike on the graph below may explain it.

    spiked

    The first thing I did was tune the server. Actually, I’d done that months ago, dropping my memory usage from 77% to about 60%, but now I went in to see how well that was working. There was a little more I could do, so I optimized a couple more settings and things eased up a little. Not enough. I scrubbed the CPU usage, too, and normally we never spiked over 1 for a load average, but that wasn’t working yesterday. Sidebar. CPU Load is a very bizarre thing to most newbie server admins, and I’m not great at sorting it out myself. Of course, I know that a ‘good’ load is anything around .3 and a ‘bad’ load is something like, oh, 9. And yes, I hit 9 yesterday on my 8 processor VPS box. I’m not going to explain it here, as I’m still learning and I’m sure I’d get it wrong, but the gist is that you don’t really worry if your load hops up to 1 or 2 for a short amount of time. When it stays there and is spiking at 4 or 5, however, you need to pay attention.

    What kept happening to me was that it would spike up to a load between 5 and 9, and the HTTP service (the bit that serves up webpages) would scream and fall over. Email, FTP, shell access and the rest were all okay, though, so I knew the server itself was fine. Thus I deduced something was sucking up the load and I knew I had three choices: JFO’s blog (it’d happened before), JFO’s gallery, or YTDaW. While I host YTDaW, I don’t actively admin it in any authoritative stance. The only ‘mod’ work I’ve done is turn off email alerts for people who are using non-existent emails (and then, only when I’m tired of getting their bouncing email). Devon pretty much keeps me on tap for server admin and security stuff, and I do my utmost best to keep my hands OUT of the pie. It’s her baby, I’m just the tech.

    And while they’re using a pretty old version of the forum software, it’s secure enough and solid enough that I didn’t think they were the culprit. The evidence (heh) supported that theory, so I went to look at JFO. It was definitely my old girl, and right away I could see that we were getting a lot of traffic from new users. Four times the traffic. Before you could say ZOMG! I was on Google Analytic and Woopra, checking out who the hell was hitting my site and the answer was surprising.

    Everyone. (Well, mostly FaceBook, AfterEllen and Twitter, but really, it was all over.)

    I’d accidentally broken news about three hot topics within a couple hours, and now everyone and their mother wanted to see JFO and, as many people have mentioned, WordPress was hemorrhaging under the ‘digg’ effect. Basically it was trying to serve up dynamic (generated on the fly) pages to too many people at once. If I was using static HTML, it would go faster, but WordPress doesn’t do that. Except … except it does if you use WP Super Cache.

    no10 As I mentioned before, I don’t (didn’t!) like that plugin. I want my app to behave correctly without it. I mean, the PM of Britain uses WordPress! I was sure they don’t need caching. They probably have a rack of servers on a co-located cluster. Except I viewed source and they were using it. The Library of Congress wasn’t, though, and neither were The Speaker of the House (Nancy Pelosi) or the Army. Honestly, I wasn’t sure how to take that, but after four hours of babysitting my server, I took a plunge and installed WP Super Cache for the fourth time.

    The first few times sucked, I admit. It was a lot of massaging and manual crap that, while I’m perfectly capable of doing, I didn’t like. This was easier. A chmod, an install, a click, another chmod and then I was done. And guess what? My loads dropped from an average of 3.45 to one of .35 by morning. On top of that, my memory had one spike since I turned it on, and that was right when I was running backups and the like.

    memoryspike

    So I’m keeping it on for now, especially with what I expect tonight, but I think that I can say … yeah. WP-Super-Cache does what it says.

  • Plug It In, Plug It In

    vilcus-plug-it-inI am not a great programmer by any means. I can hack around and muddle my way through with the best of the great net scapegraces. I’m not the genius who invents a brand new way of doing things. That said, I do, eventually, get annoyed with things enough that I force myself to learn how to code.

    Yesterday I was pissed off at WordPress because of it’s user management tools, and no plugins really did what I wanted. See, I have open registration. It lets me sync my blog and forum and let people post. But where it fails is that I can’t set users as ‘banned’ in WordPress. This is a simple thing, I feel. A user role that has no rights and is just banned from commenting. They can read all they want, but no comment. I’ve tried just about every tool out there, but they never work. In addition to that, spammers sign up to my blog.

    Since creating a ‘bozo’ user role is outside my ability, I decided what I wanted was a plugin to prevent people from registering if they were on my blacklist, similar to how I can prevent them from commenting on my comment blacklist. At first I was using TimesToCome Stop Bot Registration, which (among other things) uses StopForumSpam’s list of spammers as a stop-gap.

    The problem with TTC is that if you register with a bad email (jane132@gmail.com instead of jane123@gmail.com) and then try to register with the RIGHT email, it notes that the IP is the same and bans both emails and the IP. Which caused a couple people no end of problems on my site. It had to go.

    From there, I tried No Disposable Email, which checks against a list of known baddies. That was nice, but it was a text file list that you had to update by hand. But it got me thinking.

    I quickly converted it into Ban Hammer, which allowed me to update and edit the text file from a submenu inside my admin session. But that wasn’t enough. Why did I have to have two places to keep my jerk list? If someone was on my WordPress Comment Blacklist, I didn’t want them to comment. That implies they’re just not welcome at all. So why don’t I make Ban Hammer pull from that list. Which I did.

    I still have things I want to do to the code, like put in an option to use StopForumSpam’s list, and a way to edit the error message. But for now, Ban Hammer sits by my other plugins, Recently Registered (lists the last 25 registrations) and my bbPress plugin Spoiler Bar (adds in spoiler ‘code’ to bbPress) on my Google Code site. It’s not for ‘public’ release, but it’s there so my friends who have been helping me test out my ideas can easily download. What? I have nerd friends!