Half-Elf on Tech

Thoughts From a Professional Lesbian

Author: Ipstenu (Mika Epstein)

  • Blocking IPs – Don’t

    Blocking IPs – Don’t

    Here’s the thing. I don’t think blocking an IP address is a good idea.

    Will it prevent spammers from registering on your site? Yes. But much like CAPTCHA, I think it does more harm than good.

    See, IP addresses are numerical labels assigned to each device (e.g., computer, printer) participating in a computer network that uses the Internet Protocol for communication. Big words. Shorter version: The IP address is the phone number. DNS is Caller ID. When you dial ‘Home’ on your cell-phone, the phone translates that into a number and dials, right? Well on the internet, you say “I want to go to ipstenu.org” and it’s DNS that says “Okay, my big bad directory says that’s 67.227.208.52 so here you go!” (actually it says “You want extension on 67.227.208.52” if I can stretch that metaphor).

    The IP address for websites is pretty static. Just as most of us don’t want to change our phone numbers and teach all our parents new contact info, we don’t want to have to update all the DNS servers in the world with our new IP address. It’s a pain in the ass, it takes up to 72 hours to propogate to everyone (usually less), but that still means there’s a period of time where people can’t get to your website.

    This is all over-simplified, but you get the gist.

    Here’s where it gets weird. In order for the internet to know who YOU are and send you back the website you asked for, it has to assign you an IP address. And this changes. A lot. Most ISPs (the people who give you access to the net for moneies) have a ‘range’ of IP addresses which numbers less than the number of people they have who pay for internet access. In order to make sure everyone can get on, when you connect to the net, you get a new IP. Back in the day of dial-up, every time you dialed in, you had a new IP. This was normal, and was one of the many reasons no one bothered to block by IP. All the idiot had to do was reconnect. These days, my IP changes about once a week or so, and I have no idea when or why, but it does. That doesn’t bother me.

    Lately, I’ve gotten complaints and requests to make Ban Hammer or Register IP MS block people by IP address. And after playing with that a bit, I’ve decided I won’t. Not because I can’t (it’s really not that hard, and actually, Ban Hammer already does it for Single Site WordPress by accident), but because I don’t think it adds any value. The IP address can change too easily to make this a useful tool, and the odds are I’m going to accidentally block someone who should be able to access the site!

    This is not to say I don’t block IP addresses at all, just that I don’t do it the way people seem to want to go about it. Maintaining my own blacklist of IPs is insane, and stupid. I don’t need to waste my time clicking and banning spammers or auto-register bots. Instead, I block IPs using one of two tools that was designed to look for bad behavior. I detailed all this in Spam / Splog Wars. That’s how I stop spammers and it works.

    So no, I will not be wasting my time telling you how to edit my plugins to block people from the IP level. There are perfectly good ways of doing this that work without you having to field complaints from innocent users.

    Block the bots, not the people.

  • How the WordPress Upgrade Works

    Edited: This post was linked to by the folks at WTC, so I’m getting a lot more traffic! If you asked a question, I will try to answer it promptly, but if you need serious help fixing a problem, please consider posting in the WordPress forums for help.

    I was 90% sure about this before I started writing the post, and Andrew Nacin was nice enough to tweet me the exact file I needed to look at, so when I got home to look, I was ready to go!

    There are two kinds of automated upgrades for WordPress. The main ‘core’ upgrader and then the ‘child’ upgrades used for themes and plugins. They behave differently.

    Most of the time, we all see the plugin and theme installer, where it downloads the plugin to /wp-content/upgrade/, extracts the plugin, deletes the old one, and copies up the new one. Since this is used more often than the core updater (most of the time), it’s the sort of upgrade we’re all used to and familiar with. And we think ‘WordPress deletes before upgrading, sure.’ This makes sense. After all, you want to make sure to clean out the old files, especially the ones that aren’t used anymore.

    This is not how a core update works.

    WordPress core updates, the ones to take you from 3.0.3 to 3.0.4, do not run a blanket delete. They don’t even run a variable delete. They don’t even run a wild-card delete on files in wp-admin (which they could). Instead they have a manually created list of files to delete, files that have been deprecated, and delete only those files. Here’s a snippet of what it deletes:

    $_old_files = array(
'wp-admin/bookmarklet.php',
'wp-admin/css/upload.css',
[...]
// MU
'wp-admin/wpmu-admin.php',
'wp-admin/wpmu-blogs.php',
[...]

// 3.1
'wp-includes/js/tinymce/blank.htm',
'wp-includes/js/tinymce/plugins/safari',
[...]
'wp-admin/images/visit-site-button-grad.gif'
);

    As you can see, the files and folders are all very carefully specified. They are not ac-hoc, they are all determined based on what WordPress has removed. If you read the whole thing, the part that would impress you the most is, perhaps, the folder wp-content, where your themes and plugins are installed, is nowhere on that list. Don’t believe me? Go look at wp-admin/includes/update-core.php and search for it. It’s not not there!

    Once the old files are listed, remember that we have not deleted anything, the upgrader runs through 9 steps.

    1. Download the zip file of the new release, unzip it and delete the zip
    2. Make sure the file unzipped!
    3. Make a .maintenance file in WordPress base (this makes your blog ‘down for maintenance’ so no one can do anything and screw you up mid-stream
    4. Copy over the new files. This is a straight copy/replace. Not delete.
    5. Upgrade the database. This may or may not happen.
    6. Delete the unzipped file
    7. Delete the .maintenance file
    8. Remove the OLD files. This is where it goes through the list of deprecated and unused files and deletes them.
    9. Turn off the flag that tells you to upgrade every time you’re in wp-admin

    Nowhere in there is wp-content and your themes mentioned.

    Well, except in this one weird way.

    See, in order for WordPress to work out of the box, a theme must be included, right? WordPress has the default theme of Twenty Ten. Now, I’ve mentioned before that you should never update your themes directly, and instead make child themes. This is why. When the WordPress core files update the copy over everything. Included in everything are two plugins (Akismet, Hello Dolly) and one theme (Twenty Ten).

    Normally this isn’t a problem. Sure, someone always edits those files directly and lives to regret it, but they do live. This last cycle, with 3.0.4, Akismet was accidentally rolled back from 2.5.1 to 2.4. Again, normally? Not a problem. We just upgrade our Akismet installs, remark on the silliness and annoyance, and move on. The problem for one user is she had another plugin or theme edit that hooked into the new Akismet. With the way WordPress updates core, it only deletes the files it knows to delete (and Akismet isn’t any of them) and it copies over the ‘new’ files. Or in this case, the old ones. Which broke her site. Her fix? Just re-update Akismet manually. Not a big deal and an easy fix. (Personally I think NOT updating core, even a security upgrade, with the latest Akismet is a poor choice, but the rational is that they want to keep it small and easy to maintain. So okay, I get why, I just don’t like it. I will support that as their choice, and continue to do my manual upgrades, which include deleting Akismet, Hello Dolly and Twenty Ten before I upgrade.)

    So then, why doesn’t this work 100% of the time? (Yes, I mentioned this before in the 3.0.1 days, but apparently it bears repeating. Read Why doesn’t the WordPress Auto-Upgrade Work? for more thoughts on the subject.) Well, to start with, that’s impossible. Nothing works 100% of the time. Me submitting this post won’t work 100% of the time. You safely walking up the stairs won’t work 100% of the time. This is just how the world works. There are, simply, far too many variables out there to allow this. Even though I make part of my living ensuring that I have an automated process run in a repeatable fashion, I tell people that I only ask for a 75-80% success ratio on the process before I’ll agree to automate it. Why? Because that’s actually phenomenal.

    If you take into consideration all the moving parts, variables, and possibilities that goes into any one individual WordPress install, it’s sort of impressive this stuff works at all. In baseball, if you hit a ball and get on base 33% of the time, you’re considered a fantastic hitter. If you do it 40% of the time, you’re pretty much promised a slot in Cooperstown. Whenever I try and sort out if something is worth the risk, I quote my father “What can go wrong? How likely is it? What are the consequences?” It boils down to an understanding of risk analysis, and what that actually means. The problem is that most of the people using WordPress are users. They’re not generally expected to think about risk. Not that many don’t, but just that Joe Blogger doesn’t tend to look at an upgrade as a ‘risk.’ After all, WordPress tested this, so it should be easy.

    Most of the time, it is. And when it’s not, it was an acceptable risk. All those horrible and terrifying outcomes you read about (from a very vocal minority) are gut-wrenching when they happen to you, don’t get me wrong, but at the end of the day, you have to ask ‘If I use the default WordPress theme and no plugins, do they happen?’ If the answer is no, then WordPress has done all the testing that is required. The onus is not on WordPress to test every upgrade with every theme and plugin. That responsibility is firmly on the shoulders of the person using the themes and plugins. A good developer tests their plugins and themes the moment a release candidate comes out for a new version of WordPress, and sorts out how best to support both the current users and the future ones.

    It’s what we call ‘acceptable’ risks. You take them all the time. You took an acceptable risk brushing your teeth. You take one every time you walk out your door. You know the risks and you accept them. So when you get to arguing that ‘WordPress upgrades always break’ or ‘I hate upgrading because it means my themes/plugins won’t work’ and using these as reasons to show that WordPress is bad software, then I think you’re missing the point. The more you customize, the more things break. This is something I mentioned in When To Code, and it bears repeating. The more you customize, the more things will break when you upgrade.

    But this is an acceptable risk for most of us!

    So when the WordPress auto-upgrade breaks, and I promise you, it will for you at least once in your experience, you have to learn to accept this. It’s not going to work on every server, it’s not going to work on every host. Your server is being constantly upgraded and tweaked. Security patches for PHP, FTP and everything else are applied, most of the time automatically so you don’t have to waste time thinking about it. And when you combine all those things, yeah, it’s going to break.

    This isn’t meant to scare you off of upgrading, but an attempt to raise your awareness of what’s going on, so when things break (and they will), you have a better understanding of why, and what to do. If the automated upgrade of WordPress breaks, upgrade manually. If every single automated install (upgrades, plugins, themes) always breaks, then start to diagnose your server. But if it’s a one off, just do it manually. It is, literally, copying files up to your server. If that’s too hard for you, you may not want to run your own, self-managed, WordPress install.

    As a blanket reminder, in order to prepare yourself for an upgrade, always make backups of your database and your files. A good backup. Never edit core files (even themes) and always remember that the computer is out to get you.

  • Don’t Fear Wandering Off Topic

    Don’t Fear Wandering Off Topic

    Sometimes when you’re debugging, you haven’t the foggiest idea how someone managed to screw something up so badly that they got this particular error. Naturally they’ll tell you that they didn’t do or change anything, and sometimes they’ll even be telling you the truth! It’s at that point I try to teach my guys that it’s okay to wander off the script and start talking about anything else, in order to learn how the person thinks.

    Everyone’s thought process is different. Some of us learn by reading, some by watching, some by doing, and others by listening. There’s no one perfect way to solve a problem, but that doesn’t stop managers from trying to codify the how-tos into a script to follow. We’ve all been there, where we’re told to reboot our PC to solve a modem problem. There is no secret shibboleet call to get you to the smarter people.

    I will say that the reason the scripts exist is because they do work. I couldn’t begin to tell you how many help tickets are solved when I ask ‘Are you using the default password of changeme?’ Heck, even telling people ‘Reboot.’ is correct sometimes. That’s the thing, though. SOMETIMES. You have to learn how to tell when ‘sometimes’ is now and when it’s not. For example. I add a user to a domain group on Windows. Our setup of Windows XP systems pull down domain group info (and all permissions allocated therein) on login. User says the permissions that come with the group aren’t working. I tell them ‘Reboot so XP can apply the settings. Sometimes it gets persnickety.’ They reboot, we’re done. But notice, I know how and why the problem occurred, and I can explain it to the user.

    But. Sometimes you get yourself down the rabbit hole with a crazy problem. “Hi! Every time I add a file, it locks the system.” Okay, time to sort out what’s getting locked. What’s not cleaning itself out, what’s acting ‘weird.’ Does it work for you? It does. Interesting. What’s HE doing? Let’s ask. “I’m using a home grown script to add files.” Oh. Betcha that’s it. I actually spent three days and a call to the vendor trying to debug a problem like this because the guy refused to accept that HIS script was POSSIBLY screwing things up. Actually he didn’t even mention the script until I started chatting about workflows and productivity and he mentioned, in passing ‘Yeah, we had to write a script because it took too long.’

    This isn’t about how when you ask for help, be specific and tell them exactly what you’re doing in as clear and simple a way as possible. This is about how, when you’re talking to someone, it’s totally okay to wander off topic and ask something off the wall. “Are you using Firefox beta 4? You are? Did you know that there are problems with that and TinyMCE?” Sure the question sounded hella random, but it was the debugger’s brain pinging off the wall at a memory.

    When you’re helping people, remember that they learn differently from you as well. As much as people tout ‘thinking outside the box’ as a call to innovation, I think that’s not what they mean. What they mean is being willing to take a chance, a risk, and make a stab in the dark at a possible answer. A genius is the one who looks at a moldy cheese sandwich and thinks ‘What the hell is IN that green stuff?’ A genius is the one who says ‘It’d be easier if could clone a server.’ A genius is the one who both thinks and applies the thought to something.

    When you’re stuck troubleshooting, wander off topic. Be willing to think of the crazy things and voice them aloud. Be willing to say ‘I doubt this has anything to do with it, but what happens if you do this…’ Be willing to ask the ‘stupid’ questions. Go ahead and talk about Bad Wiener day. The scrunchy joke may make you remember something totally off the wall.

  • My WordPress Scripts

    My WordPress Scripts

    I actually have a couple little scripts I use to update my site instead of using the WordPress built in tools. Invariably, when someone has problems with the automated upgrade tool I comment that I rarely use it. That means people ask me ‘Well how do you update WordPress?’ and I tell them ‘Manually.’ Which isn’t really true. Sometimes I say ‘I wrote a script.’ So … here’s my scripts for upgrading WordPress. I don’t have one for themes, since I always do that manually and check everything. That’s the bane of having highly customized child themes.

    These are both Bash shell scripts and run perfectly happily on my server (Linux running CentOS). They’re GPL2, free, and somewhat use at your own peril.

    WordPress Upgrade Script

    WordPress Plugin Script

    As I went to post these, I took a bit of time cleaning them up, putting in comments (the ‘Blame Nacin’ bit is a joke) and formatting them nicely. When I started to look at the plugin one, I realized how freakin’ crazy it is and how many weird custom things I tucked in there. Also I made it so you always had to put in a version number, which if I wanted to ‘release’ that, is something to do away with. One of the catches with it is that a plugin’s ‘default’ zip file of plugin-name.zip is the trunk build. So you’d always have to know your version to upgrade.

    On the WordPress upgrader, I could trick that with a simple “if the version is ‘latest’, then download latest.zip”, because the WordPress most recent release is latest.zip. Now, I never call that. I like being master of my domain (pun? maybe) so I always make the time to know what I’m calling. I could have assumed that if you don’t put a version in, you want to install the latest, which is pretty easy to flip around, but I don’t like to assume like that. Still, you can take the code if you want it and do that.

    On the other hand, I think if someone wants a plugin, then they should be able to go ‘Get me the plugin’ and it should default to the latest. Probably this is because I test trunk WordPress a lot, but rarely do I trunk a plugin. Since there’s no easy to gank default like that for plugins, I came up with this really fun bit of code, shoved in an if-then check. I really like awk, by the way. It’s insanely powerful.

      # We're getting the readme from the repo and using that to calculate the latest stable release.
  wget -qO $1-readme.txt http://plugins.svn.wordpress.org/$1/trunk/readme.txt

  if ! [ -f $1-readme.txt ]
  then
    echo "FAILURE: The plugin is goobered in the WordPress repository, so we can't determine the latest stable release."
    exit 1
  else
    tr -cd '\11\12\40-\176' < $1-readme.txt > $1-readme-tr.txt
    VERSION=.`awk '/Stable/ {print $3}' $1-readme-tr.txt`
    rm $1-readme.txt $1-readme-tr.txt
  fi

    All these are free for use, and both are under GPL2. I can offer SOME support, but they’re filed under ‘Works for me!’ in my bin.

  • WordPress Plugin Script

    WordPress Plugin Script

    I don’t like the automated plugin installer. I don’t know why, I just don’t. I have this stupid simple script I use instead. I cleaned it up before posting here. It’s a lot more complicated than my WordPress Upgrade Script because it has to check for the latest release of the plugin via the subversion repository and clean up weird characters (because people write code on Windows, Linux and Mac and everything else!). This is one of the few times I assume that if you don’t specify a version, you probably want the latest and greatest.

    #!/bin/bash

####################################################################
# WORDPRESS-PLUGIN.SH - WordPress Plugin Script for BASH SHELL     #
#                                                                  #
# This script will download and copy up the specified WordPress    #
# plugin to the account. By default it gets the latest version,    #
# but you CAN specify trunk or whatever version you want.          #
#                                                                  #
# Author: Mika Epstein                                             #
# URL: https://halfelf.org/scripts/wordpress-plugin-script/    #
#                                                                  #
# Usage: ./wordpress-plugin.sh plugin [version]                    #
#                                                                  #
# plugin  == the 'ugly' name of the plugin (i.e. wp-super-cache)   #
# version == the FULL version number (i.e. 0.1.2)                  #
#                                                                  #
# This program is free software; you can redistribute it and/or    #
# modify it under the terms of the GNU General Public License as   #
# published by the Free Software Foundation; either version 2 of   #
# the License, or (at your option) any later version.              #
#                                                                  #
# This program is distributed in the hope that it will be useful,  #
# but WITHOUT ANY WARRANTY; without even the implied warranty of   #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the    #
# GNU General Public License for more details.                     #
#                                                                  #
####################################################################

if [ "$1" = "" ]
then
  echo "EXIT: FAILURE! You didn't specify a plugin name.  Kinda need that."
  echo "Syntax is ./wordpress-plugin.sh plugin [version] "
  exit
fi

if [ "$2" = "" ]
then
  # We're getting the readme from the repo and using that to calculate the latest stable release.
  wget -qO $1-readme.txt http://plugins.svn.wordpress.org/$1/trunk/readme.txt

  if ! [ -f $1-readme.txt ]
  then
    echo "FAILURE: The plugin is goobered in the WordPress repository, so we can't determine the latest stable release."
	exit 1
  else
    tr -cd '$2' < $1-readme.txt > $1-readme-tr.txt
    VERSION=.`awk '/Stable/ {print $3}' $1-readme-tr.txt`
    rm $1-readme.txt $1-readme-tr.txt
  fi

else
  # Quick check if someone wants the trunk build with a 'You sure?' 
  # double check.
  if [ "$2" = "trunk" ] # start trunk check
  then
    read -p "Are you sure you want to install the TRUNK version? (y/n) " -n 1
    if [[ ! $REPLY =~ ^[Yy]$ ]]
    then
      echo
	  echo "You have opted NOT to install the trunk build of $1."
      exit 1
    else
      echo
      VERSION=
    fi
  else
    VERSION=.$2
  fi # end trunk check
fi

# Download the plugin
wget http://downloads.wordpress.org/plugin/$1$VERSION.zip

# If the file didn't download, then you probably got the URL wrong. Fail.
if ! [ -f $1$VERSION.zip ]
then
  echo "EXIT: FAILURE! Could not download $1$VERSION.zip - Did you get the version and plugin name right?"
  exit
else
  echo
  unzip -q $1$VERSION.zip
fi

# This is ONE LAST CHANCE. If you say anything other than yes, then it cleans up.
read -p "Last chance.  You sure you want to install the plugin $1 v$VERSION for $USER? (y/n) " -n 1
  if [[ ! $REPLY =~ ^[Yy]$ ]]
  then
    rm -rf $1/
    rm $1$VERSION.zip
	echo
	echo "EXIT: You have chosen NOT to install WordPress $1 at this time."
	exit 1
  else
    echo
  fi

# This is a quick check to make the directory if it's not there.
# Change this if you want to install to a subfolder or whatever.
if ! [ -d public_html/wp-content/plugins/$1 ]
then
  mkdir public_html/wp-content/plugins/$1
fi

# Copy the files up to root public_html
# Again! Change this if you want to install to a subfolder or whatever.
cp -r $1/* public_html/wp-content/plugins/$1/

# Post install clean up with a 'I don't know what you did!' error.
if [ -f $1$VERSION.zip ]
then
  rm -rf $1/
  rm $1$VERSION.zip
  echo "SUCCESS! You've downloaded the plugin $1 (version $VERSION). Now go activate it!"
else
  echo "POSSIBLE FAILURE. Could not clean up the files, so there's a chance everything went pear shaped."
  echo "Please review your WordPress plugins and remember: Blame Nacin."
fi

exit
  • WordPress Upgrade Script

    WordPress Upgrade Script

    I don’t like the automated updater. I don’t know why, I just don’t. I have this stupid simple script I use instead. I cleaned it up before posting here. But this is what I use when I want to upgrade my various installs to the latest version, or the nightly build. I didn’t bother to put the svn stuff in here, since the script I use for that is fairly weird and particular to me.

    #!/bin/bash

####################################################################
# WORDPRESS.SH - WordPress Upgrade Script for BASH SHELL           #
#                                                                  #
# This script will download and copy up WordPress to the account.  #
# It's pretty simple, with the bare minimum of error checking. So  #
# be careful.                                                      #
#                                                                  #
# Author: Mika Epstein                                             #
# URL: https://halfelf.org/hacks/wordpress-upgrade-script/     #
#                                                                  #
# Usage: ./wordpress.sh version                                    #
#                                                                  #
#                                                                  #
# This program is free software; you can redistribute it and/or    #
# modify it under the terms of the GNU General Public License as   #
# published by the Free Software Foundation; either version 2 of   #
# the License, or (at your option) any later version.              #
#                                                                  #
# This program is distributed in the hope that it will be useful,  #
# but WITHOUT ANY WARRANTY; without even the implied warranty of   #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the    #
# GNU General Public License for more details.                     #
#                                                                  #
####################################################################

if [ "$1" = "" ]
then
  echo "EXIT: FAILURE! Call this as './wordpress.sh VERSION', you nut!"
  exit
fi

# Quick check if someone wants the nightly build with a 'You sure?' 
# double check.
# This sets the URL, as it's different for the nightly builds.
if [ "$1" = "nightly" ]
then
  DOMAIN=wordpress.org/nightly-builds
  VERSION=wordpress-latest
 
  read -p "Are you sure you want to install the NIGHTLY build? (y/n) " -n 1
    if [[ ! $REPLY =~ ^[Yy]$ ]]
    then
      echo
	  echo "You have opted NOT to install the nightly build of WordPress."
      exit 1
    else
	  echo
    fi

else
  DOMAIN=wordpress.org
  if [ "$1" = "latest" ]
  then
    VERSION=latest
  else
    VERSION=wordpress-$1
  fi
fi

# Download WordPress
wget -q http://$DOMAIN/$VERSION.zip

# If the file didn't download, then you probably got the URL wrong. Fail.
if ! [ -f $VERSION.zip ]
then
  echo "EXIT: FAILURE! Could not download $VERSION.zip - Did you get the version right?"
  exit
else
  echo
  unzip -q $VERSION.zip
fi

# This is ONE LAST CHANCE. If you say anything other than yes, then it cleans up.
read -p "Last chance.  You sure you want to install WordPress $1 for $USER? (y/n) " -n 1
  if [[ ! $REPLY =~ ^[Yy]$ ]]
  then
    rm -rf wordpress/
    rm $VERSION.zip
	echo
	echo "EXIT: You have chosen NOT to install WordPress $1 at this time."
	exit 1
  else
    echo
  fi

# Get rid of hello dolly and akismet 
rm -rf wordpress/wp-content/plugins/akismet
rm wordpress/wp-content/plugins/hello.php

# Copy the files up to root public_html
# Change this if you want to install to a subfolder or whatever.
cp -r wordpress/* public_html/

# Post install clean up with a 'I don't know what you did!' error.
if [ -f $VERSION.zip ]
then
  rm -rf wordpress/
  rm $VERSION.zip
  echo "SUCCESS! You're on WordPress $1."
else
  echo "POSSIBLE FAILURE. Could not clean up the files, so there's a chance everything went pear shaped."
  echo "Please review your WordPress install and remember: Blame Nacin."
fi

exit

    The fix to ‘Oh shit, I installed WordPress 2.1 instead of 3.1!’ is to re-run it correctly, by the way. So long as you haven’t logged in to your site, you won’t update the database, so a backout is always a re-run, even in the real world.