Half-Elf on Tech

Thoughts From a Professional Lesbian

Category: How To

  • Spam / Splog Wars

    Spam / Splog Wars

    If you run a blog where anyone can comment, no matter what the software du jour is, you’ve had spammers. There’s really no way around it. As soon as someone comes up with a great, easy, way for you to share content and open discussion with other people, the door opens for people to use that great, easy, way to spam you.

    A lot of people take spam for granted. We get junk mail, we get telemarketers, we get spam. It’s a constant of life. And spam is just like junk mail and telemarketers. They want to get your attention, they want you to click on their links, and they want your money. Some spammers link to actual products (usually sex products) and others link to sites that will infect your computer with a virus. The end result, oddly enough, is the same. They want you, and your visitors, to click on their links and somehow make a profit. The cost overhead is so low that even if just one of my visitors clicks on a link and buys something, they’ve made a huge profit. (By the way, if a post looks like spam, don’t click on their links. Only give your money to reliable companies. Research them, ask around. Be smart.)

    A spammer posts spam comments or uses your site to propagate their crap. A splogger is a spam-blog. A blog that only exists to pimp the same crap. If you have an open-registration CMS site where anyone can make a blog, you will get sploggers. Some people will argue that WordPress is less secure because it gets more splogs than Joomla. I would disagree. More people use WordPress’s built in ‘anyone can make a blog!’ feature than Joomla, so it’s a better target if you’re a spammer. You’re going to get more bang for your spam-buck, so you aim for the biggest target.

    No matter what type of tool you use for spam-trapping, remember that the best tool you have is your eyes, your brain, and your common sense. YOU are the number one, best defense, against spammers. Yes, this means you have to give up of your free time to maintain the site, to monitor the new blogs, to monitor the new comments and users, and stop them. While some posts can be hard to determine if their spam, if you check the email, the URLs and the context, usually you can sort them out.

    Unless you have a dedicated team of people monitoring your site for trouble, it’s hard to keep up. This is where I start throwing in tools to help my site. There are three levels of defense: Server, Account and Application. I’m not linking to many tools, since a lot of this is preference. I like certain tools, other people like other tools. At the end of the day, no matter how good your tools are, the human element is required to be attentive and aware of the site. I’ve blogged before on the dangers of an unchecked multisite, and they remain true. Running a website is work. If you’re not willing to put the time in to maintain, monitor, and defend your site, something bad will happen to your site.

    Server Level

    Set up a good firewall. I use ConfigServer Security & Firewall, which checks against Mod Security and bans people who hit it too hard. This prevents a lot of automated spammers and also stops them, once they GET in, from being able to send out spam emails. A good firewall does wonders for other reasons too, but only if you configure it correctly. ConfigServer has a test it can run to see if your setup is good or middlin’ or poor, and I check it every time I upgrade. Oh yeah, keep current with your firewall tool, too!

    Account Level

    I hesitate on this one, but .htaccess can be used to ban IPs. I don’t like to do this and, generally speaking, don’t do this. If someone skirts by my firewall, I’m not going to block them at the IP level, since there are probably some legit users. Also, the firewall is automated, whereas my .htaccess I’d have to manually update. The point of a good tool is that you don’t have to fiddle-fart around manually too much! That said, there are ways you can kill spammers and sploggers via the .htaccess.

    D’Arcy Norman came up with this awesome way to stop them on WordPress Multisite:

    # BEGIN ANTISPAMBLOG REGISTRATION
RewriteEngine On
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} .wp-signup.php*
RewriteCond %{HTTP_REFERER} !.yourdomain.tld. [OR]
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule (.*) http://someotherpage.tld/ [R=301,L]

    Credit: D’arcy Norman.net

    The part I like best is that if you change wp-signup.php to where ever it is your site has a signup page, you can make this work with just about anything. What it does is check the POST requests (which is the server request when you submit a form) for the page wp-signup.php. If those requests do not come from your domain (which if you click on the sign-up button, they must), then it sends them to http://someotherpage.tld/. I send them to a page that says they’ve been caught as a spammer, and to behave better.

    If you must use an .htaccess blacklist, I would strongly suggest you follow D’Arcy’s advice and block them from accessing your registration page only. His thermonuclear option is above and beyond what I need, but I can see it being useful.

    Application Level

    Once you’ve set up your server and account as best you can, you have to start modifying your application. Thankfully, any modern CMS has a good plugin or extension system, and you can leverage that. Sadly, most modern CMS have the same problem of keeping up with the spammers. The tools used are pretty much the same.

    Most CMS have a limited array of built-in tools. WordPress has a way to make all first-time posts require approval for example, but really when you get down to it, you want to throw in some plugins/extensions/modules that are designed to help.

    Content Moderation

    Blacklists are simple. Here is a list of people I don’t want to have access my site. Done. There are the usual caveats for this and the same warning applies as with .htaccess IP blocking: you may block legit users. Personally I prefer moderation lists versus delete blacklists. They put the possible spammer into a bin for me to review and approve or not. Blacklists and mod lists work best, I’ve found, for spam comments rather than splogs. I know, normally, no one on my site will be talking about viagra, but what happens if they have a question about it? The term is on my moderation list. (Funny but true story. I had the word ‘sex’ on a mod filter on another site. Suddenly people were talking about how sexy someone was, and all the posts hit my mod filter. Sometimes these terms are great to block out, but sometimes you forget how they’re really used.)

    That said. There are good, reliable, blacklists. Stop Forum Spam and Project Honey Pot both are blacklists maintained by the community, and get enough information that they can be use reliably. There’s also Spamhaus, but that’s mostly email. You can use these on most applications as well.

    Behavior

    There’s actually a plugin called Bad Behavior, which is a great example of what I mean. Bad Behavior the app analyzes the HTTP headers, IP address, and other metadata regarding the request to determine if it is spammy or malicious. What I mean by the term is simply ‘Is the visitor coming to my site from a legit browser?’ See, spammers don’t use the normal browsers that you and I use, so if someone comes from a special method to my site, they’re probably not a good person.

    Behavioral monitors learn as they go, or as they’re updated, and can be pretty effective. Sometimes people on archaic browsers (IE 6, for example) will get nailed by false positives. There’s not a whole lot I worry about with those, since any browser since 2005 is usually safe to go. There are two people who hit my sites regularly who’ve run into this problem, and once I sorted out what they were doing (Netscape Navigator 4 came out in 1998 and IE 6 came out in 2001), I told them ‘Look, the site will look like crap anyway. Upgrade.’ One guy couldn’t, the other switched to FireFox and admitted to being much happier anyway. (I don’t believe in supporting browsers pre-2005 at this point. I monitor my site stats for them, but realistically, it’s not going to happen. Upgrade, people. You’ll be happier.)

    CAPTCHA

    Personally I hate these. CAPTCHA is named after ‘capture’ and is a contrived acronym for “Completely Automated Public Turing test to tell Computers and Humans Apart.” It looks like this:

    The idea behind CAPTCHA is that it should provide a problem easy enough for all humans to solve while, at the same time, prevent standard auto-fill software from filling out the form. Sounds great, but the problem is people have created software to read the CAPTCHA files. Personally, I love the fact that we’re making AIs smart enough to parse this stuff, but it hurts CAPTCHA because in order to defeat the AIs, they tend to become harder and harder to read for real people. Also, most CAPTCHAs are not friendly to people with limited accessibility. If you have dyslexia or glaucoma, they’re of the devil. I would never consider using one on my site unless forced to.

    Human vs Computer Test

    Originally CAPTCHA just meant any challenge/response to stop automated form fillers. Since it now is used, almost exclusively, to refer to those images, I’m pulling out Human Tests. A human test is when you have a question, or form, that requires thought to answer. Like ‘Who’s buried in Grant’s Tomb?’ We all know the answer is ‘Grant’ so you type that into a text field when you register or comment, and magically you have access. I’m fond of simple math questions like ‘What is 12 + 8?’ but also good are site specific questions. One on my friend’s site is about Marg Helgenberger, and they use questions about Marg (like ‘what’s the last name of her character on CSI?’).

    The reverse is to trick the computer. Put a hidden checkbox on your site, that is NOT human readable, and if that box is checked, aha! Spammer! That’s a pretty cool trick if you can make it work.

    Do you have tips and tricks you use?

  • Dig Yourself Out of a Hole – Multisite Edition

    Dig Yourself Out of a Hole – Multisite Edition

    When I started my first MultiSite install, I didn’t want Blog to be the main ‘dashboard’ site. The reasons why don’t matter. What does is that when you make seemingly simple changes, you have to be aware of how things work, as a whole, and be willing to take risks to fix them.

    Site was my test site and has since been deleted
    Site was a vlog which still exists
    Site is my ‘main’ site, the front page of it all
    Site is my saved Tweets
    Site is a documentation site
    Site is to save my Formspring posts

    I know it’s weird. Bear with me. At first I was going to use Site as an ‘all posts!’ site, like Andrea suggests you do with sitewide tags plugin, but then I realized that I wanted a splash page and then some fancy ‘Hi! Welcome!’ stuff with links to the vlog, blog, etc. Once I decided I wanted site to be the ‘main’ site, I made that switch by editing two things.

    First I went into Super Admin >> Options and changed my main site to :

    Then I edited my wp-config.php file to say:

    define('SITE_ID_CURRENT_SITE', 3);
define('BLOGID_CURRENT_SITE', '3' );

    Now to be honest, I think that I only need one of those, but I’m totally stumped about that now. At the time, I just saw they were both set to 1 and I changed them to 3. Everything worked! Great! This was done.

    Then I went to create site . And it didn’t show up. So I made site , ditto. Decided that was weird as hell, I went into phpMyAdmin and after some poking around, deleting sites and re-adding, I found the wp_blogs table and saw this:

    Rationalizing that since I could see Blogs 2 and 3, and their site_id was 1, I changed it to this:

    And again, things seemed to work just fine. Until I started trying to use the new iPad WordPress app. Or add anything to post to my blog. Like Formspring. Nothing worked. The blogs couldn’t be found, and third-party apps sometimes crashed. Again, I thought ‘Maybe the site_id should be 3!’ So I changed it to this:

    Not only did that not work, it broke things. Quickly, I changed it back and frowned. I was obviously missing something, but what!? I started looking around for all instances of site_id and found the wp_sitemeta table! Most of the entries had site_id of 1, but some had 3.

    I changed the wp_blogs table again, knowing it would break things, exported the whole wp_sitemeta table as a backup, and then ran a quick SQL search and replace, changing 1 to 3 and got this:

    Now the whole site worked from front and back end, so I went to Formspring and tried to add my site. And guess what? It worked! It also magically fixed my iPad/WordPress testing woes.

    So the lessons learned are “Never give up!” and “Never make changes when you don’t know what you’re doing!” Actually, no, not that second one. The lesson is to be bold with your solutions. Don’t panic and be willing to take a risk. I took backups before I started playing around, and I knew at worst, I could restore my site in 30 minutes. Because I had an escape plan that took me back to where I was, I had no fears about my changes. When you have nothing to lose, that’s when you should jump forward and try something silly!

    I doubt the actual details of all this will help you, since I don’t think anyone but me was boneheaded enough to screw around like this in the first place. But seeing that someone else has seriously screwed themselves up, shot themselves in the foot, and come out feeling super smart should, I hope, encourage you to stay calm, think it through, and make a stab at something that seems right. When the wave rocks your boat, hang on and don’t be afraid to swim a bit.

  • Backup Your Data

    We all know this adage: Your data is only as secure as your last backup. But how do you backup? What do you backup? Where do you put it so that your data is safe, secure, and above all, accessible in a pinch? (more…)

  • Hotlink Protection

    Hotlinking is putting a link to someone else’s webpage’s graphic on your site. This is also called bandwidth theft. Directly linking to a website’s files (images, video, etc.) means that when someone accesses your website, they draw bandwidth from another. If you use an IMG tag to show a picture from someone else’s page on your blog, forum post, or website, that’s hotlinking. You’re stealing their bandwidth.

    There is a case in which this sort of ‘theft’ is ethically permissible, though some webhosts don’t like it. If you have multiple Yahoo! sites, and one is low on bandwidth, you can shuttle some of your content to the other site, and thus split up the bandwidth. This isn’t always a good idea, as if it’s against the Terms of Service on your host, they can kill you. Which is why you should always back up your websites on your on computer. If you own your own domains (like I do) and have multiple ‘subdomains,’ then it’s okay to share an image. ipstenu.org is considered a different website that photos.ipstenu.org, so I have to tell my server it’s okay to share between the two. But that’s code geeky.

    What the common websurfer needs to know is this: direct linking to a picture, movie file, or any other content on someone else’s site, unless it’s a simple URL link to that site, is bad form, ethically asinine, and impolite. It’s akin to stealing electricity from your neighbor by plugging into their outlets.

    But what do you do when someone’s hotlinking to your server? Most of us find out about this via a nastygram from our webhost saying we’re using too much bandwidth. Bandwidth controls how fast you can view the net from your home, as well as how much data a website can share with the world each month. Having more bandwidth is better all the time, but forcing users to use more bandwidth with image heavy sites and poorly coded web pages is not cool. Still, sometimes you have a moderate site and one image becomes super popular.

    This is where you need to learn about hotlink protection. The most basic code is this:

     
# Simple Hotlink Protection

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?yourdomain.com(/)?.*$                   [NC]
RewriteRule \.(gif|jpe?g?|png)$                             - [F,NC,L]

    This basically says ‘If you’re not from yourdomain.com, and you’re trying to see an image, you’re not me, go away. Sometimes I make that last line something like this:

    RewriteRule \.(gif|jpe?g|png)$ http://mydomain.com/hotlink.gif         [NC,L]

    Which shows them a ‘No, don’t do that’ image. If you’re going to do that, use a SMALL image, since that will use up some of your bandwidth.

    For most people, that works just fine, but I’ve run into a couple situations that were weird.

    Multiple Subdomains

    If you’re using a lot of subdomains (like, say, with WordPress MultiSite) you’ll find pretty quickly that the normal hotlink protection rule will block subdomain.yoursite.com from getting images from www.yoursite.com and we don’t want that! For one subdomain, it’s an easy fix:

     
# Simple Hotlink Protection

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?yourdomain.com(/)?.*$                   [NC]
RewriteCond %{HTTP_REFERER} !^http://(subdomain\.)?yourdomain.net(/)?.*$               [NC]
RewriteRule \.(gif|jpe?g?|png)$                             - [F,NC,L]

    But what about multiple sites? At 12 subdomains, you don’t want to have to add these links in manually every time! Thankfully, the geniuses at Perishable Press have created the Ultimate htaccess Anti-Hotlinking Strategy. You can read the whole post for the details, but here’s the basic code:

     
# ultimate hotlink protection

 RewriteEngine on
 RewriteCond %{HTTP_REFERER}     !^$
 RewriteCond %{REQUEST_FILENAME} -f
 RewriteCond %{REQUEST_FILENAME} \.(gif|jpe?g?|png)$           [NC]
 RewriteCond %{HTTP_REFERER}     !^https?://([^.]+\.)?domain\. [NC]
 RewriteRule \.(gif|jpe?g?|png)$                             - [F,NC,L]

    Simple. Elegant. Genius. All you have to do is change domain to whatever your domain is. Notice there’s no .com or .net in there? There doesn’t need to be. This is the one I use for this site:

     
# ultimate hotlink protection

 RewriteEngine on
 RewriteCond %{HTTP_REFERER}     !^$
 RewriteCond %{REQUEST_FILENAME} -f
 RewriteCond %{REQUEST_FILENAME} \.(gif|jpe?g?|png)$           [NC]
 RewriteCond %{HTTP_REFERER}     !^https?://([^.]+\.)?ipstenu\. [NC]
 RewriteRule \.(gif|jpe?g?|png)$                             - [F,NC,L]

    That’s it. Just change domain to ipstenu and I’m done.

    Letting Other Sites Use Your Images

    The other major gotcha to this is what about other sites where it’s okay if they link to you? For example, I have a livejournal site (I know) that’s a mirror of another blog. To take care of that, I added in this as my last condition:

     RewriteCond %{HTTP_REFERER}     !^http://ipstenu.livejournal\.   [NC]

    Here I specified the URL a little more, since I don’t want all of livejournal nabbing my images. Of course, ironically enough, the line where I call ipstenu has the funny side effect of allowing any URL with the name ‘ipstenu’ in it to access my site. Which is a risk I accept right now.

    If you’re using my first example, the simple protection, then just like you added in a subdomain, you add in your other URLs

     
RewriteCond %{HTTP_REFERER} !^http://ipstenu\.livejournal\.com(/)?.*$               [NC]

    This will save you some headaches down the road, but just remember which one your using. Otherwise, like me when I made a new subdomain, you’ll sit there wondering why the heck the images are broken!

  • Child Themes – Learn them, love them

    As a general rule, I don’t do much help with themes. Not because I can’t, but because themes are a style and preference issue, which means what I think looks good, you may not. Once I find a theme I like, I prefer to spawn a child there-of to modify and customize. I’m going to step a little outside my comfort zone to discuss child themes.

    The parent/child relationship in code is something that shows up everywhere. A child inherits from a parent, and only the aspects of the child you change will be different. This is most simply understood in the realm of WordPress themes. The official WordPress Codex doc on Child Themes is mostly straight forward for anyone who’s looked at a theme and played around a little. But the novice can get easily overwhelmed.

    The easiest way to get started is to make a new folder in wp-content/themes for your theme. /wp-content/themes/child-of-2010 for example. I like to name my children something that will remind me right away of their parent. For example, on another site, I have a news-style theme, taken from Justin Tadlock’s Hybrid Theme, which I called ‘Hybrid News ala JFO – Blogging’. The folder name is ‘hnj-blog‘ to differentiate it from a very similar, but functionally different ‘hnj-video‘ and ‘hnj-plain‘. Each of the three themes is a small variation on the parent theme, but because of the way I wanted to call things, I decided that it was better to have three themes instead of an extensive functions hack and slash. This was after a lot of testing and tweaking and changing, but in the end, this worked for me.

    For a proper child theme, you only need one file in that folder: style.css

    You heard me right, just one. That Style file will be what tells your code where to look for your parent theme. Here’s what Child looks like:

     
/*
Theme Name: Child of 2010
Theme URI: http://Ipstenu.org
Description: This is a child theme of Twenty Ten (2010), the 'new' default WordPress Theme.
Version: 1.0
Tags: fixed width, widgets, valid CSS, valid XHTML, SEO, SEO friendly, adsense, custom header, three columns, clean,  right sidebar, blue,white, photoblogging, widget ready, simple, gravatars
Author: Mika Epstein
Author URI: https://ipstenu.org/
Template: twentyten
*/

/* Get base CSS */
@import url('../twentyten/style.css');

    There are two really important parts to this. First is the line Template: twentyten — This is what defines the theme as a child theme. Period. Without this, you have nothing. The second is @import url('../twentyten/style.css'); which tells your CSS to pull down all the CSS from Twenty Ten and use it.

    Once you’ve done that, you have a working theme and you can activate it. I know! It’s so easy, a caveman could do it! Any CSS changes you want to make go in your new child theme and, because of how inheritance works in code, they override anything imported from Twenty Ten, so your changes magically take effect, without having to reproduce the whole theme!

    But what if you want to change a template file? I wanted to add one line to my comments.php file to turn on a subscription checkbox. This is done by copying the comments.php file from the twentyten folder into your child-of-2010 folder, and making the edits. It works almost the same way that the style.css. The main difference is that with template files, you can’t just have it contain only your changes, it must be the whole file, with your changes added in. This trips people up sometimes.

    Now that you’ve seen how easy they are to make, you may wonder what the point is? After all, they are easy, but it’s easier still to tweak the parent theme. Yeah, it is. It’s also easy to lose all your changes when you upgrade WordPress.

    For better of for worse, and yes, people argue about this all the time, WordPress upgrades overwrite the Twenty Ten theme, just like they used to overwrite the Default theme. I can’t begin to tell you how many times I’ve see angry posts ‘The upgrade blew away my theme changes!’ And for every one of those, someone’s posted ‘Yeah. It does that. Restore your theme from backup, and by the way, try a Child Theme.’ Now that WordPress lets you upgrade themes from inside the admin screen, you’re going to run into this more and more often. That’s why we’re always saying ‘Don’t change core! Don’t edit themes! Make functions/plugins/children!’ More on the other stuff later.

    A child theme is smaller than a copy, and it’s just as fast. A child theme will never be overwritten when your theme is upgraded. Your child theme is safe even from a manual upgrade. Remember the directions for that? Even when we tell you ‘delete your WordPress folders…’ we always say ‘except for wp-content!’ That’s because your uploads, plugins and themes are in that folder, and you should NEVER delete them for an upgrade. And by making your own theme folder, no one but you will use it and no other theme will overwrite it.

    So yes, use Child Themes. Learn them. Love them. Your life will be better the next time you upgrade WordPress.

  • Moving Your Images For MultiSite (Updated)

    Moving Your Images For MultiSite (Updated)

    Updated after Andrew Nacin asked me “Why are you suggesting they replace ‘wp-content/blogs.dir/1/files/’ into their post content, instead of /files/?” (Answer: Because when I did this on my first site, about a year ago, I majorly goofed my SQL search/replace and shot myself in the foot For some reason, that made me think ‘files is bad! Blogs.dir is good!’ which … it’s not. Really, blogs.dir is fewer redirects, but that’s really about all I can say. So this has been edited. Thanks!)

    In generic WordPress single installs, your images are, by default, located in a folder called uploads off the wp-content folder, and tend to look like this: wp-content/uploads/YYYY/MM/image.jpg. When you use MultiSite WordPress, the files are now in wp-content/blogs.dir/#/files (where the # is the blog number). If you upgrade from Single to MultiSite, you can leave the files for Blog , i.e. your primary blog, where they are to no ill effects. Or you can move them.

    I’m not going to be using # as a placeholder, as for most people this blog will be 1. If for ANY REASON yours is not, change it.

    Also, I can’t stress this enough, you don’t have to do this! Your images will be just fine where they are, but if you want to move them, you can. Enough people have asked on the WordPress.org support forums that I bothered to consolidate my notes on this, however.

    Oh, and backup. Always backup before you start this sort of thing, otherwise you’re a reckless fool.

    Step 1 – Move the OLD files

    This is easy. Copy or move the files to wp-content/blogs.dir/1/files

    While you’re in there, remember to set the folder writable so you can update files later. Your images will now look like wp-content/blogs.dir/1/files/YYYY/MM/image.jpg – Make a note of this, you’ll need it in a second.

    Step 2 – Teach WordPress where the OLD files are

    There are two main ways to do this:

    Edit the Database
    If you have phpMyAdmin or are savvy at command line SQL, just go ahead and run the replace command: 

     
update wp_posts set post_content = replace(post_content, 'wp-content/uploads/', 'files/');

    Now someone here might go “Hang on, I put my files in blogs.dir/1/files and you’re saying to tell WordPress to look in files! What gives?” What gives is what Andrew reminded me! WordPress MultiSite parses files from ‘files.’ The shortest explanation is ‘It’s an .htaccess trick.’ The longer explanation is it’s own blog post.

    If you don’t like the idea of SQL, get a search and replace plugin like Search and Replace or Search RegEx and install it. Then search for wp-content/uploads/ and replace with wp-content/files/ to change the database.

    In both database cases, you’ll want to check changes all over the place. For wp_posts, check post_excerpt and post_content, and then go through wp_postmeta and edit the meta_value fields.

    .htaccess
    That’s a lot of work, and odds are you’re still going to miss something. You can also be really lazy and add this to your .htaccess file, before the WordPress stuff: 

     
# Moved Images
RewriteRule ^wp-content/uploads/(.*)$ http://domain.com/files/$1 [L,R=301]

    That redirects things quietly. It’s probably not the best way, but like I said, I’m lazy. If I was doing this for a client, I’d probably do both the database fix and the .htaccess, to catch any stragglers.

    Step 3 – Tell WordPress where to put the NEW files

    Now the fun part! Go to your site’s wp-admin section, click on Super Admin and pick sites from the drop down. Hover your mouse just below the path to your main blog and click on Edit

    In that new screen, scroll down and look on the left for Upload Path – You want to change that to wp-content/blogs.dir/1/files

    You also want to change Fileupload Url to be http://domain.com/files (this will hold true even if you’re using subfolders or subdomains).

    If you see Upload Url Path, you can change if to http://domain.com/files as well, though it appears to be depreciated and no longer used.

    Step 4 – There is no Step 4

    That’s it! You’re done!