Half-Elf on Tech

Thoughts From a Professional Lesbian

Author: Ipstenu (Mika Epstein)

  • The dangers of an unchecked MultiSite?

    Blogetery was shut down, mysteriously, over the weekend. It was a WP MultiSite setup, with around 70k blogs. Not terribly abnormal to have an install that big, but the thing as an unnamed law enforcement agency shut them down. Details, such as they were, were posted at ReadWriteWeb: 70,000 Blogs Shut Down by U.S. Law Enforcement. Their shutdown reminded me of the hazards of running a website where anyone can register and make their own site and how important it is to be vigilant about what shows up on your website.

    Discussion of the situation spun up on Web Hosting Talk where it was determined that Blogetrey had been accused of hosting inappropriate content before. That probably meant they were hosting torrents or other illegal but not shut-down worthy. Copyright infringement. The site owner claimed that every copyright violation was removed within 24 hours. By the way, if you ever get slapped with a DMCA notice (i.e. a notice that your site has content copyritten to someone else), in order to be safe from a law suit, all you have to do is remove it. Done.

    So what on earth would cause BurstNET, their host, to shut down the site without warning or notice? That’s right, he had to ask ‘What happened to my site?’ and was told it was shut down, terminated, and here’s his money back.

    Turns out he had a link.

    From BurstNET’s statement:

    “It was revealed that a link to terrorist material, including bomb-making instructions and an al-Qaeda “hit list”, had been posted to the site. “

    That’s it. A link. One link. But it was enough for a warrant which then showed this:

    “Upon review, BurstNET® determined that the posted material, in addition to potentially inciting dangerous activities, specifically violated the BurstNET® Acceptable Use Policy. This policy strictly prohibits the posting of “terrorist propaganda, racist material, or bomb/weapon instructions”. Due to this violation and the fact that the site had a history of previous abuse, BurstNET® elected to immediately disable the system.”

    Now the previous ‘abuse’ was copyvio, which was all handled legally, but clearly BurstNET was feeling the pinch. They probably got slapped with a wwarrent and did the legal thing: They shut it down.

    Reagrdless of if it was fair or not to the other 69,999 sites hosted by Blogetery, it brings up the inherent problems of running an unchecked MultiSite. Anyone can make a blog/site, anyone can update it, and anyone can get you in trouble.

    It’s been a few weeks, but finally news is coming out about the whole story. CNET’s article was invectively titled Bomb-making tips, hit list behind Blogetery closure. That said, it explained this in more detail which let everyone get a grip on what was actually going on.

    I’m not going to get into the ethics of free speech and how it does (and doesn’t) apply to your website. Instead I want to use this as a reminder of the trouble you can get into, hosting websites. I host four, three are ‘mine’ and one is a site I like and visit pretty often. I’m very much aware of what’s going on all these sites and I monitor them frequently. This is not just to my benefit, but to everyone else’s on my servers. My host would be 100% within their rights to say “Ipstenu’s got a site that has kiddie porn! Kill her account!” and that would shut down everyone on my server.

    As I mentioned before, WordPress MultiSite makes it a lot easier for someone to host a thousand blogs, unchecked, but that also means it’s a lot easier for someone to post questionable content. For copyvio cases, you’re covered when you remove the material in question, but for porn and terrorism, it’s not actually under the same purview. Again. I’m NOT going to get into the why of this, nor the right or wrong about it. If you have a website, you have to accept that your host really has no interest in being involved with a legal dispute regarding kiddie porn or terrorism.

    This means it’s down to you to constantly and consistantly monitor your site for sub-sites and domains that are questionable. For me, if a site I host gets one Cease and Desist about copyvio, I take down the material, explain to the person who runs the site why, and ask them not to do it again. At this point, it’s their job to monitor their site. Should they fail to do so a second time, I give them a final warning of ‘If you can’t keep tabs on your site and your visitors, you can’t stay here.’ Third time and I close their account, refund them what’s left on their time, and offer to give them a copy of their site and database, intact.

    For the rest, though, it’s a no-warning termination, specifically because porn and terrorism are hot button topics. I’m within my rights to do so (I own the server, I make the rules) and I owe it to the other people. My ISP is in their rights to do similar, because they own the … land my server is on. If that makes sense.

    If all this sounds like too much work for you, then you shouldn’t be running an open, anyone-can-register-and-blog, multisite. Or you should hire some staff. Multisite is not a quick money scheme, it’s a job, and you have to take it seriously.

    This is not endemic solely of WordPress, but with the advent of MultiSite becoming mainstream, it’s something that’s going to start coming up more and more. Don’t say you weren’t warned.

  • Moving Your Images For MultiSite (Updated)

    Moving Your Images For MultiSite (Updated)

    Updated after Andrew Nacin asked me “Why are you suggesting they replace ‘wp-content/blogs.dir/1/files/’ into their post content, instead of /files/?” (Answer: Because when I did this on my first site, about a year ago, I majorly goofed my SQL search/replace and shot myself in the foot For some reason, that made me think ‘files is bad! Blogs.dir is good!’ which … it’s not. Really, blogs.dir is fewer redirects, but that’s really about all I can say. So this has been edited. Thanks!)

    In generic WordPress single installs, your images are, by default, located in a folder called uploads off the wp-content folder, and tend to look like this: wp-content/uploads/YYYY/MM/image.jpg. When you use MultiSite WordPress, the files are now in wp-content/blogs.dir/#/files (where the # is the blog number). If you upgrade from Single to MultiSite, you can leave the files for Blog #1, i.e. your primary blog, where they are to no ill effects. Or you can move them.

    I’m not going to be using # as a placeholder, as for most people this blog will be 1. If for ANY REASON yours is not, change it.

    Also, I can’t stress this enough, you don’t have to do this! Your images will be just fine where they are, but if you want to move them, you can. Enough people have asked on the WordPress.org support forums that I bothered to consolidate my notes on this, however.

    Oh, and backup. Always backup before you start this sort of thing, otherwise you’re a reckless fool.

    Step 1 – Move the OLD files

    This is easy. Copy or move the files to wp-content/blogs.dir/1/files

    While you’re in there, remember to set the folder writable so you can update files later. Your images will now look like wp-content/blogs.dir/1/files/YYYY/MM/image.jpg – Make a note of this, you’ll need it in a second.

    Step 2 – Teach WordPress where the OLD files are

    There are two main ways to do this:

    Edit the Database
    If you have phpMyAdmin or are savvy at command line SQL, just go ahead and run the replace command: 

     
update wp_posts set post_content = replace(post_content, 'wp-content/uploads/', 'files/');

    Now someone here might go “Hang on, I put my files in blogs.dir/1/files and you’re saying to tell WordPress to look in files! What gives?” What gives is what Andrew reminded me! WordPress MultiSite parses files from ‘files.’ The shortest explanation is ‘It’s an .htaccess trick.’ The longer explanation is it’s own blog post.

    If you don’t like the idea of SQL, get a search and replace plugin like Search and Replace or Search RegEx and install it. Then search for wp-content/uploads/ and replace with wp-content/files/ to change the database.

    In both database cases, you’ll want to check changes all over the place. For wp_posts, check post_excerpt and post_content, and then go through wp_postmeta and edit the meta_value fields.

    .htaccess
    That’s a lot of work, and odds are you’re still going to miss something. You can also be really lazy and add this to your .htaccess file, before the WordPress stuff: 

     
# Moved Images
RewriteRule ^wp-content/uploads/(.*)$ http://domain.com/files/$1 [L,R=301]

    That redirects things quietly. It’s probably not the best way, but like I said, I’m lazy. If I was doing this for a client, I’d probably do both the database fix and the .htaccess, to catch any stragglers.

    Step 3 – Tell WordPress where to put the NEW files

    Now the fun part! Go to your site’s wp-admin section, click on Super Admin and pick sites from the drop down. Hover your mouse just below the path to your main blog and click on Edit

    In that new screen, scroll down and look on the left for Upload Path – You want to change that to wp-content/blogs.dir/1/files

    You also want to change Fileupload Url to be http://domain.com/files (this will hold true even if you’re using subfolders or subdomains).

    If you see Upload Url Path, you can change if to http://domain.com/files as well, though it appears to be depreciated and no longer used.

    Step 4 – There is no Step 4

    That’s it! You’re done!

  • Understanding Plugin Stats on WordPress

    Understanding Plugin Stats on WordPress

    So you’ve have a plugin and you’ve finally released it on WordPress.org to allow the masses. It’s going well, you’ve run a couple revisions with no major issues. But how can you tell how ‘well’ your plugin is actually doing?

    Realistically, you’re going to have to wait a while, maybe even a year, for things to even out realistically. After a year, you should have something like this on your Stats panel and this CAN be explained!

    Here’s what Ban Hammer’s stats look like. It’s just about a year old right now.

    1) Initial Release – This was when I released it and started pimping it.

    2) First Major Revision – This was when I started adding in more features, making the code faster, and debugging something.

    3) Certified for 3.0 – This was … well, yeah, that.

    Look at how the spikes ramp up a lot. The first one is moderate, the second one is twice the first, and the third is another moderate change. It’s not impressive, my plugin is for a pretty niche market and I don’t expect it to be hugely popular. Impostercide gets about 80 a day right now, compared to the 10 to 30 Ban Hammer got in its first days. But the basic picture is pretty clearly painted.

    It’s not quite an exponential jump for each update, but the update is what’s telling you who is really using your plugin.

    Let’s look at the very popular WP Super-Cache plugin:

    His spikes are pretty static, standing at 11k downloads on every major release. That’s really impressive, but it gives a far more realistic understanding of your plugin and how many people are really using it.

  • The WebSVN Alternative

    The WebSVN Alternative

    Now that SubVersion is up and running, I decided to make it visible to others via WebSVN. The only reason I’m using WebSVN, which is really cool, don’t get me wrong, is because I can’t get mod_dav_svn and the default apache stuff working with my server.

    Go to http://svn.ipstenu.org and you can see my web-version of SubVersion up for you to snag whatever you want.

    Installing and configuring this was really easy. I followed the directions. Done. The only tweak was that I had to finagle some Open Basedir settings to let my install talk to SubVersion. But it works, and now you can see what I’m working on.

  • Struggling with SubVersion

    After some twittering with Mark Waterous, we decided that I probably had an out of date version of SVN. This is not a surprise. My server is CentOS, and they have this weird ‘core’ plugin repository that basically says ‘whatever version of a package we release your OS with is the version you get’. It took some heavy lifting and a stern conversation with yum that I do too want to use rpmforge as a source, but I finally upgraded everything to version 1.4.6 (previously I was on 1.1.x).

    And what happened when I tried to reactivate mod_dav_svn

    Same damn error!
    API module structure 'dav_svn_module' in file /usr/lib/httpd/modules/mod_dav_svn.so is garbled - expected signature 41503232 but saw 41503230 - perhaps this is not an Apache module DSO, or was compiled for a different Apache version?

    I took a step back and decided to make sure SubVersion really was working on my server. After a couple hours of wrangling, I determined it was. In fact, it’s working fine. For a while I had svnserve running and you could see it up on a website. But it still wouldn’t let me check in or out files from my PC at work (I had to do it all via command line on the server). That’s not what I want.

    So why won’t dav_svn work? No idea. Mark implied it’s CentOS and I’m inclined to agree. Everything else works. Not that I really have a thing against CentOS, but Red Hat (and it is Red Hat) has been weird ever since they released their IPO a decade back. Lost their street cred, I suppose.

    Mark’s suggestion:

    Pain in the butt, but uninstall current, try mod_dav_svn-1.4.6-0.1.e14.rf.i1386, then 1.3.2, then 1.2.1, see if any work?

    Thankfully, installing older packages in yum is pretty easy. It’s just yum install mod_dav_svn-1.4.6-0.1.el4.rf.i386 and done. Except I got this:

    ---> Package mod_dav_svn.i386 0:1.4.6-0.1.el4.rf set to be updated
    --> Running transaction check
    --> Processing Dependency: subversion = 1.4.6-0.1.el4.rf for package: mod_dav_svn
    --> Finished Dependency Resolution
    Error: Missing Dependency: subversion = 1.4.6-0.1.el4.rf is needed by package mod_dav_svn

    Basically, subversion and mod_dav_svn have to match for this to work. Now, since I have subversion working, I really don’t want to roll that back. I decided that maybe I just had a bad copy from the repository, so I downloaded SubVersion and installed it manually. This is always fun, but I do it often enough at work that I’m only slightly terrified. This is why I have a managed server, by the way. Since I got the same damn error after a manual install, I decided that something was wrong with my server and mod_dav_svn.

    Thankfully there are alternatives!

    svnserve is not as ‘pretty’ but it works. I followed the default installation directions and then used Subversion : svnserve over xinetd to get it running. Once I got it running, I was able to point things at svn+ssh://ipstenu.org/svn/repos/ipstenu (which will do you no good, it’s locked to my user ID/password). Now everything downloads and I say sayonara to Google Code!

    Next up, I’ll put a WebSVN site up for people to browse my code. If they really want to.

  • Switching to WordPress MultiSite Breaks Links

    As of WordPress 3.1 you can NO LONGER remove the blog slug via the method described in this post! – Feb 28 2011

    Warning: WordPress MultiSite is not for the newbie!

    Most people I know who are using MultiSite (formerly WPMU) are people who have always been using that fork of WordPress. With the advent of MultiSite, a built in way for people to move from single ‘blog’ installs of WordPress to a multiple site structure, more and more users want to move to it, which is great. But. There are some weird ‘problems’ with MultiSite that cause a lot of users to think it’s broken. It’s not, it’s just different.

    My friend Andrea wrote a nice tutorial about how to create a network, and generally when I need help, I scan WPMU Tutorials first. Or I tweet at her. One of the things we’ve talked about is the way WordPress MultiSite changes your blog links.

    When I changed https://ipstenu.org from single to multi, my links were changed from https://ipstenu.org/YYYY/MM/PostName to https://ipstenu.org/blog/YYYY/MM/PostName, which I did not want! Thankfully, I knew a lot of people had this problem beforehand and I fixed it quickly, which I will explain here, but first, here’s why it’s done that way.

    Normally, when we use pretty permalinks (that’s the part of your URL that looks ‘pretty’ instead of code-like) on a blog, all we have to worry about is not conflicting with post names and page names. If you have an about page, generally the URL is https://ipstenu.org/about. Since WordPress has to translate https://ipstenu.org/?p=2 into that to be pretty, you can’t also have a blog post with the same URL. How would the software know WHICH page you wanted? Simple enough. For details on that, you can read Otto’s post on Category in Permalinks Considered Harmful.

    When you add in MultiSite, suddenly the code has to check for something more: is this new Site named something that breaks my existing links?

    There are two kinds of MultiSite, and I use them both. You have SubFolders and SubDomains. Subfolders gets you URLs like https://ipstenu.org/code and SubDomains gets you https://halfelf.org/. If you use SubFolders, WordPress takes an extra step to make sure that pages named ‘code’ don’t break when you have a new blog with the same name. To help do that, it kicks your blog URLs from your main blog (i.e. https://ipstenu.org/) up a level to a new URL: https://ipstenu.org/blog/. For those of us who move from Single to Multi, this breaks our old links.

    Thankfully there are a couple fixes which can tide you over until WordPress 3.1 is released. The developers plan on having this ‘fixed’ or at least editable by then, so at the worst, you can go from having a blog slug to ‘news’ or ‘pasta’ or whatever.

    .htaccess

    If you want to keep the blog folder, but you want to redirect your old posts, .htaccess is your friend.

    If I wanted to redirect all YYYY/MM/DD posts to /blog/… I would add this:

     
RewriteRule ^/([0-9]{4})/([0-9]{2})/([0-9]{2})/(.*)$ https://ipstenu.org/blog/$1/$2/$3/$4 [L,R=301]

    What I actually did on another site was this:

    RewriteRule ^/([0-9]{4})/([0-9]{2})/([0-9]{2})/(.*)$ http://domain.com/blog/$4 [L,R=301]

    And I changed my permalinks to just post name. This was risky, in a way, but it matches how I use the site better.

    Plugins

    There are a couple plugins that can help you here. Thinking in WordPress made a plugin for WPMU, but I’ve heard it doesn’t quite work yet.

    Andrea (and her husband Ron) have a Remove Blog plugin that can also help you.

    I’m sure there are others, but I mostly recommend Andrea and Ron’s. And not just because I like them.

    Hack

    If you’re dead-set on doing this, and you don’t want to use a plugin, there is a manual way to do this. Remember, by doing this you will BREAK the ability of WordPress to check for conflicts between your main site’s urls and any new site. If you’re using SubDirectory SubDomain MultiSite, this should be fine. Hell, I’m doing it. If you’re using SubFolder, I really don’t recommend this at all.

    You ready?

    • Login to your admin site.
    • Go to: Super Admin > Sites > Edit
    • Scroll down to “Permalink Structure” and remove the “/blog” part
    • Save

    That’s it.