NOTE: You do not, under any circumstances, have to do this to continue using WordPress Multisite. I just wanted to see if I could.
I have been toying around with this. Since WP 3.5 doesn’t use ms-files anymore, I wanted to see how much of a pain this would be. While I was at the DMV (I now have a California Drivers License), I started sketching out the methods and possibilities for how one might do this as safely as possible, without bolluxing up your site. I got about 90% successful with it, so this is something I would never suggest you do on a massive, live site. If you compare it to what I did to move images from uploads to blogs.dir for the main site, it’s far more complex and annoying.
Do you have to do this? Again, no! Everything will work just fine. Can you do it? Sure. Why did I? Support. I already know all the mishegas with blogs.dir, but the new location, and it’s lack of ms-files, promises other, weird, errors. I want to repeat, you don’t need to do anything. The Core Team was fantastic with the work they did to turn off ms-files for fresh 3.5 and up installs is nothing short of phenomenal. This was just to assuage my curious nature, and learn more about the way things work.
You ready? Here we go.
I decided to move the images at http://test.ipstenu.org/ to start with, so everything will use that as my example. This is after I played with it on a local site and got my steps mostly solid. I knew that live is always different than test, so I backed up the DB first (always, always, backup first!) and went to town.
Move the images
This is obvious. Move your images from blogs.dir/SITENUM/files/ to /uploads/sites/SITENUM/ (or make an alias). I went and did it via command line in the /uploads/sites/ folder, doing this:
$ cp -r ~/public_html/wp-content/blogs.dir/10/files . $ mv files 10
Lather, rinse, repeat. I could have scripted it, but I was working out the kinks until I had two left.
Edit all sites
Upload Path, Upload URL Path and Fileupload Url. You can blank them all out.(Corrected thanks to Nacin.)
Since you’re blanking it out for everyone you can probably do this via SQL, but since I was doing the sites one at a time, I did them one at a time.
Fix the Database
Search/replace each posts table for each site, changing /files/ to /uploads/SITENUM/
UPDATE wp_10_posts SET post_content = REPLACE ( post_content, '="http://test.ipstenu.org/files/', '="http://test.ipstenu.org/wp-content/uploads/sites/10/');
Why did I do it that way? Because of this blog. I talk a lot about code here, and I know I’ve talked about moving files around before. If you don’t do that, you’re okay with a less precise search, but why bother? This works, it’s safe, and I’d use it again.
That got annoying really fast. I went and grabbed my favorite Search And Replace for WordPress (and any other database) tool. Seriously I love that. I used that to follow up, change everything, and it actually worked really well for me.
Another DB Fix!
One of the changes in 3.5 was turning off rewriting. This took me forever and a day to find. After I did that, my images showed up fine, but the little buggers kept uploading to /files! Turns out it was all because of the site option ms_files_rewriting
The way I got around this was by putting the following in my wp-config.php file:
define( 'UPLOADBLOGSDIR', 'wp-content/uploads/sites' );
And then I ran this in SQL to turn off ms_files_rewriting. Or so I thought. More in a second.
INSERT INTO `my_database`.`wp_sitemeta` (`meta_id`, `site_id`, `meta_key`, `meta_value`) VALUES (NULL, '1', 'ms_files_rewriting', '0');
I came up with that after reading through /wp-includes/functions.php line 1515.
For most sites, this worked, but in my later work, I determined that it actually wasn’t working. It was ignoring this. I don’t know why, but every test I did merrily ignored this setting, so I finally growled and wrote this mu-plugin function:
function stupid_ms_files_rewriting() {
$url = '/wp-content/uploads/sites/' . get_current_blog_id();
define( 'BLOGUPLOADDIR', $url );
}
add_action('init','stupid_ms_files_rewriting');
It’s stupid simple, it’s probably not a good idea, but it works for the three sites that have the stupids.
Finish up .htaccess.
.htaccess, remove the ms-files.php line for ms-files, or comment it out. This is pretty simple.
Why not move the main site to /uploads/?
Because of the way I fixed the uploadblogsdir. It defaulted everyone to /sites/ and after an hour I said fuck it.
Any weird problems?
Yeah, two sites (this one and my grandmothers) decided that they wanted to be repetitious and spat out URLs like this: wp-content/uploads/sites/8/sites/8/
Since that wasn’t right at all, and I was a little too eggnoggy to parse why, I did this:
RewriteCond %{HTTP_HOST} ^taffys\.org
RewriteRule ^wp-content/uploads/sites/8/sites/8/(.*) /wp-content/uploads/sites/8/$1 [L,R=301]
I swear I have no idea why three sites got stuck with /files/ and two more decided to double down, Vegas style, but frankly I’m pleased I got through this far on my own.
I can’t stress enough that you do not have to do this!
Comments
8 responses to “Dumping ms-files”
FYI, if you follow Nacin and me on Twitter, we’ve hashed out things I did wrong and could do better, and wondering why the rewrite call isn’t disabling.
Tweets from Nacin for posterity:
Of note, BLOGUPLOADDIR is needed because ms_files_rewriting isn’t working, upload_path and upload_url_path are all blank, this is all one network, and if I ever did this again, I’d use symlinks and get to the drinking part of the night faster.
Now that I’ve got more coffee in me, why didn’t I just symlink goes back to ms_files_rewriting not working. In fact… all of this is because of that. I tried symlinks and redirects, but they didn’t work.
I might try this with my multisite too. Maybe make a plugin to do it for me automatically. Need to setup a fresh MS instance though, to examine the differences.
I can envision a plugin now that I’ve see it in action (Jaquith mentioned wanting to make one). My concern is if ms_files_rewriting fails for everyone or just me. Looking at both of my upgraded sites, neither have that site option set. That’s normally okay, the absence is what keeps old sites working.
Hello There
I have to ask one thing how did you get pass the linux file limit file limit of i think 32K files.
I hit this a long time ago and my fix was to ask “buddy” to tweak the code in:
ms-default-constants.php
http://pastiebin.com/?page=p&id=50e1db9f4587b
So all that was done is to move the new files in batches.
This is working well on bloglines.co.za as were at about 100K in users and site i just wish the WordPress Core will add this some how to there next version or even one of your clever lads here can maby make a plugin to do this 🙂
Ps Mika i must say well done on all the flack you take over WordPress.org some gays can be asses http://wordpress.org/support/topic/wordpress-35-upgrade-multisite-issues?replies=31
Any way just my thoughts
Thank you Kindly
Mark de Scande
32k limit… Do you mean the file size limit, or the how many files in a folder limit? Neither have ever been a problem for me (PHP filesize limits, yes, annoying, but easy enough to fix) but I only have 10 blogs on this network. I know there’s a limit of 32k directories and I imagine at that point you’d be getting pretty creative with distributed file-systems like NFS, but there you’re into massive amounts of scaling configurations. And that’s a little out of my league right now. Large systems vs huge blog-base. I know the former, but not as much the latter 🙂
There’s an interesting post on the scaling shenanigans here: http://wpmu.org/scaling-wordpress-wpmu-buddypress-like-edublogs/
I don’t mind being called names. I actually appealed to Twitter for help at that point and stepped away until two other people pointed out the same things I had. Magic.
Hey Mika
Thx for getting back to me 🙂 i did not get a email from your website 🙁
The 32k Limits Cent OS / Cpanel / WHM / Linux
SO what i was saying is that if you have a folder with 32000 folders in it then you can not add any more in there.
So if you have multi site site running with 32000 blogs your stuffed there is no way to “cheat” the system .
The only way to do it is to have some custom code added to ms-default-constants.php so now your dir will look like follow.
wp-content/blogs.dir/ First 31K Blogs
The all new blogs will go in here:
wp-content/blogs.dir/assets/00/
wp-content/blogs.dir/assets/01/
I just wish this was added to the core WordPress so that any one out there can have as big as they want networks with out the need of modding stuff to fit in to os and so on.
Thank you Kindly Good Friend
Mark de Scande
The email is sent by jetpack so … y’know look for wordpress.org in there 🙂
You’ve got a pretty extreme and one-off use case. In another post I talked about how multisite is still the new kid? Well, because of that we haven’t caught up with everything everyone wants. Yet. You need to grab someone to write that plugin and share 🙂 Then it may slip into core if a lot of people use it. I think many sites would.