How To Duplicate Content

I’ve talked about this before. 100% duplication of content on multiple sites is bad. So why am I going to tell you how to do it? And better, why am I going to tell you how to do it without Multisite? Because as a proof-of-concept, it was interesting.

The rest of this post tells you how to do something I don’t advocate, nor will I support. If you have a better way, or improvements, please leave comments. Otherwise, you’re on your own when you do this. I will not help you do it in any way, shape, or form.

3402955869_5c79c8c7efHonestly, I still think this is a pretty silly idea. Duplicating content is a terrible user experience, and I still flat-out decline to accept any work for doing this. Sharing content is one thing, but 100% duplication of sites makes no sense at all to me. Yoast also says it’s a bad idea. But, if you really are totally 100% dedicated to do this, and you absolutely are going to, damn the torpedos, then you should do this in the least computationally expensive way possible. And that would be a single install.

Now all that said, this means you’ll need to do a lot of monkey-work, so why do I call this ‘easier’? In many ways, easy is relative and this will be hard, complicated, and may I stress, entirely unnecessary. You’re going the hard way around for something that good planning and a solid understanding of the Internet totally negates. Remember the absolute rule of the Internet: Use one URL per page and never change that URL. 1

The way to make all this work, without Multisite, is by tricking your domain a little. There’s a neat trick with parking (or mirroring, depends on your host) domains, that lets you keep the other domain URL in your browser’s address bar. That’s what I do with this site, actually, halfelf.org is parked on top ipstenu.org. And with a park, the URL always stays as halfelf.org. Hey look! Two URLs, one site! Multisite has secret sauce to know “Someone’s coming to HalfElf, send ‘em to site #2.” But on a single site, all my links would still be ipstenu.org and not halfelf.org.

Now how do you use this to duplicate content? You use relative URLs.

So here’s a real example. I have twofer.elftest.net set up to mirror plugins.elftest.net (which will give you a coming soon page, it’s just where I like to blow things up for tests).

In the beginning of this post, when I linked to my old post about Duplication Dilution, the URL was http://halfelf.org/2012/duplication-dillution/ and that is what we call an absolute URL. Because I’m mapping domains, I can leave those in without worry, but if I wasn’t, I’d change that URL to /2012/duplication-dillution/ instead. Right away this makes my URLs entirely relative, no domain name included, and I’m off to the races.

This doesn’t solve everything, though. See, WordPress really wants to use absolute URLs. There are plugins like root relative URLs, and those will help a lot. None of them back-ports your existing posts, though, so for that it’s nothing for it but to search/replace the DB and change your post content. 2 I really like those plugins because now for a new post, when I add a link and chose to link to existing content, it happily works:

addinglink

And when I add an image, it too smartly handles as I want it to:

addingimages

That’s the easy part of all this, though. Now you have to disable canonical URLs, so that you don’t end up with even more of the dread duplicate content penalty in WordPress.

remove_action('wp_head', 'rel_canonical');

This also stops WP from redirecting things like http://plugins.elftest.net/?p=1 as well, however, so keep that in mind. Of course, that’s what you wanted. But they don’t address the problem of your source code. If I view source on twofer.elftest.net, it still showed plugins.elftest.net, and that would be a problem for images and themes. You’ll need to toss in this to your wp-config.php, which will dynamically change your URL to be whatever URL I’m visiting from, so that changes automatically. Awesome.

define('WP_HOME', 'http://' . $_SERVER['HTTP_HOST']);
define('WP_SITEURL', 'http://' . $_SERVER['HTTP_HOST']);
define('DOMAIN_CURRENT_SITE', $_SERVER['HTTP_HOST']);

Now I want to tell WordPress that wp-content is not in URL/wp-content/ so let’s just put this in and make it relative too!

define('WP_CONTENT_URL', '/wp-content');

I’m still going to have to search and replace my old post content (I used Velvet Blues for this):

Velvet Blue Update

But that didn’t address the problem of the source code. 90% of WP now thinks it’s all on twofer, which is what I wanted, but look at XMLRPC:

sourcecode

the_more_you_know2And even better, when I try to log in via twofer, it still says I’m going to plugins. Oh and it doesn’t pass through cookies, so really, I never log in to Twofer. Realistically? This isn’t a problem. I’m always going to use plugins.elftest for all this when I log in on the backend, and since the convenience of all this was meant for the front end, and it’s just pingbacks. And why is that? Honestly, I don’t know. I have a guess that since, at WordPress’ heart, the site is always plugins, the absolute URL there has to be what it is, but in so far as all that goes, I think it meets the needs of why most people want to do this.

Conclusions? You can do this. If you wanted to, you can hardcode the theme so the domain you visit the site with will dynamically change the header image, or widgets, or anything else you want. PHP is pretty cool that way and WordPress is too. But I would never do it, except as an experiment to see what I could do it at all.

About these ads

Notes:

  1. With all rules, there are exceptions, of course.
  2. ONLY change your post content. Do not change GUIDs!
StudioPress Theme of the Month

Comments

  1. Hi Mika
    I just finished reading your post, and I don’t know if its a coincidence or my luck, that I needed something similar to read currently, as I have two new websites both of them gonna have almost same content (both of them are off course different client sites but purpose is same, build regional sites (city based)), but as I have experienced and know the client will not gonna be providing new content for new sites, they always insist on using same content (no matter how much I convince them :().

    Anyway, my point is, how I can achieve this, I mean how and what I should do to not get penalized as duplicate content, and the purpose of having these new sites is
    1) Having regional sites (both sites)
    2) Boost SEO which is not present in main site (1 site)
    3) Get regional market (1 site)

    I have read your: Duplication Dillution

    and both of them (including the current article) are useful, and I know I should not use duplicate content (for the sake of SEO, using same content on different domains) but my need is different, as I get sometimes client wanting their region based sites means they will have few pages different (their locations page, their staff (if any), may be their services, hmmm may be testimonials / reviews etc) but company related content will be same (about us, services, products, privacy policy, and so on).

    So what are the necessary steps I can do to make content unique (I know little about article spinning :)).

    It will be a v. helpful if you can point me to right direction (I personally prefer asking questions on stackexchange but usually in my experience I get late responses and specially rarely I get expert opinion or guidelines :()

    Hope you don’t mind answering my query.

    Thanks,
    Cheers :D

    • You already know this answer, right? Because you read where I say I will not help you with this?

      Don’t duplicate content. Make it unique by writing unique content.

      It really is that simple. And yes, it’s work. If it wasn’t work, no one would pay copywriters. The ‘right’ way is to have a corporate site with all the me information, and then a page or smaller section for each local place. Look at how every single successful hotel and resturant chain in the known world handles it. Do that.

      If the client won’t, walk away. There’s no shortage on work out there for websites right now, and people won’t stop doing it wrong if you keep letting them.

    • @Mika
      You already know this answer, right? Because you read where I say I will not help you with this?

      Don’t duplicate content. Make it unique by writing unique content.

      Yes I read it, but still thought should get expert opinion, as I am not doing this for the sake of duplicate content, instead its a necessity that needs to be done :(.

      If you are asking my opinion I never do it when I have opportunity.

      @Mika
      It really is that simple. And yes, it’s work. If it wasn’t work, no one would pay copywriters. The ‘right’ way is to have a corporate site with all the me information, and then a page or smaller section for each local place. Look at how every single successful hotel and resturant chain in the known world handles it. Do that.

      Totally agree regarding copywriters, but sadly I don’t get enough budget to even build websites (cheap clients :(), and I still provide extra services within same budget (wish I could find client who understand this).

      Thanks for the suggestion, I will look into this. May be find some hints.

      @Mika
      If the client won’t, walk away. There’s no shortage on work out there for websites right now, and people won’t stop doing it wrong if you keep letting them.

      Well its not that simple, when I deal with direct clients I don’t accept these kind of projects, but the thing is I work with somebody who lives in USA and he handles all project dealings etc because he have direct access to clients :(.

      Yes I agree. We need to educate clients but that’s a long debate, its not a simple matter to deal with :(.

      I will look around and try to find a inspiration for my problem.

      Note: After posting comment here, I search on the web for tools help me avoid duplicate contents and I found YOAST duplicate content post, and glad I did and found couple of plugins to help me avoid duplicate content.

    • You claim this is all a ‘necessity.’ I disagree. It’s a choice you made to follow this road by the following:

      (a) not educating your partner who acquired the client
      (b) not having the technical expert, you, be involved in the decisions as to what clients and contracts you agree to
      (c) choosing money and time over doing things correctly and sustainably for the long term
      (d) perpetuating something you know isn’t the best practice

      Look, I really am not kidding. I will not help you do this. Not even for Matt or my CEO, Simon. I will hold firm that this is a bad idea, it’s going to hurt you and your brand and your SEO and your standing in the web. I won’t put my name on that.

      My advice is simple: Teach your partner why this is bad. Educate him. Then work along side him when reviewing clients, so that you two can provide fantastic work. You are the expert, not the client, and not your partner. They hired you because of that and they respect you because of your skills. Now you need to prove that you deserve it by being the expert.

    • Well I don’t have much of a choice when picking clients, I found v. few clients who not only understand what they need (in my experience), but also try to listen to me, and respect my suggestions and allow me to work the way I feel best.

      But sadly apart from few clients, none of the other clients understands what exactly they are asking for, or what I am trying to explain to them.

      (a) First thing I do is educate my clients (but its another story whether they listen to me or not)
      (b) We try to communicate before taking over new project, and at first all of the clients seems understanding but after a little they show their true self :D.
      (c) That’s not true, so far how much we have worked together we don’t remember doing this, we preferred clients needs over own, we made sure we give our best to client (no matter how much he / she paid, as we have agreed to provide them good service, and we knew (kind of) what we are getting into)
      (d) Agree.

      Heard your message loud and clear.

      Yes recently we are working on this, try to make decisions mutually, but it still need to go a long way.

      One thing I will do for sure (wish me luck), is ask client to let us hire a professional copywriter so he gets not only a quality content but also a relief of not getting bannned ;) (by google).

      Thanks for your time and suggestions,

    • Point A was educate your partner – The guy you said you used to contract out. HE needs to understand this. I spend time here at DreamHost educating sales and marketing so they understand what they’re selling. It’s a huge part of any company or collaboration. If he knows what he’s talking about, then before the customer becomes a client, they’re educated as to the expectations. Basically, if you’re teaching them after you’ve agreed to the project, it’s too late. Hook in sooner ;)

      As for point C – Understanding and meeting the client’s needs does not mean allowing them to do what they want. Learn to tell the difference between wants and needs. It’s not easy, but it’s worth it. (Remember: A person wants the candy bar, but rarely needs it) But clearly Wants vs Needs needs it’s own post….

  2. After reading this post and discussion above i do hope to get an answer to my question. As I think I have a situation where duplicating content does make sense.
    Say I have a great site in production and still developing new functionality. How could I now duplicate live data to my development site to support testing? Or, maybe more frequent, copy the new plugin data and settings without compromising the post and client data?
    Currently this is a hell of a manual job…

  3. I think that looking at Moz, Koozai and Search Engine Land to name but a few resources to get an understanding of why duplicating content is a massive ‘no’ along with all of the other many factors that impact SEO on site and off site.

  4. Well never say never lol..

    I just finished up a project where duplicate content on thousands of sites was an absolute must and was actually the proper way of doing it. For the job I needed to be able to provide thousands of affiliates their own shopping carts where all the content was directly derived from our primary store. Think Amazon and affiliates. Our affiliates all have their own basic websites, but very few of them have any idea how to build a store. We also had to maintain 100% control over inventory levels as we didn’t want the affiliates selling products that we had run out of.

    So the scope was simple, deliver a fully functional WordPress website to our affiliates complete with a pre-populated, dynamic auto-product-generated WooCommerce storefront where the affiliate had to do ZERO work to maintain it. Pretty much the only thing that changed on the affiliate sites was that they got to choose their theme for the site. The products were the same on all of the thousands of storefronts with the exception of the product urls containing their affiliate id# which was both related, and not related to WordPress.

    To date, we have seen ZERO penalty for the duplicate content, and if anything our sales have skyrocketed beyond anything we could have imagined. Because we give the affiliates a free website where they get the commissions on the products and they have to do zero work, they love it. In fact, we find that they write much more unique content than we could have ever hoped for, thus increasing the the google rankings of the sites, even though they have hundreds of ‘duplicate’ content pages.

    I won’t get into the details on how we pulled it off as that would take to long and isn’t relevant to the comment here, other than to say, there are instances where duplicate content is a HUGE BONUS, so NEVER SAY NEVER :)

    • To every rule there is an exception.

      However …. Store merchandise is not ‘content.’ Content is this post. Content is your comment. Content is not “This shirt.” You, and everyone else, expects the description of a shirt to be the same on 100 sites. Content is your ‘We have some cool new products!’ articles to draw in people. So I get what you’re saying, but you’re not doing what those other short-sighted people are doing, with pushing the same articles on fifty sites. You’re fine.

    • Ehhh…
      Even in this case, I would opt for another solution.

      How about having the duplicate ‘store’ data in a separate database apart from the thousands of WP databases? Each dedicated WP shop site should then attach to the external database to get the data to populate the item pages.

      In this solution you would have an additional database to connect to, but I think this would weigh up to the risk involved trying to keep the duplicate data synchronised.

    • Oh true, I would lean towards a central ‘merch’ DB myself, but in so far as my massive ‘Thou shalt not duplicate content’ argument does, merch ain’t content ;)

    • I do agree that in an optimal setup it would have been more effective to simply use a single database for all the storefronts. In fact, should I do it all over again today, I probably would do it that way. The issue was in that we were using a number of WooCommerce plugins including the ability for users to leave comments on the products. Due to specific laws in different countries, we didn’t want to fight a battle of ‘who owns the comments’, and the user registrations for each site. In this instance we decided it would be much better to have each site independent as the ‘customer would expect it to be’, instead of porting the customer registration and comments to a central location. Just a can of worms not worth opening.

  5. I am doing it and here is why. I want several website for the company with their own TLD and with local pricing in their currency.

    • I mentioned in another comment that stores are not the same thing. Stores are not content :)

      If these are pricing tables only, however, it’s a little messy about how to BEST handle this for SEO. I would probably have a page for pricing in general, explaining our strategy “We aim to keep your prices low blah blah marketing”, and then links below a header for ‘Pricing in your locale…’

    • Peter Vandever says:

      I am planning to have a completely localized website for each market with their local pricing. Also talk about things that matter to that market. What matters to a US market is not what matters to the Hong Kong market.

    • Yes, but then your content isn’t 100% the same, now, is it? :) Which is my point.

Trackbacks

Half-Elf? Try Half OFF WordPress ebooks!