Tag: open source

Dependency Disaster

Over the last few weeks and months, the nightmare that is WordPress plugin dependency hell has waxed and waned with the ire of a thousand burning suns. It flared up when I, perhaps naively, posted a reminder of a rarely called upon guideline to not submit Frameworks for hosting on WordPress’s official plugin repository.

This brought up the perfectly valid arguments that these repositories are useful as they allow for deployment and updates that were hitherto unavailable when rolled up into one package. And at its heart, I understand their point and agree with them as to the benefits. But since WordPress core doesn’t have true plugin dependencies at this time, it’s exceptionally complicated and a hassle to include them.

Personally, I feel that the NPM or Composer style development of plugin is the way to go. Development. Not deployment. With few exceptions, I feel the best way to release a plugin is all in one, rolled up together. The exceptions are, of course, plugins that are add-ons to others, like a plugin that adds a new email integration hook to MailChimp, or a new payment gateway for WooCommerce.

The rest of our plugins should be self contained, reliant on naught but ourselves. I have two plugins which contain the same AWS SDK libraries, written in a way to check if the library is already loaded and to make sure it only loads what’s needed. Why did I do that? Because then someone can have one or the other or both without conflicts.

The user doesn’t have to care about dependancies. They are invisible. That is as it should be. Users don’t have to care.

But there’s also a danger with dependancies, as recently came to light in the JS world. Azer Koçulu liberated his modules from NPM after the threat of a trademark based lawsuit had NPM remove one of his projects.

Sidebar: Open Source has no right to impinge on trademark law. Period. However some lawsuits are frivolous and daft and should be contested. Sadly most communities (NPM, WordPress.org, etc) do not have the money or resources to fight that battle for a developer. If you chose to fight, please contact the EFF.

As is his right, Azer pulled all his packages from NPM. The fall out from this package removal is that a lot of automated builds failed. This has to do with the way Composer is often bundled. Instead of a wrapped up package like an exe or a dmg, it’s a stub that reaches out and downloads all its requirements on the fly. Just like the TGM Plugin Installer. Right? And what happens when those requirements are missing? The build fails.

Perhaps worse, by unpublishing the name slugs used can be taken over by anyone else, and used to push more nefarious code. This did not happen. NPM checked everyone and verified they were as decent as one can before handing over the old names to new owners.

My first thought was “Could this happen to WordPress?”

Yes and no. First up, we don’t reuse names. If you have the plugin ‘foobar’ and ask to have it closed, the name is still reserved. In extremely rare cases we’ve turned over names to trademarked owners (like how Facebook owns the facebook slug) but always with communication of both parties and always with awareness to how many users might be impacted.

But could we pull a plugin for trademark infringement and screw up package dependancies? You bet your ass.

We’ve been lucky that all legal parties involved in similar arguments have been accepting of the ‘grandfathered’ ruling. That’s why we still have plugins with slugs like ‘google-analytics-by-joe’ out there. But it’s also why we don’t allow any more. And yes, when a plugin with a unique name is submitted, we do take the time to check if it’s already trademarked. To spare people this drama.

But yes. It could totally happen. And since we have to name our dependancies and rely on those slugs, there’s no easy way out.

I suggest reading the Hacker News thread on the matter. They weigh both sides of the issue, and show you how the pros and cons make this even more complex.

28 March, 2016
Trust the Changelog
Recently there were a couple WordPress plugins with fairly major security fixes. But you wouldn’t know it by looking at their changelogs.

The changelog is a section of a product’s readme that describes what changed. For most people, it’s a list of items like this:
- Added feature X
- Corrected typo
- Security fix
The problem many people have is that last one is often left rather vague. I’m guilty of this myself. In a recent fix, I simply said “Security fix: Sanitizing _POST calls to prevent evil.” and “Security Fix: Implementing nonces.”

The primary reason we keep change logs a bit vague is because we don’t want to open the door to alert hackers as to vulnerabilities. People don’t update their code right away, so every time we publicize a security issue, the people who haven’t updated immediately are at greater risk of being hurt.

But if we don’t tell people how important it is to update, how do they know how important it is to update?

There’s the real issue. There’s not yet a proper balance between “You should upgrade as soon as possible” and “You need to upgrade now, or you’re doomed.” My security issue was only accessible by people with admin access. It would be possible to trick an admin, with a cleverly crafted page, but … The effort it took me to apply a nonce check and sanitize things is minimal. From my end, it’s very minor of a fix. From a user’s end, it’s an exceptionally rare hack and unlikely to occur.

The right answer here is “Always upgrade to the latest version of code as soon as possible.” The problem is “as soon as you can” gets bumped out if it’s not mission critical. A patch that adds in a filter? Not a big deal. A patch that secures my site? Should be a big deal. I would argue that any time anyone says “This is a security fix” then you shouldn’t have to concern yourself about how likely the hack is to impact. Instead, security is a watchword to tell you to update the software “immediately.”

Which brings us to two agreements we need to start making with people. The agreement of developers to do things ‘easier’ for users and the agreement of the users to trust developers. If we want people to upgrade, they have to trust us. And if they’re going to trust us, we have to be reliable and consistent.

As developers, we promise not to flag something as a critical security fix that isn’t just security fix. If there’s a major issue with our code, we will push a patch as soon as possible that only deals with that issue. There will be no feature changes, no little fixes, no minor tweaks. A security release will only be a security release.

Furthermore, to enable people to update properly, we will properly use semantic versioning. This will allow us to update minor releases as far back as logical, because you can know that version 1.2.8 is the latest version of the (old) 1.2 branch, and 1.5.3 is the latest of the (current) 1.5 branch. The next time we add in new features, we will properly version our code as 1.6 so that you know what branch is current.

As users, we promise to trust your security-only releases and upgrade our copies of your code when a minor release that is a security issue is released. If you release a version 1.5.4 and not a version 1.4.4, we will trust that either the 1.4 branch is not subject to this security issue, or the fix could not be back-ported. If you inform us that we must upgrade to the 1.5 branch because there’s no way to secure 1.4, we will expedite our upgrade.

In order to enable ease of upgrade, we will not edit our code to make it impossible to change. We will properly use functions and actions and filters and hooks. We will make regular backups as well as immediate ones before upgrading.

Of course… That’s a perfect world. But I’m going to do my part as a developer and start versioning better. If I do that, and if I as a user hold up my other end, then we can get to a place where all disclosures of security issues happen in tandem with a release, as we know that everyone will upgrade immediately.

A place of trust.
24 June, 2015
URL Validation
I was writing a script for a rather complex series of checks on a website. I wanted to do what I thought would be simple. I wanted to grab the website headers and parse them to check if a site was WordPress or not.

That was a weird and wild trip.

In theory, this is all the code you need:

filter_var($url, FILTER_VALIDATE_URL)

But as it turns out, that’s actually not the best thing! I started using it but found that I could break it pretty easily and, since I was writing a tool I knew would be used by end-users (who are exceptionally creative when it comes to breaking things), I googled around and found this blog post on Why URL validation with filter_var might not be a good idea.

Yikes! All I was mad about was that FILTER_VALIDATE_URL thinks http://foo is okay, even when you tell it you want the damn host.

In the end I used this Strict URL Validator code but even then I had to wrap it around this:
```
	// Sanitize the URL
	$gettheurl  = (string) rtrim( filter_var($_POST['url'], FILTER_SANITIZE_URL), '/' );
	$gettheurl  = (string) $_POST['url'];

	if (preg_match("~^https://~i", $gettheurl)) {
		$gettheurl = "https://" . $gettheurl;
	} elseif (!preg_match("~^(?:f|ht)tp?://~i", $gettheurl)) {
	    $gettheurl = "http://" . $gettheurl;
	}

	// Is it a real URL? Call StrictURLValidator
	require_once 'StrictUrlValidator.php';

	if ( StrictUrlValidator::validate( $gettheurl, true, true ) === false ) { 

		// Do the needful
	}
```
In many ways, this makes some sense. What is and isn’t a URL can be tetchy to check. http://foo is real. I can use it locally. That’s why http://localhost can exist. And we can’t just say “if not .com/org/net” anymore (if we ever really could). But boy is my code clunky.
10 June, 2015
Self Ghosted
I had to restart everything.

You see, getting Node.js and apache to work together wasn’t going to fly, so I went and built a new VPS with Ubuntu over on DreamHost. Obviously not everyone has this luxury, but since I do, and since DreamHost made this as easy as clicking on a box, I thought it would be perfect.

Requirements

You have to be shell savvy for this. Sorry. There’s no other way. Just accept that and move on.

You need a VPS. The new Ubuntu SSD VPSes all have a one-click ability for Node.js which makes life easier.

You should use nginx and not apache.

Ghost requires Node.js 0.10.x (not 0.12.x). They recommend Node.js 0.10.36 & npm 2.5.0. Of course, DreamHost installs node 0.12.x and npm 2.5.x. You can check by typing node -v and npm -v to see what versions you have. Does this mean you can’t run Ghost? No, it’ll work, but it’s not what they like.

You’ll also need an ‘admin’ account on your server. From panel, click on “VPS” and then “Manage Admin Users” to add a new user to that box.

Get Node.js on your VPS

This is quite simple. First, get an Ubuntu VPS. All the new boxes on DreamHost are Ubuntu, so this is the easy part. Once you have the VPS on DreamHost, make a new domain. I made ghost.elftest.net and in the settings, I checked a box to use node.js:

That will force my file path to be ~/ghost.elftest.net/public (normally on DreamHost it’s ~/domain.com you see) but that’s not an issue since this is a new box. That path will be my ‘home’ folder for the domain and where I install everything (if someone asks you ‘where is X installed on your server?’ you tell them that path). That path also means that had I already been using that domain, it would mess up my folders and paths. If you’re doing this on an existing domain, keep that in mind.

Install Ghost

Like I said before, you have to install on your server, not your desktop. Every single doc I read told me I HAD to do this as the server admin, someone with sudo access. I really, really, really did not like that. I mean, epic levels of hate for the idea that I couldn’t have my user account own things, and have the code in my darn web folder.

Me being me, I hatched a plan. I was just going to do it. I was just going to install it in my webfolder and see how it goes. I came up with this theory because when I read How To Install Ghost on DreamHost I noticed it wanted me to have a specific user chown the folders. Not the admin user, but a special ghost user. So what if I just made ‘elftestghost’ the ghost user, and what if I just used ~/ghost.elftest.net/public/ghost/ as my install directory instead of /var/www/ghost/? And this way, I’d be able to pop in and edit things as my normal user.

So here’s what I did.
```
$ curl  -L https://ghost.org/zip/ghost-latest.zip -o ghost.zip
$ unzip -uo ghost.zip -d ghost
$ cd ghost/
$ npm install --production
```
That’s it. Ghost is installed. I expected that last step to fail and I’m still a little surprised it didn’t. If it does fail for you, run it as your admin account with sudo, and then chown it over to your ‘owner.’

Configure Ghost

Now that Ghost is installed, you may wonder where it is. We haven’t finished the install yet as it happens. We need to configure it and to do that, we copy the file config.example.js to config.js and open it up.

Once you look at it you see that there are multiple ‘config’ options. Since we called --production in our start command, logically we’re going to edit that section. And lo, there’s the bad URL:
```
config = {
    // ### Production
    // When running Ghost in the wild, use the production environment
    // Configure your URL and mail settings here
    production: {
        url: 'http://my-ghost-blog.com',
        mail: {},
```
Change that to the right URL. Mine is http://ghost.elftest.net

Also look for this:
```
        server: {
            // Host to be passed to node's `net.Server#listen()`
            host: '127.0.0.1',
            // Port to be passed to node's `net.Server#listen()`, for iisnode set this to `process.env.PORT`
            port: '2368'
        }
```
Change the host to 0.0.0.0 and save the file. You can also set it to the IP of your server, but as that might change, it’s not required.

Finally you’ll want to set up mail which can be complex. I set up a special account on my own server.

Setup Ghost

Seriously this is not a five minute install. Your site starts with a Hello World type post which tells you all about the glory of Markdown. If you go to your admin page (http://ghost.elftest.net/ghost/) it sends you to http://ghost.elftest.net/ghost/setup/ where you get this:

This actually makes more sense than it did on the Ghost Pro site, where I was wondering why I had to make a user and then a user. Here I’m clearly making a user for the site. There I was making a user for their network (Ghost Pro) and then one for my site.

Setup Nginx Proxy

Will it never end? The reason we want to do this, and this is why we’re on nginx and not apache, is in order to have a pretty URL without the port number.

Log in to your server as your ‘normal’ account (the one who owns the domain, not the admin) and make an nginx folder for config:
```
cd ~
mkdir -p nginx/ghost.elftest.net
```
Then make a file called ghost.conf with the following:
```
location / {
    proxy_set_header   X-Real-IP $remote_addr;
    proxy_set_header   Host      $http_host;
    proxy_pass         http://localhost:2368;
}
```
Restart nginx (which yes, you need the sudo power for so yes, you need an admin account) and it’ll magically work. If you get a “502 Bad Gateway” error, it’s because Ghost isn’t running. Speaking of…

Make Ghost Always Start

Okay so I actually do need my admin account. I went with the init script approach to keeping the Ghost always on.
```
$ sudo curl https://raw.githubusercontent.com/TryGhost/Ghost-Config/master/init.d/ghost \
  -o /etc/init.d/ghost-elftest
```
Then I edited the ghost-elftest file:
```
GHOST_ROOT=/var/www/ghost
GHOST_GROUP=ghost
GHOST_USER=ghost
```
I changed that to where ghost was really installed, but it brings up an interesting thought. What if I wanted multiple ghost instances? Well that’s actually why I named the file ‘ghost-elftest’ instead of ‘ghost’ (which they recommend). With this setup, I can name an init file for each instance and run the commands as sudo ghost-elftest start and so on. Keep in mind, you also have to pick a custom port for each instance.

There are three final commands to run in order to force Ghost to start up on reboot:
```
$ sudo chmod 755 /etc/init.d/ghost-elftest
$ sudo update-rc.d ghost-elftest defaults
$ sudo update-rc.d ghost-elftest enable
```
Now I can use Ghost to my heart’s content.

Conclusion?

A WordPress killer this is not. It’s just not. If they can make it run without the need for admin/root/sudo access, it has a chance. Once it’s set up, it’s quite nice, but the 5 minute install this is not, and it’s going to need that to beat the beast.
11 May, 2015
NUX: Setting Up Ghost (Self Hosted)
Once I used Ghost Pro, I thought about self hosting. I have a WP site that’s basically wasted as a WP site. It’s small, it’s static, and it rarely changes. I thought it would be perfect for Node. There are also a couple of small, basically HTML, sites I run in the back of things. This would be fine to manage that.

But first I had to address a major misconception.

You Install Ghost on Your Server

For some reason in my head I had this working like Jekyll, which I would install on my computer and push up to my server. No, I’m not so much installing Ghost as Deploying Ghost.

But I Installed Ghost On Your Computer

I decided to do this anyway, just to see what I was getting into.

To install Ghost you must install Node.js first. Since I have Homebrew, this is two commands:
```
$ brew unlink node
$ brew install node
```
I had an older version of Node.js installed for whatever reason.

Sadly I can’t install Ghost this way.

Next you download Ghost (I was download 564,730) and at this point I hesitated. The directions don’t tell you where to put the files. It just says this:

Next, grab the newly extracted ‘ghost-#.#.#’ folder and drag it onto the tab bar of your open terminal window, this will make a new terminal tab which is open at the correct location.

Since I know that upgrading involves replacing the files, I’m no fool, and I made a new folder setup: ~/Sites/ghost/sitename.com/ That’s where I ran node commands:
```
$ npm install --production
$ npm start
```
Done. Now I have Ghost up and running locally.

Install Ghost on My Server

In a word? Ow.

The main issue is Node.js and Apache both want to use the same ports. That’s impossible. And I want to keep Apache running port 80 because this VPS runs… well… WordPress. This is where I stopped the first time I tried to do all this and tossed this post into a long draft.

There are directions on How to Host Ghost on an Apache Subdomain, which luckily is what I wanted to do. Except it was complicated and messy and required root.

So the NUX here? Absolute crap. It’s just not something a new user would want to do, be able to do, or be able to maintain.

And that sucks.

Ghost’s got a great interface, one that I like better than WordPress for blogs and simple sites. It’s nailed simple in a way we crave. But it came at a cost. WordPress’s simple to install is fraught by it’s IDIC complex. Infinite diversity in infinite combinations, with the themes and plugins, lends WordPress amazing abilities but a pretty insane learning curve. Ghost I could sort out in a couple hours but you really can’t do too much with. I wouldn’t use it for a store. I might use it for a blog if I had to start over.

Except I can’t (easily) self host it because of stupid Node.js.

If they can sort that out, make it so I can easily, without root, install and manage Ghost, I’ll be back.

Until then, Managed Ghost Hosting is the way to go. Or WordPress. Take your pick.
29 April, 2015
Mailbag: Homogenous websites?

I have a site for my business that has multiple physical locations that have online booking for each location. Right now we basically have separate websites that look very similar for each location (except for some content) with separate domains [URLs redacted]. Is this what you meant by homogenous websites? If WPMU ins’t a good option, can you steer me in the right direction for how you would design this type of site?

Yes. That’s pretty much exactly what I mean when I call a site “homogenous.”

You can’t see it, but each URL I removed had identical design. Same layout, pretty much the same splash page.

When I talk about ‘sameness’ with websites, I really do mean exactly this. Each site on the network has the same information, the same about page, and actually pretty much the same everything except for the booking page.

Right now, each site has a locationdomain.com/book-now URL. Without looking at the code, I’m going to guess that the book-now pages are inserts. Either templates or shortcodes, but something that doesn’t need to be on the domain URL to be unique. Just some content in the page.

And I would have maindomain.com/book-location Or better still maindomain.com/location/LOCATION/book

That second example looks weird, I know. You see, I’d do it with Custom Post Types for each location. My CPTs would be pages and their slug would be /location/. Then each page would be LOCATION, giving me URL formats like /location/lexington/ and so on.

My thought process is that if the majority of the content of each site is the same, and the design of all sites are the same, then I don’t need a multisite unless there’s a specific need to silo data.

There are very few cases, in a homogenous network, where you need to silo data. Exceptions are pretty much all based on legal requirements.

And if you have a song in your head, here it is:

24 April, 2015