Tag: security

HSTS and Chromium
I use Chromium, which is like the super most beta beta of Chrome, for one reason: It lets me run Chrome ‘twice’ in separate environments. I have a reason for it, but it doesn’t matter for the purpose of this post. Neither does the reason why I was editing a subdomain I rarely use. What matters is this.

I went to http://test.ipstenu.org and I got this:

My bp must have dropped because my first thought was “Oh shit, I’ve been hacked!” and I got the cold sweats. After a rant on Twitter and a plea for test help, I determined this was only happening on Chromium, which was interested, and my friend Benny pointed me to this Chromium Issue: Issue 350912: Chrome fetches https URL even when http URL explicitly specified which sure sounded right.

Now the rest of the error was telling me that test.ipstenu.org was asking I use HTTPS (which it was not) and that the SSL Cert didn’t match that domain (which was correct, but again, there is no SSL cert for that subdomain). Weird, right? Why would it force https when I didn’t ask it to! But what if I was? As I read through the ticket, I saw that you could see the headers better at chrome://net-internals/#hsts so I ran that and got this:
```
mode: STRICT sts_include_subdomains:true pkp_include_subdomains:false sts_observed:1402609861.497659 pkp_observed:1402608721.515596 domain:test.ipstenu.org pubkey_hashes: [...]
```
That was odd. I knew I’d set this in my .htaccess file:
```
Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains"
```
But from what I’d read that was getting set only if you accessed the site via HTTPS (or anything on the page was HTTPS). After all, no other browser was acting like this, not even Beta Chrome. Still I turned it off, flushed my server cache, flushed my browser cache, and deleted the TransportSecurity files from Chrome. Now it worked. Turned it back on? Error came back.

It was here I realized “Chromium has decided that if you set HSTS, you mean that everything should be HTTPS regardless of the domain…” Well that’s just peachy! I pulled the HSTS line from .htaccess and instead tossed it into a PHP file for the subdomain(s) where I needed it:
```
header("Strict-Transport-Security:max-age=15768000; includeSubDomains");
```
Now. There are two situations particular to me that make this an epic pain in the ass:

First, I’m using WordPress multisite and I have SSL turned on for one domain. Second, I’m using WMH, which means editing my vhosts file to make this work properly is more of a hassle than it’s worth.

Really wouldn’t it make more sense to check “Is this an HTTPS page? No? Okay, skip!” But this is a (to me) departure from the logic train. As my friend James (and I) understood, the logic of HSTS was “If any aspect of the page is HTTPS, force HTTPS” Instead of that happening, if you went to, say, ‘fake.ipstenu.org’ instead of going to my about page (yes, that’s on purpose), it gave me the error. If you go to ‘https://fake.ipstenu.org’ you should get a specific warning page and then a ‘shucky darn!’ page. And yet here I saw that no matter if I picked HTTPS or HTTP, I was forced to HTTPS and that’s just silly.

Does my fix work? Of course. Am I happy? Nope! Oh, and yes, I filed a bug report with Chromium. You can read it at Issue 384069: HSTS in .htaccess forces https even when not specified
13 June, 2014
Reset the Net Gotchas
All my domains will not be HTTPS by the end of 2014.

Sorry. It’s one of those things that just isn’t (at this time) something I can pull off. If I only had one domain and everything was subs, I could get one wildcard subdomain cert and be done with it. But with the number of domains I have it’s not feasible. Which brings me to what I think one of the major issues with our desire to protect the net is… But let’s step back!

Yesterday, as you may have noticed, was Reset The Net day. It was a call to action, much like we did when we went dark one day.

Now on this site, I’m using the Internet Cat Signal, which cleverly updates itself as I need to alert people to crap like this. The tldr is that the NSA is spying on us. I leave that plugin on all the time, it fires up when there’s something people need to know. It doesn’t slow down my site, and I hope it brings awareness to folks who otherwise have no idea about this stuff. About 75% of my traffic on this server can be described as people who don’t know about any of this.

What have I done for this? The recommendations are to use HTTPS, HSTS, and PFS. Since HeartBleed, I enabled PFS. This is a non-logical sort of thing to do, in that few people seem to explain how to do it. On my box, which uses WHM, it was pretty easy. In my WHM Panel, I went to Apache Configuration -> Global Configuration -> SSL Cipher Suite. Then I picked the PCI Recommended suite, not the default, and rebuilt the configuration. Then I went to Apache Configuration -> Include Editor -> Pre Main Include and, for all builds of Apache, added this:
```
# Enabling PFS
SSLHonorCipherOrder On
SSLProtocol All -SSLv2
# CVE-2011-3389
SetEnvIf User-Agent &quot;.*MSIE.*&quot; nokeepalive ssl-unclean-shutdown
```
The last bit lets me support any IE 6 users who visit my store. But as I said, I don’t have SSL on for all my domains. So what are my HTTPS issues?

The cost is insane. Let’s look at wildcard ssl, which is what you want for *.example.com situations. It’s pretty much $100 a year. That’s not too bad until you factor in how many domains I have on this server. Six family members, six of my own sites (including short domains like helf.us). So that’s either $1200 a year, which is obscene, or $145 a year at the cheapest I could find, and that’s for the simple green lock and no wildcards. For the big green bar, it’s back to around $1000 a year. Oh and I forgot one of my domains, so that’s $164 and $1047. Now I could totally afford the $164 a year, it’s doable with my ad revenue (which pretty much breaks me even at the end of a year) but….

It’s slower. Look, I get it how it’s important to be secure, but right now, the nginx proxy setup I’m using doesn’t work on HTTPS. That sets me back some since using it has sped up my site considerably. I know how to (and have) set Google Pagespeed to play nicely with HTTPS, so I’d be back to where I was before. This isn’t bad, it’s just not a great experience. Right now I have a secure login, secure email, a fully secure store, and ssh/sftp only, so the only place your data could get ‘sniped’ is when you’re leaving a public comment on my public site, which makes me less worried than I might be. Even my git repo is secured.

Also it’s hard. And no, that’s not an excuse. PFS (Perfect Forward Secrecy) isn’t easy to add to your servers, and it’s way outside the realm of what most people can do. Hell, it’s outside the realm of what I’m comfortable doing. It took until my server had the specs for OpenSSL that will support PFS for me to do it. The point is, this part has to be done by the webhost for most people, and that is a big issue. It’s not easy or fast to upgrade servers, and it’s far, far more persnickety than updating WordPress. It’s complex, and you have to think about everyone on the server. Again, not an excuse, just a caution that it takes a while to finish up.

Speaking of WordPress, multisite isn’t great at it. In fact, it’s less great than normal WP. I have two sites with SSL right now, ipstenu.org and store.halfelf.org. Ipstenu is only SSL on the back end, but even with that, there are inconsistencies. First, all the links are HTTPS, so when I click on “My Sites” the link to NON HTTPS sites are using HTTPS, which doesn’t work. Also if I made a new domain, it defaults to HTTP and not HTTPS. So I have to edit that manually. This is annoying, thought not insurmountable, and I know it’s something being worked on.

In the end, the absolute biggest reason I’m not switching to HTTPS is that the only person who needs secure communication are the people logging in or the people buying things, and I’ve taken care of that. For the rest of you, know that my store is secure, my logins are secure, and if you’re commenting on the site, for god’s sack, don’t post anything you don’t want people to know!

I’m sure in a few years if not months all this will change, but this is where I am today. The racket with SSL certs costing that much needs an easier solution, and then the rest will fall into place.
6 June, 2014
You Are The Weakest Link

I sit by people who are on the phone with customers about their accounts all day. Each and every day I hear them talk to customers who are past due or need to activate a new credit card or had an account closed by not paying. In general, I marvel at how consistently nice these coworkers of mine are, considering the number of times I hear people screaming at them. But I also get to hear some fairly impressive conversations.

Most of the time, the conversations are mild, a reminder that you actually have to pay, here’s how you pay, off you go. But once in a while you get to hear the tale of someone who wants to cancel an account. This is only interesting because we don’t cancel your account for you. You have to log in and cancel the charges and billing. About once a day, someone asks why we can’t just accept they are who they say they are and close the account, and I hear my coworker explain over and over that it’s not secure. We can’t verify you over the phone, we called you, and… well there’s a reason you have to call your bank and not the other way around.

Recently I had a credit card jacked, causing me to miss a payment I didn’t know I owed. I was called by a bill collector after 60 days and they asked for my account information. I balked and asked what card he was calling about, what was the amount, when was it charged, and so on. Then I hung up (I tried to tell him thank you, he kept talking) and called the card company directly. They confirmed the situation, I explained it wasn’t me, we got the bill reversed, everyone was happy… except me. I pointed out having some random number call me and claim to represent them was not safe or secure. They agreed, and also remarked that I shouldn’t have had a collection agency after me at only 60 days, so clearly they had a billing problem.

I’m a little thoughtful about my security, if you can’t tell. So when I read my friend Gary’s story about Paypal’s lack of security, it surprised me. There’s no true Two Factor authentication on something that has access to my bank account, just a dongle I’d have to pay for (and keep handy) or SMS. And when you read more into why Gary turned on SMS (see the story about how @N lost his user name and $50k), it gets more disturbing.

Social engineering to figure out passwords and pose as other people is easy. I do it all the time when I need to set up something for myself and my wife, and the company says she has to call too. I happen to have all of her information, down to access to her email (after all, I’m her sysadmin), and we actually made a specific joint email address for things like our bank account. If you have the information, it’s not hard to do.

We can blame GoDaddy and Paypal all we want for this. Should they accept the last four digits of my credit card as identification? Should they accept my social security number? What about my password (which means they can read it, by the way), or what about a special password used only for verification? Now I have to remember more, carry more, and know more all the time. It’s information overload. And because of that, because we’ve complained, they do less.

People see our credit cards all the time. You handed your debit card to someone to pay your eye-doctor’s co-pay, or for that latte. We tweet about our first pets and mother’s maiden names, and we Facebook everything. It’s pretty obvious that the weakest link in security, and the reason social engineering can easily exploit them with companies, is us.

There isn’t a perfect way to protect ourselves, though. Last year my phone got wiped and I lost my Google Authenicator settings. I had secondary login codes for Google itself, but not WordPress.com (I’d never set them up) so I was unable to log in. After skyping and emailing two of my good friends who work there, I was able to get back in, but had they not been able to know it was me, what could have happened?

Every time someone asks me what I do to be safer online, I tell them this “I am working to eradicate my ability to be stupid.”

30 April, 2014
Save Bacon With ModSecurity
Earlier this month, my company DreamHost had a small snafu with ModSecurity. The tl;dr is that we had a typo and it stopped some people from being able to access or properly use Jetpack. Thankfully, the WordPress community (including everyone at Jetpack, whom I owe drinks and/or dinner) is filled with amazeballs awesome people who helped us figure out everything and sort out customers who, upon getting what appeared to be a Jetpack error, went there when they needed to go to DreamHost.

These things happen. Code isn’t perfect, people aren’t perfect, and everyone makes mistakes. Of course, on the internet it’s unreasonable to assume a legit gaff, and I’ve seen people call out “Why was DreamHost pushing out these tweaks?” and “Didn’t they test?” so I thought perhaps it was time to explain why we use Mod Security and why, even though it’s my nemesis, I like it a lot.

What is ModSecurity anyway?

ModSecurity (aka modSec or mod_sec) is an open source web application firewall (WAF). That means it sits between your website and the world, blocking all the bad people. When we have those brute force attacks, ModSecurity is key in blocking them. It blocks people who attempt code injection attacks like this:
```
http://www.example.com/wp-login.php?username=admin'">DROP%20TABLE%20wp_users--
```
Now that would never work on WordPress core, but the Bobby Tables attacks have the potential to kill your site if you have a plugin or theme that’s insecure. Most hosts have customized their rules to check for things like hitting the wp-login page improperly or passing through credentials directly. That means if someone tries to log in to your site without clicking the submit button (yes, you can code that), it will block them.

One of my favorite things about ModSecurity is that you can hook it into another firewall like ConfigServer, Fail2ban, or even the built in linux feature of IPTables, and block any IP that routinely trips your security rules.

So why was DreamHost monkeying with it anyway?

Every host is constantly monkeying with ModSecurity. As attack patterns grow and change, your host has to adapt. There’s a team at pretty much every host on the planet who watches logs, studies them, and improves the ModSecurity rules. Heck, we even share our rules with other hosts when the situation calls for it, like that Brute Force attack back in 2012. It was brand new, we were all surprised at the aggressiveness, and we quickly shared information.

On any given week, your host is creating new rules and testing them in their dev environment, or on specific real servers that are designated “Go ahead, blow me up.” After all, we all know nothing beats real-world testing. And if push came to shove and one specific site was being hammered, we may push an experimental rule to them before we’re done testing everything, because it’s that or your site is down.

We’re always working and improving. Security is a moving target after all.

How come a typo slipped through?

If you can find me someone who makes a 100% perfect product every single time, I’ve got a bridge for you to buy. We tested everything we could think up, and interestingly enough, that Jetpack error didn’t impact all Jetpack users! We have a test box, with Jetpack, and it worked fine there. Go figure.

But I’ve often said your website is a pretty snowflake. It’s unique, and what you do with it is different from what everyone else does. Things I have and do on this server and this domain are wildly different from my other sites on this same server! The need for the site is different, and what it uses is different, so what it does when it communicates with the world? Different.

I’ve had days where one domain is acting like a prat, but the others are fine. And I’ve sat there thinking “But it’s the same on these domains! They’re on the same server for God’s Sack!” only to realize that the usage pattern of the sites were very much not the same. And that takes everything longer to fix because you have to narrow things down over and over until you actually find out what the heck you did wrong.

I can’t even tell you this will never happen again because I’m pretty sure someone will make a mistake again sometime in the future.

Conclusion?

I wouldn’t run a site without ModSecurity, but there are options.

In February 2013, Zero Science Lab released a study comparing it to Incapsula and Cloudflare. While ModSecurity came out on top (though it was noted to be more aggressive and caused more false positives), Incapsula has been working hard to fix it’s issues. There was actually a Round 2: Incapsula vs Cloudflare study in October 2013, and in this one, Incapsula is the clear winner. Of note, you won’t get the WAF protection on either for free.

The studies say, to me, that if you’re master of your own domain and want the firewall on your server to run yourself, use ModSecurity. If you’re going to farm that security out to the cloud, use Incapsula. There are, of course, benefits to putting the firewall on the cloud, and the major one is that you’ll be spared high CPU since the processing of the naughty people is done on their server, not yours. But of course, if they go down, you’re at risk, so you should probably have ModSecurity anyway.

After all, your website is important, right?
28 April, 2014
Is This Plugin Bad?

I get asked this a lot, in part because of my job (WordPress Support Guru and Manager) but also because I’m a know-it-all busy-body. The problem with the question is that it’s very subjective, and the answer highly depends on why someone’s asking the question.

I’m sure it annoys my co-workers when they ask “Is this plugin bad?” and I ask “What problem is the customer reporting?” If the answer is that the customer has a slow site, then MY reply will be different than if they were hacked. Making matters worse, sometimes the answer depends on what other plugins they’re using, or what their theme is, or how they use everything together! You see, the issue is rarely “This is a horrible, evil, terrible plugin and no one should ever use it!” It’s generally more “Well in this case, I would say this is the best plugin, but you have to take this into consideration…”

As a customer, it’s annoying. I just want a yes or no answer. But this is like that gas milage situation I talked about in my explanation of Shared Hosting. How many tanks of gas does it take to drive from Chicago to Cleveland? For me, it’s one. For my cousin, it was two and some change. Same distance, same day, same weather! What was different? The car and how we drove.

Your site and my site are different. This site and this other site on my network are different. They run different plugins, though the same theme, and sometimes one of those different plugins causes a problem. Like I found out the custom prices plugin caused my background image not to display. Oops. Does that make it bad?

There are a few types of ‘bad’ plugins to consider.

Evil Plugins

This is the easiest to explain. A plugin that is created to do evil things, like leave backdoors into your site, is bad, no matter what. Don’t use it.

Holey Plugins

This plugin has the best intentions in the world, but for whatever reason has a security hole. Maybe they forgot, maybe they missed it, but it happens to everyone. In general, this is not a bad plugin, unless the dev refuses to fix it. Or worse, can’t fix it! Now it’s a bad plugin.

Broken Plugins

Pretty common, this is a plugin that once worked but now, with the new upgrade of your theme/plugin/WordPress it stopped. This one sucks, and not much can be done except try and fix it, unless the developer comes around.

Works For Everyone But You Plugins

This is the brunt of what people mean when they ask me “Is this a bad plugin?” but they just don’t know it yet.

If you haven’t noticed, most of the ‘bad’ plugins are really just unfortunate plugins in bad situations. Determining if a specific plugin is bad for you isn’t as simple as going “Yes, I know that plugin is crap!”

What I do know, but I have to be circumspect in saying, is some plugins are better than others for specific server situations. You’re on shared? You probably don’t want W3 Total Cache right away because the best parts of it (that hooks into server side caching) aren’t available for you. On a VPS? You can probably use that YARPP (yet another related posts…) plugin just fine! Oh, but you’re using it with BuddyPress and bbPress and a whole mess of other plugins with a high degree of interactivity? You may need more memory.

And that’s the real answer. Is any individual plugin I named ‘bad’? No! In fact I’ve used them all and they’re wonderful in their use case. But they also require me to be aware of my whole situation. What kind of server am I using, what kind of environment am I in, what other plugins am I using?

It all comes back to being aware.

28 February, 2014
Your Username is Not A Secret

I keep seeing this pop up. “Your CMS is not secure because it makes your username/id public! Once a hacker has that, they can try to break in!” At one point I snapped “Sure, and your house isn’t secure because someone knows your address.”

It’s one of those logical fallacies that seems vaguely accurate on the surface, but really are just plain wrong. On some level, you’d think that if a hacker doesn’t know your ID, they can’t get in, but the reality is most hackers, the surface level idiots who are trying to break into any site available aren’t checking for your user ID/Name, they’re looking specifically for a vulnerability, like they did with the TimThumb accidental (D)DoS.

In addition, they’re not usually looking for your ID when trying that brute force login attack. The practical difference between someone trying to log in with “admin” and someone trying to log in with “ipstenu” is pretty negligible, since they’re killing my server before they get in anyway.

As I wrote this, I thought what it would be like if there was a mod_security rule that checks if you’re trying to log into a site with the username ‘admin’ and, if so, blocks you from being able to log in. Of course, there are millions of sites with millions of CMS tools, and for some you actually cannot change the admin account name away from admin.

WordPress is not alone in thinking your username isn’t a secret. Drupal also thinks disclosure of usernames/id is not a security risk. In fact, Google doesn’t think your ID is a secret. After all, you can log in to Google’s devices with your email, and everyone whom you’ve ever emailed kinda knows that. “Oh, you emailed me from ipstenu@gmail.com? I’ll attack that!”

Now of course, if you try to log in with that too many times, you lock your IP out. And similarly, if you try to log in to my server via SSH too many times, the same thing happens. Have I ever locked myself out? You bet. Less since I switched to 1Password and SSH keys, but it still is very effective.

Why isn’t this built into the core of most CMSs? Because a CMS like Drupal and WordPress is not as volatile as, say, the healthcare.gov site. The danger that comes from someone getting into my blog is minimal compared to someone getting into my email. But again, everyone knows my email account, so they’ve always got one half to the puzzle right then and there.

One of the other primary reasons this isn’t built in to WordPress is that it’s hard to do right, and in a way that will work on all servers, and in a way that will be easy for someone to undo. I said I locked myself out a couple times, right? I can unlock myself with a device on another IP, or I can call up my webhost and tell them my IP and can they please unlock me. Now flip that to your blog. How do you handle it? Who do you call? Do you make this a ‘solvable by the host only’ problem? Can you envision your host being happy about handling that?

Not that I’m passing the buck here. There are plugins and extensions that do this, but they’re still best used by people who already understand security than by the common man, because the people who know what to do when they have to edit a .htaccess are the ones who probably already know how to pick a secure password, or install two-factor authentication already.

All this comes back to something blindingly obvious though. Everyone is going to know part of your access. The reason we tell people not to use ‘admin’ as a login ID is not because it’s more or less secure, but because it makes it easy for script kiddies to target. Remember, most of the time when you’re being attacked it’s nothing you did personally, it’s just a script running. When it’s someone who has an absolute vendetta against you, your userID is the least of your concerns.

The crux of the matter here is that your username is not a point of authentication, it’s a point of identification. Giving you an identification (I am Ipstenu) is not the same as giving you data that can be used to authentication (my mother’s maiden name is Jones; I was born in Battlesboro, VT; My favorite superhero is the Flash). There’s a reason we call them ‘Secret Questions’ as they’re both identification and authentication. Only I would know these things. And no, that’s not true, which is why secret questions are pretty useless. The more obscure they are (my first maths teacher) the less likely I am to remember them correctly. “His name was Smith… Now did I put in Dr. Smith, Mr. Smith, or Smith? Oh wait, how did he spell Smith? Smythe? Smyth? I know people with all those spellings! Which was he?”

So no. Your user ID is not a secret, nor should it be. I spend no time hiding it.

6 January, 2014