Half-Elf on Tech

Thoughts From a Professional Lesbian

Author: Ipstenu (Mika Epstein)

  • When Sites Go Down

    When Sites Go Down

    My webhost had a bad day. They ran a regular, normal, upgrade to some switches and a switch failed. It decided it would be fun to reboot over and over. Since running on one switch is rather dangerous, they decided to roll back so we could have two and everything would be okay. It didn’t work. In fact, nothing worked. It was one of those days where you put out a fire only to have an earthquake, and as soon as you got the place cleaned up there was a flood. And then the sprinklers thought they should go off.

    We have, every last one of is, had a day like that. A day where absolutely everything went wrong when it was possible to do so. It was clearly one of those days.

    I can certainly complain that my host wasn’t really the best when it came to explaining things, too. They would send us emails, which was fine, except my emails were on my server which was inaccessible to me. So many people went to their live-chat that it was impossible to get a rep. Tweets? Unanswerable due to volume. They actually replied by (heh) email. And when I went to their management panel on their website, it just said that it would be updated in ‘an hour.’

    I resorted to opening a ticket. And I hated doing it, since I knew better than many exactly how shitty their say was. So I asked “Status Update?” and explained that I was unable to check my email but what was the status? I got the form replies, which I expected, and I pushed back for one detail. Just one:

    “It is our aim to have this completed within the next hour by proceeding with this fix immediately.”

    Hour from when?

    Suggestion for your website: Can you timestamp things so we know when to kind of expect things?

    By the end of that day, they had a static.html page with the information, and times(!), on it. They’re not ‘great’ at keeping it updated, but I know how annoying that aspect is, and I don’t fault them one bit. Once the work was done and everything was back to normal, I inquired as to the offered credit, which actually I’d forgotten about but they had mentioned to me in the support ticket! I think it works out to being about $4, clearly not very much, and honestly I don’t care about it very much.

    A thought that never crossed my mind? Leaving them.

    It’s not because my server’s been there for over a decade, and it’s not because I like the more and more ‘grown up’ corporate tone of their communications. It’s certainly not that I agree with everything they do. But what I do agree with is that I pay them for a service and, for the most part, I get it. When I don’t, after the dust settles, they’re as responsive as every other host.

    I’m not paying them just for server space, after all. I pay for backups, some cloud services, and most of all, I’m paying them for help when I screw up. Not to be my consultants, certainly, but I do pay them for technical support and advice like “Can you tell me how to install Ruby on my server?” because there’s no KB article … yet. Also when I needed help tuning httpd.conf they helped out. They don’t do the work for me, they do their limit, and they’re generally friendly about it.

    Someone playing an acoustic guitar

    So how bad does an incident have to be to make me leave it?

    I’ve only ever left a host when they didn’t offer the services I needed (SQL, PHP 5, so on and so forth). If I was paying less for a bare-bones host, I’d have to pay someone to help me with server stuff anyway, so for me the all-in-one matters.

    As for outages, I’m pretty relaxed about it, At an hour of downtime, I pay attention. I had a total of about 4 hours over the course of a day, which is annoying, and bad, but not horrible because no content was lost, just traffic. It’s not that my website isn’t my life, it’s that I’m realistic about situations. If the host explains what happened and are working on fix it as fast as is reasonably possible, I’ll suffer up to 6 straight hours before getting really upset. I’ve never had an outage of more than 75 minutes in a row, though, and before this one, I never had one more than 45. So yes, this was the worst outage I’ve ever had with them (that wasn’t my fault).

    Other incidents that may make me leave would be a deletion of my server without warning. That, hands down, is time to go. Any service promise that isn’t regularly met is grounds for a chat about expectations. I don’t count ‘support response time’ as a service promise, mind you, since when shit gets bad, that’s always going to drop. I mean things like backups or uptime. I’ve never been one to care about 99% uptime, but if the server’s always crashing no matter what I do, and they’re not willing to help me, then I have a problem. In general, I feel that if my site in particular is having issues, it’s probably me and my snowflake more than them. If all sites have the same problem, then it’s probably our needs don’t match the host services.

    The funny thing is I don’t know of many hosts that fits that bill. Sometimes a host has to tell you no, they can’t offer a service, and sometimes they tell you that you’ll have to pay more to do something. But in general, most hosts want to keep you, they want to help you, and they sometimes have to be the bearer of bad news. I’ve actually met hosts who have told me “We won’t be able to provide you the quality of uptime you need due to the way your site is being accessed.” That was a fancy way to say “You get too much traffic for our small node to handle.” And then they handed me a discount for another host. Another Host. This small host was bought out years ago, but I will always remember Greg for that moment. He was awesome.

    My point is that it’s not my host’s job to manage my website, so if I let the spam on my site go wild and it causes my server to crash, well that was my fault. Not theirs. Don’t like the way my plugins make my site work? That’s on me. And if they tell me “You’re getting hit by Reddit, we need to increase your CPU/memory to handle it, and that will cost you more money” I know they’re not just upselling me.

    There are some hosts, sure, out to make a buck, but in general I find that if they know that I understand our relationship, things go well.

    This isn’t meant to be a love song to any one host. None of them are perfect, and they all have weird quirks. This is a love song about remembering my relationship with my host, respecting that, and holding up my end of the deal. I’m not naming any host names (even though it would take you about 2 seconds to sort out who mine is, and who I work for, and yes, I’m ecstatic about both), because it doesn’t matter. I’ve had an experience like this with hosts that are maligned and vilified. My choice not to use them is not based on quality of service but on my morals and ethics. I chose not to fund people I am diametrically opposed to, for my own peace of mind.

    But I find, for the most part, that when I make it clear I know how our relationship works, and better yet, I know how their job works, I get both the support I want and the results I expect. It’s funny how that goes. They keep my faith and I keep trusting them.

  • Cold Calling Support

    Cold Calling Support

    Recently a coworker said I was mean to support, because I was firm and annoyed with someone on the phone. “Every time I’ve heard you take a support call in the office, you’ve been mean.”

    I corrected him “Those were cold calls. When I call support, with the exception of the idiot I got at NetSol, I usually walk away from my desk so we can have a long, friendly, chat.”

    Basically he only saw me talking to cold callers and thought I was mean. And I get why. Are not cold-calls a part of support? Given that cold-calling people who didn’t pay for (and who paid but never used) hosting, I can see where he might be taking it personally. I didn’t mention, since we both agreed that surveys calling you and offering to pay you was shady at best, but I don’t see what he does as a cold call. Debt collection maybe, but he’s not a cold call. He’s calling you because he has your information and you already started a relationship with us.

    So why do I hate cold calls? Well it’s the same reason I generally hate the emails “Can I ask you for a debugging favor?” You’re trying to get something off me without compensation, or generally thanks (no, Anne and Benny, you were fine). A cold call is even worse, though, because it really is just an out of nowhere call.

    Whats an example of the worst kind of cold call? Phone scams. I actually get a lot of those for services I don’t use, like Microsoft, and it’s actually made me tell people when there are legit calls “If you’re calling from X company, I’m going to call you back at the main contact number. What case number can I reference?” I did that with the cold-call for a debt collection which I argued. I didn’t recognize them, I didn’t know them, and I was not about to give them my credit card info over the phone.

    I got into an argument with a fake Microsoft call recently. “Sir, let me stop you. You’re calling someone who works in IT. I don’t have a Windows computer, you’re working for a scam company. I know many good companies in India–”

    And he shouted at me “I am not in India. Please listen, your computer has a virus.”

    So I raised my voice, “Sir, no it does not. You are working for a scam–”

    And he screamed, “YOU ARE A SCAM!” My wife could hear him. I tried to cut him off and explain, he shouted insults (I used to work with people from India, I know some insults) and I hung up.

    Hand holding a phone

    I’m sure I could have been nicer. Equally I could be nicer to the salespeople from DirecTV who call. “We’d like to upgrade your service.” and I say no thank you. “But it’s free for 3 months.” And I know that, but I know in 3 months I have to remember to cancel the service. No thank you. Again, no thank you. It’s around the second ‘no thank you’ that I start to lose my patience. Certainly I try to be firm, so they don’t think I’m easy to convince, but I’m not trying to be mean.

    It’s possibly a side effect of ‘Bitchy Resting Face.’ Whenever I’m firm and direct and say “No, I don’t want that service.” I get push back that I’ve been mean to the person on the phone. But if I say the same thing in a sweet and kind voice, I’m told I’m being too soft and that encourages the hard sell. From my end, it’s a no win.

    This is probably why I’m a bad salesman. If I say “You may be interested in product ABC, it can do these things.” and the person says “No, thank you.” I stop. I may say “Okay, if you change your mind or have later questions, please just ask.” and I move on. Because I, personally, hate the hard sell. I don’t want someone convincing me I want something I don’t need. Sales calls are not something I appreciate.

    On the other hand, of the few times I’ve been called by companies for support (not the other way around), I’m cautious and then pleasent. When I moved to California, my bank and credit cards called me. “Hi, we’re seeing a lot of charges from your card in a new location.” I laughed and asked if it was my new city and if it was the Target. They said it was, I assured them that was me. “Well, we’re going to hold those transactions until you update your account with your new address.”

    Boom. That was awesome. Security and support in one. I had to update it, of course, but in doing to, I confirmed for them it was me, and I helped them out. This was good because the next week someone in Kentucky tried to use my card numbers and they knew it wasn’t me. Of course, the amount of travel I do makes this hard, but they keep checking with me when appropriate.

    And that support? I always smile for.

  • Mailbag: Multisite Files

    Mailbag: Multisite Files

    Another mailbag! This one is a few people… No, this is the one most people ask me. I understand why.

    I really do understand it’s intent, it’s goals, and I try to keep up with it. I use it daily. When I write plugins for it, I spend hours trying to decide how to properly support it. I may even write in checks on the ones that don’t work on Multisite to auto-fail when you activate and prevent you from using them. I will argue till the cows come home that Multisite is for multiple separate sites. But that said, there’s also an exception to every single rule.

    A bunch of envelopes

    Let’s get to the letter! Kevin in Canada has a Multisite with a uploaded files conflict:

    The site is setup as default /files for the upload dir. Problem is, my client already has a directory in the root called /files. It cannot be changed as it houses software exe’s and needs to remain as /files. I need to set the upload path to /media for the images and not mess the site up. I read your blog post about this issue but wondering if you could clarify the steps. Right now, software download requests are redirected to the homepage!

    The steps are, alas, complicated because the situation is complicated. The blog post Kevin’s referring to is called Dumping ms-files. If you’ve installed WordPress as of WP 3.5, this is the new default anyway. That would mean that Kevin has an old Multisite.

    The ‘easiest’ option would be to start over with Multisite, use a fresh install of 3.9 and go from there. Second to that? Well the dumping MS files stuff is not easy and it really can’t be easy. You can try trepmal’s directions but after that you’re getting into writing a little bit of your own custom directions, and that is going to be really complicated.

    I’m not 100% clear on why these can’t be moved, so I’m guessing that the issue is the lost redirections. While I believe good URLs never change, I think that redirecting them is okay. These are .exe files, which means if, before the .htaccess rules for WordPress, you were to put in a rewrite rule and check ‘If you’re looking for /files/*.exe, go to /otherlocation/filename instead’ that might get around it, or even to say ‘if files and NOT exe.’ Maybe something like this:

    # uploaded files
RewriteCond $1 !\.(exe)
RewriteRule ^files/(.+) wp-includes/ms-files.php?file=$1 [L]

    Mind you it’s weird. Either you had Multisite first, which would explain the /files/ for uploads, or you had the EXEs in /files/ first, which doesn’t explain how you got this far into it at all, since 3.5 came out in December 2012, and that would imply you built the Multisite in 2012 and have had this problem for at least 18 months, give or take, which is a long time to have a problem and do nothing about it.

    If you started with WP 3.5 or later, you just make a /files/ folder and off you go, no code needed. If you started before that, you can follow the directs to undo MS files and then make a files folder and be done. Make sure not to forget about the .htaccess rules!

  • “Oh yeah. That’s a bug.”

    “Oh yeah. That’s a bug.”

    “You’ve been saying you’re working on it for two years! How hard could it be?”

    The post made my blood pressure rise a little. I was uncharitable, in my own mind, thinking Well if you know how to do it, why don’t you get off your whingy ass and do it? I was snippy, I was snide, I was … let’s face it, I was mean.

    But I didn’t say it out loud. I stood up from the laptop and played a round of ping pong with a coworker who is a bazillion times my better (seriously, he’s great) and tried to learn to return spin-serves. I still can’t, but I understand them now. Then, refreshed, I went back and replied.

    “I know it sounds really dismissive, but there are more things at play then just slapping a new coat of paint on the system. We have to take into account the following things [list]. On top of that, we have to do it all in a way that won’t crash the system and that’s backwards compatible!”

    I looked at that for a moment and I sighed. No matter how I explained it, no matter which language I used to demonstrate the complications, the reply was still going to be ‘Don’t make an excuse, just fix it.’

    Complex art

    There’s a weird truth about software and development that is the people writing or fixing the code generally don’t get fired up about it in a reactionary way. That’s not to say they’re not passionate, or that I never see them angry or excited, but that they’re sort of a Cool Hand Luke about a lot of things, including their job.

    From 1997 to 2012, I worked at a bank. I tapped out just shy of 15 years, and through that time I went from desktop applications to operating systems to server based desktop applications (remote apps) to server deployment. I did a lot. I worked on everything, but I was never what you’d call ‘tech support.’ I did support for developers for about 5 years. At one point, when I was working on the OS side of things, my manager asked if I could help one of the senior VPs with an email issue. I agreed, went to his computer, looked at it, calmly minimized the app, fixed the resolution (yes), and was about to go when they screamed.

    “You closed my email!”

    I blinked a few times, clicked on maximize, and pointed out I had only minimized it.

    My manager apologized for over-reacting, but suggested next time I maximize before I leave the desk. I did not point out that what I was doing was not in my job description at all, but I did agree he had a point and said that I would be sure to leave the PC the way I found it, except fixed. This lesson stayed with me. When I fix your site, I try to leave it the way YOU wanted it.

    The next day I had a complaint land in my lap, from the Sr.VP, that I didn’t take the issue serious enough. This was tacked on to the heels of another incident that, at the time, I’d forgotten about, where a server went down, people flipped out and called me, and I said “Uh huh, I see what’s wrong. *type* There we go. All better, I’ll fill in the ticket. Sorry about that.”

    What was this complaint? I was too cavalier.

    I did argue that complaint, and ‘won’ as much as anyone can, because I noted you don’t want the panicking person to run around and scream when they fix your broken stuff, you want them to be calm and collected. At the same time, I did learn something important, and that is people need to feel that you do empathize with them and feel their pain. I’m much better at that now than I was in 2001, but in the last thirteen years, I’ve blown my site up enough to know that level of terror in a visceral way.

    But for the person who is having the horrible, no good, very bad, day, there’s a lesson for you too! Sometimes when someone says “Yes, we know.” or “Yes, we’re fixing that.” they can sound far more calm and casual than you are, to the point where they seem dismissive, not because they are dismissive, but because of their experiences. It’s the same logic behind anyone who handles high-stress situations regularly: we get used to it, and we have managed to overcome our panic.

    What does all this have to do with the time you asked for a change and we said “We’re working on it?” We’re calm because we are working on it. We sound dismissive because we’ve said many times we’re working on it. We know we don’t have an answer and how much that sucks, but we know (and you know in your heart) that yelling at people to fix it now has never, in the history of ever, actually made someone come to that genius moment faster. Even critical bugs, like Heartbleed, are things everyone tries to fix as soon as possible, but not in a rush or a panic. And if they’re not critical, then we’re generally not going to rush and fix it unless we’re certain we can do it in a way that is safe for everyone, sustainable, and we agree with it.

    That sounds like a lot, and it also sounds like an excuse. It’s not. If someone replies “Yes, we’re working on it.” it’s okay for you to ask “Is there anything I can do to help?” It also means you may have to accept that some annoyances are going to stick around for a long time, because they’re complicated, or maybe they require a total rewrite. But as long as they’re communicating with you, they’re not ignore you.

  • Genesis Favicons Multistyle

    Genesis Favicons Multistyle

    For the longest time I used .htaccess to shunt my favicons to the right place based on the requesting domain. This basic logic worked well, but was starting to get weird and it didn’t always work depending on if I was using pagespeed to trim out leading domains or not, and of course when I tightened security it got weirder.

    A whole mess of favicons

    Now I’ve know you can use custom favicon in Genesis and that’s pretty simple. But … I have to make my life complicated. See I use Multisite and I don’t edit my themes. No, really, this theme? I only edited it to update it. I don’t edit them, I use custom CSS and functions to edit via hooks and filters, the way you’re supposed to.

    So how do I override favicons for five sites on a Multisite? Why with an MU Plugin!

    wp_head() and toss in the link for shortcut icon.

  • Mailbag: Multisite Caching

    Mailbag: Multisite Caching

    Dave has a question abtou Multisite and caching!

    I asked about caching on multisite setups. I’ve got 10 of these, with between 10-50 sites added to each- and the content is static- never going to change it much. I have a feeling I could benefit from caching, you recommended server side. I could use some help here. I know your site says you’re maxed out time wise- I’m thinking I just need someone to help me with one site, and show me what to do. Or, if they want to set things up on all of them. Do you have any recommendations?

    Thanks again for the help.

    Dave

    Caching! There are three main types: plugin, server, and proxy.

    Plugin caching is WP Super Cache, or W3TC, and it’s letting WP make a static cache file on demand. Server caching is APC/Memecached and such, which lets the server make a dynamic cache. Proxy caching is Varnish and the like, which puts a cache before the WordPress server, kind of like a static file on another server.

    Which one is best for Multisite with static content? Probably a proxy cache there. Though I’m going to go out on a limb and say if you’re not using WordPress for all it’s useful for, maybe you don’t need it at all. If you have a pretty static site, where you’re not changing content all that much, there’s nothing wrong with static HTML. But if you want to use WordPress (like you’re a WordPress only shop) then you have to consider the options. What is your site doing?

    Dog catching a frisbee

    This is actually not a Multisite Question so much as a cache management question. The part that’s multisite specific is who to you let manage what aspect of the cache. I maintain a plugin called Varnish HTTP Purge, which is a pretty basic plugin that checks for common actions, like posting or commenting, and flushes your cache selectively on Varnish when those things happen. One of the Multisite specific issues with that code came up when I added in a new feature: Purge Cache.

    That feature lets you press a button to flush the whole cache. Pretty cool, right? The problem was Multisite with Subfolders. See, the way the cache flushes is to say “Dump everything under domain.com!” which works fine with subdomain sites, as that will only flush domain.com and not foo.domain.com. On a subfolder, though, it’ll flush domain.com, domain.com/foo/ and domain.com/bar/ and everything else. That would suck! So the best thought I had was I should lock the flush down to Super Admins only on the main site on a subdomain setup. Normally I allow the site admin to flush their own cache, but this way only the Super Admin could make a full flush.

    And that’s the crux of the issue with caching on Multisite. Do you want each admin to maintain their own cache? Do you want it to be something you control for everyone? W3 Total Cache can do that, by the way. You can do it both ways with that plugin, and only control what you want to control for the network. But what about if even I don’t want to do that? What if I want to be lazy and trust my tools? That’s when you look at the server itself and try to make it as brainless as possible. Decisions, not options, remember?

    And that brings up another question. Why do I want to manage who flushes at all? In my Varnish example, it’s because careless flushing clears the whole cache, which means my server and site have to recache everything and it slows everyone down. Obviously I want to limit that as much as possible. The other reason is work. People who have a managed hosting site generally aren’t the most technical savvy users in the world. Some are, of course, but many are more likely the ones who don’t want to know about the crazy server stuff. This is great, but it means I want to make their life easier so I have to provide less support, then I want things that do their job and do it well, and that means taking options away and making decisions for them.

    How would I cache a static Multisite network and fully control it as a network admin? Probably server side caching like Varnish or ZendAccelerator Memcache. If I can’t do that, I’d use W3TC and lock it down so only I, the super admin, in my fancy cape, can edit settings and purge the cache. And always, always, remember to only catch what should be caught in your cache. If you have a site that’s a store, account for that.