Why We Hate Your Security Reports

I was having a day where a bunch of security reports were dumped in my lap. There have been days where those have been hundreds. Thankfully this wasn’t, but it did make me sit back and think about why I hated so many reports.

In general, everything can be summed up as “The person reporting doesn’t provide all the relevant information.” Sometimes they don’t know, and that’s okay. People don’t learn by osmosis, they have to be taught. But a lot of the people submitting reports do know and just don’t. And the real crux of it all? People aren’t explaining why things matter.

Let me explain…

The Uninformed

These are the people who report an app and say “It’s this app because I installed it and I was hacked.” Another variant is “It’s this app because that’s where the hacked file lives.”

And the email will have no other information. With those, I have to explain not only are they probably wrong, but I have to dig and get at the information.

  • Why do you think it’s this app?
  • Do you have any evidence?

Those are pretty basic questions at the heart, but it’s on me to explain why correlation isn’t causation, and just because (say) Hello Dolly was infected does not mean that it was vulnerable.

This has no solution other than education. I dislike those, but since most people are pretty cool about it when I explain how I know it’s not the app they think it is, it’s okay.

Semi-related are people who email us security logs from their services, and those are always messy, since it’s information without enough context. Which I’ll get to in a minute.

The Too Terse

Sometimes reports are explanatory but not enough. “It’s an XSS in this app.”

Now, the good news is I know what I’m looking for. The bad news is that someone who actually knows what the issue is has decided to not share, which means I now have to figure out how they figured it out. It used to be I’d do that, but then they’d email back all snarky and bitchy that I didn’t find the one they found. Nowadays I push back. “Can you please provide details?”

That has a weird hit-and-miss. Sometimes people are pretty chill and explain. The majority do what I think of as a pre-teen eyeroll. You can actually tell they’re huffing in annoyance that someone dared ask them to unpack what was in their heads.

To put this differently, have you ever had someone say “Hey, the website’s down.” and just … not give you an error message? You know how maddening that is? Right that’s what we’re talking about.

The Non Explainers

You’d think this is the same as Terse, but it’s not. The non-explainers don’t explain WHY something is a security issue.

I know, someone’s reading this going “Hang on, but if I tell you it’s a SQL injection vulnerability, and where it is, isn’t that enough?” And the answer is, most of the time, yes! The people who give a great proof of concept, with exactly how to replicate it, in clear English, are my favourites. They break down how things happen so you can see “Oh that’s why.” But… When they don’t, it means someone (read: me) has to go and figure out “Okay, why is this bad, and how bad is it?”

The Hater Reporters

I can’t believe I had to add this one in, but here we are in 2021 and there are some ‘security reporter firms’ who think the best way to report an issue is

  1. make it public
  2. attempt to make other people feel bad
  3. dogshame developers

For them, the only way forward is their road or no way at all, and they cannot be reasoned with. Eventually someone will sue them for releasing a 0-day vulnerability without even trying to privately disclose first, and when that happens, I’ll make the popcorn.

The Issue Is Education

If you go back through, you can see the real issues are people not unpacking what they actually know and sharing in a digestible manner! And this is terribly endemic of security companies more than anything else.

For example, recently a security company reported a local file inclusion (LFI) issue. Now for those who don’t know, the issue is the code in question could be used to include any file on the server. Including a hacked file. But if someone just told you “Hey that’s an LFI and you’re bad!” then, even if they take the time to tell you where the issue is, if they’re not explaining to you how it’s exploitable, you may not know!

And then, even when people explain it, they explain as if they’re talking to a developer of their caliber. I certainly am, but the people I’ve got to pass the report to (the actual devs) are not always. Even when they are, sometimes they’re total berks who will snark that it’s not worth the time to escape things at that low a risk.

Understanding Risk

Security is massively important but the reality is that it’s not the first thing on most people’s minds when they write code (sorry folks). Usually people concentrate on making the code work first. Then, once it works, they go back to make it safer. I’m not casting aspersions here. There’s nothing wrong with making it work first. The issues begin when people don’t take security with the proper seriousness.

Just the other day I saw someone who had to be told that yes, you always escape content you’re echoing. Why? Because users are humans, and humans do some really stupid things. Even if you think of yourself as average, that means roughly half of your users are not as smart as you, which means that half is who you’ve got to look out for. You sanitize content you save, you scape content you echo. All. The. Time.

And yes, I’ve seen people who are experienced developers, people with plugins whose user count is in thousands, reply that it’s not needed to escape because … it’ll make their code slower.

Sometimes I tell people “This is why I drink.”

Proper Education

Now. Part of this is on the community/company. WordPress, where I do a lot of work, has decent documentation about security. In fact, as of late WordPress’ docs have been phenomenal about this!

Here’s what the plugin dev docs say about nonces:

If your plugin allows users to submit data; be it on the Admin or the Public side; you have to make sure that the user is who they say they are and that they have the necessary capability to perform the action. Doing both in tandem means that data is only changing when the user expects it to be changing.

Now. That kind of explains why you want to do this, but does it explain why it’s needed for security? Only from a high level. For the crux you have to scroll down a little:

The capability check ensures that only users who have permission to delete a post are able to delete a post. But what if someone were to trick you into clicking that link? You have the necessary capability, so you could unwittingly delete a post.

Now that makes a lot more sense, right? That is a good doc, assuming people read it.

And look at the bolded intro for escaping:

Escaping means stripping out unwanted data, like malformed HTML or script tags.

Whenever you’re rendering data, make sure to properly escape it. Escaping output prevents XSS (Cross-site scripting) attacks.

Whenever. Not sometimes, not when it’s convenient. WHENEVER.

And yes, this means every single time you echo anything as a variable, you damn well escape it. No questions asked.

But when I say proper education, I mean in explaining why a specific issue is, in fact, an issue.

Communication Is Queen

If you’re a regular person who sent a report you weren’t sure about, you’re fine. This next bit is not about you. This is about security experts and other developers.

Did you contact developers, privately, about issues with their code? If you’re a security company, do you have documentation on your site to explain how something in an XSS or LFI vulnerability? Did you explain in the contact why something is a risky LFI? Did you take a minute to share a Proof of Concept to illustrate how you knew something was a risk? Did you describe why and how the POC shows the risk?

That’s what you have to do.

You want other developers to be better and to write better code? Then you communicate, clearly, and take time to ensure what you’ve said is understandable by the recipient.

I hate your security reports because they’re not reports at all. They’re dumping a problem on someone else and not giving them the tools to progress. You’re expecting them to do all the work you already did, which by the way is a waste of everyone’s time. It wastes my time because now I have to do everything you already did, and it wastes yours since I’ll probably ask you for the details.

But. If you start with a private, polite, report of “Hey, I found this. Here’s how and here’s why I know it’s an issue.” then you, you my friend, are heroes. You’re actually making the entire world better for everyone.

Thank you.




%d bloggers like this: