Category: How To

Interlude: Gutenberg Moves Fast
I’m taking a pause on my plugin posts to talk about Gutenberg.

I really love Gutenberg. I’m not kidding! I find it far more enjoyable to write (stories) in plain apps (I used Apple Pages because it syncs between laptop and iPad, yes, I am often using my iPad to write my novel, yes, I will let the world know when it’s done). But when I write for the web, it’s a more visual medium, and Gutenberg is fantastic to represent what I’m writing as it will be properly seen by all!

But.

Gutenberg moves fast. Hella fast. So fast it can leave you in the dust, and it has a critical flaw that I feel has been stifling it’s growth and usage among developers.

JS isn’t your Momma’s PHP

This is obvious. Javascript ain’t PHP. PHP is a simple language that can be coerced into doing complex things if you understand basic algebra. Surprise! Everyone who considers themselves good at PHP? You’ve mastered the concepts of algebra! Alegrba is one of the easier ‘complex’ mathematic concepts to wrap your head areoud. You get “if a + b = c and a = 10 and c = 11 then b = 1” and you win!

Javascript though, it’s a little more like calculus and trig, in that you have to understand the formulas a little deeper, and they have that thing where not just numbers and letters appear, but weird symbols.

[nb: Like all analogies, this falls apart at scale, don’t read too much into it.]

For the thousands of developers who whet their teeth on PHP, jumping into JS feels like you’re a first-year high schooler in senior maths! It’s scary, it’s complicated, and worst of all … it isn’t actually documented at the micro level, because it’s generally compiled.

Micro vs Macro / Interpreted vs Compiled

The macro scale is, more or less, the big picture of what your code is supposed to do. Micro would be each individual element. For PHP, you can clearly identify both the macro (the overall function) and the micro (each teeny process in that function). This is less so for JS because the languages are different.

There are two primary types of code languages. PHP is what we call an interpreted language, because while the PHP binary is a compiled app, what you write is interpreted by the compiler. Basic JS (like jQuery) is also an interpreted language!

Compiled languages need a “build” step – they need to be manually compiled first. And if that suddenly made you think “Wait, Gutenberg is JS but I have to build it!” then you have spotted the quirk! The JS we use in Gutenberg is actually JSX!

JSX was designed for React (which is what we use to build in Gutenberg) and while it may contain some plain Javascript, it’s impossible to use the code without React. That’s why we have the build process, it takes the JSX, compiles it into JS, and saves it to a file.

The Compilation Downfall

This is where it gets messy … messier.

When there’s an error in PHP, we get the error message either on the page or in our logs, depending on how we set up our environment. I personally pipe things to debug.log and just keep that file up as I bash on things. Those errors tend to be incredibly helpful!

$mastodon not defined on /path/to/file.php:123

In that example, I know “Ooops, I’m calling the variable $mastodon on line 123 of file.php and forgot to declare it!” Either I need an isset() check or (in this case) I brain farted and copied a line but forgot to rename the variable so I was setting $tumblr twice. Mea culpa, pop in, edit, save, done.

On the other hand, I was testing out some blocks and modernizing them a little when suddenly … the block didn’t load. I got the WP notice of the block had an error. You’ve probably seen this if you’re a dev:

or this:

And if you’re like me, you used foul language and wondered ‘well… now what.’

Enter the Console

Unlike PHP, the errors don’t go to a nice debug.log file, it goes to your in-browser console. This is because, again, PHP is being directly interpreted on the server, and the server happily converts the PHP to HTML and Bob’s your uncle.

JS (and JSX in this case) aren’t processed by the server. They’re processed on the fly in the browser. If you’ve ever wondered why too much JS, or bad JS, cause your browser to hang, that’s why. We moved the processing from the server (PHP) to the browser. On top of that, it’s also why JS content isn’t really cachable by traditional methods! But that’s another story.

In this case, I got the first error (cannot be previewed) and being somewhat savvy with the world of Gutes, I popped open the console and saw this gem:

wp.blockEditor.RichText value prop as children type is deprecated

The rest of the message was warning me that the thingy would be removed in WP 6.3, and it had a link to ‘help’ resolve it. Spoilers? It didn’t. But take a deep breath. Let’s debug.

Debugging Gutenberg

The first issue was that the error came on a page with multiple blocks. I happened to be using a custom plugin I wrote that contains about 6 blocks, you see, so I opened a new page on localhost and added each block, one at a time, until I determined the issue was my incredibly simple spoiler block.

How simple is this block? It’s basically a custom formatted paragraph, so everyone could use the same design without having to remember the exact colours. I could have made it a ‘reusable block’ on the site but, at the time, I wanted the practice.

Next I went to that link, which was for “Introducing Attributes and Editable Fields“. I admit, I was a little confused, since I was already using attributes and editable fields! But I did the logical thing and searched that page for the word ‘children.’ My thought process was that if something was being deprecated, it would have a warning right?

Okay, maybe I was looking in the wrong place. This error is specific to RichText so I clicked on the link to read the RichText Reference and again, looked for “children.” Nothing. Zip. Nada. I followed the link for the more in-depth details on GitHub and still nothing.

At this point, I ranted on Mastodon because I was chapped off. I also popped open the Gutenberg Deprecations page, and looked for “children” but all I could find was a message to use children!

RichText explicit element format removed. Please use the compatible children format instead.

Logically there should be a note that “children is deprecated, please use…” but there is not.

Now, here is where I accidentally stumbled on a fix, but after I made my fix is when I found the Github issue about this!
If you are still using “children” or “node” sources in the block attribute definition, like so:
content: { type: 'array', source: 'children', selector: 'p', }
Then change it to use the “html” source instead to get rid of the deprecation warning:
content: { type: 'string', source: 'html', selector: 'p', }
And in fact, that was the correct fix.

Here’s the Flaw

None of that was properly documented.

The link to ‘help’ fix the error didn’t mention the specific error, it talked about attributes at the MACRO level. I was (obviously) already using attributes, else I wouldn’t have had that error at all.

There is no proper documentation that could help someone fix the issue on their own UNLESS they happened to be trawling through all the issues on GitHub.

As I put it to my buddy, the reasons developers are salty about Gutenberg are:
1. It changes pretty much every release
2. There’s no real way to tell people if it impacts you so you have to check every release and read the console logs, which is not what devs are used to
3. The JS console won’t tell you (I don’t know if it can) what file caused the warning, so finding it is a crap shoot
4. The documentation is high level, which is not helpful when you get micro level errors
Okay, can we fix it?

At this point, if you’ve made a WordPress Block for Gutenberg, make sure you test every single release with the JS console open. If you don’t do this, you will have a rude awakening until things are made a little better.

How can things be made better? It will have to begin with a culture shift. Traditionally WordPress has used a “release and iterate” model. With Gutenberg, we’ve become “move fast and break things,” but that is only sustainable if everything broken can be documented for a fix.

That means I see only one way to correct this, and it’s to slow down Gutenberg enough that deprecations AND THEIR CORRECTIONS are properly documented, and the error messages link to a page about deprecations.

We need to not link to the general “here’s how attributes work” page, but instead to a specific page that lists those deprecations along side the WordPress versions impacted.

Another matter is we should be posting in the Field Guide about these things. Currently the 6.3 field guide links to all the various pages where you can find information, but that means you have to click and open each one and hopefully find your exact usage. In my case, those links to the ten Gutenberg versions being ported to core never mention the issue I had.

If we don’t start slowing down and paving the road for developers, we will begin haemorrhaging the very people who make WordPress a success.
27 July, 2023
Zaptodon
On a site, I use Zapier to automate a specific set of tasks. Every day, the website sets up a show/character of the day (I think you know what site this is…) and it posts that show/character to Twitter, Tumblr, Facebook and …. Mastodon.

Or at least it does now.

Zapier

Caveat! I pay for this service. The webhooks used are in the starter package at $19.99/month annually.

There’s a service, Zapier, that allows you to make incredibly complex if/then/else checks and performs an action. It’s not cheap, but at $250 a year it’s not expensive for my needs. Way back when, I picked it because I needed something that would let me actually script, and I recognized that running pushes like that all from my own server was a lot more work than it should be, what with all the libraries and so on.

Seeing as that wasn’t driven by a post or any action except time, it had its own quirks. But then one day in May we all woke up and saw how dumb Twitter was. They priced WordPress.com and Automattic out of their own API!

But not Zapier.

Note: Automattic made the right choice here. With the API cost starting at $42,000 a month (yes, a month, and I can remember when I made that a year and thought I was hot shit), knowing how inexpensive Jetpack Pro/Premium/Whatever is, there was no way they could justify it.

Zapier’s business model has a large chunk invested in pushing things to social (among all sorts of cool things, like calendar integration). So when I had to revisit how I posted new articles to Twitter anyway, I figured I’d wrangle Mastodon as well.

The Zap Flow

Overall, my flow looks like this:

But what goes in each?

The first one is a built in and easy trigger. It follows the RSS for the site and, when there’s a new article, off it goes.

The third one is the tweet, which is about as straightforward as you might expect.

The second one is Mastodon. That’s where we’re going to concentrate today.

Add an App

To do this, you need to create an ‘app’ for your Mastodon account. Log in to your instance (mine here is mstdn.social) and edit your profile. On that page, on the menu to the left, is an item called Development.

On that page you’ll see a list of Applications, if you have any, and a button to create a New Application. That’s what we want to do today. Click on that button and you’ll get the basic data:

I put in “Zapier” and “https://zapier.com” as my

Scroll further down and there are a bunch of options. You only need to have these two checked:
- read:accounts
- write:statuses
The read will let us know things worked, and write … I really hope that one is obvious for you, but you have to be able to write to post.

Click create and you will be redirected to the Application page where it will now list your app to Zapier. Click on that and you’ll be show a page with the set up info, but now it has a Client Key and a Client secret.

I clicked regenerate right after I took this screenshot.

You’ll be able to get at this whenever you want, so don’t panic.

Back to Zapier

Over on Zapier, pop open your Zap. In my case, I had a pre-built one for Twitter, so I added this in by telling it I wanted to add an Action. Since I pay for Zapier, I have access to their premium webhook to post:

Pretty clear I think. I need “Webhooks by Zapier” and the Event is POST. That’s telling Zapier what to do with the hook.

The next part of the step is the Action and that has a lot of stuff. The first two are the URL and the Payload:

The URL is going to be https://yourinstance.com/api/v1/statuses?access_token=[YourToken] — What’s your token? Remember that as the third item shown on the edit page for your Application over on your instance? Yep! Paste that in.

I picked JSON for my payload since I’d been using it elsewhere. For the next part, I have to dig out the data. For Mastodon, you want your data ‘type’ to be status since that is literally telling it “I wanna make a status post!” and the content I made the description and the link.

If you click on that box where I have description etc, it’ll pop up with more options!

Pretty nifty! I left the rest as default:
- Wrap Request In Array – no
- File – empty
- Unflatten – yes
- Basic Auth – empty
- Headers – empty
Click on continue and you can test.

Done!

That’s it! Now your RSS feeds will auto post to Mastodon.

I’m sure someone’s wondering “Why aren’t you using ActivityPub, Mika!?!” And the answer is… It doesn’t actually work on all hosts. ActivityPub requires you to be able to write to your .well_known/ folder and, currently, you cannot do that on DreamHost because it’s managed at the server level.

This is not a wrong choice by either party! DreamHost (especially on DreamPress, the managed WP solution) wants to prevent you from breaking your SSL. Now, thanks to @diziara, there is a workaround if you can edit the .htaccess file in your .well_known folder:
```
# Permit access to the challenge files but nothing else
Order allow,deny
Allow from all

RewriteCond %{REQUEST_URI} ^/[.]well-known/webfinger+$
RewriteRule .* /wp-json/activitypub/1.0/webfinger [L]

RewriteCond %{REQUEST_URI} ^/[.]well-known/acme-challenge/[a-zA-Z0-9_-]+$
RewriteRule .* - [L]

RewriteRule .* - [F]
```
Assuming your install is in root (mine is) you put that into the .htaccess and it works! I was surprised that it also let me edit on DreamPress, but I’m not sure if that will last. I’ll keep my support-thread updated though.

And the other thing… I don’t want people to ‘follow’ my blog like that. I mean, you could, but also people follow me as me, and if I auto-post to ‘me’ then it works. Telling people to follow my blog and me is tricky since people are lazy (seriously we all are). But if that’s your thing, then yes, you absolutely can follow @ipstenu@halfelf.org and get all my articles.

I’m still going to use a combination, since while I do want people to follow my blog, I suspect more will follow me instead. Also it’s easier to follow up on engagements (questions etc) if I’m watching ‘me’ and not two places. The other problem is it’s letting you follow ME at my blog. My other site has many more authors, and this isn’t quite right for that.

The nice thing, though, is that there isn’t a single perfect answer for everyone’s use case. For most people, ActivityPub will work and they can be made discoverable. For the others, though, hold on until end of June. My friends at Automattic are planning to have post-to-mastodon support in their next iteration.

I’ll still need my zaps since I post things that aren’t blog posts, but I’m looking forward to one less.
24 May, 2023

Stopping Jerks in Gravity Forms

“Hang on,” I hear you say. “Didn’t you already do this?”

Well, kind of. I did with Ninja Forms (and actually that resulted in me removing Ninja Forms and going back to Jetpack, and isn’t that a pill?). And I’ve mentioned Gravity Forms before with disallowed keys.

This is really an extension of the disallowed keys and some very specific complex things I’ve changed since those posts.

Yeah, tech changes.

What’s a “Disallowed Key”?

Disallowed keys are basically all those terms you put in your Disallowed Comment list (formerly the Blacklist):

A screenshot of the disallowed comment keys field from Wordpress, with a string of examples like 'foo.bar@gmail' and so on

Now if you look at my example above, I have a bunch of terms in there. Each line is a ‘key,’ meaning @example.net is a key.

WordPress built this in for handling comments, specifically, but I’ve long advocated for things to be global, and since this is saved in my site, I can easily re-use it by telling Jetpack or anything else “If this email is on the list of disallowed keys, I don’t want to see it.”

The reality of doing that is, of course, harder than it looks.

For example, if you use Gmail, then foobar@gmail.com and foo.bar@gmail.com and foobar+ihatemika@gmail.com are all the same bloody thing! And if I block generaljerkyperson do I then want to block generaljerkyperson@nonjerky.com ?

You have to make some decisions. Now solving the email one with periods and plus signs was easy. By comparison.

Jerky People Aren’t Spammers

This is the other thing to get into our heads. Jerky people ain’t spammers. Spammers are, weirdly, easier to spot and handle. Someone posts about viagra? Spam. Someone posts about SEO? Probably spam (though not always on a tech site). But those are all pretty self evident once you look at them!

What is not self-evident is something like the world I have.

Over on a site I run, we have a group instance (Slack, Discord kind of thing) and we let people sign up via Google Sheets for a while. But we ended up with one person in the group who was an antagonist. Let’s call her Bertie.

Bertie isn’t a terrible human, but she has “Opinions” and they are the right ones. She likes a very specific thing, and does not like variations or changes. And if you don’t like what she does, you’re wrong, she’s right, and damn you. She picks fights, constantly, she disrespects anything anyone else likes… To put it how my friend Tracy might say “She yucks on other peoples’ yums.”

We attempted to negotiate with her. Point out “Hey you’re in a GROUP and if you cannot respect people enough to tolerate their opinions when they differ, then this ain’t the group for you.”

Bertie didn’t. And she didn’t change. So we banned her. Fine.

She tried to sign up via Google Sheets again. We said no. And again. No. Then she tried new emails, new bios, new fake names. One actually got in. But as soon as we learned it was her, we kicked her.

And at this point, we were fucking tired of playing whack-a-mole. So I decided to make something better.

Catch and Release

My overall thought process boils down to this: Every single jerky person has tells.

They just do. And a human can spot them and go ‘wait a second…’ because we’re actually really good at recognizing patterns that are similar. A computer has to be taught that “If the submission is from this region or has this kind of email, it’s probably them.”

So I started to build out some logic that would check all my flagged emails and IPs. Then I took advantage of Gravity Forms’ API to make a note in the entry so I could have a record of why someone was flagged.

Here’s what it looks like for someone who’s email and IP was on the naughty list:

Seeing that come in overnight, by the way, delighted me. It shunted the annoying Bertie to spam for two reasons, and none of us had to deal with her.

How it Works

This is the code part. The down and dirty here is I have two classes, one for finding spammers, and then the Gravity Forms that calls it. I did this because at the start, I had both Jetpack and Gravity Forms AND Google Forms. Obviously I can’t block someone hitting the Google Form directly, but I used to have this prevent that from loading the form. It was easy to get around. I know I know.

Even though I now exclusively use Gravity Forms on the site, I left these separate to be future friendly. Also it means you can steal that to put it into whatever you’re doing. It’s GPLv2.

By the way, everything goes to spam for a simple reason: It lets me clear up false positives.

Find Spammers

This class is what hooks into disallowed keys and checks if the email is banned, the domain is banned, or the IP used it banned. It also has an option to check for people who are moderated! That means if you wanted to flag people who might be jerks, you can do it.

class Find_Spammers {

	/**
	 * List of disallowed Keys
	 *
	 * We check for emails, domains, and IPs.
	 *
	 * @return array the list
	 */
	public static function list( $keys = 'disallowed_keys' ) {

		// Preflight check:
		$valid_keys = array( 'disallowed_keys', 'moderation_keys' );
		$keys       = ( in_array( $keys, $valid_keys, true ) ) ? $keys : 'disallowed_keys';

		// Time for the show!
		$disallowed_keys  = array();
		$disallowed_array = explode( "\n", get_option( $keys ) );

		// Make a list of spammer emails and domains.
		foreach ( $disallowed_array as $spammer ) {
			if ( is_email( $spammer ) ) {
				// This is an email address, so it's valid.
				$disallowed_keys[] = $spammer;
			} elseif ( strpos( $spammer, '@' ) !== false ) {
				// This contains an @ so it's probably a whole domain.
				$disallowed_keys[] = $spammer;
			} elseif ( rest_is_ip_address( $spammer ) ) {
				// IP adresses are also spammery people.
				$disallowed_keys[] = $spammer;
			}
		}

		return $disallowed_keys;
	}

	/**
	 * Is someone a spammer...
	 * @param  string  $email_address The email address
	 * @param  string  $plugin        The plugin we're checking (default FALSE)
	 * @return boolean                True/False spammer
	 */
	public static function is_spammer( $to_check, $type = 'email', $keys = 'disallowed_keys' ) {

		// Default assume good people.
		$return = false;

		// Get disallowed keys & convert to array
		$disallowed = self::list( $keys );

		if ( 'email' === $type ) {
			$email_address = $to_check;

			// Break apart email into parts
			$emailparts = explode( '@', $email_address );
			$username   = $emailparts[0];       // i.e. foobar
			$domain     = '@' . $emailparts[1]; // i.e. @example.com

			// Remove all periods (i.e. foo.bar > foobar )
			$clean_username = str_replace( '.', '', $username );

			// Remove everything AFTER a + sign (i.e. foobar+spamavoid > foobar )
			$clean_username = strstr( $clean_username, '+', true ) ? strstr( $clean_username, '+', true ) : $clean_username;

			// rebuild email now that it's clean.
			$email = $clean_username . '@' . $emailparts[1];

			// If the email OR the domain is an exact match in the array, then it's a spammer
			if ( in_array( $email, $disallowed, true ) || in_array( $domain, $disallowed, true ) ) {
				$return = true;
			}
		}

		if ( 'ip' === $type ) {
			$ip      = $to_check;
			$bad_ips = false;
			foreach ( $disallowed as $nope ) {
				if ( rest_is_ip_address( $nope ) ) {
					if ( ( strpos( $ip, $nope ) !== false ) || $ip === $nope ) {
						$bad_ips = true;
					}
				}
			}

			// If they're a bad IP, then they're a bad IP and we flag.
			if ( false !== $bad_ips ) {
				$return = true;
			}
		}

		return $return;
	}

}

new Find_Spammers();

Gravity Forms Check

You may have noticed that the spammer checker is really just that, a checker. You have to call it. How I call it is via a Gravity Forms function. This does a couple kind of redundant things, and I know it can be optimized.

The IP checker has some extra stuff to help me record where an IP is from when people submit, in order to try and catch other ‘common traits.’ It’s using ip-info, and amusingly I’ve found it’s mostly right. For some reason, it got the IP location of the same IP as being from 3 separate locations. I suspect it’s Bertie trying to be smarter and use a VPN. The fact that the Location is not the only measuring stick I use though means she can change her IP and email a bunch of times, but I have other checks.

One improvement on my list is that if someone has a certain number of red-flags, it treats it like a jerk and sends to spam.

class My_Gravity_Forms {

	public function __construct() {
		// Check all Gravity Forms ... forms for spammers.
		add_action( 'gform_entry_is_spam', array( $this, 'gform_entry_is_spam' ), 10, 3 );
	}

	/**
	 * Mark as spam
	 *
	 * If someone on our block-list emails, auto-mark as spam becuase we do
	 * not want to hear from them, but we don't want them to know they were rejected
	 * and thus encourage them to try other methods. Aren't assholes fun?
	 *
	 * @param  boolean  $is_spam  -- Is this already spam or not?
	 * @param  array    $form     -- All the form info
	 * @param  array    $entry    -- All info from the entry
	 * @return boolean            true/false if it's "spam"
	 */
	public function gform_entry_is_spam( $is_spam, $form, $entry ) {

		// If this is already spam, we're gonna return and be done.
		if ( $is_spam ) {
			return $is_spam;
		}

		$spam_message = 'Failed internal spam checks';
		$warn_message = '';
		$is_spammer   = false;
		$is_moderated = false;
		$is_bot       = false;
		$is_vpn       = false;

		// Loop and find the email:
		foreach ( $entry as $value => $key ) {
			if ( is_email( $key ) && ! $is_spammer ) {
				$email        = $key;
				$is_spammer   = Find_Spammers::is_spammer( $email, 'email', 'disallowed_keys' );
				$is_moderated = Find_Spammers::is_spammer( $email, 'email', 'moderated_keys' );
			}

			if ( rest_is_ip_address( $key ) && ! $is_spammer ) {
				$ip           = $key;
				$is_spammer   = Find_Spammers::is_spammer( $ip, 'ip', 'disallowed_keys' );
				$is_moderated = Find_Spammers::is_spammer( $ip, 'ip', 'moderated_keys' );
				$is_bot       = self::check_ip_location( $ip, 'hosting' );
				$is_vpn       = self::check_ip_location( $ip, 'proxy' );
			}
		}

		// If this was a bot...
		if ( true === $is_bot ) {
			$warn_message .= 'Likely submitted by a bot or someone scripting. ';
		}

		// If a VPN...
		if ( true === $is_vpn ) {
			$warn_message .= 'Using a VPN. This may be harmless, but it\'s also how people evade bans. ';
		}

		// And if it's a spammer...
		if ( $is_spammer ) {
			$message = $spam_message;

			if ( isset( $email ) ) {
				$message .= ' - Email ( ' . $email . ' )';
			}
			if ( isset( $ip ) ) {
				$message .= ' - IP Address ( ' . $ip . ' )';
			}

			$result = GFAPI::add_note( $entry['id'], 0, 'My Robot', $message, 'error', 'spam' );
			return true;
		} else {
			if ( ! empty( $warn_message ) ) {
				$add_note = GFAPI::add_note( $entry['id'], 0, 'My Robot', $warn_message, 'warning', 'spam' );
			}
		}

		// If we got all the way down here, we're not spam!
		return false;
	}

	/**
	 * IP Checker
	 */
	public function check_ip_location( $ip, $format = 'full' ) {
		$return    = $ip;
		$localhost = array( '127.0.0.1', '::1', 'localhost' );

		if ( in_array( $ip, $localhost, true ) ) {
			$return = 'localhost';
		} else {
			$api     = 'http://ip-api.com/json/' . $ip;
			$request = wp_remote_get( $api );

			if ( is_wp_error( $request ) ) {
				return $ip; // Bail early
			}

			$body = wp_remote_retrieve_body( $request );
			$data = json_decode( $body );

			switch ( $format ) {
				case 'full':
					// Return: US - Chicago
					$return .= ( isset( $data->countryCode ) ) ? ' ' . $data->countryCode : ''; // phpcs:ignore
					$return .= ( isset( $data->countryCode ) ) ? ' - ' . $data->city : ''; // phpcs:ignore
					$return .= ( isset( $data->proxy ) && true === $data->proxy ) ? ' (VPN)' : '';
					break;
				case 'hosting':
					$return = ( isset( $data->hosting ) && true === $data->hosting ) ? true : false;
					break;
				case 'proxy':
					$return .= ( isset( $data->proxy ) && true === $data->proxy ) ? true : false;
					break;
			}
		}

		return $return;
	}
}

new My_Gravity_Forms();

Any Issues?

Two.

I use an ‘approval’ feature (forked from “Gravity Forms Approvals” to allow for multiple approvers optional but only one has to approve to be a go – the original requires all approvers to approve) – for some reason this is not properly moving anything in spam or trash to a ‘rejected’ status
IP-Info got a ‘different’ IP location than I see from the IP in two cases. I believe that’s due to the individual trying to juke the system and being caught anyway, but it needs some debugging.

Oh and clearly I have some optimization I could stand to work on, but that’s for another day.

This code is live, in production, and has been merrily blocking Bertie for some time.

2 February, 2023

Stopping Jerks in Ninja Forms
I don’t have a spam problem, I have a jerky people problem. I have people who, no matter how many times I explain I cannot help them, or I don’t want to talk to them, will continue to email.

Right now, I have some absolute weirdo in Europe who emails me every day via a contact form. I don’t know what the heck he’s thinking, but I do not need advice about how to live my life nor can I help him talk to a celebrity. The problem though is I can’t delete the form. I can (and did) set his email to auto-bin via my mail server, but he still fills the form in and I am just tired of cleaning this up.

This site happens to use Ninja Forms, and really what I want to do is auto-cycle his emails to the bin so he can rant all he wants and never knows I don’t see a thing.

(Note: This is not the same person as my serial harasser.)

Warning: Their Documentation is Rough

The biggest headache to all this is the fact that Ninja Forms’ documentation kinda sucks. For example, you cannot search their ‘codex‘! That’s just basic level for a documentation service, and on top of that if you try googling, it wants to send you to the non-developer pages.

Now to their credit they know this:

Admin note: we have not been able to give this site the attention it needs or deserves for a while. Most of the Codex documentation is still applicable, but please be aware that you will find some outdated material here that will need to be adapted for Ninja Forms in its current, more modern, state.

But that doesn’t make it really any better for me today, and it’s been like that for a while.

Which means thinking “I can search for how to auto-flag a submission as spam/trash!” is impossible. It doesn’t work, it doesn’t exist in current NF format, and it’s a pain to the point that I seriously considered dumping the whole plugin over this!

Folks. I know documentation is incredibly hard, but if you want people to make plugins to extend yours, and thus help make you even more popular, hire someone to do this. It’s only gonna get harder as time goes on.

The Initial Code

The first step is, of course, can I even do this, and of course I can:
```
<?php
/**
 * Prevent anyone from my blocklist from spamming me.
 */

// Exit if accessed directly.
defined( 'ABSPATH' ) || exit;


class FLF_NinjaForms {

	/**
	 * List of disallowed emails
	 *
	 * We omit anything that isn't an email address or has an @ in the string.
	 *
	 * @return array the list
	 */
	public static function list() {
		$disallowed_emails = array();
		$disallowed_array  = explode( "\n", get_option( 'disallowed_keys' ) );

		// Make a list of spammer emails and domains.
		foreach ( $disallowed_array as $spammer ) {
			if ( is_email( $spammer ) || ( strpos( $spammer, '@' ) !== false ) ) {
				// Anything with an @-symbol is probably an email, so let's trust it.
				$disallowed_emails[] = trim( $spammer );
			}
		}

		return $disallowed_emails;
	}

	/**
	 * On load.
	 */
	public function __construct() {
		add_filter( 'ninja_forms_submit_data', array( $this, 'comment_blocklist' ) );
	}

	/**
	 * Ninja Forms: Server side email protection using WordPress comment blocklist
	 * https://developer.ninjaforms.com/codex/custom-server-side-validation
	 *
	 * @param array $form_data Form data array.
	 * @return array $form_data email checked form data array.
	 */
	public function comment_blocklist( $form_data ) {
		$disallowed = self::list();

		foreach ( $form_data['fields'] as $field ) {
			// If this is email, we will do some playing.
			if ( 'email' === $field['key'] ) {
				$email_address = sanitize_email( strtolower( $field['value'] ) );

				// Break apart email into parts
				$emailparts = explode( '@', $email_address );
				$username   = $emailparts[0];       // i.e. foobar
				$domain     = '@' . $emailparts[1]; // i.e. @example.com

				// Remove all periods (i.e. foo.bar > foobar )
				$clean_username = str_replace( '.', '', $username );

				// Remove everything AFTER a + sign (i.e. foobar+spamavoid > foobar )
				$clean_username = ( false !== strpos( $clean_username, '+' ) ) ? strstr( $clean_username, '+', true ) : $clean_username;

				// rebuild email now that it's clean.
				$email = $clean_username . '@' . $emailparts[1];

				// If the email OR the domain is an exact match in the array, then it's a spammer
				if ( in_array( $email, $disallowed, true ) || in_array( $domain, $disallowed, true ) ) {
					$form_data['errors']['fields'][ $field['id'] ] = 'Error: Invalid data.';
				}
			}
		}
		return $form_data;
	}

}

new FLF_NinjaForms();
```
This code takes the email, strips out any periods (since Google allows you to put those in anywhere in your username) and then also removes anything after a + sign (since… Google lets you add in random whatever after a + sign) and builds a sanitized email. Then it checks that email on my block list. It also checks if I banned the domain.

The Problem

The only problem?
```
				// If the email OR the domain is an exact match in the array, then it's a spammer
				if ( in_array( $email, $disallowed, true ) || in_array( $domain, $disallowed, true ) ) {
					$form_data['errors']['fields'][ $field['id'] ] = 'Error: Invalid data.';
				}
```
That tells them “Error: Invalid Data” for the email. Which will suggest to them to try something else.

I don’t want that!

So I thought what if I changed that error to this:
```
$form_data['fields']['is_spam'] = true;
```
Which sets a new field for me! Is spam.

The only problem? Well I thought I could use that with ninja_forms_after_submission to then say “Submission is in but we are going to treat it as trash and not email it.” Ever tried to look up ‘don’t email Ninja Forms’? Or any form? Yeah, all you get it help if Ninja Forms isn’t sending email.

Then I thought I could set it as actual spam, but Ninja Forms has the most useless advice:

If you’ve used all the methods above and you still receive spam submissions, maybe it’s time to change your hosting provider. Ideally, they can help you minimize spam and provide you a web application firewall to keep those spambots off your website.

My host isn’t the issue. This jerk is. And both Jetpack and Gravity Forms have an _is_spam filter/action you can hook. I find it very odd that native mark-as-spam isn’t a think in Ninja Forms, and honestly it’s putting more fuel to the ‘change tools’ fire. This is some basic stuff, ain’t it? If you can hook into Aksimet and have it catch spam, why can’t you mark as spam and send that data back to make everyone’s life better?

Is this the end?

Well. For now it is. For next it’s not. I think the real answer will be to create an action like they have for Akismet, and in there rebuild my spam tool.

Maybe as a plugin for all.

But for right now, I actually turned off the forms entirely. No more contact form. The only person really using it was that yahoo, and instead I have the email address up there for now. Likely I’ll go back to Jetpack for a while, or maybe write a whole, complex, add on plugin. Later.
25 October, 2022
CLI Like a Webhost
For the last ten years, I worked for DreamHost, which meant I had access to a lot of awesome commands that everyone ran to diagnose things.

Well now I’m gone and I’m still a webadmin for my domains. And I have, as you all know, a weird guy who keeps going after me. I also have been running fandom sites for longer than WordPress has existed. I’ve had to learn a lot of tricks to sort out ‘Is this person so-and-so again?!’

Now… I’m going to tell you a secret. You ready? Okay, most of those scripts hosts run? They’re just cleaned up shell commands you run on the server via command line (aka command line interface aka cli). And those commands? They’re actually pretty common, well known, and public.

So here are some of the ones I use and why!

Before You Begin…

I have to step back a moment.

Do you know where your log files are? DreamHost posted in their KB how you do that, but you will want to check your hosts:
- Access.log via SSH on DreamHost
- Error.log via SSH on DreamHost
There are three caveats, and I know one is weird.

1. Logs rotate

Server space matters, so logs are regularly deleted to prevent your data from killing things.

Right now I see this:
```
-rw-r--r-- 1 root      root      3.5M Sep 16 09:08 access.log
lrwxrwxrwx 1 root      root        21 Sep 16 00:49 access.log.0 -> access.log.2022-09-15
-rw-r--r-- 1 username  server    1.6M Sep 12 00:51 access.log.2022-09-11.gz
-rw-r--r-- 1 username  server    1.4M Sep 13 00:54 access.log.2022-09-12.gz
-rw-r--r-- 1 username  server    1.6M Sep 14 00:11 access.log.2022-09-13.gz
-rw-r--r-- 1 root      root      9.7M Sep 15 00:21 access.log.2022-09-14
-rw-r--r-- 1 root      root       11M Sep 16 00:49 access.log.2022-09-15
```
Tomorrow I’ll loose the 9-11 log.

2. You need to know what your logs look like

Every host tweaks the format of apache logs in a different way. You’ll see I use things like print $1 in my code, and for me I know that means “$1 is the IP address.” But that may not be what your host does.

Look at the logs:
```
192.0.114.84 - - [16/Sep/2022:00:49:05 -0700] "GET /wp-content/uploads/2019/10/Pure.jpg HTTP/1.1" 200 257552 "-" "Photon/1.0"
```
And then count things. IP is #1, URL is #7, and so on.

It can be a pain so please feel free to experiment and mess with it to get exactly what you want.

3. You may need to use http logs for everything

This is specific to DreamPRESS (the managed WP hosting) and is the weird thing, you always have to use the http folder even if you use https.

Why? Well that has to do with how the server processes traffic. DreamPress (as of the time of this post) uses Varnish to cache and Nginx as an SSL proxy. That means when you go to https://example.com the server has nginx check the HTTPS stuff and passes it to Apache, which runs HTTP. Those logs are your apache logs, not your Nginx ones.

Can you view the Nginx logs? Not at this time. Also they really are pass-throughs, so you’re not missing much. If you think you are, please open a ticket and tell them what you’re looking for in specific. Those help-desk folks are awesome, but the more clear you are about exactly what you’re looking for, the better help you get.

Okay! On with the show!

Top IPs

Sometimes your site is running super slow and you want to know “Who the heck is hitting my site so much!?”
```
awk '{ print $1}' access.log | sort | uniq -c | sort -nr | head -n 10
```
This command will list the top 10 IPs that hit your site. I find this one super helpful when used in conjunction with an IP lookup service like IPQualityScore, because it tells me sometimes “Hey, did you know Amazon’s bots are hitting the heck out of your site!?”

You can change that 10 to whatever number of top IPs you want to look for. That tends to be enough for me.

If you know you have a lot of ‘self’ lookups (like you wrote something that has your server do a thing) you’ll want to try something like this to exclude them:
```
awk '{print $1}' access.log | grep -ivE "(127.0.0.1|192.168.100.)" | sort | uniq -c | sort -rn | head -10
```
Popular pages (excluding images/css/js)

Sometimes you just want to know what pages are being hit, right?

Remember how I said you actually need to know what your log looks like? For me, $7 is the 7th ‘item’ in my access log:
```
192.0.114.84 - - [16/Sep/2022:00:49:05 -0700] "GET /wp-content/uploads/2019/10/Pure.jpg HTTP/1.1" 200 257552 "-" "Photon/1.0"
```
Counting is weird, I know, but the 7th is ‘/wp-content/uploads…’ so I know that the command has to use $7. BTW Photon there just means I use WordPress’s image stuff via Jetpack.
```
awk '{print $7}' access.log | grep -ivE '(mod_status|favico|crossdomain|alive.txt)' | grep -ivE '(.gif|.jpg|.png|.js|.css)' | \
 sed 's/\/$//g' | sort | \
 uniq -c | sort -rn | head -25
```
That returns a unique list:
```
    862 /xmlrpc.php
    539 /wp-admin/admin-ajax.php
    382 /wp-login.php
     75 /wp-cron.php?doing_wp_cron
```
And it’s not a shock those are the high hits. Nice try folks. I use things to protect me. But before we get into that…

IPs Hitting a Specific Page

Now let’s say you’re trying to figure out what numb nut is hitting a specific page on your site! For example, I have a page called “electric-boogaloo” and I’m pretty sure someone specific is hammering that page. I’ll do this:
```
awk -F'[ "]+' '$7 == "/electric-boogaloo/" { ipcount[$1]++ }
    END { for (i in ipcount) {
        printf "%15s - %d\n", i, ipcount[i] } }' access.log
```
That spits out a short list:
```
   12.34.56.789 - 3
  1.234.567.890 - 4
```
It’s okay that the command spans multiple lines. Check those IPs and you might find your culprit.

What ModSecurity Rule Hates Me

I have a love/hate relationship with ModSecurity. My first WP post (not question) in the forums was about it. It’s great and protects things, especially when you tie it into IPTables and have it auto-ban people… Until you accidentally block your co-editor-in-chief. Whoops!

For this one, you’ll need to ask the person impacted for their IPv4 address. Then you can run this:
```
zgrep --no-filename IPADDRESS error.log*|grep --color -o "\[id [^]]*\].*\[msg [^]]*\]"|sort -h|uniq -c|sort -h
```
That will loop through all the error logs (on DreamHost they’re in the same location as the access logs) and tell you what rules someone’s hitting. Then you can tweak the rules.

Of course, if you’re not the root admin, you’ll want to ping your support reps with “Hey, found this, can you help?” They usually will.

Don’t feel bad about this, and don’t blame the reps for this. ModSecurity is constantly changing, because jerks are constantly trying to screw with your site for funzies and profit (I guess). Every decent host out there is hammering the heck out of their rules constantly. They update and tweak and change. Sometimes when they do that, it reveals that a rule is too restrictive. Happens all the time.

Long Running Requests

Another cool thing is “What’s making my site slow” comes from “What processes are taking too long.”
```
awk  '{print $10,$7}' access.log | grep -ivE '(.gif|.jpg|.jpeg|.png|.css|.js)'  | awk '{secs=0.000001*$1;req=$2;printf("%.2f minutes req time for %s\n", secs / 60,req )}' | sort -rn | head -50
```
That gets me the 25 top URLs. For me it happened to list MP4s so I added that into my little exclusion list where .gif etc are listed.

Who’s Referring?

A referrer is basically asking “What site sent people here.”
```
awk '{print $11}' access.log | \
 grep -vE "(^"-"$|/www.$host|/$host)" | \
 sort | uniq -c | sort -rn | head -25
```
This one is a little weird to look at:
```
  15999 "-"
     31 "www.google.com"
      8 "example.com"
      4 "binance.com"
```
The ‘example.com’ means “People came here from here” which always confuses me. More impressive is that top one. It means “People came here directly.” Except I know I’m using Nginx as a proxy, so that’s likely to be a little wonky.

What are your favourite cli tools?

Do you have one? Drop a line in the comments! (Be wary about posting code, it can get weird in comments).
21 September, 2022

Open Dungeons

I play a D&D game. We recently transitioned into playing in person (yay vaccines!) and after much discussion, we have acquired a 3D printer.

Blame my friend Jan.

Anyway. One of the things I wanted to build, besides maybe minis, was the modular dungeon! I knew about the brilliant Devon Jones and his OpenForge dungeons, and recently he’s pushed out a version 2. This was exactly what I wanted and needed. The problem … all the tutorials were for version 1. So I had to sit down and figure out what I really needed.

What’s OpenForge?

OpenForge is an open source (heh) dungeon set for D&D which lets you build…. oh this:

Example build by the creator – Devon Jones

Originally we used images on a screen, or (badly) drawn lines on a mat. And all my D&D life, we used graph paper and ‘theatre of the mind.’ We’ve also been playing online for almost a year, and now that we’re in person, I find myself wanting minis and a set for people too SEE the evil that is in my mind.

What Connections

There are three main types of connections:

Magnets – Most common, it lets things be compatible with Dwarven Forge
OpenLock – this is a kind of lego-esque clip connection from Printable Scenery. It’s open source.
Dragonbite – Proprietary (licensed) but compatible with their system
Infinitylock – Compatible with the DungeonWorks system
Glue – … As it says.

Magnets are a little weirdly expensive, since you have to buy the magnets to go in. Glue is messy. What I wanted was OpenLock — it’s more stable and my supplies are upstairs.

Originally I was really anti-glue, but as time went on, I came to appreciate it. Especially since I can throw things in the freezer to separate them! But day to day I wanted to be able to click things together like lego and change up sets when needed.

Sets

The next bit is where I got miles of miles confused. Here are the recommended starter sets for the Stone setup. I figured stone is 90% of what I’ll be using for now (there are other places where we may be on mats, like for the beach or wood battles (yes, beware players! More adventures are coming!):

Stone Floor

16 dungeon_stone_floor.inch.E
8 dungeon_stone_floor.inch.F
2 dungeon_stone_floor.inch.R
4 dungeon_stone_floor.inch.U

Stone Walls

20 dungeon_stone_wall.inch.A
4 dungeon_stone_wall.inch.BA
8 dungeon_stone_wall.inch.G
2 dungeon_stone_wall.inch.IA
4 dungeon_stone_floor.inch.Q
4 dungeon_stone_column.inch.I.stl
8 dungeon_stone_column.inch.L.stl
4 dungeon_stone_column.inch.O.stl
4 dungeon_stone_column.inch.X.stl
4 dungeon_stone_column.inch.T.stl

If you look at that, though, you wonder what the heck E, F, R and U are. Oh and then there’s Openforge vs Triplex!? And some of those files have SIDE in there!?!

Bases

First, we’ll cover the bases. Depending on how you want to connect, depends on the base you want. The one we’re going to use is triplex – which has many openlock ports, one per edge between squares and one per square. So here’s what Triplex looks like:

The non Triplex is just a ‘topper’ which you will then glue or magnet on to your bases (hence why I came to love the glue). The Triplex comes with holes on the sides, which are for using OpenLock clips! That said, you will need to prep your printed whatnots for the clips. Which is also a pain in the ass.

This means I looked at the toppers and then I looked at the bases. There were a lot of options, and I finally settled on the one that had the least amount of work for me: OpenForge 2.0 Plain OpenLOCK Base

Related to that, the ‘side’ versions of walls are ones with click holes on the sides, as well as front/back.

OpenLock

I like this. It’s a double ended clip you stick in the holes in the above. They look like this:

Stick ’em in the hole, click and done. Now if you don’t want to use the clips, you can put magnets in some of those holes. If you’re using OpenLock, you want about 2 clips per tile you print, so 54 for the starters. For magnets you’ll need 256. Big difference!

Many people recommend the Springy OpenLock Clip by Marcel Tool, who makes a loose, medium, and firm springy version (he recommends the medium). For whatever reason, I could not get it to build properly reliably (sometimes it was okay), so I used the ones made by Jones and the official ones until I could figure out why my slice was wrong (I’m very new at this!).

Letters

Now! What the bell end does “dungeon_stone_wall.inch.A” mean?

Well after some serious digging I found the documentation about filename! [Texture]#[Shape]+[Shape Options].[Letter].[Connection system]+[Options].stl

Now… that’s the new naming convention, but as you can see on Thingiverse, the names aren’t that. But! That clued me in to the fact that those letters, A and AS and so on all relate to the OpenLock code!

This is made worse by the ThingiVerse display:

If you hover over the name, you can see the whole name but you cannot click on the small picture and see the big one. I had to turn off ‘max width’ to get the full names, and everything looks like this:

dungeon_stone_floor.inch.AS.openforge.stl
dungeon_stone_floor.inch.AS.triplex.stl

Obviously I don’t need openforge AND triplex. But that still was a pain in the butt to get the list of. And those tiny photos meant I was going mad to figure out what the hell was what Thankfully someone else felt my pain, and with that I made a key!

Keys

Filename	Size	#
AS	3″ x 1″ rectangle (wide)
E	2″ x 2″ square	16
EA	3″ x 3″ square
F	2″ x 2″ curve/V	8
I	1″ x 1″ square
R	2″ x 4″ rectangle	2
S	1″ x 2″ rectangle (tall)
SA	1″ x 3″ rectangle
SB	1″ x 4″ rectangle
U	4″ x 4″ square	4

# means the recommended number to print from the starter set

That makes a lot more sense, right? I tend to use sprawling dungeons and large rooms (they’re fighting in the castle as I started all this) it would work for me. Except for the 8 F’s. I don’t use curves like that because I’m hand-making maps and I hate curves. I can’t make straight lines.

Walls are a little weirder since a number of items have ‘side’ options and ‘pegs’ options.

Filename	Size	Side	Pegs	#
A	0.5″ x 2″	Yes	Yes	20
B	0.5″ x 1.5″	Yes	No	4
G	0.5″ x 2″ x 2″ curved corner	Yes	Yes	8
IA	0.5″ x 1″	Yes	Yes	2
Q	0.5″ x 4″	Yes	Yes	4
I	1″ x 1″	No	No	4
L	1″ x 1″	No	No	8
O	1″ x 1″	No	No	4
X	1″ x 1″	No	No	4
T	1″ x 1″	No	No	4

# means the recommended number to print from the starter set

You’ll notice new columns!

Pegs is if there’s an alt version with pegs for making a second layer (not needed for starters)
Sides is if there’s an alt version with clip slots on the sides

All columns are 1″x1″, but have pegs in different places.

What did I Print?

The weird part about this was figuring out if I really wanted Triplex (supports all three) or just plain bottoms and glue on tops… I will likely use the same floors over and over and over again. And I don’t plan on stacking layers…. Yet. Not until they have fights in a house. Otherwise I’ll pop upstairs, get the next set, and pop back down and have them scream their delight at me.

So what did I end up building?

54+ OpenLock clips — these are a mix of springy and official and Devon’s
36 2×2 bases (E) — these are a mix of topless (10) and the old plain ones (26)
16 2×2 curved bases (G) – topless
2 2×4 bases (R) — topless
8 4×4 bases (U) — topless
16 cut stone 2×2 toppers (E)
20 cut stone 2×2 walls (G)
20 cut stone 2×2 wall floor (G)
4 cut stone 4×4 toppers (U)
4 cut stone 4×4 wall floor (U)
4 cut stone 4×4 walls (U)

Example of how to do an interior hallway by **Manfred G**

You’ll notice things are more than they recommended. Once I went with toppers, while it does mean a lot of gluing, it also let me use the new Wall Floor system, which obviates the drama of a wall being a half inch. Instead of having a half-inch gap between wall and the floor behind it (like if you’re doing internal walls), you can make a wall that takes up a half inch, and then a floor for an inch and a half.

The example image I included is weird, I know, but basically those are TWO towne_wall.floor.inch.2x1, which are both 1 inch wide. Then two walls which are each a half inch for a total of three inches. Plug them onto an AS or SA (3 inches by 1 inch) and you have a hallway!

Since I’m building a specific campaign, I mathed out what I needed to mimic the boss fight. In the end, I decided to change the size a little from paper map to cardboard to real, to compensate for what I was doing.

I had to wait to post this until (a) the printer showed up and (b) I had successfully built the set. My wife knew I was doing this. The others did not. And due to the time crunch we did not paint them. Still… it worked! Here are some of the ones I’ve made (or am making) and a cardboard to compare:

So far, it’s all been well received.

19 April, 2022

Category: How To

Interlude: Gutenberg Moves Fast

JS isn’t your Momma’s PHP

Micro vs Macro / Interpreted vs Compiled

The Compilation Downfall

Enter the Console

Debugging Gutenberg

Here’s the Flaw

Okay, can we fix it?

Zaptodon

Zapier

The Zap Flow

Add an App

Back to Zapier

Done!

Stopping Jerks in Gravity Forms

What’s a “Disallowed Key”?

Jerky People Aren’t Spammers

Catch and Release

How it Works

Find Spammers

Gravity Forms Check

Any Issues?

Stopping Jerks in Ninja Forms

Warning: Their Documentation is Rough

The Initial Code

The Problem

Is this the end?

CLI Like a Webhost

Before You Begin…

1. Logs rotate

2. You need to know what your logs look like

3. You may need to use http logs for everything

Top IPs

Popular pages (excluding images/css/js)

IPs Hitting a Specific Page

What ModSecurity Rule Hates Me

Long Running Requests

Who’s Referring?

What are your favourite cli tools?