Welcome to another edition of The Lab Report!
What a fun couple of weeks it’s been, eh?
It’s been years since I last had a website get compromised.
Legit… it might’ve been close to 10 years.
Back when I was young, dumb, and using $3 cPanel hosting with shitty and possibly-nulled plugins.
Since then I’ve upped my stack considerably.
Cloudflare, GridPane, 7G Firewall, fail2ban and NinjaFirewall as a base – along with Fortress and Virusdie for some cases.
And yet, I finally got bit in the ass by the Bricks Builder RCE vulnerability that’s been spreading like a corona.
It’s kinda my fault.
I updated all of my Bricks sites the day of the disclosure and even told my brother to update his – but then forgot about one tiny microsite 🤦♂️.
I didn’t have NinjaFirewall installed on the site either.
The next day while I was at the gym (because bad things NEVER happen when you’re at your computer), I got a notification that my server was running at max capacity.
I messaged support… of course it’s the microsite overloading the server with cron jobs.
Luckily it wasn’t stopping the server from serving pages, so it wasn’t a code red.
And within an hour, I restored a backup, updated Bricks, killed any existing cron jobs, scanned with Virusdie, and everything was back to normal.
While this incident didn’t result in anything major for me personally, it’s had me thinking a lot about website security.
I don’t like that I had a site get compromised.
🔒 Reviewing my site security stack moving forward
“How can I prevent this from happening again?“
There is no such thing as perfect security. Even the freakin’ White House gets compromised.
But you can always take steps to add one extra barrier.
Each barrier reduces the odds that an attack makes it all the way through.
I thought I had enough barriers – and I have had enough for 10 years – but this recent exploit makes for a good time to revisit.
As a non-security expert who just wants his sites to not be hacked, here’s how I’m looking at things moving forward.
1) CloudFlare
Unless you require a complicated network setup, there’s really no reason to not being using Cloudflare.
Even the free version.
It stops a ton of bad actors from reaching your server to begin with.
2) Server-Level Firewall & Hardening
Some people ask why you can’t just fire up a WordPress instance directly through Digital Ocean, Vultr etc for $5/mth.
Technically, you can, but this is where hosting management panels like GridPane add a lot of value.
You get access to a firewall and critical hardening measures tailored specifically for WordPress.
In my particular case, the only reason the vulnerability didn’t spread to every site on the server is because GridPane isolates sites from each other.
On a cheap host, the exploit has access to your entire root folder.
Yes, it still caused stress on server resources, but having to go through every single site, restoring a backup from a few days prior, and notifying each client would have been a lot more annoying.
You can see many cases of this happening in the Bricks Facebook group.
3) Application Firewall
This one tends to spark a lot of debate.
It’s true that many security plugins give a false illusion of safety because they sit on top of WordPress.
This means that if the exploit gains control of your file system, it can easily just disable your security plugin.
Not very effective is it.
However, there are options like NinjaFirewall that can be configured to sit the layer beneath WordPress.
This means it can intercept any PHP script before it reaches your site.
This is why I’ve been a huge fan of NinjaFirewall (plus, the free version has all the essential features).
Like I mentioned above, the site that got hacked wasn’t running NinjaFirewall.
I’d be so curious to know if it would’ve prevented this specific attack.
I’ve been tempted to A/B test by loading an older version of Bricks on a test server and seeing which site gets hacked, but for the fear of the unknown, I might hold off on that idea for now lol.
I asked their support if it would’ve, but the response was a little vague:
“NinjaFirewall has several rules to block PHP code injection. In addition, it would have alerted you by email that your plugin was vulnerable.”
4) Vulnerability Monitoring
Sometimes when there are very specific plugin attack vectors, there are no existing firewall rules to stop the attack.
That’s when applying the latest plugin/theme update or a manual patch becomes essential.
Of course that means you first need to be made aware of the vulnerability.
The Bricks team was quick to respond and sent out a mass email, but not everyone does this.
As mentioned by the support response, NinjaFirewall does have vulnerability alerts built-in as well.
5) Vulnerability Patching
This is where I think I could make a significant improvement to my stack.
The Bricks vulnerability was a very special case, but on average, there are hundreds of active vulnerabilities at any given time.
When I get an email notification in the middle of the night, or when I’m laying in the sand in the Algarve sippin’ on a margarita, I’m not exactly wanting to pull out my laptop and update a plugin across a number of sites.
That’s not the life I want for myself.
But as seen here, even just a few hours can make all the difference.
This is what has me intrigued by Patchstack.
The paid version monitors vulnerabilities in real-time AND automatically applies a virtual patch.
That would reduce the need for human involvement and mean no more racing to the laptop.
My only hesitation with Patchstack at this point is that the cost is not insignificant.
Yes, you can purchase it at $5/mth per site which would work for some cases, but if I’ve learned anything from my case, it’s that any single site can overload a server.
So if you’re going to protect a site, you should probably just protect ’em all.
That means you’re looking at the $89/mth for 50 sites plan for starters.
Is it worth it for the piece of mind? Yeah, probably.
But a tad harder to justify when your existing stack hasn’t had any issues for 10 years.
It’s something I’m currently strongly considering.
6) Malware Scanning
This is another area I could improve.
Sometimes a compromised site is very obvious about it, but other times you only realize it once you’ve dug deeper.
And by that point, restoring an old backup would be painful.
That’s where automated scanning can let you know if something doesn’t look right.
GridPane has Maldet and ClamAV malware scanning available, but for some reason, I’ve never installed it before.
(There are some things like this that I wish they’d just build into the UI instead of having to log in to each server via SSH).
I have a 10-license pack to Virusdie that I use for one-offs.
There are plugins that offer malware scanning as well (such as NinjaScanner), but I’m not as familiar with how effective they are.
The key is to have something that scans automatically based on a schedule.
If you’re doing a manual scan, it’s probably because it’s already too late.
All of this sounds too complicated. Wouldn’t simply using a good Managed WordPress host make more sense?
This has definitely been on my mind.
I like managing my own servers because of the flexibility and cost efficiency.
I don’t want to pay $10+/mth for every single micro site I spin up.
At the same time, you inherit a lot more risk.
And spend time & money trying to mitigate it.
If I had all my sites with a managed WordPress host like Rocket.net (who would absolutely be my first choice btw), I wouldn’t have to think about most of this.
They had the vulnerability patched within minutes.
I could also choose to bring mission-critical sites there, but I don’t want to fall out of touch with the servers that I’m still managing.
But again, still something that I’m considering.
☁️ Closing Comments
Just as a reminder, I’m not a security expert, so make sure you do your own due diligence as well.
I’m just going by what I’ve learned from reading stuff on the internet.
I know everyone’s stack is different, and some parts may not be relevant to you, but I think it’s always a good idea to make sure you have each area covered.
Or at least be aware of your weakpoints.
That’s all from me this week. Thanks for reading until the end!
If you ever want to discuss any of the topics in this newsletter, I usually post a dedicated thread each week inside of the NewPulse Labs Facebook group.
You can also find previous issues of The Lab Report over at newpulselabs.com/newsletter.
Talk soon!
