ds106 Hacked and WordFence to the Rescue

If you have used WordPress for any length of time, chances are you’ve gotten hacked at least once or twice. That certainly has been my experience, and it never helps when you let old plugins and themes sit around for a while and stink up the fridge. In fact, before I went on vacation over two weeks ago I realized ds106.us was down, and that was the clue that the site had been hacked. We have rapid response scripts to lock sites down when this happens and change passwords and cordon off everything so that we can work through cleaning up any remnants. There were malicious scripts left-over in several of the theme and plugin files that I manually cleaned up, but after doing some work on ds106.us with the WP offload Media plugin, I discovered that the hack had left scripts in almost all of the nearly 100,000 posts in ds106.us archive.

Not fun, and the worst part was I discovered this while on vacation and the Reclaim Hosting crew had their hands full with a more pressing issue. When Chris suggested Wordfence as a way to get me out of Slack, I knew he was right and decided to reach out to see if they would/could clean it up given the scope. I’m glad I did because not only did I get a sense of what Wordfence can and cannot do as part of the Wordfence Care offering, but also because Gert—who works with Wordfence and was my point of contact— was amazing. Like pretty much everyone, I love good customer service. In fact, Reclaim Hosting was built on a foundation of responsive support, so it was really nice to work with a company that also values a great support experience. I mean Wordfence have become synonymous with WordPress security for years now, they’ve built an amazing niche for themselves. Even more remarkable, is despite their growth and obvious success they’ve maintained such a high-level of support for the one-off user like myself. Major kudos to those running that ship given I have some idea how hard that balance can be.

So, a couple of things I knew would be issues that Wordfence confirmed, and one I didn’t:

• Site running PHP 7.3 is a no-go
• Can’t have old WordPress core files anywhere on server
• A WordPress Multisites with more than 5 sites cannot be part of the Wordfence Care product I signed-up for

Points one and two were not a surprise, and I bit the bullet and bought the recent version of the Salutations’s Paralellus theme from ThemeForest so that I could get the site running cleanly on PHP 8. Removing the tmp folder with old core files was not an issue given it was only there when we were replacing core files after the original hack. The third point was not one I expected, but given the implications of cleaning a huge WordPress Multisite, it should not be all that surprising.

Luckily, ds106 only has about 9 sub-sites, so I temporarily “archived” (archive is the term WordPress uses for making them inaccessible to the web) a few that were tests or not in regular use. There are two sites I archived—namely the original Daily Create and the Re-Mixer—that I want to bring back online here shortly, but first I need to figure out what other sites might be a good candidate for flat-file archiving. After taking four sites offline I asked Gert if that would allow us to continue on the Wordfence Care package for the next year, and luckily it did! After that, they went ahead with the scan and clean-up of any and all offending malicious scripts. Whew!

The clean-up took several hours given the size of the database, and I’m still waiting on the post-mortem given this was all done over the weekend on Saturday and Sunday!—did I mention I love Wordfence? As of now all the offending scripts have been removed and I’ve been going through and removing out-dated plugins and themes in an attempt to avoid any re-occurrence. That said, I’m happy to remain on the Wordfence Care plan for another year to ensure all is good. Part of this is because I have an idea for a new ds106 course and you can’t start something new until the existing infrastructure is solid. Not to mention it just makes good sense to continue the clean-up and archiving of large parts of this site to future-proof its survival. I think that was the big take-away from the UMW Blogs archival project, and it’s work we should continue to push on.

It might be worth noting the thing that tipped me off there were still hacked files was browsing ds106.us on the phone, something I normally would never do. But given I was on a vacation from my problems (well, not really in the end!),  I had been testing the WP Offload Media plugin for ds106.us using the phone. When clicking a link on the site via mobile a new tab gets created that opens a crypto spam site, but this happens only on mobile devices. So it was hard to find, and probably impacted next to no one given the site is fairly dormant, that said this aggression will not stand, and hence Wordfence did the sweep and things are cleaner than a fresh “Hello World!” site.

There’s still a PHP conflict on assignments.ds106.us that needs to be resolved, and not able to get the wp-cli to play nicely to find and replace some strings to ensure all older embedded Youtube videos play, but if those are the worst of my problems right now then we’re on easy street!

Something else that might be worth noting is that ds106.us is hosted on Reclaim Cloud, which made giving Wordfence access to a single container in a clean way separate from my other environments pretty easy. We can use the collaborator tool for this, and I have to say it made giving server-level access to a container that much cleaner. That said, Wordfeence also needed a user with SSH access beyond the built-in web interface SSH client, which is possible if they share a public key. The one issue we hit is when someone shares a public key and we try and use the SSH Gate created by Reclaim Cloud there can be issues when connecting server-to-server. Taylor suggested adding the public key directly to the server environment (so not through the SSH key interface in Reclaim Cloud) and then using ssh user@ipaddress to access once the public key was added, which worked perfectly. Gert was patient with me while we worked through this, and this is very good to know for any future issues we may have. because, let’s face it, when using WordPress you’re a big target on the web in this day and age. And while preventative security is crucial, hacks will happen and response-time and effective clean-up services are increasingly becoming necessities when you host your site using WordPress.