UMW Blogs was feeling its age this Spring (it officially turned ten this month—crazy!) and we got a few reports from the folks at DTLT that performance was increasingly becoming an issue. Since 2014 the site had been hosted on AWS (Tim wrote-up the details of that move here) with a distributed setup storing uploads on S3, the databases on RDS, and running core files through an EC2 instance. That was a huge jump for us back then into the Cloud, and the performance jump was significant. AWS has proven quite stable over the last two years, but it’s never been cheap—running UMW Blogs back then cost $500+ a month, and I’m sure the prices haven’t dropped significantly since.
While not as active as back in the day, the consistent traffic on UMW Blogs remains fascinating to me.
One theory Tim floated around the issues we were running into this Spring was that having all the pieces separate, namely the RDS database and Ec2 instance, may be contributing to the slowdown. We couldn’t test that theory during the semester though, so UMW Blogs limped through the Spring. But this week we were able to test that theory by trying a new setup that has the site running about as fast as I can ever remember. Rather than the distributed server setup we had on AWS, this time around we brought all the pieces back together under the same roof on Digital Ocean. We setup an Ubuntu 16.04 instance and then installed a LEMP stack (Apache is replaced with Nginx). After that, we installed WordPress on the LEMP server and we had basic shell of a site to start importing the files and databases from AWS. We made sure we were running PHP 7.1 given its speed, and also installed the http accelerator Varnish. This is a very similar to the setup to what we ran for VCU’s Rampages, and given the success we’ve had there, we had a good feeling about going this route. We were right.
We spent the earlier part of the week syncing files over to the new server and making sure we had everything on the new server before shutting UMW Blogs down on Thursday and Friday to pull over the databases (UMW Blogs is sharded into 16 databases). Turns out we only needed until 5 Pm on Thursday before the site was back online.
It was a fairly smooth migration, the only real issue we ran into during the migration was figuring out why we were getting a database connection error Thursday morning after importing the WPMU_global and VIP databases to the new server. We got the same error when pointing the database from RDS at the new server, so we were stumped. Tim, wondering if it might be a PHP error, updated PHP 7 to 7.1 and that fixed it—there are few better troubleshooters I have ever met when it comes to this stuff!
The only issue we’ve had since the site has gone back online were reports of uploaded images not resolving. Turns out the plugin sending all files to S3 after the 2014 move was re-writing the URLs, what’s more they were being pulled from a directory other than blogs.dir on S3 (they were being pulled from s3://umwblogs.org/sites to be exact). Once realizing this, Tim turned the S3 plugin back on and all uploaded files worked, which was a huge relief. So, at least for right now, the only piece not on Digital Ocean are the uploaded files that remain in Amazon’s S3.
I’m sure Tim can and will fill in on any details I may have missed or misrepresented, he managed this migration like a champ—as usual. I did the early setup of the server and the LEMP environment, as well as syncing files from S3 (which turned out to be unnecessary) using S3cmd sync. Tim synced the core files, exported and imported the databases, got Varnish running smoothly (I failed on that one), and more generally cleaned up any messes I made during the initial setup.
It was a fun project, and part of that was setting UMW Blogs to ludicrous speed after a decade of service to the UMW community. Fact is, there is still a lot of amazing work on that site, and the fact that it had almost 3 million pageviews served up to 1.7 million users over the last year alone suggests it’s not dead yet. But costs and sustainability for keeping a site like UMW Blogs around is always a question, so being able to make the site significantly faster for less than half the price of the previous server infrastructure is something to smile about. Also, I get to blog about UMW Blogs again, which never sucks.
