Speeding Up WordPress

I started messing around with my WordPress by first adding in a layer of security in Adding Nginx in Front of WordPress. After putting Nginx in front of my WordPress, I decided that I would further secure it by also Building a Static WordPress. That’s great and all but maybe it was time to make Nginx give me some performance gains rather than just some security controls. That is exactly what we’re going to do in this blog post. Now that Nginx is sitting in front of WordPress, we can use it to control some of the performance aspects.

Generating a Baseline Performance Report

First thing’s first though. Let’s first get us a baseline of where the site is at and what needs work. Google’s PageSpeed is a great tool for finding out what’s slowing down your site. Below is the report for this blog.

PageSpeed mobile numbers

I guess those numbers aren’t terrible but I’m sure they could be better.

Figuring Out What to Fix

As you scroll down the report, there are a number of things to correct. An example of such things would be the Opportunities section:

Opportunities section of report

In addition, there are some diagnostic items that show up:

Example cache policy items

Fixing Some of the Items

Adding a Caching Policy

An initial first step to correct some performance issues, would be to enable caching policies on the Nginx server. Given that we’re serving mostly all static content now, there’s no need to cache any content that we serve up. Nginx is already serving static data so we don’t need to rely on a backend. Let’s modify the static path’s caching policy for clients by adding the cache-control response header:

...
         location /status {
                 return 200 "healthy\n";
         }
 
         location / {
                 try_files $uri $uri/ /index.html;
                 add_header 'Cache-Control' "public,max-age=31536000,stale-while-revalidate=360";
                 #proxy_pass https://wordpress;
                 #proxy_ssl_verify off;
                 #proxy_set_header Host blog.shellnetsecurity.com;
                 #proxy_set_header X-Forwarded-For $remote_addr;
         }
 
         location /sitemap {
                 proxy_pass https://wordpress;
                 proxy_ssl_verify off; 
                 proxy_set_header Host blog.shellnetsecurity.com;
                 proxy_set_header X-Forwarded-For $remote_addr;
         } 
...

This example configuration snippet shows that we are adding the Cache-Control response header to the requests to “/”. This means we’re doing what we planned and are only telling clients to cache data that isn’t sent to the backend WordPress server. Additional parameters that can be supplied to Cache-Control are documented here.

Enable Gzip Compression

By default, even with gzip on, Nginx will not compress all files. Let’s add some additional content to our http config block (note the additional gzip directives bolded listed below gzip on):

    http {
         proxy_set_header X-Real-IP       $proxy_protocol_addr;
         proxy_set_header X-Forwarded-For $proxy_protocol_addr;
         sendfile on;
         tcp_nopush on;
         tcp_nodelay on;
         keepalive_timeout 65;
         types_hash_max_size 2048;
         include /usr/local/nginx/conf/mime.types;
         default_type application/octet-stream;
         access_log /dev/stdout;
         error_log /dev/stdout;
         gzip on;
         gzip_vary on;
         gzip_min_length 1000;
         gzip_proxied expired no-cache no-store private auth;
         gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml;
         gzip_disable "MSIE [1-6]\.";
         resolver kube-dns.kube-system.svc.cluster.local;
 
         include /etc/nginx/sites-enabled/*;
     } 

With those changes added to our Nginx configuration, restart Nginx for the changes to take effect.

Testing Our Page Again

Now that those changes should be live in your Nginx, let’s check how we did again on PageSpeed.

Updated PageSpeed details.

The numbers aren’t amazingly stellarly awesomer but they are better. If you look at the overall scoring, we jumped from a 66 to a 72. The final problem left is not something we can correct using Nginx. There are a number of first and third party scripts that are loading and slowing the site down. Next steps will involve researching those scripts and attempting to determine if there are any that can be removed. Until next time!

Automating Static WordPress Updates

Photo by Alex Knight from StockSnap

In my previous post, Building a Static WordPress, I setup my Nginx sitting in front of WordPress to load static content from a private repo. This is great but could become tedious long term. Most notably, this becomes challenging as you begin to post more content. Each time content is posted, we need to fetch all of the update pages including category updates and any new images. Sure, we can just run a few wget commands manually and then update our repo and all is better. Just because you “can” doesn’t mean you should.

From that previous post, you’ll note that I had a bunch of unanswered questions. Some of those questions might remain unanswered. By the time you get to the end of this post, you might be able to address them yourself. I’m going to focus on automating static WordPress updates whenever a new post is published. This similar logic should be possible to replicate when it comes to needing to update static content based upon WordPress and WordPress plugin updates.

Setting Up Slack Notifications

I guess the secret is out! I’m going to be using Slack as part of this. I’m going to assume you have your own slack setup with a channel dedicated to wordpress notifications. In my case, that’s going to be called #wordpress. Given the way my site is configured, I figured Slack notifications would be the best method of triggering my automation. I’m not going to reinvent the wheel here so please refer to the wpbeginner post, How to Get Slack Notifications From Your WordPress Site, on how to configure the Slack Notifications plugin.

After walking through the wpbeginner post, you should have Slack Notifications installed, configured, and successfully tested. For the purposes of this post, you’ll want to create a Posts Published notification like below:

Building the Automation

I wanted to keep everything self contained in my Kubernetes so I decided to build a nice little Node service to do everything that I wanted. We need a service to connect to the #wordpress slack channel and watch for updates. Depending upon the update type it has received, it should commit the new article content to our private repo.

Follow the Existing Tutorial

Slack has created a Javascript related bot framework called Bolt. For this reason, I’m not going to spend too much time explaining how to build a bot when Slack already created a great tutorial, Building an app with Bolt for JavaScript. When setting up the bot, make sure you add the following permissions:

  • message.channels
  • message.im
  • message.groups
  • message.mpim

Go ahead, get familiar with Bolt. I’ll wait while you make the example bot. Once you’re done, come back and continue to the next section.

Extending the Existing Tutorial

This next section takes the above tutorial and extends that to include what we actually need for our bot to handle Post Published notifications. The first step is to generate a SSH Key that has read write to the static content repo. Remember, this was created as a private repo so we’ll need a key that has read write access and that is used by our bot. If you need a little refresher on the process of creating SSH Keys and adding them to your private repo, checkout the previous post on this topic, Creating a Private GitHub Repo. You can store the key where ever you like just keep it handy.

With the SSH Key created, you need to add nodegit to the bot project. Make sure you are in the project root of the bolt bot project you created above.

$ npm install nodegit

Next, we’ll add some variables and constants to the app. Edit your app.js and add in the following:

 const BLOG_HOSTNAME = 'blog.shellnetsecurity.com';
 const WORDPRESS_CONTAINER_URL = 'wordpress_container.default.svc.cluster.local';
 const cloneURL = "git@github.com:my_account/wordpress_content.git";
 const clonePath = "/tmp/clone"; 
 const sshEncryptedPublicKeyPath = "/opt/slackapp/testKey.pub";
 const sshEncryptedPrivateKeyPath = "/opt/slackapp/testKey";
 const sshKeyPassword = "abcd1234";
 const { exec } = require("child_process"); 
 var NodeGit = require("nodegit");
 var Repository = NodeGit.Repository;
 var Clone = NodeGit.Clone;
 const fs = require('fs'); 

You’ll see how we leverage these variables later but the below table explains how we’ll use them.

BLOG_HOSTNAMEThis should be the URL of your blog
WORDPRESS_CONTAINER_URLThis should be the kubernetes DNS hostname of your wordpress container
cloneURLThis will be the SSH link to your static content repo
clonePathThis path will be used for staging our replicated repo.
sshEncryptedPublicKeyPathPath to the location of the public SSH key you created earlier
sshEncryptedPrivateKeyPathPath to the location of the private SSH key you created earlier
sshKeyPasswordPassword for the SSH key you created earlier
explanation of added const

The other remaining items should be self explanatory so we’ll move on by adding some functions of use. We’ll start by adding a new feature to our bot with the botMessages function:

 // Listener middleware - filters out messages that have subtype 'bot_message'
 async function botMessages({ message, next }) {
   if (message.subtype && message.subtype === 'bot_message' && validBotMessageByText(message.text) === true) {
     removeRepo(clonePath, cloneURL, BLOG_HOSTNAME, WORDPRESS_CONTAINER_URL);
     await next();
   }
 } 

Since the notifications will be coming in from a bot, we check that the message contains a subtype of “bot_message”. The validBotMessageByText function is used to confirm that the message is supported by our flow:

 function validBotMessageByText(text) {
   let re = RegExp('The post .* was published'); // Test for a post scheduled message
   if(re.test(text)) {
     return true;
   }
   return false;
 } 

This is a simple function that contains a regex looking for the Post Published message. If the message is valid, then botMessage executes removeRepo:

 function removeRepo(clonePath, cloneURL, blogHostname, wpUrl) {
   // delete directory recursively
   try {
       fs.rmdirSync(clonePath, { recursive: true });
 
      console.log(`${clonePath} is deleted!`);
       this.clonePrivateSite(cloneURL, clonePath, blogHostname, wpUrl);
   } catch (err) {
       console.log(`Error while deleting ${clonePath}.`);
   }
 } 

The removeRepo function attempts to delete the clonePath directory if it exists and then runs clonePrivateSite.

 function clonePrivateSite(cloneURL, clonePath, blogHostname, wpUrl) {
     var opts = {
       fetchOpts: {
         callbacks: {
           certificateCheck: () => 0,
           credentials: function(cloneURL, userName) {
             return NodeGit.Cred.sshKeyNew(
               userName,
               sshEncryptedPublicKeyPath,
               sshEncryptedPrivateKeyPath,
               sshKeyPassword
             );
           }
         }
       }
     };
 
     Clone(cloneURL, clonePath, opts).catch(function(err) {console.log(err);}).then(function(){ this.mirrorSite(blogHostname, 'https://' + wpUrl, cloneURL, clonePath);} );
 } 

clonePrivateSite creates an options object to configure nodegit with SSH credentials created earlier. The Clone command clones the latest version of the repo to the clonePath. Next, mirrorSite is used to pull down a copy of the current dynamic WordPress site running on our backend:

 function mirrorSite(blogHostname, blogURL, cloneURL, clonePath) {
   var cmd = `wget -q --mirror --no-if-modified-since --follow-tags=a,img --no-parent --span-hosts --domains=${blogHostname} --directory-prefix=${clonePath}/html/ --header="Host: ${blogHostname}" --no-check-certificate ${blogURL}`;
   console.log('Executed Command : ' + cmd);
   var child = exec(
     cmd,
     function (error, stdout, stderr) {
       this.fixUrls(cloneURL, clonePath);
       if (error !== null) {
         console.log('exec error: ' + error);
       }
     }
   );
 } 

This is the lazy man’s method but it does work. mirrorSite is simply calling the wget command I had in the previous article and saving the output to the html directory of our clonePath. I still haven’t taken the time to fix my site so fixUrls is doing that for me:

 
 function fixUrls(cloneURL, clonePath) {
   var cmd = `find ${clonePath} -type f -print0 | xargs -0 sed -i'' -e 's/http:\\/\\/blog/https:\\/\\/blog/g'`;
   console.log('Executed Command : ' + cmd);
   var child = exec(
     cmd,
     function (error, stdout, stderr) {
       this.commitPrivateRepo(cloneURL, clonePath, 'Some Commit Message Here');
       if (error !== null) {
         console.log('exec error: ' + error);
       }
     }
   );
 } 

Because of a rip in the space time continuum, all of my wgets clone URLs like https://blog.shellnetsecurity.com so fixUrls runs find to swap all of those out to https://blog.shellnetsecurity.com. Once it completes, it calls commitPrivateRepo to commit all of our changes.

 function commitPrivateRepo(cloneURL, clonePath, commitMsg) {
     repoFolder = clonePath + '/.git';
     
     var repo, index, oid, remote;
     
     NodeGit.Repository.open(repoFolder)
       .then(function(repoResult) {
         repo = repoResult;
         return repoResult.refreshIndex();
       })
       .then(function(indexResult) {
         index  = indexResult;
         index.read(1);
         var paths = [];
         return NodeGit.Status.foreach(repo, function(path) {
           paths.push(path);
         }).then(function() {
           return Promise.resolve(paths);
         });
       })
       .then(function(paths) {
         return index.addAll(paths);
       })
       .then(function() { 
         index.write();
         return index.writeTree();
       })
       .then(function(oidResult) {
         oid = oidResult;
     
         return NodeGit.Reference.nameToId(repo, 'HEAD');
       })
       .then(function(head) {
         return repo.getCommit(head);
       })
       .then(function(parent) {
         author = NodeGit.Signature.now('Slack App', 'author@email.com');
         committer = NodeGit.Signature.now('Slack App', 'commiter@email.com');
     
         return repo.createCommit('HEAD', author, committer, commitMsg, oid, [parent]);
       })
       .then(function(commitId) {
         return console.log('New Commit: ', commitId);
       })
     
       /// PUSH
       .then(function() {
         return NodeGit.Remote.createAnonymous(repo, cloneURL)
         .then(function(remoteResult) {
           remote = remoteResult;
 
           // Create the push object for this remote
           return remote.push(
             ["refs/heads/main:refs/heads/main"],
             {
               callbacks: {
                 credentials: function(url, userName) {
                   return NodeGit.Cred.sshKeyNew(
                     userName,
                     sshEncryptedPublicKeyPath,
                     sshEncryptedPrivateKeyPath,
                     sshKeyPassword
                   );
                 }
               }
             }
           );
         });
       })
       .then(function() {
         console.log('remote Pushed!')
       })
       .catch(function(reason) {
         console.log(reason);
       })
 } 

This function goes through the clonePath directory and adds all new and changed files to the commit. After committing all changes to the local repo, it pushes those changes to the remote repo again using the SSH credentials created previously.

What’s Next?

After making all of the above changes, restart the bolt bot and do some testing. If you publish any posts, you should receive a notification to Slack and eventually an update to your repo. Also, you should be able to extend this bot to handle other types of WordPress notifications coming into Slack.

Building a Static WordPress

Photo by Vidsplay from StockSnap

Now that I have Nginx in Front of WordPress, I thought the next logic step was to try and hide my WordPress even more. What exactly would this mean? In my mind, I figured that I would restrict access to all of the backend functions of my WordPress site to just my IP Addresses. From there, I would simply serve static versions of the content.

Part of the reason that I can do this is because my site is mostly static. I don’t allow comments or other dynamic plugins. The site is only used to publish my blog posts and that’s about it. I also setup WordPress to use the permalink format of /%year%/%monthnum%/%post_id%/

First Step, Mirror the Site to a Private Repo

Just as the heading states, I needed to first get all of my content available outside of WordPress. Luckily, I realized that I had a few previous blog posts:

that could help me accomplish the initial steps. I won’t completely bore you with the details contained in these posts. I’m going to assume that you can get a basic idea of how to setup the private repo using Creating a Private GitHub Repo. You can setup your repo however you like but for future planning purposes, I decided to create a html directory inside of it to house the website files. My initial repo looked like the following:

 % ls -al
 total 8
 drwxr-xr-x   5 salgatt  staff   160 Dec 31 08:46 .
 drwxr-xr-x  49 salgatt  staff  1568 Jan  7 12:32 ..
 drwxr-xr-x  15 salgatt  staff   480 Jan  7 09:05 .git
 -rw-r--r--   1 salgatt  staff    18 Dec 30 18:57 README.md
 drwxr-xr-x   4 salgatt  staff   128 Jan  5 21:31 html 

With the private repo created, I needed to get all of my content into the repo for later use by Nginx. I just did a wget to pull only the page content down. The reason I did this is because there were a number of js and css files that are required for the admin pages and possibly for other “things” that I might not use right away:

 % cd html
 % wget --mirror --follow-tags=a,img --no-parent https://blog.shellnetsecurity.com
 --2021-01-07 16:37:24--  https://blog.shellnetsecurity.com/
 Resolving blog.shellnetsecurity.com (blog.shellnetsecurity.com)... 157.230.75.245
 Connecting to blog.shellnetsecurity.com (blog.shellnetsecurity.com)|157.230.75.245|:443... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: 17266 (17K) [text/html]
 Saving to: ‘blog.shellnetsecurity.com/index.html’
 

 blog.shellnetsecurity.com/index.html       100%[=======================================================================================>]  16.86K  --.-KB/s    in 0.09s   
...
 --2021-01-07 16:37:41--  https://blog.shellnetsecurity.com/author/salgatt/page/2/
 Connecting to blog.shellnetsecurity.com (blog.shellnetsecurity.com)|157.230.75.245|:443... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: 41746 (41K) [text/html]
 Saving to: ‘blog.shellnetsecurity.com/author/salgatt/page/2/index.html’
 

 blog.shellnetsecurity.com/author/salgatt/p 100%[=======================================================================================>]  40.77K  --.-KB/s    in 0.1s    
 

 2021-01-07 16:37:44 (398 KB/s) - ‘blog.shellnetsecurity.com/author/salgatt/page/2/index.html’ saved [41746/41746]
 

 FINISHED --2021-01-07 16:37:44--
 Total wall clock time: 19s
 Downloaded: 56 files, 2.7M in 3.4s (821 KB/s) 

My wget command runs the –mirror command to ummm mirror the site. I do the –follow-tags=a,img so that I only nab the html plus images and follow only href tags. Finally, I want to stay within my site and not download any other sites’ content by issuing –no-parent. With that, I now have a blog.shellnetsecurity.com directory in my repo’s html directory.

 % ls -al
 total 0
 drwxr-xr-x   4 salgatt  staff  128 Jan  5 21:31 .
 drwxr-xr-x   5 salgatt  staff  160 Dec 31 08:46 ..
 drwxr-xr-x  18 salgatt  staff  576 Jan  7 08:38 blog.shellnetsecurity.com 

Now, I need to get all of my static content into the repo as well. In order to do that, I just did a simple copy of the static files from my container running wordpress using kubectl cp:

 % kubectl cp -n wordpress wordpress-85589d5658-48ncz:/opt/wordpress/wp-content ./blog.shellnetsecurity.com/wp-content
 tar: Removing leading `/' from member names
 % kubectl cp -n wordpress wordpress-85589d5658-48ncz:/opt/wordpress/wp-includes ./blog.shellnetsecurity.com/wp-includes
 tar: Removing leading `/' from member names 

These copy commands grab ALL files in these two directories. The idea is that I’m grabbing the js and css for any plugins running in my WordPress and any theme related files. Since these directories contain PHP files and other files I don’t need in my static repo, I remove them with a nice little find command:

 % find blog.shellnetsecurity.com/wp-includes -type f -not -name '*.js' -not -name '*.css' -not -name '*.jpg' -not -name '*.png' -delete
 % find blog.shellnetsecurity.com/wp-content -type f -not -name '*.js' -not -name '*.css' -not -name '*.jpg' -not -name '*.png' -delete 

At this point, I now have a repo that should have all of the content ready to go. I commit all of the changes and push the changes to main.

Serve the Static Repo

Like I said before, I’m not going to clutter this post with the details that can be found in Building a Kubernetes Container That Synchs with Private Git Repo. Assuming you have this all ready to go, I’m going to cut straight to the configuration portion. I’m assuming the nginx container is mounting the private repo at /dir/wordpress_static. I am also going to build upon the nginx configmap that was created in Adding Nginx in Front of WordPress. I’m first going to change the root directory to be the static WordPress blog:

         root /dir/wordpress_static/html/blog.shellnetsecurity.com; 

I also need to change some of my original reverse proxy mappings to serve most content from static but still leave a few requests go to my WordPress

         location /status {
                 return 200 "healthy\n";
         }
 
         location / {
                 try_files $uri $uri/ /index.html;
         }
 
         location /sitemap {
                 proxy_pass https://wordpress;
                 proxy_ssl_verify off;
                 proxy_set_header Host blog.shellnetsecurity.com;
                 proxy_set_header X-Forwarded-For $remote_addr;
         }
 
         location /wp-sitemap {
                 proxy_pass https://wordpress;
                 proxy_ssl_verify off;
                 proxy_set_header Host blog.shellnetsecurity.com;
                 proxy_set_header X-Forwarded-For $remote_addr;
         }
 
         location /wp-json {
                 allow 1.1.1.1;
                 allow 2.2.2.2;
                 deny all;
                 proxy_pass https://wordpress;
                 proxy_ssl_verify off;
                 proxy_set_header Host blog.shellnetsecurity.com;
                 proxy_set_header X-Forwarded-For $remote_addr;
         }
 
         location /wp-login {
                 allow 1.1.1.1;
                 allow 2.2.2.2;
                 deny all;
                 proxy_pass https://wordpress; 
                 proxy_ssl_verify off;
                 proxy_set_header Host blog.shellnetsecurity.com;
                 proxy_set_header X-Forwarded-For $remote_addr;
         }
 
         location /admin {
                 allow 1.1.1.1;
                 allow 2.2.2.2;
                 deny all;
                 proxy_pass https://wordpress;
                 proxy_ssl_verify off;
                 proxy_set_header Host blog.shellnetsecurity.com;
                 proxy_set_header X-Forwarded-For $remote_addr;
         }
 
         location /wp-admin {
                 allow 1.1.1.1;
                 allow 2.2.2.2;
                 deny all;
                 proxy_pass https://wordpress;
                 proxy_ssl_verify off;
                 proxy_set_header Host blog.shellnetsecurity.com;
                 proxy_set_header X-Forwarded-For $remote_addr;
         }

Through some trial and error, I found that I needed to have all of the following paths allowed for my admin functionalities:

  • /wp-admin
  • /admin
  • /wp-login
  • /wp-json

Since these are required for admin functions, I have made sure to run my IP restrictions on them and only allow my addresses to access them. For now, I am managing my sitemaps from within WordPress so I also allowed requests from any clients to go directly to my WordPress server still (something I’ll correct in a future post when I talk about automation). Aside from these exceptions, I’m using try_files to find the other content. This means that requests for any other content will be sent into the root directive, aka /dir/wordpress_static/html/blog.shellnetsecurity.com, aka the private repo! Notice the trailing /index.html on the directive? That just means that I’ll serve /index.html whenever the page isn’t found.

With that, I am now serving content from my mirrored content that is running from the private repo. I can still manage my WordPress site like I normally do from the backend and generate content and make changes and life is mostly good.

I am an idiot

Yes, you don’t need to tell me this! I know there are some obvious flaws in what I’ve setup like:

  • What happens when I post a new article?!
  • What do I do when WordPress is upgraded?
  • What happens when a plugin is upgraded?
  • Do you know that doing a wget for just pages won’t download pretty little images?
  • Did you know that serving /index.html for css/jpg/png/js files is ugly?
  • This manual process is terrible!

I know! I have already begun to tackle these and I’ll have more details on that when I write my Automating Static WordPress Updates (Currently in Draft). As a sneak peak to all of this, there’s a really cool WordPress plugin that will send various notifications to Slack. Oh the fun that we will have when talking about using Slack as a message bus and writing and app and and …. ok I’ll contain my excitement for now!