Month: February 2021

In my previous post, Building a Static WordPress, I setup my Nginx sitting in front of WordPress to load static content from a private repo. This is great but could become tedious long term. Most notably, this becomes challenging as you begin to post more content. Each time content is posted, we need to fetch all of the update pages including category updates and any new images. Sure, we can just run a few wget commands manually and then update our repo and all is better. Just because you “can” doesn’t mean you should.

From that previous post, you’ll note that I had a bunch of unanswered questions. Some of those questions might remain unanswered. By the time you get to the end of this post, you might be able to address them yourself. I’m going to focus on automating static WordPress updates whenever a new post is published. This similar logic should be possible to replicate when it comes to needing to update static content based upon WordPress and WordPress plugin updates.

Setting Up Slack Notifications

I guess the secret is out! I’m going to be using Slack as part of this. I’m going to assume you have your own slack setup with a channel dedicated to wordpress notifications. In my case, that’s going to be called #wordpress. Given the way my site is configured, I figured Slack notifications would be the best method of triggering my automation. I’m not going to reinvent the wheel here so please refer to the wpbeginner post, How to Get Slack Notifications From Your WordPress Site, on how to configure the Slack Notifications plugin.

After walking through the wpbeginner post, you should have Slack Notifications installed, configured, and successfully tested. For the purposes of this post, you’ll want to create a Posts Published notification like below:

Building the Automation

I wanted to keep everything self contained in my Kubernetes so I decided to build a nice little Node service to do everything that I wanted. We need a service to connect to the #wordpress slack channel and watch for updates. Depending upon the update type it has received, it should commit the new article content to our private repo.

Follow the Existing Tutorial

Slack has created a Javascript related bot framework called Bolt. For this reason, I’m not going to spend too much time explaining how to build a bot when Slack already created a great tutorial, Building an app with Bolt for JavaScript. When setting up the bot, make sure you add the following permissions:

• message.channels
• message.im
• message.groups
• message.mpim

Go ahead, get familiar with Bolt. I’ll wait while you make the example bot. Once you’re done, come back and continue to the next section.

Extending the Existing Tutorial

This next section takes the above tutorial and extends that to include what we actually need for our bot to handle Post Published notifications. The first step is to generate a SSH Key that has read write to the static content repo. Remember, this was created as a private repo so we’ll need a key that has read write access and that is used by our bot. If you need a little refresher on the process of creating SSH Keys and adding them to your private repo, checkout the previous post on this topic, Creating a Private GitHub Repo. You can store the key where ever you like just keep it handy.

With the SSH Key created, you need to add nodegit to the bot project. Make sure you are in the project root of the bolt bot project you created above.

$ npm install nodegit

Next, we’ll add some variables and constants to the app. Edit your app.js and add in the following:

 const BLOG_HOSTNAME = 'live-blog.shellnetsecurity.com';
 const WORDPRESS_CONTAINER_URL = 'wordpress_container.default.svc.cluster.local';
 const cloneURL = "[email protected]:my_account/wordpress_content.git";
 const clonePath = "/tmp/clone"; 
 const sshEncryptedPublicKeyPath = "/opt/slackapp/testKey.pub";
 const sshEncryptedPrivateKeyPath = "/opt/slackapp/testKey";
 const sshKeyPassword = "abcd1234";
 const { exec } = require("child_process"); 
 var NodeGit = require("nodegit");
 var Repository = NodeGit.Repository;
 var Clone = NodeGit.Clone;
 const fs = require('fs'); 

You’ll see how we leverage these variables later but the below table explains how we’ll use them.

The other remaining items should be self explanatory so we’ll move on by adding some functions of use. We’ll start by adding a new feature to our bot with the botMessages function:

 // Listener middleware - filters out messages that have subtype 'bot_message'
 async function botMessages({ message, next }) {
   if (message.subtype && message.subtype === 'bot_message' && validBotMessageByText(message.text) === true) {
     removeRepo(clonePath, cloneURL, BLOG_HOSTNAME, WORDPRESS_CONTAINER_URL);
     await next();
   }
 } 

Since the notifications will be coming in from a bot, we check that the message contains a subtype of “bot_message”. The validBotMessageByText function is used to confirm that the message is supported by our flow:

 function validBotMessageByText(text) {
   let re = RegExp('The post .* was published'); // Test for a post scheduled message
   if(re.test(text)) {
     return true;
   }
   return false;
 } 

This is a simple function that contains a regex looking for the Post Published message. If the message is valid, then botMessage executes removeRepo:

 function removeRepo(clonePath, cloneURL, blogHostname, wpUrl) {
   // delete directory recursively
   try {
       fs.rmdirSync(clonePath, { recursive: true });
 
      console.log(`${clonePath} is deleted!`);
       this.clonePrivateSite(cloneURL, clonePath, blogHostname, wpUrl);
   } catch (err) {
       console.log(`Error while deleting ${clonePath}.`);
   }
 } 

The removeRepo function attempts to delete the clonePath directory if it exists and then runs clonePrivateSite.

 function clonePrivateSite(cloneURL, clonePath, blogHostname, wpUrl) {
     var opts = {
       fetchOpts: {
         callbacks: {
           certificateCheck: () => 0,
           credentials: function(cloneURL, userName) {
             return NodeGit.Cred.sshKeyNew(
               userName,
               sshEncryptedPublicKeyPath,
               sshEncryptedPrivateKeyPath,
               sshKeyPassword
             );
           }
         }
       }
     };
 
     Clone(cloneURL, clonePath, opts).catch(function(err) {console.log(err);}).then(function(){ this.mirrorSite(blogHostname, 'https://' + wpUrl, cloneURL, clonePath);} );
 } 

clonePrivateSite creates an options object to configure nodegit with SSH credentials created earlier. The Clone command clones the latest version of the repo to the clonePath. Next, mirrorSite is used to pull down a copy of the current dynamic WordPress site running on our backend:

 function mirrorSite(blogHostname, blogURL, cloneURL, clonePath) {
   var cmd = `wget -q --mirror --no-if-modified-since --follow-tags=a,img --no-parent --span-hosts --domains=${blogHostname} --directory-prefix=${clonePath}/html/ --header="Host: ${blogHostname}" --no-check-certificate ${blogURL}`;
   console.log('Executed Command : ' + cmd);
   var child = exec(
     cmd,
     function (error, stdout, stderr) {
       this.fixUrls(cloneURL, clonePath);
       if (error !== null) {
         console.log('exec error: ' + error);
       }
     }
   );
 } 

This is the lazy man’s method but it does work. mirrorSite is simply calling the wget command I had in the previous article and saving the output to the html directory of our clonePath. I still haven’t taken the time to fix my site so fixUrls is doing that for me:

 
 function fixUrls(cloneURL, clonePath) {
   var cmd = `find ${clonePath} -type f -print0 | xargs -0 sed -i'' -e 's/https:\\/\\/blog/https:\\/\\/blog/g'`;
   console.log('Executed Command : ' + cmd);
   var child = exec(
     cmd,
     function (error, stdout, stderr) {
       this.commitPrivateRepo(cloneURL, clonePath, 'Some Commit Message Here');
       if (error !== null) {
         console.log('exec error: ' + error);
       }
     }
   );
 } 

Because of a rip in the space time continuum, all of my wgets clone URLs like https://live-blog.shellnetsecurity.com so fixUrls runs find to swap all of those out to https://live-blog.shellnetsecurity.com. Once it completes, it calls commitPrivateRepo to commit all of our changes.

 function commitPrivateRepo(cloneURL, clonePath, commitMsg) {
     repoFolder = clonePath + '/.git';
     
     var repo, index, oid, remote;
     
     NodeGit.Repository.open(repoFolder)
       .then(function(repoResult) {
         repo = repoResult;
         return repoResult.refreshIndex();
       })
       .then(function(indexResult) {
         index  = indexResult;
         index.read(1);
         var paths = [];
         return NodeGit.Status.foreach(repo, function(path) {
           paths.push(path);
         }).then(function() {
           return Promise.resolve(paths);
         });
       })
       .then(function(paths) {
         return index.addAll(paths);
       })
       .then(function() { 
         index.write();
         return index.writeTree();
       })
       .then(function(oidResult) {
         oid = oidResult;
     
         return NodeGit.Reference.nameToId(repo, 'HEAD');
       })
       .then(function(head) {
         return repo.getCommit(head);
       })
       .then(function(parent) {
         author = NodeGit.Signature.now('Slack App', '[email protected]');
         committer = NodeGit.Signature.now('Slack App', '[email protected]');
     
         return repo.createCommit('HEAD', author, committer, commitMsg, oid, [parent]);
       })
       .then(function(commitId) {
         return console.log('New Commit: ', commitId);
       })
     
       /// PUSH
       .then(function() {
         return NodeGit.Remote.createAnonymous(repo, cloneURL)
         .then(function(remoteResult) {
           remote = remoteResult;
 
           // Create the push object for this remote
           return remote.push(
             ["refs/heads/main:refs/heads/main"],
             {
               callbacks: {
                 credentials: function(url, userName) {
                   return NodeGit.Cred.sshKeyNew(
                     userName,
                     sshEncryptedPublicKeyPath,
                     sshEncryptedPrivateKeyPath,
                     sshKeyPassword
                   );
                 }
               }
             }
           );
         });
       })
       .then(function() {
         console.log('remote Pushed!')
       })
       .catch(function(reason) {
         console.log(reason);
       })
 } 

This function goes through the clonePath directory and adds all new and changed files to the commit. After committing all changes to the local repo, it pushes those changes to the remote repo again using the SSH credentials created previously.

What’s Next?

After making all of the above changes, restart the bolt bot and do some testing. If you publish any posts, you should receive a notification to Slack and eventually an update to your repo. Also, you should be able to extend this bot to handle other types of WordPress notifications coming into Slack.