Testing Out the Digital Ocean Container Registry

Disclosure: I have included some affiliate / referral links in this post. There’s no cost to you for accessing these links but I do indeed receive some incentive for it if you buy through them.

Photo by Guillaume Bolduc from StockSnap

The house use to be full of random computers and networking gear but I’ve reduced the home presence over the years. I’ve messed with a number of cloud providers both inexpensive and expensive. My base for the majority of my toys reside in Digital Ocean. I’ve really liked what they’ve done over the years. Recently, they announced a Container Registry. If you follow this blog, then you remember my post, Posting a Custom Image to Docker Hub. In that post, I explained how to build an image and push it up Docker Hub. Some images might not need to be public for whatever the reason. Needless to say, Digital Ocean’s Container Registry announcement, intrigued me. With the move to WordPress, I figured that I should also build a custom nginx build to run in my Kubernetes cluster on Digital Ocean.

Building the Custom Nginx

This part was pretty easy. I simply created a Dockerfile for the build.

FROM ubuntu

ENV DEBIAN_FRONTEND noninteractive

MAINTAINER Scott Algatt

RUN apt-get update \
    && apt-get install -y libjansson-dev libcurl4-openssl-dev libapr1-dev libaprutil1-dev libssl-dev build-essential devscripts libtool m4 automake pkg-config libpcre3-dev zlib1g-dev\
    && apt -y upgrade \
    && apt -y autoremove \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
    && curl -o /tmp/nginx.tgz http://nginx.org/download/nginx-1.18.0.tar.gz

WORKDIR /tmp

RUN tar zxf nginx.tgz \
    && cd nginx-1.18.0 \
    && ./configure --with-http_realip_module\
    && make \
    && make install

EXPOSE 80
CMD ["/usr/local/nginx/sbin/nginx"]

As you can see from the Dockerfile, this is a really super simple build. It is also not very custom aside from my compile command where I’ve added –with-http_realip_module. This little addition is something that I will use later in a future post (I know everything will be in the future) but you can see what it does by visiting the nginx documentation. Anyhow, there you go. Aside from the configure command, I’m just setting up ubuntu to compile code and I download nginx and compile it. Then expose port 80 and run nginx.

Once you have created the Dockerfile, you can run a build to generate your docker image. You’ll see that my build command tags the build with a name, c-core-nginx, and specific version, 1.1. I would suggest doing this to help keep versions straight in your repository.

% docker build -t c-core-nginx:1.1 .
Sending build context to Docker daemon  21.72MB
Step 1/9 : FROM ubuntu
 ---> 4e2eef94cd6b
Step 2/9 : ENV DEBIAN_FRONTEND noninteractive
 ---> Using cache
 ---> decc285ce9e4
Step 3/9 : MAINTAINER Scott Algatt
 ---> Using cache
 ---> 197e4c81b654
Step 4/9 : RUN apt-get update     && apt-get install -y libjansson-dev libcurl4-openssl-dev libapr1-dev libaprutil1-dev libssl-dev build-essential devscripts libtool m4 automake pkg-config libpcre3-dev zlib1g-dev    && apt -y upgrade     && apt -y autoremove     && apt-get clean     && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*     && curl -o /tmp/nginx.tgz http://nginx.org/download/nginx-1.18.0.tar.gz
 ---> Using cache
 ---> d5c8a70c412f
Step 5/9 : COPY ./perimeterx-c-core /tmp/perimeterx-c-core
 ---> Using cache
 ---> d325026c19b6
Step 6/9 : WORKDIR /tmp
 ---> Using cache
 ---> 8fb23db246a3
Step 7/9 : RUN tar zxf nginx.tgz     && cd nginx-1.18.0     && ./configure --add-module=/tmp/perimeterx-c-core/modules/nginx --with-threads --with-http_realip_module    && make     && make install
 ---> Using cache
 ---> 25af69d04a9f
Step 8/9 : EXPOSE 80
 ---> Using cache
 ---> e74b4cc64160
Step 9/9 : CMD ["/usr/local/nginx/sbin/nginx"]
 ---> Using cache
 ---> 6f10e3bebefc
Successfully built 6f10e3bebefc
Successfully tagged c-core-nginx:1.1

After the build completes, you can confirm that your image is listed on your local docker repo

% docker images c-core-nginx
REPOSITORY     TAG       IMAGE ID       CREATED       SIZE
c-core-nginx   1.1       6f10e3bebefc   2 weeks ago   584MB
c-core-nginx   1.0       b3673b4bf518   2 weeks ago   584MB

Pushing Your Image to the Container Registry

I’m not going to spend a ton of effort in this section because the Digital Ocean Container Registry announcement I posted above explains the setup really well. At a high level, you simply complete the following steps:

  1. Install and configure doctl (assuming you had never done this like me)
  2. Login into your Digital Ocean account
  3. Go to the Container Registry link
  4. Create the Container Registry
  5. Login to your registry using the doctl command
  6. Push your desired container(s) to the registry

The below image shows a screenshot of my c-core-nginx images that I uploaded to my Container Registry.

Notice something really cool? The size of those images in my local registry is 584MB but they are roughly 194MB when uploaded. They are being compressed in the registry. This is a really nice feature since the initial free tier of Digital Ocean’s Container Registry is a single repo of 500MB.

In the future, you will see how I actually used this new feature for fun and zero profit.

Making the Lights Dance

My previous post, Making the Little Lights Twinkle, covered my coding of the NodeJS server that could take simply relay numbers and command (on|off) and put them to use. Now that I have a server/service up and running, it was time for me to be able to control the lights. I removed my Orchestra of Lights and put my new Raspberry Pi hardware device in its place. The only downside is that my setup has 8 outlets and the Orchestra of Lights only had 6. As I built out some of my light sequences, I noticed there’s a delay due to relays 7 and 8 being triggered but nothing being connected to them. Maybe that’s something I can do for next year. I’ll plan for 8 lighting areas instead of just 6.

In addition to needing to be able to plan for two more strands of lights, I still hadn’t figured out my audio configuration yet. I’ve only had the time to focus on the lights themselves. For the time being, I’ve got a weatherproof bluetooth speaker outside that I sync my iPad to for music. I just tell my iPad to loop through a playlist that my wonderful wife has put together. Thank you deer!

The Initial Script

While I still struggled to find some time to continue to research making the lights dance to music, I figured it was a really good idea to get something up and running. I thought randomness was key. I wanted it to be random because I was sick of the Orchestra of lights running through the same/similar light pattern over and over and over and over…..and over. With that, I built the initial script control my lights:

#!/bin/bash

COMMANDS=("off" "on")
i=0

trap exitout SIGINT

exitout() {
  echo "We Are Done Here!"
  exit
}

while :
do
  CMD=$(( ${RANDOM} % 2 ))
  RELAY=$(( ${RANDOM} % 8 + 1 ))
  curl 127.0.0.1:8080/light/$RELAY/${COMMANDS[$CMD]}
  echo $RELAY ${COMMANDS[$CMD]}
  sleep 0.2
done

Obviously, this is a really quick hack of a script but it works. It’s also obvious that it is a simple shell script. I created the array “COMMANDS” with two elements “off” and “on”. I ended up not using “i” but oh well it’s still here. I have the script setup to exit “cleanly” by calling exitout whenever someone presses CTRL+C on the keyboard. You need this because as you can tell the script is created with a while loop that never exits.

Let’s talk about what is going on inside that while loop. With each iteration, I’m setting CMD to a random number modulo 2 which gives us either 0 or 1. I’m also setting RELAY to a random number modulo 8 which gives us something 0 – 7 on each iteration. You’ll notice I add +1 so that I actually make RELAY be 1 – 8. Next, I run a curl command with my relay number and the element 0 or 1 aka off on from the COMMANDS array. From there, I just echo out what I sent to the server and then sleep for 0.2 seconds. This loop runs forever until something crashes or the user does CTRL+C.

Oh My Goodness This is Ok

The above simple script did the trick. This made my lights randomly turn on and off. I was quite happy with the results we got with this. I had light strands turning on and off. Sometimes, the lights were doing something that appeared like it was in tune with the music. There was one little flaw in this. The flaw was that because it was random, you could have the same relay being given the same command or the randomness would focus on a single light strand turning on and off.

Ultimately, my major concern was that we had too many lights out at the same time. So, this was a really good initial step and worked well for my immediate wants and needs. I wanted more. I got bored with random and put a little time into something a little more.

Building the More Interesting Client

Ok like I said before, I was getting bored with the original client script. I wanted to be able to do just a little bit more. I built a script that does a few sequences as you can see below.

#!/bin/bash

COMMANDS=("blinkUp" "blinkDown" "danceUp" "danceDown" "blinkAll" "crazyRun")
i=0

trap exitout SIGINT

exitout() {
  echo "We Are Done Here!"
  exit
}

allLights() {
  a=1
  while [ $a -le 8 ]
  do
    curl 127.0.0.1:8080/light/$a/$1
    echo ""
    a=$(( a + 1 ))
  done 
}

blinkUp() {
  echo "Blinking Up"
  allLights on
  a=1
  while [ $a -le 8 ]
  do
    curl 127.0.0.1:8080/lights/$a/off
    echo ""
    sleep 0.5
    curl 127.0.0.1:8080/lights/$a/on
    echo ""
    sleep 1
    a=$(( a + 1 ))
  done
}

blinkDown() {
  echo "Blinking Down"
  allLights on
  a=8
  while [ $a -ge 1 ]
  do
    curl 127.0.0.1:8080/lights/$a/off
    echo ""
    sleep 0.5
    curl 127.0.0.1:8080/lights/$a/on
    echo ""
    sleep 1
    a=$(( a - 1 ))
  done
}

danceUp() {
  echo "Dancing Up"
  allLights off
  a=1
  while [ $a -le 8 ]
  do
    curl 127.0.0.1:8080/lights/$a/on
    echo ""
    sleep 0.5
    curl 127.0.0.1:8080/lights/$a/off
    echo ""
    a=$(( a + 1 ))
  done
}

danceDown() {
  echo "Dancing Down"
  allLights off
  a=8
  while [ $a -ge 1 ]
  do
    curl 127.0.0.1:8080/lights/$a/on
    echo ""
    sleep 0.5
    curl 127.0.0.1:8080/lights/$a/off
    echo ""
    a=$(( a - 1 ))
  done
}

blinkAll() {
  echo "Blinking All"
  allLights on
  allLights off
  allLights on
  allLights off
  allLights on
  allLights off
}

crazyRun() {
  echo "Doing Crazy Shit"
  danceUp
  danceDown
  allLights off
  sleep 0.5
  allLights on
  sleep 0.5
  danceDown
  danceUp
  blinkAll
  danceUp
  danceDown
  danceUp
  danceDown
}

while :
do
  CMD=$(( ${RANDOM} % 6 ))
  ${COMMANDS[$CMD]}
  sleep 0.2
done

I’ve added a bunch of different functions to this new script:

allLightsThis function takes an argument of “on” or “off”. When called, it will either turn on or off all of the lights by issuing curl commands for all relays and either on or off.
blinkUpThis function starts by turning on all of the lights and then every 0.5 seconds it turns off and then on a light staying at 1 and continuing through 8.
blinkDownThis function is similar to blinkUp but it just works backwards from 8 through 1.
danceUpThis function starts by turning all of the lights off and then works its way up from 1 through to 8. It works up by turning each light on and then off.
danceDownThis function is similar to danceUp but it just works backwards starting at 8 and continuing through to 1.
blinkAllThis function just flashes all of the lights on and off.
crazyRunThis function just takes each of the above options and runs them all as a sequence that I’ve picked out.

Finally, the while statement in this script is now being used to randomly select one of the predefined sequences. So now I’ve got something a little more sophisticated to run my light show. I still want to up the game on this to sequence on its own to music and add more lights. Here’s an example below.

This got us through this Christmas season so more upgrades for next year and I can’t wait!

Making the Little Lights Twinkle

My previous post, Building the RaspberryPi Christmas Light Box, explained at a hight level building out the hardware. That step was a little scary than I think it should be but it all worked out just fine in the end. Now that I had everything put together and powered on, I was stuck here:

pi@raspberrypi:~ $ 

What are the next steps? I’ve got this box all wired up and ready to go but now I’m just sitting at a prompt waiting. As I mentioned before, I broke this down into a few parts to make my life easier and not get overwhelmed. After doing some more and more reading, I figured I had two options. I could program everything in Python or I could program everything in NodeJS.

I’m comfortable either language and no matter how hard I tried I kept going in circles. Something told me that I should write it in NodeJS because I felt that I should consider a client-server model. There were TONS of examples of people that had written all kinds of programs and libraries for handling sound and music and lights and GPIOs. I ended up throwing myself a curve ball. I decided that the client-server model was indeed what I should consider for future expansion of my new found hobby of Christmas tree lighting so I decided on NodeJS.

Well Folks, Here’s the Start of the Code

It all started pretty easy. I wanted to first make sure I had all of my GPIOs hooked up correctly. I wanted to make sure things blinked on and off like I expect. It seems that the only thing I needed to make NodeJS work was the onoff package. I popped into a directory on my Pi and installed it:

npm install onoff --save

Great! I guess the next step was to steal one of the sample JS scripts that gives you an example of how to make a relay turn on and off (my blink.js is born):

var Gpio = require('onoff').Gpio; //include onoff to interact with the GPIO
var LED = new Gpio(23, 'out'); //use GPIO pin 23, and specify that it is output
var blinkInterval = setInterval(blinkLED, 250); //run the blinkLED function every 250ms

function blinkLED() { //function to start blinking
  if (LED.readSync() === 0) { //check the pin state, if the state is 0 (or off)
    LED.writeSync(1); //set pin state to 1 (turn LED on)
  } else {
    LED.writeSync(0); //set pin state to 0 (turn LED off)
  }
}

function endBlink() { //function to stop blinking
  clearInterval(blinkInterval); // Stop blink intervals
  LED.writeSync(0); // Turn LED off
  LED.unexport(); // Unexport GPIO to free resources
}

setTimeout(endBlink, 5000); //stop blinking after 5 seconds

That seemed to work just wonderfully so I wanted to make sure that I could get all of the relays to go click click for me so enter the flowled.js (This is a great way to annoy ANYONE within ear shot. Remember from my previous post that I have the analog relays so they go *click click* when they turn on and off). While my wife was excited at the thoughts of our new Christmas lighting show, she was getting annoyed of the various clicking combinations that I came up. Thank you for your patience and appearing to be just as excited as I was deer!:

var Gpio = require('onoff').Gpio; //include onoff to interact with the GPIO
var RELAY01 = new Gpio(24, 'out'), //use declare variables for all the GPIO output pins
  RELAY02 = new Gpio(25, 'out'),
  RELAY03 = new Gpio(23, 'out'),
  RELAY04 = new Gpio(22, 'out'),
  RELAY05 = new Gpio(12, 'out'),
  RELAY06 = new Gpio(13, 'out'),
  RELAY07 = new Gpio(16, 'out'),
  RELAY08 = new Gpio(26, 'out');

//Put all the RELAY variables in an array
var leds = [RELAY01, RELAY02, RELAY03, RELAY04, RELAY05, RELAY06, RELAY07, RELAY08];
var indexCount = 0; //a counter
dir = "up"; //variable for flowing direction

var flowInterval = setInterval(flowingLeds, 100); //run the flowingLeds function every 100ms

function flowingLeds() { //function for flowing Leds
  leds.forEach(function(currentValue) { //for each item in array
    currentValue.writeSync(0); //turn off RELAY
  });
  if (indexCount == 0) dir = "up"; //set flow direction to "up" if the count reaches zero
  if (indexCount >= leds.length) dir = "down"; //set flow direction to "down" if the count reaches 7
  if (dir == "down") indexCount--; //count downwards if direction is down
  leds[indexCount].writeSync(1); //turn on RELAY that where array index matches count
  if (dir == "up") indexCount++ //count upwards if direction is up
};

function unexportOnClose() { //function to run when exiting program
  clearInterval(flowInterval); //stop flow interwal
  leds.forEach(function(currentValue) { //for each RELAY
    currentValue.writeSync(0); //turn off RELAY
    currentValue.unexport(); //unexport GPIO
  });
};

process.on('SIGINT', unexportOnClose); //function to run when user closes using ctrl+c

Time to Build the REST API!

Now that I had sufficiently annoyed everyone in the house by showing them the little LED on the relays blink and click, I think it was time to actually put this to work for me. All good client server models work via APIs, right? I had dreams of multiple Pis being setup around the yard and house being controlled by a central Pi that played the music and made all of the magical lighting happen. This will indeed be the case in a few years I’m sure. But we’re crawling before we can dead sprint. With that, I added http to my project with a little:

npm install http --save

So now I had onoff and http installed and saved to my package.json. With onoff and http ready to go, it was time for me to create the REST API server. I started with a few constants:

var Gpio = require('onoff').Gpio; //include onoff to interact with the GPIO
var RELAY1 = new Gpio(24, 'out'), //use declare variables for all the GPIO output pins
  RELAY2 = new Gpio(25, 'out'),
  RELAY3 = new Gpio(23, 'out'),
  RELAY4 = new Gpio(22, 'out'),
  RELAY5 = new Gpio(12, 'out'),
  RELAY6 = new Gpio(13, 'out'),
  RELAY7 = new Gpio(16, 'out'),
  RELAY8 = new Gpio(26, 'out');

const NUM_RELAYS = 8;
const RELAYS = [RELAY1, RELAY2, RELAY3, RELAY4, RELAY5, RELAY6, RELAY7, RELAY8];

const COMMANDS = [ 'on', 'off' ];

const PORT = process.env.PORT || 8080

Of course, the first var is to bring in the GPIO control and then I mapped the various RELAY vars to the appropriate gpios that I was using on my Pi. I have a total of 8 relays to choose from and then I also created two arrays, RELAYS and COMMANDS. These will make more sense later. Finally, I’m defining a default port for my API server to run on:

var http = require('http').createServer(handler); //require http server, and create server with function handler()

console.log(`Server Running on ${PORT}`);
http.listen(PORT);

Then we fire up the http server with my “handler” middleware. The handler function is below:

function handler (req, res) { //create server
  if (req.url.startsWith('/light') && req.method == 'GET') {
    getCommands(req.url, (err, commands) => {
      if(err) {
        var message = `{"test": "Failed","message": "${err}"}`;
      } else {
        var message = doCommand(commands.relay, commands.command);
      }
      res.statusCode = 200;
      res.setHeader('Content-Type', 'application/json');
      res.end(message);
    })
  } else if ( req.url == '/status' && req.method == 'GET' ) {
    res.statusCode = 200;
    res.setHeader('Content-Type', 'application/json');
    res.end('{"status": 200, "message": "test ok"}');
  } else {
    //Set the response HTTP header with HTTP status and Content type
    res.statusCode = 200;
    res.setHeader('Content-Type', 'application/json');
    res.end('{"status": 200, "message": "ok"}');
  }
}

I setup handler to push two routes that will accept GET requests, /light and /status. The /light route is where everything happens and /status was future proofing to make sure that we could possibly check the status of the server when we build the monolithic light show like Clark W! Of course, the final “else” is my garbage eat all where I just return “ok” to any request.

For those that don’t know, I’m a security geek so I built my getCommands function into the /light route:

function getCommands(string, cb) {

  var data = string.split('/');

  if(data.length != 4) {
    return cb('Wrong Number of Arguments Provided');
  }

  if(!COMMANDS.includes(data[3])) {
    return cb('Unsupported Command');
  }

  if(data[2] > NUM_RELAYS || data[2] < 1) {
    return cb('Sorry We Cannot Control That One');
  }

  result = {
    "relay" : data[2],
    "command" : data[3]
  }

  return cb(null, result);
}

The purpose of this function is to make sure we’re not fed garbage by anyone. I’m checking to make sure we get the right number of items in the request path (aka /light/<RELAY #>/<COMMAND>). If these fail, then I fail the request and do nothing. Assuming we pass validation, we get to the work horse, doCommand:

function doCommand(relay, command) {
  var myIndex = relay - 1;
  try {
    RELAYS[myIndex].writeSync(COMMANDS.indexOf(command));
    var message = 'OMG This is Great!';
  } catch (e) {
    var message = e;
  }
  var resp = `{"status": "Ok", "message": "${message}"}`

  return resp;
}

This function just takes the received command (on or off) and runs it against the specified relay (1 – 8). In curl the command would look a little something like this to turn on relay 4:

$ curl localhost:8080/light/4/on
{"status": "Ok", "message": "OMG This is Great!"}

To turn off the same relay, we would issue:

$ curl localhost:8080/light/4/off
{"status": "Ok", "message": "OMG This is Great!"}

Now I have a NodeJS server that can handle REST API calls to be able to turn on and off certain relays. What an accomplishment! Next post will cover how I put this all together to at least do some crappy light shows.

Building the RaspberryPi Christmas Light Box

Disclosure: I have included some affiliate / referral links in this post. There’s no cost to you for accessing these links but I do indeed receive some incentive for it if you buy through them.

Let’s Cover Some Background Here

I have always enjoyed Christmas lights. For quite some time, I was very intrigued at the notion of putting the lights to music or at least making them dance in motion. About 3 years ago, my wife and I bought a string of lights that had a mind of their own. It was neat to watch it random go through the different patterns of blinking, dancing, and chasing. Last year was the year that we got the really neat ones. They were icicles that changed colors AND danced and chased and more.

All of this was great but I wanted more this year. I wanted to be able to set the lights to music just like the fancy light shows you go to see. I figured the easiest way to do this was to just simply buy something at the store, right? So that’s what we did. We ended up buying an Orchestra of Lights (sure a plug for them yaay). The concept is really cool. We bought the speak box that comes with 6 outlets that are all controlled by the wifi hub. We’ll just say that if you’d like to buy one of these, ask me about purchasing a barely used one for a deep discount.

As part of a little additional background, I’ve always wanted to get my hands on a Pi or Arduino but I could never justify buying one. I could never figure out what legitimate project to apply such an amazing device. Enter the friend….

The Friend Made Me Do It

This is what friends are for right? As we were stringing up all of the lights outside, a friend of ours stopped by who is also a geek. I explained what we were doing and he very promptly asked if I was using a Raspberry Pi to do all of it. As the gears began to turn in my head, I know my face gave it all away. We weren’t doing it at the time, but we would be in just about a week!

The Planning Phase

I’m an over planning and over thinker so I was looking everywhere for what I needed how I needed to do it and what I should do next. My end goal was to be able build a device I could put outside that would play music and control the lights automatically to whatever little tune was playing at the time. This turned out to be a little more difficult than I bargained for but not a big deal. I looked at a bunch of sites and decided that I should probably break this project down into parts.

  1. Build out the hardware
  2. Make it do “something”
  3. Look at how I could possibly get the music to control the action

During this planning phase, I happened upon a really great article that gave amazing details on hooking up the hardware (https://system76.com/weekend-project/holiday-light-show). I mostly ignored everything but the pretty pictures. This site along with a bunch of other sites helped me put together my shopping list.

The Shopping List

Now remember that I said I had already ordered the Pi but let’s still list it here so that you know what I had coming:

This would be the very the basic shopping list. I also bought some little connectors and such so that I could conduit all of the metal pieces together. I also bought some tiny screws and bushings so that I could install the Pi and relay into the breaker box.

Putting it all Together

I lined up the Pi and Relay and marked my holes. I drilled them out. I added my bushings. I mounted everything and was quite proud of it all. Next steps were to wire everything up. Ok why reinvent the wheel here. As I noted before, I basically did exactly what was done in Steps C – K in here. This was a really great write up on how to wire everything.

After everything was done I had me a nice little system. This would be a GREAT spot to add a picture but I already have it hooked up on the porch, plugged in, and nicely hidden away so the neighbors can’t see it. I’ll remember to take some pictures next time.

Next, I’ll go through the code that I put together to get the thing clicking like crazy!

The Move to WordPress

I think I stopped blogging because I just wasn’t quite sure I liked the blogger platform. I guess WordPress is the place to be in the blogging world. After agonizing for a very long time, it seems that it was not time to make the big switch. Turns out, it wasn’t that hard at all. The best thing is that my hosting provider made it totally easy to integrate into my existing infrastructure. On top of that, WordPress has a nice little utility that can migrate your blogger blog write into WordPress. Fancy that!

In case you wanted to learn more about this migration, you can read about it on the WordPress support site here.

Now that I’m onto a platform that seems to be a little more friendly, I hope that I can find some time to write about my latest challenge, my Raspberry Pi 4!

Stay tuned for the next set of posts where I walk through my first project with a Pi in hand.