CI/CD server behind a firewall

The goal of this post is to show how to create a CI/CD server that sits behind a firewall.  When we do a git push then we would like a webhook to be fired off to our CI/CD server.  The CI/CD server would then run our jobs and create our artifacts.  Normally this is pretty simple when you have a server that sits at a public IP address on the internet.  But then I’d have to pay for that.  Why pay for it when I have lots of computing resources behind my corporate firewall?

DRONE

First we set up our CI/CD server.  I’ll use drone just cause I think its cool the way it works using containers to do each step of the CI/CD pipeline.

NGROK

ngrok is recommended by Github as a way to get our private server an internet address.  Create an account by going to their homepage.

We download the app and put it on our server

After signing up we get our own authtoken with ngrok.  Run the command to get it set up

Now let’s start this server.  On the free plan you get a new domain name everytime you start it so ideally we’d like to keep this session up for good.  One way we can do that is to create a service that does this.  The other way is to create a screen session.  To be quick and pedantic we will use a screen session to make this work.

Now in this screen session let’s start our service

This now shows that our public IP address is ldb27123.ngrok.io.  Cool. We’ll use this now to set up our CI/CD server as well as Github.  You can detach from the screen using the Ctl-a-d command and the connection will stay up as long as the server is up. Notice that we told this to point to port 9001 on our server.  This is the port where we will have Drone run.

Github

We now go to our project and we will register a new application (drone in this case).  To do this on Github go to settings, developer settings , and register a new OAuth application.

On the next screen you’ll be shown the variables for client ID and client Secret.  Make note of those!

Environment Variables for CI/CD server

We’re almost ready to bring up our CI/CD server.  First we will put in environment variables into the ~/.bash_profile file of the server. We define them as follows:

 

From there you need to ‘source’ the bash_profile by running

Test that its running by doing

You should see the environment variables print out.

Install Drone

To start up our Drone service on this server you’ll need to have docker-compose installed.   We follow the instructions and created a docker-compose file that looks as follows:

Now we can start with

If you have problems, check the logs of the docker containers.  It may be that the environment variables are not set correctly.

We made it so anyone can register and will need to change this to something different so only authorized users of our org can use our CI/CD server.  You can see how to do this with more settings.  Check out the documentation on Drone’s web page.

Configure Drone

Now that you are installed you should be able to navigate to the drone web interface by going to the grok page.  You’ll be redirected to your CI/CD server and be ready to accept push events.  Next up, you’ll need to configure drone on your application to do all the wonders you like it to do!  Drone, unlike Jenkins, has a paradigm where you put the configuration of the CI/CD job into the github repo.  This works great as it allows jobs to be configured individually by the different applications. I’ll probably write about that in a different post as we explore that with our team!

Notes

  • I did find I had a few errors when putting it in production.  To start out with I was using an older docker version so I got an error that was resolved with this post.
  • Because my docker images were behind a proxy I had to configure proxy settings.  This was done in the environment variable inside the .drone.yml file. See the example here.