{"id":3319,"date":"2015-06-04T18:01:57","date_gmt":"2015-06-05T00:01:57","guid":{"rendered":"http:\/\/benincosa.com\/?p=3319"},"modified":"2015-06-15T12:02:40","modified_gmt":"2015-06-15T18:02:40","slug":"continuous-delivery-of-a-simple-web-application-tutorial-part-1","status":"publish","type":"post","link":"https:\/\/benincosa.com\/?p=3319","title":{"rendered":"Continuous Delivery of a Simple Web Application Tutorial &#8211; Part 1"},"content":{"rendered":"<p>This will be the first of a series of posts showing how we can do continuous delivery of a simple web application. \u00a0This will be written tutorial style to show all the different components used. \u00a0We are using Open Source tools on Cisco OpenStack private cloud, but the majority of the instructions here could be used in any cloud. \u00a0This first part in the series is going to introduce the architecture and the components.<\/p>\n<p><a href=\"http:\/\/benincosa.com\/?p=3330\">In Part 2<\/a> we configure the build environment with Ansible.<\/p>\n<p><a href=\"http:\/\/benincosa.com\/?p=3342\">In Part 3<\/a>\u00a0we configure the load balancers and the web site.<\/p>\n<p><a href=\"http:\/\/benincosa.com\/?p=3352\">In Part 4<\/a>\u00a0we configure Jenkins and put it all together.<\/p>\n<p>Code for this can be found on <a href=\"https:\/\/github.com\/vallard\/CiscoLive2015\">Github in my Cisco Live 2015 repo<\/a>.<\/p>\n<p>If you want to see the end result of what we&#8217;re building, check out this <a href=\"http:\/\/youtu.be\/RxLiPD7zfBs\">video<\/a><\/p>\n<p>&nbsp;<\/p>\n<h3>A New Startup!<\/h3>\n<p><a href=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-2.54.28-PM.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-3322\" src=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-2.54.28-PM.png\" alt=\"Screen Shot 2015-06-04 at 2.54.28 PM\" width=\"1319\" height=\"841\" \/><\/a><\/p>\n<p>Our startup is called\u00a0<a href=\"http:\/\/lawngnomed.com\">Lawn Gnomed<\/a>. \u00a0We specialize in lawn gnomes as a service. \u00a0Basically, suppose your friend has a birthday. \u00a0You engage with our website and on the day of their birthday they wake up and there are 50 lawn gnomes on their front yard with a nice banner that says: &#8220;Happy Birthday you old man!&#8221;. \u00a0We set up the gnomes, we take them down. \u00a0Just decorations.<\/p>\n<h3>The Requirements &amp; the Stack<\/h3>\n<p>We need to react fast to changes in our business model. \u00a0Right now we&#8217;re targeting birthdays, but what if we want to target pool parties? \u00a0What about updates to our site for Fathers&#8217; Day or Mothers&#8217; Day? \u00a0Instead of listening to the HiPPO (The Highest Paid Person&#8217;s opinion) we need to be able to react quicker. \u00a0Try things out fast, if we&#8217;re wrong change, and do it fast.<\/p>\n<h4>Requirements<\/h4>\n<p>1. \u00a0We have a private cloud. \u00a0We are part of a large corporation already. \u00a0This app fits under the category of &#8220;System of Innovation&#8221; as defined by Gartner. \u00a0We&#8217;re going to develop this on our Private Cloud. \u00a0In this case Cisco OpenStack Private Cloud (formerly Metacloud) fits the bill for this nicely.<\/p>\n<p>2. \u00a0Our executives think we should leverage as much internally as possible. \u00a0Our goals are to keep things all in our private data center. \u00a0Most of these tools could use services and cloud based tools instead, but there are already plenty of tutorials out there for those types of environments. \u00a0Instead, we&#8217;re going to focus on keeping everything in house.<\/p>\n<p>3. \u00a0We want to use containers for everything and keep everything ephemeral. \u00a0We should be able to spin this environment up as quickly as possible in other clouds if we decide to change. \u00a0So we are avoiding lockin as much as possible. \u00a0This may be a bad idea <a href=\"http:\/\/blog.bwhaley.com\/the-fallacy-of-lock-in\">as some argue<\/a>, but this is the choice we are going with.<\/p>\n<h4>The Stack<\/h4>\n<p>So here is our stack we&#8217;re developing with:<\/p>\n<ul>\n<li><strong>OpenStack<\/strong>. \u00a0In this example we are using Cisco OpenStack Private Cloud (formerly Metacloud) but we may instead decide that we want to do this on a different cloud platform, like Digital Ocean, AWS, or Azure.<\/li>\n<li><strong>CoreOS.<\/strong> \u00a0CoreOS is a popular lightweight operating system that works great for running containers.<\/li>\n<li><strong>Docker<\/strong>. \u00a0Our applications will be delivered and bundled in Docker Containers. \u00a0They are quick and fast.<\/li>\n<li><strong>Gitlab<\/strong>. \u00a0We are using the free open source version of what Github offers to most organizations. \u00a0We will be using Gitlab to check in all of our code.<\/li>\n<li><strong>Jenkins<\/strong>. \u00a0Our Continuous Integration service will be able to listen to Gitlab (or Github if you used that) and not only do our automated test cases when new changes are pushed, but will also be able to update our live servers.<\/li>\n<li><strong>Slack<\/strong>. \u00a0This is the only service we don&#8217;t host in house. \u00a0Slack allows our team to be alerted anytime there is a new build or if something fails.<\/li>\n<li><strong>Ansible<\/strong>. \u00a0We are using Ansible to deploy our entire environment. \u00a0Nothing should have to be done manually (where possible) if we can automate all the things. \u00a0We&#8217;ve mostly followed that in this article, but there are some hands on places that we feel are ok for now, but can automate later.<\/li>\n<\/ul>\n<p>In this series, we will not be concentrating so much on what the app does nor the database structure, but in an effort to be complete, we will add that for now we are using a simple Ruby on Rails application that uses BootStrap with a MariaDB backend.<\/p>\n<h3>The Application Stack<\/h3>\n<p>The application will be a set of scalable web services behind a pair of load balancers. \u00a0Those in turn will talk to another set of load balancers that will house our database cluster.<\/p>\n<p>The diagram below gives a high level view of our application.<\/p>\n<ul>\n<li>Blue circles represent the instances that are running in our cloud.<\/li>\n<li>Red circles represent the containers<\/li>\n<li>Green circles represent the mounted volumes that are persistent even when containers or instances go away.<\/li>\n<\/ul>\n<p>We will probably add multiple containers and volumes to each instance, but for simplicity we show it running this way.<\/p>\n<p><a href=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/LG-Application.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-3320\" src=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/LG-Application.png\" alt=\"LG-Application\" width=\"1142\" height=\"573\" \/><\/a>\u00a0We have several choices on metacloud as to where we put the components. \u00a0Cisco OpenStack Private Cloud has the concept of Availability Zones which are analogous to AWS Regions. \u00a0If we have more Metacloud has theWe could if we were to do A\/B testing put several components inside a different availability zone or a different project. \u00a0Similarly, we could put the database portion inside its own project, or separate projects depending on what types of experiments we are looking to run.<\/p>\n<p><a href=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-2.57.57-PM.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-3323\" src=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-2.57.57-PM.png\" alt=\"Screen Shot 2015-06-04 at 2.57.57 PM\" width=\"913\" height=\"398\" \/><\/a><\/p>\n<p>Diving in a little deeper we can make each service a project. \u00a0In this case the application could be a project and the database could be a separate project within each AZ.<\/p>\n<p><a href=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-3.22.14-PM.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-3324\" src=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-3.22.14-PM.png\" alt=\"Screen Shot 2015-06-04 at 3.22.14 PM\" width=\"1236\" height=\"516\" \/><\/a><\/p>\n<h4>Autoscaling the Stack<\/h4>\n<p>Cisco OpenStack Private Cloud does not come with an Autoscaling solution. \u00a0Since Ceilometer is not part of the solution today, we can&#8217;t use that to determine load. \u00a0We can, however use third party cloud management tools like those that come from Scalr or RightScale. \u00a0These communicate with Cisco OpenStack Private Cloud via the APIs as well as agents installed on the running instances.<\/p>\n<p>There is also the ability to run a poor mans autoscaling system that can be cobbled together with something like Nagios and scripts that:<\/p>\n<ol>\n<li>Add or Remove instances from a load balancer<\/li>\n<li>Monitors the CPU, memory, or other components on a system.<\/li>\n<\/ol>\n<h4>Anti-Affinity Services<\/h4>\n<p>We would like the instances to run on separate physical hosts to increase stability. \u00a0Since the major update in the April release we have that ability to add anti-affinity rules to accomplish this.<\/p>\n<pre class=\"lang:sh decode:true\">nova server-group-create \u2013policy anti-affinity group-1\r\nnova boot \u2013image IMAGE_ID \u2013flavor FLAVOR_ID \u2013hint \\\r\n\t group=SERVER_GROUP_UUID web01\r\nnova boot \u2013image IMAGE_ID \u2013flavor FLAVOR_ID \u2013hint \\\r\n\t group=SERVER_GROUP_UUID web02\r\nnova server-group-list\r\n<\/pre>\n<p>This rule will launch web01 and web02 on different physical servers. \u00a0We mention this now as we won&#8217;t be going over it in the rest of the articles.<\/p>\n<h4>Logging and Analytics<\/h4>\n<p>Something we&#8217;ll be going over in a future post (I hope!) is how to log all the activity that happens in this system. \u00a0This would include a logging system like Logstash that would consolidate every click and put it into place where we can run analytics applications. \u00a0From this we can determine what paths our users are taking when they look at our website. \u00a0We could also analyze where are users come from (geographically) and what times our web traffic gets hit the hardest.<\/p>\n<p>Cisco OpenStack Private Cloud allows us to carve up our hypervisors into aggregates. \u00a0An aggregates is a collection of nodes that may be dedicated to one or more projects. \u00a0In this case, it could be hadoop.<\/p>\n<p><a href=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-4.29.00-PM.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-3325\" src=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-4.29.00-PM.png\" alt=\"Screen Shot 2015-06-04 at 4.29.00 PM\" width=\"672\" height=\"454\" \/><\/a><\/p>\n<p>The blue arrow denotes the collection of servers we use for our analytics.<\/p>\n<h3>Continuous Delivery Environment<\/h3>\n<p>A simple view of our Continuous Delivery environment is shown below<\/p>\n<p><a href=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-4.36.37-PM.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-3326\" src=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-4.36.37-PM.png\" alt=\"Screen Shot 2015-06-04 at 4.36.37 PM\" width=\"1224\" height=\"580\" \/><\/a>Let&#8217;s go over the steps at a high level.<\/p>\n<ol>\n<li>A developer updates code and pushes it to Gitlab. \u00a0This Gitlab server is the place where all of our code resides.<\/li>\n<li>When Gitlab sees that new code has been received he notifies Jenkins. \u00a0Gitlab also notifies Slack (and thus all the slackers) that there was a code commit.<\/li>\n<li>Jenkins takes the code, merges it, and then begins the tests. \u00a0Jenkins also notifies all the slackers that this is going to happen.<\/li>\n<li>As part of the build process, Jenkins creates new instances on Cisco Openstack Private Cloud \/ Metacloud. \u00a0Here&#8217;s what the instances do when they boot up:\n<ol>\n<li>Download the code from gitlab that was just checked in.<\/li>\n<li>Perform a &#8216;Docker build&#8217; to build a new container.<\/li>\n<li>Run test cases on the container.<\/li>\n<\/ol>\n<\/li>\n<li>If the tests are successful, the container is pushed to a local Docker Registry where it is now ready for production. \u00a0Slack is notified that new containers are ready for production.<\/li>\n<li>A second Jenkins job has been configured to automatically go into each of our existing web hosts, download the new containers, and put them into production and remove the new ones. \u00a0This only happens if a new build passed.<\/li>\n<\/ol>\n<p>This whole process in my test environment takes about 5 minutes. \u00a0If we were to run further test cases it could take longer but this illustrates the job pretty quickly.<\/p>\n<h3>The Build Environment<\/h3>\n<p><a href=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-4.46.57-PM.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-3327\" src=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-4.46.57-PM.png\" alt=\"Screen Shot 2015-06-04 at 4.46.57 PM\" width=\"533\" height=\"401\" \/><\/a><\/p>\n<p>Our build environment is pretty simple. \u00a0It consists of a single instance with a mounted volume. \u00a0On this instance we are running 4 containers:<\/p>\n<ol>\n<li>NGINX. \u00a0This does our reverse proxying so that subdomains can be hit.<\/li>\n<li>Jenkins. \u00a0This is the star of our show that runs the builds and puts things into production.<\/li>\n<li>Registry. \u00a0This is a local docker registry. \u00a0We&#8217;re using the older one here.<\/li>\n<li>Gitlab. \u00a0This is where we put all our code!<\/li>\n<\/ol>\n<p>This shows the power of running containers. \u00a0Some of these services need their own databases and redis caches. \u00a0Putting that all on a single machine and coordinating dependencies is crazy. \u00a0By using containers we can pile them up where we need them.<\/p>\n<p>The other thing to note is that all of the instances we create in OpenStack are the same type. \u00a0CoreOS 633.1.0 right now.<\/p>\n<h3>Getting Started<\/h3>\n<p>The last piece of this first part is that we&#8217;ll need to gain access to our cloud. \u00a0Not just GUI access but command line access so that we can interface with the APIs.<\/p>\n<p><a href=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-4.55.51-PM.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-3328\" src=\"http:\/\/benincosa.com\/wp-content\/uploads\/2015\/06\/Screen-Shot-2015-06-04-at-4.55.51-PM.png\" alt=\"Screen Shot 2015-06-04 at 4.55.51 PM\" width=\"1670\" height=\"617\" \/><\/a><\/p>\n<p>Once you login to your project you can go to the Access &amp; Security menu and select API Access. \u00a0From there you can download the OpenStack RC file.<\/p>\n<p>Test it out with the nova commands:<\/p>\n<pre class=\"lang:sh decode:true \">$ nova list\r\n+--------------------------------------+-----------------+--------+------------+-------------+------------------------------------+\r\n| ID                                   | Name            | Status | Task State | Power State | Networks                           |\r\n+--------------------------------------+-----------------+--------+------------+-------------+------------------------------------+\r\n| ad4dfbc4-3048-4f8a-a104-6adaffbeab45 | ci              | ACTIVE | -          | Running     | trial2-3003=10.2.3.7, 38.84.67.179 |\r\n| d97f38d7-ce0d-47dd-a105-281de586e6c8 | demo-server     | ACTIVE | -          | Running     | trial2-3003=10.2.3.5, 38.84.67.185 |\r\n| 445d4040-e1bf-4af4-b2c9-4f60c5bd64a5 | load-balancer-a | ACTIVE | -          | Running     | trial2-3003=10.2.3.2, 38.84.67.175 |\r\n| 2d5c5436-6dae-4c6a-a36f-093ac853f704 | load-balancer-b | ACTIVE | -          | Running     | trial2-3003=10.2.3.3, 38.84.67.176 |\r\n| 1ae2e4bd-9936-43a9-a870-c12f465312b5 | web01           | ACTIVE | -          | Running     | trial2-3003=10.2.3.4               |\r\n| 31356f78-f068-4b3b-8c66-c1b3bdac00e7 | web02           | ACTIVE | -          | Running     | trial2-3003=10.2.3.6               |\r\n| 25c2d915-5dae-4105-af58-73189ca7748a | web03           | ACTIVE | -          | Running     | trial2-3003=10.2.3.8               |\r\n+--------------------------------------+-----------------+--------+------------+-------------+------------------------------------+<\/pre>\n<p>While you may not see all the instances that I&#8217;m showing here, you should at least see some output that shows things are successful.<\/p>\n<h3>What&#8217;s Next<\/h3>\n<p>The next sections will be more hands on. \u00a0Refer back to this section for any questions as to what the different components do. \u00a0The next section will talk about:<\/p>\n<p><a href=\"http:\/\/benincosa.com\/?p=3330\">Part 2: Setting up the Development machine. \u00a0Ansible, CoreOS, Docker, Jenkins, and all the other hotness.<\/a><\/p>\n<ul>\n<li>Getting Image for CoreOS<\/li>\n<li>Ansible Configuration<\/li>\n<li>Cloud Init to boot up our instances<\/li>\n<li>Deploying load balancers<\/li>\n<li>Deploying web servers<\/li>\n<li>Deploying Jenkins Slaves.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This will be the first of a series of posts showing how we can do continuous delivery of a simple web application. \u00a0This will be written tutorial style to show all the different components used. \u00a0We are using Open Source tools on Cisco OpenStack private cloud, but the majority of the instructions here could be&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1013,744,990,745,676],"tags":[731,1011,730,729,176,740,743],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/3319"}],"collection":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3319"}],"version-history":[{"count":5,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/3319\/revisions"}],"predecessor-version":[{"id":3378,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/3319\/revisions\/3378"}],"wp:attachment":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3319"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3319"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3319"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}