May 27, 2015
by Lakshmi Kannan
In this blog post, we cover CI part of CICD with Docker as an example. At a high level, this blog post will demonstrate how to create a docker image and push it to a registry on every commit to a github repo. You have probably read our own James Fryman’s excellent blog post on continuous delivery with StackStorm in a more traditional environment. If not, I highly recommend reading it. Also, this recent blog post from docker is a great introduction to using docker in continuous delivery. Before diving into the details, below is some motivation for switching to container based immutable infrastructure.
As Michael Dehaan of Ansible fame notes, immutable infrastructure is the way of the future. Containers (especially, Docker) facilitate this by offering users a way to define their infrastructure through simple files (Dockerfile, for example). The role of orchestration tools changes when dealing with immutable infrastructure. A relevant quote from Michael Dehaan’s blog:
“To keep up, I view orchestration systems as needing to embrace workflow, merging with build systems, producing images, and perhaps attempting to provide those higher “cluster level” APIs prior to clouds more natively doing this.”
We agree with this, and also believe StackStorm is the right tool to do this. StackStorm provides a flexible, infrastructure as a code, approach to provide orchestration workflows. With StackStorm, you can define high level tasks as actions and glue these tasks together into a workflow. This will help avoid vendor lock-in. For example, “spin up this image in a container on a cloud host” is an abstract task that is still valid irrespective of whether you use “docker” or any other format and also whether the host is on AWS or Rackspace or Azure or Google compute. It’s a good time to checkout some of the packs like aws, azure, rackspace and softlayer in StackStorm community repo StackSTorm Exchange.
Hopefully, with the sensors, rules and actions in those packs you are convinced that StackStorm is the tool you want to invest your time in. Let’s get to how to get an image built and pushed to a registry now, shall we? To recap, here is what we want to achieve as steps:
Note that what we have above are vendor specific tasks. You’d soon realize that you can replace github with BitBucket or any preferred source control system (hosted or on premises). Similar argument can be made for Docker.
To restate the tasks in bash:
At a high level, this is probably three lines of bash script and you’d go write it. There starts the problem. Let’s say you want to handle failures on steps and do something different or notify users on slack that an image has been pushed. The three line bash code will now become bigger and unwieldy. You cannot run any of these steps in isolation. You’d have code duplication of some of these steps in every script you write. These scripts will live on multiple computers without any kind of source control. Of course, not everyone does this but you get the drift. StackStorm solves these problems elegantly. You’d write StackStorm actions for these steps (Scripts can be migrated to StackStorm actions). These steps can now be run independently. You can also organize them in their respective packs so they can be reused later. You can write a workflow that wires tasks together. This will achieve your end goal.
You are one persistent person. Thanks for sticking around and reading more. I can hear you saying this.
Let’s get to it then. Here’s the git clone action meta.
---
name: "git_simple_clone"
runner_type: "remote-shell-script"
description: "Clone a git repo"
enabled: true
entry_point: "git_simple_clone.sh"
parameters:
repo:
type: "string"
description: "Git repository to clone"
required: true
position: 0
target:
type: "string"
description: "Where to clone the repo to"
required: true
position: 1
dir:
immutable: true
default: "/home/stanley/"
sudo:
immutable: true
default: false
cmd:
immutable: true
default: ""
kwarg_op:
immutable: true
default: "--"
This meta tells us that the script git_simple_clone.sh takes in couple of required arguments namely repo and target. This action is very specific and does one thing. Once you have this action registered (actual shell script), you can run it anytime for any repo. You can think about putting this in a git pack. Note that in this example, the runner used is a remote-shell-script. This is done so the script can be run on any host. It is not necessary if you want to run this on same box as st2 action runners.
Feel free to browse the meta and the scripts for the other tasks.
Now that we have the constituent tasks in StackStorm ready, we can wire them together. Here is the action chain workflow that achieve’s our goal – Build a docker image on commit and push it to docker hub on git commit.
---
chain:
-
name: "git_clone"
ref: "cicd-docker.git_simple_clone"
params:
repo: "{{git_repo}}"
target: "{{target}}"
hosts: "localhost"
on-success: "git_checkout_branch"
on-failure: "notify_clone_error_slack"
-
name: "git_checkout_branch"
ref: "cicd-docker.git_checkout_branch"
params:
target: "{{target}}"
branch: "{{branch}}"
hosts: "localhost"
on-success: "build_docker_image"
on-failure: "notify_co_error_slack"
-
name: "build_docker_image"
ref: "cicd-docker.build_image"
params:
dockerfile_path: "{{dockerfile_path}}/"
tag: "{{docker_repo}}:{{docker_tag}}"
on-success: "push_docker_image"
on-failure: "notify_build_image_failed_slack"
-
name: "push_docker_image"
ref: "cicd-docker.push_image"
params:
repo: "{{docker_repo}}"
tag: "{{docker_tag}}"
on-success: "notify_image_pushed_slack"
on_failure: "notify_image_push_failed_slack"
-
name: "notify_image_pushed_slack"
ref: "slack.post_message"
params:
message: "Docker image pushed for `{{project}}/{{branch}}`..."
channel: "#lakshmi"
on-success: "cleanup_repo"
on-failure: "cleanup_repo"
-
name: "notify_build_image_failed_slack"
ref: "slack.post_message"
params:
message: "Docker build image failed for `{{project}}/{{branch}}`..."
channel: "#lakshmi"
on-success: "cleanup_repo"
on-failure: "cleanup_repo"
-
name: "notify_image_push_failed_slack"
ref: "slack.post_message"
params:
message: "Docker image push failed for `{{project}}/{{branch}}`..."
channel: "#lakshmi"
on-success: "cleanup_repo"
on-failure: "cleanup_repo"
-
name: "notify_clone_error_slack"
ref: "slack.post_message"
params:
message: "Clone failed for `{{project}}/{{branch}}`..."
channel: "#lakshmi"
-
name: "notify_co_error_slack"
ref: "slack.post_message"
params:
message: "Checkout failed for `{{project}}/{{branch}}`..."
channel: "#lakshmi"
on-success: "cleanup_repo"
on-failure: "cleanup_repo"
-
name: "cleanup_repo"
ref: "core.remote"
params:
cmd: "rm -rf {{target}}"
hosts: "localhost"
default: git_clone
Notice that the workflow has notification as tasks too. It uses slack pack to post notifications to Slack – our favorite communication tool. In future releases of StackStorm, we are building out notifications as first class concept for tasks so your workflow will not be as verbose.
For this blog post, we picked action chain because of it’s simplicity. You can write a mistral workflow too (See our mistral documentation for some examples.). The associated meta for this action chain allows you to run this chain with StackStorm CLI.
st2 run cicd-docker.docker_image_build_and_push \
git_repo=moc.buhtignull@tig:lakshmi-kannan/mongo-docker.git \
project=mongo-docker \
docker_tag=0.3 \
docker_repo=lakshmi/mongo-unofficial -a
You can also see the progress of the workflow using the CLI by doing an st2 execution get ${id}.
(virtualenv)/m/s/s/st2 git:master$ st2 execution get 55563b5032ed3533611a94d4 --tasks id: 55563b5032ed3533611a94d4 action.ref: cicd-docker.docker_image_build_and_push status: running start_timestamp: 2015-05-15T19:30:40.059640Z end_timestamp: None +--------------------------+-----------+---------------------+----------------------+----------------------+ | id | status | task | action | start_timestamp | +--------------------------+-----------+---------------------+----------------------+----------------------+ | 55563b5032ed3533648a9fcb | succeeded | git_clone | cicd-docker.git_simp | Fri, 15 May 2015 | | | | | le_clone | 19:30:40 UTC | | 55563b5132ed3533648a9fce | succeeded | git_checkout_branch | cicd-docker.git_chec | Fri, 15 May 2015 | | | | | kout_branch | 19:30:41 UTC | | 55563b5232ed3533648a9fd1 | succeeded | build_docker_image | cicd- | Fri, 15 May 2015 | | | | | docker.build_image | 19:30:42 UTC | | 55563b5332ed3533648a9fd4 | running | push_docker_image | cicd- | Fri, 15 May 2015 | | | | | docker.push_image | 19:30:43 UTC | +--------------------------+-----------+---------------------+----------------------+----------------------+
If you are not a CLI person and you want a shiny UI, we got you covered. Here’s some mouth watering screen shots for you:
UI showing parameters typed in.
UI showing the workflow execution in progress.
Also if you are into chatops, you can kickoff this workflow from within your chat client. We love us some slack (I meant the tool). If you are a curl person, please use httpie. Oh, I was supposed to say you can use our APIs.
There is only one step remaining – automation. I walked you through how you would decompose the goal on hand into sub-tasks and how to wire them together to achieve the goal. The goal isn’t complete until you get away from the picture and let computers do their thing. Your only missing step is a webhook that listens for github events and kicks off the docker image workflow. Here is that rule.
---
name: "cicd-docker.github_incoming"
enabled: true
description: "Webhook listening for pushes to our CI/CD Pipeline from GitHub"
trigger:
type: "core.st2.webhook"
parameters:
url: "cicd-docker/github"
criteria:
trigger.headers['X-Github-Event']:
pattern: "push"
type: "eq"
action:
ref: "cicd-docker.docker_image_build_and_push"
parameters:
project: "{{trigger.body.repository.name}}"
branch: "{{trigger.body.ref}}"
user: "{{trigger.body.pusher.name}}"
commit: "{{trigger.body.after}}"
detail_url: "{{trigger.body.compare}}"
Note: To operate docker without sudo, it is best to add the system_user in StackStorm (stanley is the default.) to be added to docker group. Also for docker push to work, your docker hub credentials should already be available for docker daemon. Otherwise, you might see the workflow hang in last step.
You have basically seen how few YAMLs can essentially replace things like Packer and Container Factory with tools you are already familiar with – some bash and some YAML.
Well, hold your horses! We’ll write a follow up blog post about how to promote these images from being dev ready to staging to production ready. Then you’ll have continuous delivery (CD) part of CICD. We are building it out ourselves so we can walk you with opinionated ways to do this. We certainly believe StackStorm can help a lot there.
We also video blogged about our openstack integrations during our automation happy hour. Our happy hours will give you a good idea about using StackStorm for some commonly used infrastructure operations automation. You can follow @Stack_Storm for our happy hour announcements. Your participation will be so vital to make us (you and us) successful.
If this blog post piqued your interests, good or bad, we believe in community participation and would request you to share feedback either via github issues or IRC. You’re welcome to open pull requests.