April 3, 2015
by James Fryman
Day in and day out, the team at StackStorm is building tools that take away pain from the daily lives of Operations and Developers in IT Departments everywhere. We do this by focusing our efforts on absorbing all the Glue Code, the small snippets of code that tie together tools in your organization, and relieving the developer and operator of much of the traditional management overhead associated with automations. We also need to make sure that the tools we provide around StackStorm are equally enjoyable and frictionless to use. To that end, I would like to share with you how to rapidly build integrations with our integrated development environment,
If you’re just getting started with StackStorm, or are curious what it is, we have a great primer on the product… Take a moment, head over there, and give it a quick read or watch. We’ll be right here when you get back.
In a nutshell, StackStorm provides Event Driven Automation. StackStorm integrates with your various tools, and manages orchestration based on events that occur in your environment.
The st2workroom is a self-contained project that has some custom additions that allow us to quickly and easily prototype infrastructures of all types and sizes. This project can be downloaded from our GitHub repository at https://github.com/StackStorm/st2contrib. In this project, there are a few primary components:
Using this project, we can quickly and easily set up a fresh StackStorm server, associate our code with that node, and rapidly prototype/develop. Let’s build our first pack.
In this example, I am going to work on the trello integration with StackStorm. I have a few things already set up on my host computer. These steps really should only have to be done one time. Let’s go over them:
st2incubatorprojects are downloaded to my computer: These projects are where we keep our main packs. We encourage community members to submit Pull Requests against this repository. To learn more about Pull Requests, head over to https://help.github.com/articles/fork-a-repo/.
st2workbenchproject is downloaded to my computer: This is the actual place work will be done.
Now, in this example, let’s work on developing the
trello pack. What we’ll want here is to set up
st2workroom to allow us to develop on our local machine with all of our environment settings, but still run the code in our clean-room development environment. This consists of two steps: the first, to tell
st2workroom where the pack files are located, and the second, to optionally configure any configuration for the integration itself.
Setting up the Pack File Location
In this example, I’ll be contributing to the
st2contrib repository. Let’s take advantage of our Stacks and tell the development environment where it can expect to find our
trello pack files, which on my computer is located at
stacks/st2.yamland add the file-mount to the
st2expresskey. We want to target the pack directory to
st2express: <<: *defaults hostname: st2express sync_type: nfs puppet: facts: role: st2express mounts: - “/opt/stackstorm/packs/trello:/Volumes/Repositories/stackstorm/st2contrib/packs/trello”
Note: By default in the
st2 stack is loaded up (
stacks/st2.yaml). If you want to create your own infrastructure definitions, simply create a new Stack, and change the environment variable
stack to your newly named stack. For example, if I wanted to create a new stack called
frymanio, I would create a new file
stacks/frymanio.yaml, and update my environment variable. See the file STACKS.md in the root of the
st2workbench repository for more information.
Setting Pack Configuration
To bootstrap StackStorm,
st2workbench leverages our Puppet module. (https://github.com/StackStorm/puppet-st2). Because of this, we can leverage Hiera to set configuration variables about our packs that will be downloaded. The benefit here is that once you are done developing against this repository, the hiera data is easily portable to a production environment, reinforcing the Infrastructure as Code concept.
trello pack, we have determined before hand that we need two configuration variables:
token. Let’s set those now.
hieradata/workbench.yamlfile, and add the pack configuration to the
# hieradata/workbench.yaml st2::packs: trello: ensure: present config: api_key: "" token: ""
This is an optional step. However, if you find yourself wanting to develop on StackStorm, but your local computing environment is not up to the challenge? No fear, you can quickly and easily get a quick development environment setup.
To enable Digital Ocean support, you’ll need to do the following:
1) Install the Digital Ocean Vagrant plugin: vagrant plugin install
2) Add the following environment variables to the file
.env at the root of the
# .env DO_SSH_KEY_PATH=XXX # Path to the SSH Key to be used for installation. (e.g.: ~/.ssh/id_rsa) DO_TOKEN=YYY # Digital Ocean API Token sync_type=rsync # Copies all bootstrap items to the Digital Ocean node, and provisions.
3) Make sure the attribute
sync_type: nfs is removed from the file
You can get your Digital Ocean API Token from https://cloud.digitalocean.com/settings/applications and generate a new token.
Start it up!, paying specific attention to the
To enable AWS support, you’ll need to do the following:
1) Install the AWS Vagrant Plugin:
vagrant plugin install vagrant-aws
2) Download the
dummy AWS box:
vagrant box add dummy <a href="https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box" target="_blank">https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box</a>
3) Setup a New Keypair in AWS. Download this keypair and store it locally.
4) Add the following environmental variables to the file
.env at the root of the
# .env AWS_ACCESS_KEY_ID=XXX # AWS Access Key AWS_SECRET_ACCESS_KEY=YYY # AWS Secret Access Key AWS_SSH_KEY_PATH=ZZZ # Path to the SSH Key Generated in Step #3 (e.g.: ~/.ssh/aws_ssh) AWS_KEYPAIR_NAME=NNN # Use the keypair generated in Step #3 sync_type=rsync # Copies all bootstrap items to the Digital Ocean node, and provisions.
5) Make sure the attribute
sync_type: nfsis removed from the file
Head to the AWS Web Console and look for the “Security Credentials” page. You may need to get in touch with your administrator for key.
Start it up!, paying specific attention to the
Note: By default, this provisions an Ubuntu 14.04 node in
us-west-2. If you would like to change this, set the
as detailed above.st2express
<h6>Start it Up!</h6>
At this point, we're all set to get started! Go ahead and spin up
via the workroom, and let's get started. You can do this with thevagrant up
command in the root of thest2workroom` directory.
$ cd ~/stackstorm/st2workroom $ vagrant up st2express (--provisioner
Note: If you are leveraging the DO/AWS Cloud Provisoner, take note of the provisioner parameter. You will need to supply this. Use the
--provisioner digital_ocean for DO, and
If this is your first time booting, it will take a few minutes depending on the speed of your computer and internet connection. On average, first boot takes approximately ~5 minutes. During this process, StackStorm is being downloaded to the latest version, setup on your computer in the development environment, and validated. It is rare that you’ll need to re-create the
st2workbench after the first boot.
Guard is our Filesystem monitor. Its job is to monitor the
/opt/stackstorm directory (and children, which in this case includes our new
trello pack) for filesystem changes and automatically reloads. Anytime that you save a file in your pack directory, Guard will automatically restart the appropriate components of StackStorm to have your changes live loaded.
To do this, log into your new StackStorm box, and click on
$ cd ~/stackstorm/st2workroom $ vagrant ssh st2express $ cd /opt/puppet $ bundle exec guard -i -w /opt/stackstorm/packs
This will now automatically restart the appropriate component of StackStorm as you save files to the filesystem. Focus on development!
The hard work is over! We now have a full fledged development environment to work with. If you happen to be on OSX, or have the
avahi tool installed on your Linux box, you can head over to https://st2express.local:9101 and gain access to the WebUI. Likewise, you now have access to the StackStorm
st2express environment, and can kick off commands from there.
Head over to our documentation website at http://docs.stackstorm.com, and read up on creating Actions and Sensors. From here, you can begin creating all the integrations that you desire.
In coming articles, we’ll have tutorials and walk throughs on developing your first action, developing ActionChain workflows, developing Mistral workflows, sensor development and more! In the meantime, be sure to take a look at our existing community packs online at https://github.com/StackStorm/st2contrib and https://github.com/StackStorm/st2 incubator for example packs.
Until next time!