April 3, 2015
by James Fryman
Day in and day out, the team at StackStorm is building tools that take away pain from the daily lives of Operations and Developers in IT Departments everywhere. We do this by focusing our efforts on absorbing all the Glue Code, the small snippets of code that tie together tools in your organization, and relieving the developer and operator of much of the traditional management overhead associated with automations. We also need to make sure that the tools we provide around StackStorm are equally enjoyable and frictionless to use. To that end, I would like to share with you how to rapidly build integrations with our integrated development environment, st2workroom.
If you’re just getting started with StackStorm, or are curious what it is, we have a great primer on the product… Take a moment, head over there, and give it a quick read or watch. We’ll be right here when you get back.
In a nutshell, StackStorm provides Event Driven Automation. StackStorm integrates with your various tools, and manages orchestration based on events that occur in your environment.
The st2workroom is a self-contained project that has some custom additions that allow us to quickly and easily prototype infrastructures of all types and sizes. This project can be downloaded from our GitHub repository at https://github.com/StackStorm-Exchange. In this project, there are a few primary components:
Using this project, we can quickly and easily set up a fresh StackStorm server, associate our code with that node, and rapidly prototype/develop. Let’s build our first pack.
In this example, I am going to work on the trello integration with StackStorm. I have a few things already set up on my host computer. These steps really should only have to be done one time. Let’s go over them:
Now, in this example, let’s work on developing the trello pack. What we’ll want here is to set up st2workroom to allow us to develop on our local machine with all of our environment settings, but still run the code in our clean-room development environment. This consists of two steps: the first, to tell st2workroom where the pack files are located, and the second, to optionally configure any configuration for the integration itself.
Setting up the Pack File Location
In this example, I’ll be contributing to the trello repository. Let’s take advantage of our Stacks and tell the development environment where it can expect to find our trello pack files, which on my computer is located at /Users/jfryman/stackstorm/stackstorm-trello.
<
pre>
defaults: &defaults
domain: stackstorm.net
memory: 1024
cpus: 1
box: puppetlabs/ubuntu-14.04-64-puppet
st2express: <<: *defaults
hostname: st2express
sync_type: nfs
puppet:
facts:
role: st2express
mounts:
- “/opt/stackstorm/packs/trello:/Volumes/Repositories/stackstorm-trello”
Note: By default in the st2workroom, the st2 stack is loaded up (stacks/st2.yaml). If you want to create your own infrastructure definitions, simply create a new Stack, and change the environment variable stack to your newly named stack. For example, if I wanted to create a new stack called frymanio, I would create a new file stacks/frymanio.yaml, and update my environment variable. See the file STACKS.md in the root of the st2workbench repository for more information.
Setting Pack Configuration
To bootstrap StackStorm, st2workbench leverages our Puppet module. (https://github.com/StackStorm/puppet-st2). Because of this, we can leverage Hiera to set configuration variables about our packs that will be downloaded. The benefit here is that once you are done developing against this repository, the hiera data is easily portable to a production environment, reinforcing the Infrastructure as Code concept.
In the trello pack, we have determined before hand that we need two configuration variables: api_key and token. Let’s set those now.
# hieradata/workbench.yaml
st2::packs:
trello:
ensure: present
config:
api_key: ""
token: ""
Cloud Configuration
This is an optional step. However, if you find yourself wanting to develop on StackStorm, but your local computing environment is not up to the challenge? No fear, you can quickly and easily get a quick development environment setup.
Digital Ocean
To enable Digital Ocean support, you’ll need to do the following:
1) Install the Digital Ocean Vagrant plugin: vagrant plugin install vagrant-digitalocean
2) Add the following environment variables to the file .env at the root of the st2workroom directory.
# .env DO_SSH_KEY_PATH=XXX # Path to the SSH Key to be used for installation. (e.g.: ~/.ssh/id_rsa) DO_TOKEN=YYY # Digital Ocean API Token sync_type=rsync # Copies all bootstrap items to the Digital Ocean node, and provisions.
3) Make sure the attribute sync_type: nfs is removed from the file stacks/st2.yaml.
You can get your Digital Ocean API Token from https://cloud.digitalocean.com/settings/applications and generate a new token.
Proceed to Start it up!, paying specific attention to the --provisioner parameter.
AWS
To enable AWS support, you’ll need to do the following:
1) Install the AWS Vagrant Plugin: vagrant plugin install vagrant-aws
2) Download the dummy AWS box: vagrant box add dummy <a href="https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box" target="_blank">https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box</a>
3) Setup a New Keypair in AWS. Download this keypair and store it locally.
4) Add the following environmental variables to the file .env at the root of the st2workroom directory.
# .env AWS_ACCESS_KEY_ID=XXX # AWS Access Key AWS_SECRET_ACCESS_KEY=YYY # AWS Secret Access Key AWS_SSH_KEY_PATH=ZZZ # Path to the SSH Key Generated in Step #3 (e.g.: ~/.ssh/aws_ssh) AWS_KEYPAIR_NAME=NNN # Use the keypair generated in Step #3 sync_type=rsync # Copies all bootstrap items to the Digital Ocean node, and provisions.
5) Make sure the attribute sync_type: nfsis removed from the file stacks/st2.yaml.
Head to the AWS Web Console and look for the “Security Credentials” page. You may need to get in touch with your administrator for key.
Proceed to Start it up!, paying specific attention to the --provisioner parameter.
Note: By default, this provisions an Ubuntu 14.04 node in
us-west-2. If you would like to change this, set theAWS_REGIONand “AWS_AMIvariables in.envas detailed above.st2express
<h6>Start it Up!</h6>
At this point, we're all set to get started! Go ahead and spin upvia the workroom, and let's get started. You can do this with thevagrant upcommand in the root of thest2workroom` directory.
$ cd ~/stackstorm/st2workroom $ vagrant up st2express (--provisioner)
Note: If you are leveraging the DO/AWS Cloud Provisoner, take note of the provisioner parameter. You will need to supply this. Use the --provisioner digital_ocean for DO, and --provisioner aws.
If this is your first time booting, it will take a few minutes depending on the speed of your computer and internet connection. On average, first boot takes approximately ~5 minutes. During this process, StackStorm is being downloaded to the latest version, setup on your computer in the development environment, and validated. It is rare that you’ll need to re-create the st2workbench after the first boot.
Guard is our Filesystem monitor. Its job is to monitor the /opt/stackstorm directory (and children, which in this case includes our new trello pack) for filesystem changes and automatically reloads. Anytime that you save a file in your pack directory, Guard will automatically restart the appropriate components of StackStorm to have your changes live loaded.
To do this, log into your new StackStorm box, and click on guard.
$ cd ~/stackstorm/st2workroom $ vagrant ssh st2express $ cd /opt/puppet $ bundle exec guard -i -w /opt/stackstorm/packs
This will now automatically restart the appropriate component of StackStorm as you save files to the filesystem. Focus on development!
The hard work is over! We now have a full fledged development environment to work with. If you happen to be on OSX, or have the avahi tool installed on your Linux box, you can head over to https://st2express.local:9101 and gain access to the WebUI. Likewise, you now have access to the StackStorm st2express environment, and can kick off commands from there.
Head over to our documentation website at http://docs.stackstorm.com, and read up on creating Actions and Sensors. From here, you can begin creating all the integrations that you desire.
In coming articles, we’ll have tutorials and walk throughs on developing your first action, developing ActionChain workflows, developing Mistral workflows, sensor development and more! In the meantime, be sure to take a look at our existing community packs online at https://exchange.stackstorm.org for example packs.
Until next time!