August 14, 2015
by Joe Topjian
For the past nine months or so, some of us at Cybera have been using a system called StackStorm. StackStorm is a very powerful tool that provides a hub for building automated workflows. That’s a pretty vague description, but StackStorm’s power comes from its amorphous character.
A core feature of StackStorm is the ability to store a library of “commands”. These commands can be anything: creating a ticket in Jira, executing an action on a remote server, doing a Google search — anything. We already had our own library of everyday commands, so our first task was to port this library into StackStorm. This process felt awkward. It quickly became obvious that most of our commands were focused on single-phase information reconnaissance. StackStorm seemed to work best with multi-phase workflows. The StackStorm team was very receptive to this feedback and worked with us on some simple changes that made our library a bit less awkward to use.
Once we had our original library stored inside StackStorm, we then began exploring how we could change our library to take advantage of StackStorm’s other features. It was around this time that I came across their original ChatOps instructions.
A quick Google search shows that ChatOps is a term that came out of GitHub. It’s a methodology that enables collaborative development and troubleshooting through a “chat” medium such as IRC or Slack.
This sounded like an interesting feature to explore. Instead of having each team member running the same command in one window, getting the same output, and then discussing the interpretation of the results in another window, we could just do everything in one window. It sounded like such an obvious thing to do.
The above-mentioned instructions describe how to integrate StackStorm with Slack. Fortunately, Slack is what we use at Cybera, so the process of integration was quite easy.
Once it was set up, the benefits were immediately obvious. On the same day that integration was in place, we held our weekly team meeting for the Rapid Access Cloud. Our meetings usually involve everyone sitting around a table with their laptops. Whenever the topic of a certain project came up (how many virtual machines the project was using, if a new project had begun using the Rapid Access Cloud, etc), someone would run a command in Slack that would print the report for all of us to see.
Instead of:
“Can someone lookup how many instances that project is using?”
pause
“It looks like they’re using three.”
“How big are the instances?”
pause
wash, rinse, repeat.
There was now:
“So as everyone can see in Slack, that project is using three instances.”
ChatOps integration was the key to our library modifications. It allowed us to see how our original monolithic reports could be broken down into smaller atomic pieces. These pieces are then mixed and matched like LEGOs, building multi-phase workflows that either help us collaborate in Slack or do some behind-the-scenes automations.
LEGO-building may be the best way to describe how we’re currently using StackStorm. StackStorm provides a repository of community-contributed packs. By using these packs in conjunction with our own in-house Cybera-specific pack, we can build different workflows and actions for our different projects.
“Workflows” has been mentioned several times already and it deserves more explanation. A great, in-depth article on workflows can be found here. Basically, a workflow can be thought of as a multi-step process. We’ve only scratched the surface of using workflows, but can already see their power.
As an example, we have a command that will generate a report of a project’s usage in the Rapid Access Cloud. This command accepts a project in its unique ID form.
Projects are internally referenced by a unique ID such as “9aa5f9f66b4b417d84d778a23acdf45b”, as well as a common name like “jtopjian”. When referring to projects in conversation, it’s easier to use the latter form. However, for automated processes, the unique ID is used. This is because a project can sometimes have special characters in the common name, or even the same common name as another project. The unique ID will always have alpha-numeric characters and be guaranteed to be unique.
So if I want to run a report of the “jtopjian” project, I first need to look up the unique ID and use that to run the report command. Why not just combine the two steps into a workflow?
Step 1: Take a project’s common name as an input and output the unique ID.
Step 2: Take the unique ID from step 1 as input, run the reporting command, and print the result as output.
Even more beneficial is that Step 1 is written in Ruby, our common language for internal tools. The reporting tool is written in Golang (as an exercise to further explore this language). So pieces that make up workflows can be completely unrelated.
StackStorm has been a very exciting tool for us. It’s enabled us to discover new ways of collaborating as well as building automated workflows for our environments. As we continue to use it every day, we look forward to the new discoveries we’ll make.
The Return Of Workflows
So, What is ChatOps? And How do I Get Started?
ChatOps at GitHub
StackStorm And “ChatOps For Dummies”
Enhanced ChatOps From StackStorm
Integrating ChatOps With StackStorm
Ansible and ChatOps. Get started