July 10th, 2018
by Lindsay Hill
It has been 3 months since our last confession release. That’s why there’s a lot happening in this release. We are very excited to present the first public release of our new workflow engine, Orquesta. The Web UI has a new look & apps, and we’ve added Python 3 action support. We’ve also added metrics collection to help understand exactly what your system is doing. Read on for more!
We have been busy this year writing a new Workflow engine: Orquesta. This is a new StackStorm-native workflow engine, written by the StackStorm team. We’ve taken what we’ve learned over the years working on Mistral and ActionChains, and improved upon it. We’ve then tightly integrated it with StackStorm.
Why do this, and what does it mean for you? We’ve done it to provide a better, simpler experience for creating, running and debugging complex workflows. Orquesta runs as a StackStorm sub-component, and uses the same configurations, database and log locations as the rest of the StackStorm services. This makes deployment and troubleshooting simpler.
We can also write a new DSL, addressing gaps, and making it easier to understand. We will be able to address all Action Chain & Mistral use-cases, and more.
OK, enough talk about what & why: What does it look like? Here’s an example workflow definition:
| version: 1.0 | |
| description: > | |
| A sample workflow that demonstrates how to handle rollback and retry on error. In this example, | |
| the workflow will loop until the file /tmp/done exists. A parallel task will wait for some time | |
| before creating the file. When completed, /tmp/done will be deleted. | |
| vars: | |
| - file: /tmp/<% str(random(1000000, 9999999)) %> | |
| tasks: | |
| init: | |
| action: core.local cmd="rm -rf <% ctx().file %>" | |
| next: | |
| - when: <% succeeded() %> | |
| do: check, create | |
| check: | |
| action: core.local | |
| input: | |
| cmd: > | |
| echo 'Do something useful here.'; | |
| if [ ! -e <% ctx().file %> ]; then exit 1; fi | |
| next: | |
| - when: <% succeeded() %> | |
| do: delete | |
| - when: <% failed() %> | |
| do: rollback | |
| rollback: | |
| action: core.local | |
| input: | |
| cmd: > | |
| echo 'Rollback something here.'; | |
| sleep 1; | |
| next: | |
| - when: <% succeeded() %> | |
| do: check | |
| create: | |
| action: core.local cmd="sleep 3; touch <% ctx().file %>" | |
| delete: | |
| action: core.local cmd="rm -f <% ctx().file %>" |
Pretty easy to follow, right? Written in YAML, and some of the layout & constructs will be recognizable to Mistral users. As with all StackStorm actions, you’ll need to define the metadata. This is very similar to other actions, but with runner_type: orquesta
Check the docs to learn more, and the examples directory to see more example code.
Note: Orquesta is currently in public beta. It is stable, but is not yet feature-complete. We’re going to be adding more features in the next couple of releases. We’re aiming for GA in StackStorm 3.0. Expect to see a massively improved Workflow Designer that works with Orquesta then too.
Astute readers will be asking themselves: “What does this mean for Mistral and Action Chains?” In the short term, they are not going anywhere. But in the longer term, they will be deprecated. Don’t worry: There will be plenty of notice before this happens, and we’ll have tools to help convert your workflows.
In January we migrated our Web UI from Angular to React. We kept the look and feel the same, but promised there was a reason for doing it – for future developments. The first fruits of that are now visible:
Two things you’ll notice when you login: The colors are different, and hey, what’s this “Triggers” icon here?
This page is designed to help answer the age-old question: “Why did my rule not fire like I expected?” In earlier versions, you had to work through the troubleshooting docs to figure out what was going on, tracing trigger-instances, rules logs, etc.
Now you can do this from the Web UI. Let’s take a simple example, the sample_rule_with_webhook rule in the examples pack.
This matches the webhook URL of /webhooks/sample, and the criteria is looking for {“name”: “st2”} in the body.
First I’ll doing a test with curl:
lindsaysmacbook:~ lhill$ curl -X POST -H 'Connection: keep-alive' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'User-Agent: python-requests/2.14.2' -H 'X-Auth-Token: 8d6adce7b59c4605ac381d937d0b6e47' -k https://localhost/api/v1/webhooks/sample -H "Content-Type: application/json" --data '{"key1": "value1"}'
{
"key1": "value1"
lindsaysmacbook:~ lhill$
My rule says that it should dump the {{trigger.body}} contents to ~/st2.webhook_sample.out file, but that doesn’t exist. What’s happened?
Let’s check the Triggers app, under core.st2.webhook:
We can see “Instance has never been enforced” – that means that we saw something match this trigger, but it didn’t result in any rule enforcement. Clearly we don’t have the right combination of trigger + criteria.
Our rule criteria says this:
criteria:
trigger.body.name:
pattern: "st2"
type: "equals"
In our body we had { “key1”: “value1”}. Let’s try our curl call again:
lindsaysmacbook:~ lhill$ curl -X POST -H 'Connection: keep-alive' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'User-Agent: ython-requests/2.14.2' -H 'X-Auth-Token: 8d6adce7b59c4605ac381d937d0b6e47' -k https://localhost/api/v1/webhooks/sample -H "Content-Type: application/json" --data '{"name": "st2"}'
{
"name": "st2"
}lindsaysmacbook:~ lhill$
This looks promising:
[email protected]:~# ls -l ~stanley/
total 4
-rw-r--r-- 1 stanley stanley 18 Jun 29 01:03 st2.webhook_sample.out
[email protected]:~# cat ~stanley/st2.webhook_sample.out
{u'name': u'st2'}
[email protected]:~#
And now look at our Web UI:
We can see the rule that matched, and the Enforcement ID.
There are many other smaller changes to all aspects of the Web UI too. Layouts have been adjusted for better usability, widths changed to display more useful content, etc. Expect to see more tweaks here, and more new apps.
Finally, if you have impaired color vision, you’ll appreciate this change: better visual indications of success or failure:
Want to use Python 3 to run StackStorm actions? Now you can! Use the --python3 flag when running st2 pack install. That pack will then use python3 when creating the virtual environment.
Note: This is still an experimental, opt-in feature. It does not represent complete StackStorm support for Python 3.
We are working on full Python 3 support for the whole platform. We’ve been putting in a lot of work on the backend, updating our code and tests to work with Python 3. Fingers crossed, we’ll have complete GA-ready Python 3 support for the whole platform early next year.
/etc/st2/st2.conf by default.basename and dirname
All the rest of the gory details are in the changelog.
Packages are available in our apt and yum repos. Follow the standard instructions to install this version, or upgrade following the General Upgrade Procedure.
You know the drill by now: backup first.
And as always, thanks to everyone who contributed in some way, with code, bugs, or even just fixing typos in our docs.