New Feature: Output Schema

April 9, 2019
by Lindsay Hill

We added support for Output Schema in StackStorm 2.9. This feature has been “under the radar” for a while. Time to shed a little light, explain what it is, how to use it, and why we added this feature. Read on!

Output Schema – What’s That?

Anyone who has created their own custom StackStorm action knows all about adding parameters. This defines the expected parameters and their type. For example, the examples.orquesta-basic action has this:

parameters:
  cmd:
    required: true
    type: string
  timeout:
    type: integer
    default: 60

We define the input structure, but we have not defined the output structure. Our action is free to return whatever it wants. This is particularly important for Python actions, which can return any custom object they want.

That’s no big deal when you’re running a single action. But when you are combining multiple actions into a workflow, it’s really, really useful to know what one action’s output looks like, so you can then use it in another action.

That’s where Output Schema comes in. It lets us define the expected structure of our action outputs.

OK, so How Do I Use It?

The first thing we need to do is enable schema validation. In current versions, this is disabled by default.

Edit /etc/st2/st2.conf, and set this:

[system]
validate_output_schema = True

Restart all services (sudo st2ctl restart).

We’re going to use this simple python action for testing:

import sys
import platform

from st2common.runners.base_action import Action


class PrintPythonVersionAction(Action):

    def run(self):
        version = platform.python_version()
        executable = sys.executable

        print('Using Python executable: %s' % (version))
        print('Using Python version: %s' % (executable))


        return { 'version': version,
                 'executable': executable }

You can see that it will normally return a result object with a version and an executable string.

So let’s use this metadata:

---
name: python_runner_print_python_version
runner_type: python-script
description: Action which prints version of Python executable which is used.
enabled: true
entry_point: pythonactions/print_python_version.py
parameters: {}
output_schema:
  version:
    type: string
    required: true
  executable:
    type: string
    required: true

Register the action, then run it.

[email protected]:~$ st2 run examples.python_runner_print_python_version
.
id: 5ca6d4430761290dc50f1652
status: succeeded
parameters: None
result:
  exit_code: 0
  result:
    executable: /opt/stackstorm/virtualenvs/examples/bin/python
    version: 2.7.12
  stderr: ''
  stdout: 'Using Python executable: 2.7.12
    Using Python version: /opt/stackstorm/virtualenvs/examples/bin/python
    '
[email protected]:~$

OK, all good so far.

What happens if we change our return: line in the Python code?

Let’s make it this:

    def run(self):
        version = platform.python_version()
        executable = sys.executable

        print('Using Python executable: %s' % (version))
        print('Using Python version: %s' % (executable))


        return { 'version': version }
        # return { 'version': version,
        #          'executable': executable }

Note how we’re just returning version now.

[email protected]:~$ st2 run examples.python_runner_print_python_version
.
id: 5ca6d4940761290dc50f1655
status: failed
parameters: None
result:
  error: "u'executable' is a required property

Failed validating 'required' in schema['properties'][u'executable']:
    {'additionalProperties': False,
     'properties': {u'executable': {u'required': True,
                                    u'type': u'string'},
                    u'version': {u'required': True, u'type': u'string'}},
     'type': 'object'}

On instance[u'executable']:
    {u'version': u'2.7.12'}"
  message: Error validating output. See error output for more details.
[email protected]:~$

You can see that schema validation failed – executable is a required property, and we didn’t get that in the result object. So the action failed. If this was part of a workflow, it would take the failure path for that task, or if no failure path was defined, the whole workflow would fail.

What’s the Point?

Doesn’t this just add complexity? More config to add when creating a new action?

Yes – but it is for good reasons. Defining the expected output structure lets us do two things:

1. Detect errors earlier, so a workflow can fail when actions don’t behave as expected. The earlier you pick up the error, the less it propagates.
2. If we know the structure of an action’s outputs, we can make it easier to map those to inputs in the next action in our workflow. If you’re creating a workflow in Workflow Designer today, you need to know what the output structure looks like. If this is predefined, we can make this mapping much easier. Watch out for this in a future Workflow Designer update.

What’s Next?

Watch out for output_schema to start popping up in packs on StackStorm Exchange. Try it out with your own packs too. Let us know if you run into any problems.

In future we will enable output schema validation by default. This will be well-signposted when we do so. We’ll also continue to ignore actions that don’t have any output schema defined.