Monitor Twitter And Fire Automations Based On Twitter Keywords Using StackStorm

December 22, 2014

by Tomaz Muraus

The StackStorm automation platform is very powerful and flexible. Our users most commonly use it to perform tasks such as Continuous Integration & Continuous Deployment, Facilitated Troubleshooting and Remediation as a Service.

123

Continuous Integration & Continuous Deployment, Facilitated Troubleshooting and Remediation as a Service, three mostly common use-cases.

Because StackStorm is an extensible automation platform, you are not limited to these three use cases. You can use it to perform almost any kind of automation you can think of.

Today, let’s look at a slightly different use case. I’ll demonstrate how to monitor Twitter in near real-time for matching keywords and relay tweets matching those keywords to a Slack channel.

There are many scenarios where doing something like this comes in handy. A popular scenario is monitoring social media for messages which mention your company or a product name. This allows you to provide a better user experience and support because you are able to react to those messages faster, and potentially also in an automated manner.

Examples of things you can do when a matching tweet is detected include forwarding that message to the customer department, providing an automated answer for simple questions, kicking off basic troubleshooting or at least event investigation, or relaying the tweet to some other medium that is constantly monitored by your company employees.

In this blog post we will have a look how do the last scenario I mentioned — relaying matching tweets to a Slack channel.

Using sensor from Twitter pack to monitor Twitter for matching keywords

We have recently added a Twitter pack to our StackStorm Exchange. This pack includes a sensor that allows you to monitor Twitter for tweets matching a particular query. Once a matching tweet is detected, a trigger that contains all the information about that tweet is dispatched.

In the background, the sensor works by periodically polling the Twitter search API.

Getting and configuring the pack

To get and configure the pack, simply follow the instructions below.

Step 1: Download the pack

st2 pack install twitter

Step 3: Edit the configuration file

Copy /opt/stackstorm/packs/twitter/twitter.yaml.example to /opt/stackstorm/configs/twitter.yaml and edit the configuration to match your needs.

All the options are explained below and in the README.

  • consumer_key – Twitter API consumer key
  • consumer_secret – Twitter API consumer secret
  • access_token – Twitter API access token
  • access_token_secret – Twitter API access token secret
  • query – A query to search the twitter timeline for. You can use all the query operators described at https://dev.twitter.com/rest/public/search
  • count – Number of latest tweets matching the criteria to retrieve. Defaults to 30, maximum is 100.
  • language – If specified, only return tweets in the provided language (e.g. en, de, etc.).

To obtain API credentials, you need to first register your application on the Twitter Application Management page.

4

Step A – Application registration

After you have done that, go to the “Keys and Access Tokens” tab where you can find your consumer key and secret. On the same page you can also generate a read-only access token.

5

Step B – Obtaining consumer key and consumer secret

6

Step C – Generating and obtaining access token and secret

As noted above, in the query field you can use all of the query operators supported by Twitter. This gives you a lot of flexibility and makes the solution powerful.

Step 4: Register the sensor, restart sensor_container service

For the StackStorm to pick up the new sensor, you need to first register it and then restart the sensor_container service.

You can do that using the commands listed below.

sudo st2ctl reload
sudo st2ctl restart
Creating a rule

Now that we have a sensor in-place, we just need to create a rule which will relay matched tweets to our Slack channel.

Example 1: Using slack.post_message action

Using slack.post_message action

In this rule we are going to use a “post_message” action from the slack pack. This action willpost a message which is defined in the parameters to a Slack channel.

To download and configure the slack pack, followed the steps 1-4 described above (just replace Twitter with Slack). You can find the description of all the available settings in the README.

https://gist.github.com/Kami/c13b5b47e974c572e218

As you can see, the rule is pretty simple and straightforward. We don’t define any additional criteria and simply relay every matched tweet to our Slack channel. If you want to perform additional filtering, you can to that using the criteria field.  

For information on how to use this field and which operators are supported, see the Rules section in the documentation.  

To create this rule, run the following commands:

curl https://gist.githubusercontent.com/Kami/c13b5b47e974c572e218/raw/52c703a2f98ded48853bc5cba1ffeae1fea85727/gistfile1.yml -o rule1.yaml
st2 rule create rule1.yaml

Example 2: Using core.http action

To show how flexible and powerful StackStorm really is, we are also going to have a look at how to perform exactly the same task using a built-in core.http action.  

core.http action allows you to perform an arbitrary http request and you can control everything from headers, to body and cookies.  

https://gist.github.com/Kami/7675287d81365cc8dac4

As you can see, the rule is the same as the upper one, but instead of using slack.post_message action, we are using core.http action.

Using this action means we need to define some more parameters inside the rule – we need to define the webhook url, specify correct headers and pass data in an expected format.

To create the rule, you run the same commands as above:

curl https://gist.githubusercontent.com/Kami/7675287d81365cc8dac4/raw/8f3ca0e5ab8b842be9e693b67aef2b09cb7f6645/gistfile1.yml -o rule2.yaml
st2 rule create rule2.yaml
Putting it all together

Now that we have a sensor and rule in place, all we need to do is wait for a matching tweet to be posted and the tweet will be relayed to our channel.

7

Our little solution in action. 

Keep in mind that this blog post just briefly touches the surface of what is possible to do with StackStorm. For example, you can modify this solution to relay the tweet to an IRC channel or email instead of Slack, or you can use a Workflow to perform more advanced automation consisting of multiple steps – the possibilities are almost endless!

Conclusion

I hope this blog post gave you a better idea of how powerful StackStorm really is and showed you that because StackStorm is an extensible automation platform, it’s not just limited to three most commonly used use-cases described above — it can be used  to perform almost any kind of automation you can think of.

Additionally, I hope it inspired you to think outside of the box and build something unusual, but cool and useful with StackStorm yourself.

And if you have already done that, please get in touch – we are more than happy to feature your use case and / our solution on our blog.

  • Pingback: StackStorm 0.7 Is Here! | StackStorm()

  • Eugen C.

    Corrected commands:

    
    st2 packs.download packs=twitter
    ->
    st2 run packs.download packs=twitter
    
    st2 packs.setup_virtualenv pack=twitter
    ->
    st2 run packs.setup_virtualenv packs=twitter
    
  • Eugen C.

    One more tip for those who new with Slack Chat:
    To have that nice looking view with embedded images (on screenshot) you have also to enable Twitter integration in Slack settings.

    It wasn’t obvious for me.

    • Our Twitter and Slack integrations are standalone and not related to the new Slack Twitter integration.

      Screenshots should still be displayed as long as there is long enough delay between the time when tweets are relayed to Slack. If there are many Tweets relayed to Slack in a short time frame (e.g. when starting a sensor for the first time or if the sensor was not running for a long time and many Tweets have accumulated in the queue), Slack won’t automatically parse and display links and image in the messages.

      Now back to the new Slack Twitter Integration – Slack recently introduced a first-class citizen Twitter integration. Our integration predates that.

      Even though there is a first class integration available now, using StackStorm for monitoring Twitter still has many benefits – in addition to Slack you can relay Tweets to other mediums (IRC, email, hipchat, you name it – if we don’t support it yet, it’s easy enough to write a new integration for it).

      On top of that, monitoring and consuming events from Twitter into StackStorm also allows you to kick off other processes based on the matching Tweet(s) – e.g. open a JIRA ticket if a customer complains, many users complaining about your website being offline? you can correlate this with your monitoring system, trigger a deployment via Twitter, post monitoring system notifications to your Twitter status account to let customers know what is going on and more. Possibilities are almost endless. 🙂

  • Pingback: Automating with Mistral Workflow - StackStorm()