Integrating Runscope with CircleCI

In this tutorial, we're going to show you how to include Runscope API tests in your CircleCI builds. We'll use a small Python script and Trigger URLs, and we'll change the build status based on the results of your Runscope API test runs.

We'll cover how to:

Let's get started!


Getting Your Runscope Variables

Trigger URL

When running this script from the command line you can pass one parameter to it, which is a Runscope Trigger URL. If you want to run a single test, you can find its trigger URL under your environment settings:

Runscope environment settings menu expanded, showing the contents of the Trigger URL tab

If you want to run all the tests in a bucket, you can find a separate trigger URL in your bucket settings:

Runscope dashboard view, with an arrow pointed to the Bucket Settings link on the top-right corner

Runscope Bucket Settings page, showing the Trigger URL section the user can use to trigger all tests in a bucket

Make sure to copy the Trigger URL you wish to use, so we can use it with our script in the next steps.

Generating Your Runscope API Key

We need a Runscope personal access token to interact with the Runscope API and retrieve the results from our test run.

To get your access token, head over to your account's application tab, and click on Create Application. In the next screen, give your application a name, website URL, and callback URL. You can use dummy URLs if you're just using this app for your CI integration (e.g. http://example.org):

Runscope Create Application page, with the required fields containing the dummy URL http://example.org

Click on Create Application to finish the process. Then, scroll down to the bottom of your new application page and copy the personal access token value. We're going to use that in our next step:

Runscope Application detail page, highlighting the Personal Access Token section found at the end of the page which includes the Access Token variable

Setting up the Runscope Python script

You can find the sample script in our GitHub:

The two files we're interested are requirements.txt and app.py. For this tutorial, I'm just going to work with the raw links from our GitHub repository.

If you're integrating this into your project, I highly recommend either forking it to your own repository, or adding these files to a separate folder. That way, you can prevent your build from breaking in case there's an update to the repository.

Integrating the Script with CircleCI

In your CircleCI account, select the Build tab on the left-hand side menu, and click on the gear icon next to the project you want to integrate with Runscope to open its settings:

CircleCI Builds page, with an arrow pointing to the gear icon next to a project

On the left-hand side Settings menu, click on Environment Variables under Build Settings, then click on Add Variable. We only need to add one environment variable here named RUNSCOPE_ACCESS_TOKEN. Paste the access token that you copied in our previous step under Value, and click on Add Variable:

CircleCI project settings page, with the Environment Variables menu selected on the left-hand side, with an arrow pointing to the Add Variable button and the RUNSCOPE_ACCESS_TOKEN variable added

Now, let's go to Dependency Commands under the Test Commands menu.

The CircleCI environment already comes with python and pip pre-installed. The first thing we need to do is make sure the necessary packages for the script are installed. Add the following command to your Pre-dependency commands window:

pip install -r https://raw.githubusercontent.com/Runscope/python-trigger-sample/master/requirements.txt

Note: Remember to change the requirements URL above to your fork or local file.

Next, let's add another command just below it to download our app.py file (you can skip this step if you copied the file to your project):

wget https://raw.githubusercontent.com/Runscope/python-trigger-sample/master/app.py

CircleCI project settings page, with the Dependency Commands tab selected on the left-hand side, and highlighting the pre-dependency commands textbox with the pip install and wget commands added to it

For the final step, let's head to Test Commands under the Test Commands menu. In the Post-test commands window, we can run our app.py script. It takes one parameter, which is the Trigger URL you copied at the beginning of this tutorial. So we can just run the command as:

python app.py https://api.runscope.com/radar/your_test_trigger_id/trigger?runscope_environment=your_runscope_environment_id

Note: Remember to use the correct directory for app.py if you copied it to a folder inside your project, and change the URL after `app.py` to your tests Trigger URL.

CircleCI project settings page, with the Test Commands tab selected on the left-hand side, and the Post-test commands textbox highlighted with the python app.py command added to it

Continuous Integration Complete

In your next build runs, you should be able to see an extra step running the Python script, and hopefully returning a green checkmark ✅:

CircleCI build detail page, showing the python app.py step command being run, and an expanded tab showing its output and the message 'All test runs passed'

You're all set!


Having trouble configuring CircleCI? Contact our support team.