Integrating Runscope API Monitoring with New Relic Insights

New Relic Insights is a real-time analytics platform. By connecting New Relic Insights with Runscope API monitoring you can collect metrics from your API tests and transform them into actionable insights about your applications.


Connecting Your Accounts

We need two get two variables from New Relic to integrate it with Runscope: an Account ID, and Insert Key.

Getting a New Relic Account ID and Insert Key

Log in to your New Relic Insights account, and click on Manage Data on the left-hand side menu:

New Relic Insights highlighting Manage Data option on left-hand side menu

Then on the top bar, click on API keys:

New Relic Insights highlighting API Keys option on top bar

Next, click on the "+" button next to Insert Keys:

New Relic Insights with an arrow pointing to the + button next to Insert Keys

In the next page you will find an Account ID, and Key. Copy these two values and we're going to use it in the next steps:

New Relic Insights highlighting the account id and key variables in the new insert key creation page

Activating New Relic Insights in Runscope

In your Runscope account, click on your profile on the top-right, and select Connected Services:

Runscope account highlighting the Connected Services option on the dropdown after clicking on the user's profile on the top right

Find the New Relic Insights logo and click on Connect New Relic Insights:

Runscope connected services page, highlighting the New Relic Insights integration and the button Connect New Relic Insights

Here you have to provide two items from your New Relic account: the Account ID, and Insert Key. Paste the values that we just got in the previous step:

Runscope New Relic Integration page, showing the two textboxes where the user has to add their New Relic Insights account id and key from the previous steps

For the last step, make sure to enable the integration for any API test that you have that you wish to send the information back to New Relic Insights. You can toggle the integration on/off by opening an API Test, expanding the Environment Settings, and clicking on the Integrations tab on the left-hand menu:

Runscope API test, showing an expanded environment settings with the Integrations tab selected, and the newly connected New Relic Insights integration toggled on


Example NRQL Queries

To try these sample NRQL queries, select the 'Query' screen from within your New Relic Insights dashboard. You can filter any of these by bucket (WHERE bucket_name = 'Silly Swan') or test (WHERE test_name = 'My API Tests') as needed.

Test Result Counts for Past Week
SELECT count(*) FROM RunscopeRadarTestRun FACET result TIMESERIES SINCE 1 week ago
Assertion Pass/Fail Counts by Day for Past Week
SELECT sum(assertions_passed), sum(assertions_failed) FROM RunscopeRadarTestRun SINCE 1 week ago TIMESERIES 1 day
Response Times Percentiles Over Past Day
SELECT percentile(response_time_ms, 5, 50, 95) FROM RunscopeRadarTestRequest TIMESERIES SINCE 1 day ago
Response Times Histogram Over Past Week
SELECT histogram(response_time_ms, 1000) FROM RunscopeRadarTestRequest SINCE 1 week ago
Response Status Codes for Past Day
SELECT count(*) FROM RunscopeRadarTestRequest SINCE yesterday FACET response_status_code TIMESERIES

Event Reference

There are two types of events generated at the completion of every test run: RunscopeRadarTestRun which summarizes the entire run and Runscope RadarTestRequest for each individual request. An explanation of each of the fields is below. The identifier is the name to use for the given attribute in NRQL queries.

RunscopeRadarTestRun

Friendly Name Identifier Description
Test Name test_name The name of the test.
Test ID test_id The unique ID for the test.
Test Run ID test_run_id The unique ID for the individual test run instance.
Result result The outcome of the test run, either 'passed' or 'failed'.
Started At finished_at The timestamp when the test run started.
Finished At finished_at The timestamp when the test run completed.
Assertions Defined assertions_defined The number of assertions defined for all requests in the test.
Assertions Passed assertions_passed The number of assertions that passed in this test run.
Assertions Failed assertions_failed The number of assertions that failed in this test run.
Variables Defined variables_defined The number of variables defined in this test run.
Variables Passed variables_passed The number of variables that passed in this test run.
Variables Failed variables_failed The number of variables that failed in this test run.
Scripts Defined scripts_defined The number of scripts defined in this test run.
Scripts Passed scripts_passed The number of scripts that passed in this test run.
Scripts Failed scripts_failed The number of scripts that failed in this test run.
Requests Executed requests_excuted The number of requests executed for this test run.
Environment Name environment_name The name of the environment used in this test run.
Environment ID environment_id The ID for the environment used in this test run.
Bucket Name bucket_name The name of the Runscope bucket the test belongs to.
Bucket Key bucket_key The unique key for the Runscope bucket the test belongs to.
Team Name team_name The name of the Runscope team this test belongs to.
Team ID team_id The unique ID for the Runscope team this test belongs to.
Agent agent The agent used to execute this test run, or null if a default Runscope location was used.
Region region The region code for the location the test was run from, or null if an agent was used.
Region Name region_name The full region name and location the test was run from, or null if an agent was used.
Initial Variables Each of the initial variables (as they were after the initial scripts and variables have been processed but prior to the execution of the first request) is added as an attribute using the name of the variable as the key, and the value of the variable.

RunscopeRadarTestRequest

Friendly Name Identifier Description
Test Name test_name The name of the test.
Test ID test_id The unique ID for the test.
Test Run ID test_run_id The unique ID for the individual test run instance.
Request URL request_url The URL that was requested for this test step.
Request Method request_method The HTTP method used for this request. e.g. GET, POST, PUT, PATCH, DELETE, etc.
Response Status Code response_status_code The HTTP status code for the response, if the request completed.
Response Size Bytes response_size_bytes The size of the HTTP response body.
Response Time response_time_ms The time it took for the server to respond, in milliseconds.
Started At finished_at The timestamp when the test run started.
Finished At finished_at The timestamp when the test run completed.
Assertions Defined assertions_defined The number of assertions defined for this request.
Assertions Passed assertions_passed The number of assertions that passed for this request in this test run.
Assertions Failed assertions_failed The number of assertions that failed for this request in this test run.
Variables Defined variables_defined The number of variables defined for this request.
Variables Passed variables_passed The number of variables that passed for this request in this test run.
Variables Failed variables_failed The number of variables that failed for this request in this test run.
Scripts Defined scripts_defined The number of scripts defined for this request.
Scripts Passed scripts_passed The number of scripts that passed for this request in this test run.
Scripts Failed scripts_failed The number of scripts that failed for this request in this test run.
Environment Name environment_name The name of the environment for this request used in this test run.
Environment ID environment_id The ID for the environment used for this request in this test run.
Bucket Name bucket_name The name of the Runscope bucket the test belongs to.
Bucket Key bucket_key The unique key for the Runscope bucket the test belongs to.
Team Name team_name The name of the Runscope team this test belongs to.
Team ID team_id The unique ID for the Runscope team this test belongs to.
Agent agent The agent used to execute this test run, or null if a default Runscope location was used.
Region region The region code for the location the test was run from, or null if an agent was used.
Region Name region_name The full region name and location the test was run from, or null if an agent was used.