HTTP unit tests using ExVCR

What is ExVCR?

Record and replay HTTP interactions library for elixir. It’s inspired by Ruby’s VCR, and trying to provide similar functionalities.

ExVCR allows you to automatically record any HTTP request/response to a JSON text file. Subsequent requests matching the same URL are served up by the cached response from disk. The feedback loop is minimised when following a test-driven development approach due to the instant HTTP responses. The recorded fixtures support offline development, help other developers, and allow test execution without external HTTP requests in continuous integration environments.

Motivation

I authored an Elixir wrapper for the Strava API. During the development I wrote unit tests to build the API and verify functionality. Most of these tests made HTTP calls to the Strava API.

Strava enforces a rate limit to its REST API. The default rate limit allows 600 requests every 15 minutes, with up to 30,000 requests per day. I also use the mix test.watch task to execute the test suite after any file is saved. The rate limit and frequent test runs meant that ExVCR was an ideal fit.

The Strava library uses HTTPoison as the HTTP client. This uses hackney to execute the HTTP requests. Therefore I had to use the ExVCR.Adapter.Hackney adapter in my tests.

Usage

Using ExVCR in a unit test requires the following code changes.

1. Add exvcr to the project’s dependencies in config/mix.exs as a test-only dependency.

defp deps do
  [
    {:exvcr, "~> 0.8", only: :test},
  ]
end

2. Disable ExUnit’s async support.

use ExUnit.Case, async: false

3. Use the ExVCR mock macro with the ExVCR.Adapter.Hackney adapter.

  
use ExVCR.Mock, adapter: ExVCR.Adapter.Hackney

4. Configure HTTPoison to start for all tests.

setup_all do
  HTTPoison.start
end

5. Wrap the lines of code making the HTTP request inside a use_cassette block.

use_cassette "club/retrieve#1" do
  # ... HTTP request/response handling
end

The string provided to use_cassette is used to build the path to the recorded JSON file, relative to fixture/vcr_cassettes. So you can use the path separator character to group related fixtures together in the same directory.

Sample test

The full code for the sample test is given below.

defmodule Strava.ClubTest do
  use ExUnit.Case, async: false
  use ExVCR.Mock, adapter: ExVCR.Adapter.Hackney

  setup_all do
    HTTPoison.start
  end

  test "retrieve club" do
    use_cassette "club/retrieve#1" do
      club = Strava.Club.retrieve(1)

      assert club != nil
      assert club.id == 1
      assert club.name == "Team Strava Cycling"
      assert club.member_count > 1
      assert club.sport_type == "cycling"
    end
  end
end

Running tests

The first time you run the test there will be no ExVCR fixture data. So HTTP requests are made and the response is cached to disk at fixture/vcr_cassettes. Subsequent test runs will use the cached fixture data if it matches the requested URL.

For the example test above, the cached fixture JSON data is given below.

[
  {
    "request": {
      "body": "",
      "headers": {
        "Authorization": "<<access_key>>"
      },
      "method": "get",
      "options": [],
      "request_body": "",
      "url": "https://www.strava.com/api/v3/clubs/1"
    },
    "response": {
      "body": "{\"id\":1,\"resource_state\":3,\"name\":\"Team Strava Cycling\",\"profile_medium\":\"https://dgalywyr863hv.cloudfront.net/pictures/clubs/1/1582/4/medium.jpg\",\"profile\":\"https://dgalywyr863hv.cloudfront.net/pictures/clubs/1/1582/4/large.jpg\",\"cover_photo\":\"https://dgalywyr863hv.cloudfront.net/pictures/clubs/1/4328276/1/large.jpg\",\"cover_photo_small\":\"https://dgalywyr863hv.cloudfront.net/pictures/clubs/1/4328276/1/small.jpg\",\"sport_type\":\"cycling\",\"city\":\"San Francisco\",\"state\":\"California\",\"country\":\"United States\",\"private\":true,\"member_count\":114,\"featured\":false,\"verified\":false,\"url\":\"team-strava-bike\",\"membership\":null,\"admin\":false,\"owner\":false,\"description\":\"Private club for Cyclists who work at Strava.\",\"club_type\":\"company\",\"post_count\":21,\"following_count\":0}",
      "headers": {
        "Cache-Control": "must-revalidate, private, max-age=0",
        "Content-Type": "application/json; charset=UTF-8",
        "Date": "Thu, 20 Oct 2016 09:10:18 GMT",
        "ETag": "\"1276cfd913500278eea28e8b4d029a5d\"",
        "Status": "200 OK",
        "X-FRAME-OPTIONS": "DENY",
        "X-RateLimit-Limit": "600,30000",
        "X-RateLimit-Usage": "33,33",
        "X-UA-Compatible": "IE=Edge,chrome=1",
        "Content-Length": "768",
        "Connection": "keep-alive"
      },
      "status_code": 200,
      "type": "ok"
    }
  }
]

To execute the unit tests against the real web services you simply delete the stored fixture files.

Query string parameters

By default, query params are not used for matching with URLS recorded in ExVCR fixtures. You must specify match_requests_on: [:query] in order to include query params.

test "stream members" do
  use_cassette "club/stream_members#1", match_requests_on: [:query]  do
    member_stream = Strava.Club.stream_members(1)

    assert Enum.count(member_stream) > 1
  end
end

Filtering sensitive content

The Strava API uses an access key included in the HTTP Authorization request header to authenticate requests. As I wanted to include the recorded fixture data in the public Git repository I had to remove this header. This is supported by the ExVCR config setting filter_sensitive_data added to config/test.exs. As shown below, I replace the Bearer token containing the access key with a placeholder value using a regular expression.

use Mix.Config

config :exvcr, [
  filter_sensitive_data: [
    [pattern: "Bearer [0-9a-z]+", placeholder: "<<access_key>>"]
  ],
  filter_url_params: false,
  response_headers_blacklist: ["Set-Cookie", "X-Request-Id"]
]

The config blacklists the Set-Cookie and X-Request-Id response headers to ensure that these are also not recorded to prevent revealing sensitive data.

Continuous integration

Including fixture data in source control allowed me to run the test suite on Travis CI.

Without using ExVCR, the tests would require a valid access key to use the Strava API. I would have to register a new Strava developer account, since each account only has a single key, and then use an environment variable to configure this setting. With recorded HTTP fixture data included in source control the tests suite runs quickly without being affected by external service availability.

In the config/test.exs mix configuration file I optionally include a config/test.secret.exs file, if present. This file contains the private Strava API settings including my own access key. The file is excluded from source control by adding /config/test.secret.exs to the .gitignore file.

use Mix.Config

if File.exists?("config/test.secret.exs") do
  import_config "test.secret.exs"
end

This approach also supports other developers who might want to contribute changes without requiring their own Strava API access key. They can clone the Git repository and immediately run the full test suite.

Conclusion

Running the Strava test suite without recorded HTTP fixture data.

$ mix test
....................

Finished in 43.5 seconds
21 tests, 0 failures, 1 skipped

Now running the same tests with fixture data present results in a significant speed up.

$ mix test
....................

Finished in 4.1 seconds
21 tests, 0 failures, 1 skipped

The total time reduced by a factor of ten: from 44 seconds down to 4 seconds.