Req API Client Testing

• Wojtek Mach
• April 2nd, 2024
• req, http client, tests

In recent discussions with one of our Elixir Development Subscription clients, the topic of HTTP API client testing came up. In this article we’ll walk through an imaginary third party API integration and discuss testing approaches and affordances provided by Req.

Imagine we’re building an app that displays the weather for a given location using a third-party HTTP weather service. We might have code similar to this:

defmodule MyApp.Weather do
  def get_temperature(location) do
    options = [
      url: "https://weather-service/temperature",
      params: [location: location]
    ]

    # error handling left out for brevity
    {:ok, %{status: 200, body: %{"celsius" => celsius}}} = Req.request(options)
    {:ok, %{celsius: celsius}}
  end
end

iex> MyApp.Weather.get_temperature("Krakow, Poland")
{:ok, %{celsius: 10.0}}

Unfortunately this function is not so easy to test because it performs an external API request on every call. Such network requests can be flaky, slow, expensive, etc. A popular solution is to use mocks and stubs: just stub out a function call or two and move on, right? However, exactly because these network requests can be slow & flaky, they tend to be the source of many, if not most, performance and reliability issues in applications. Making these pain points stand out can be helpful. There are also real issues with mocks and stubs described next.

Testing with Explicit Contracts

Per Mocks and Explicit Contracts:

Because a mock is meant to replace a real entity, such replacement can only be effective if we explicitly define how the real entity should behave. Failing this, you will find yourself in the situation where the mock entity grows more and more complex with time, increasing the coupling between the components being tested, but you likely won’t ever notice it because the contract was never explicit.

Following the advices from that article, let’s define the contract and the implementations. Deviating from the article a bit, let’s implement the behaviour and the public API in the same module:

defmodule MyApp.WeatherAPI do
  @callback get_temperature(String.t()) :: {:ok, %{celsius: float()}} | {:error, term()}

  @impl_module Application.compile_env!(:myapp, :weather_api)

  def get_temperature(location) do
    @impl_module.get_temperature(location)
  end

  # or even more succinctly:
  #
  # defdelegate get_temperature(location), to: @impl_module
end

defmodule MyApp.WeatherAPI.Req do
  @behaviour MyApp.WeatherAPI

  @impl true
  def get_temperature(location) do
    options =
      [
        url: "https://weather-service/temperature",
        params: [location: location]
      ]

    {:ok, %{status: 200, body: %{"celsius" => celsius}}} = Req.request(options)
    {:ok, %{celsius: celsius}}
  end
end

defmodule MyApp.WeatherAPI.Stub do
  @behaviour MyApp.WeatherAPI

  @impl true
  def get_temperature(_), do: {:ok, %{celsius: 30.0}}
end

Let’s use the “real” implementation by default and the stub one in tests:

  # config/config.exs
  config :myapp,
    ...
+   weather_api: MyApp.WeatherAPI.Req

  # config/test.exs
  config :myapp,
    ...
+   weather_api: MyApp.WeatherAPI.Stub

As we grow our app and the test suite, pretty quickly we’ll run into an issue: our stub is completely static. What if we want to test different temperatures in different tests?

An easy solution is to switch from Application.compile_env!/2 in module body to Application.fetch_env!/2 in function body, allow changing implementation module at runtime by changing the application environment:

defmodule LowTemperature do
  @behaviour MyApp.WeatherAPI

  @impl true
  def get_temperature(_), do: {:ok, %{celsius: -2.5}}
end

test "low temperature" do
  old_module = Application.fetch_env!(:myapp, :weather_api)
  on_exit(fn -> Application.put_env(:myapp, :weather_api, old_module) end)
  Application.put_env(:myapp, :weather_api, LowTemperature)

  # ...
end

There are drawbacks to this however:

• application environment is global so our tests can no longer be async: true as we might run into race conditions.

• defining one module per test requires a fair amount of boilerplate.

While we can build some infrastructure by hand to tackle these concerns, there are existing projects that tackle them for us.

Testing with Mox

Both of these shortcomings can be resolved by using Mox. With just a couple of changes:

  # test/test_helper.exs
+ Mox.defmock(MyApp.WeatherAPI.Mock, for: MyApp.WeatherAPI)

  # config/test.exs
  config :myapp,
    ...
-   weather_api: MyApp.WeatherAPI.Stub
+   weather_api: MyApp.WeatherAPI.Mock

We have a much shorter test:

test "low temperature" do
  Mox.stub(MyApp.WeatherAPI.Mock, :get_temperature, fn _ ->
    {:ok, %{celsius: -2.5}}
  end)

  # ...
end

which can be running concurrently too.

Let’s recap. We currently have the following boundaries:

[MyApp]
 |
[MyApp.WeatherAPI (contract)]
 |\
 | [MyApp.WeatherAPI.Req] -> [Network] -> [Weather Service]
  \
   [MyApp.WeatherAPI.Mock]

I believe the biggest benefit of this solution is making an important part of the system, the contract between our app and the outside world, explicit.

On the flip side, while contracts are great when we get them right, they can cause a lot of additional work if we get them wrong. A contract with a lot of churn and/or many functions can become hard to maintain and turn from an asset to a liability.

There’s also the matter that we are skipping the whole HTTP client during tests. The more code replaced by the mock, the more likely bugs will creep in.

What if we could push our mock/stub further down, increasing the amount of code shared between dev/test/prod? Let’s see about some other ways to arrange things.

Testing with Req :plug

What if we had the contract for the HTTP client? Clients such as Req and Tesla already have a way to swap the underlying adapter (which makes the actual network request) so it’s easy to perfectly control responses in tests.

In Req, the most convenient way to configure a custom adapter for tests is via its Plug integration and the :plug option. The Plug API is well known by Elixir developers so using it for writing server stubs should be familiar and straightforward.

First, let’s make our production code accept some options:

  defmodule MyApp.Weather do
-   def get_temperature(location) do
+   def get_temperature(location, options \\ []) do
      options =
        [
          url: "https://weather-service/temperature",
          params: [location: location]
        ]
+       |> Keyword.merge(options)

      {:ok, %{status: 200, body: %{"celsius" => celsius}}} = Req.request(options)
      {:ok, %{celsius: celsius}}
    end
  end

and now let’s create a plug stub and configure Req to use:

test "low temperature" do
  plug = fn conn ->
    conn
    |> Plug.Conn.put_resp_content_type("application/json")
    |> Plug.Conn.send_resp(200, ~s|{"celsius":25.0}|)
  end

  assert MyApp.Weather.get_temperature("Krakow, Poland", plug: plug) ==
           {:ok, %{celsius: 25.0}}
end

Now our tests exercise even more of the production code. Under the hood it uses Plug.Test, the same foundation used in for example Phoenix controller tests. Effectively we have:

[MyApp]
 |
[MyApp.Weather]
 |
[Req]
 |
[Req Adapter (contract)]
 |\
 | [Finch] -> [Network] -> [Weather Service]
  \
   [Plug]

Making our production code just a little bit more flexible allows us to easily introduce stubs into our tests. On the flip side, passing the options through multiple layers might be impractical, especially if a given function already accept options that it cares about (as opposed to options its dependencies care about). Another idea is to mimic what Mox does, which is to use an ownership mechanism to configure the system under test on a test-by-test basis concurrently. Enter Req.Test.

Testing with Req.Test

First, let’s change the global configuration.

  # config/config.exs
  config :myapp,
    ...
+   weather_req_options: [
+     plug: {Req.Test, MyApp.Weather}
+   ]

plug: {Req.Test, <name>} tells Req to find a plug stub with the given name and run the request against it.

Let’s change the production code:

  def get_temperature(location, options \\ []) do
    options =
      [
        base_url: "https://weather-service",
        url: "/temperature",
        params: [location: location]
      ]
+     |> Keyword.merge(Application.get_env(:myapp, :weather_req_options, []))
      |> Keyword.merge(options)
    ...
  end

And now in the test we can easily stub out the plug:

  test "get_temperature" do
-   plug = fn conn ->
+   Req.Test.stub(MyApp.Weather, fn conn ->
      conn
      |> Plug.Conn.put_resp_content_type("application/json")
      |> Plug.Conn.send_resp(200, ~s|{"celsius":25.0}|)
    end)

-   assert MyApp.Weather.get_temperature("Krakow, Poland", plug: plug) ==
+   assert MyApp.Weather.get_temperature("Krakow, Poland") ==
             {:ok, %{celsius: 25.0}}
  end

And that’s it!

There’s even a Req.Test.json(conn, body) to conveniently create JSON responses:

Req.Test.stub(MyApp.Weather, fn conn ->
  Req.Test.json(conn, %{celsius: 25.0})
end)

See Req.Test module documentation for more information.

Testing the Network

For the sake of the discussion, what if we tried to exercised basically all of the production code? In other words, what if we stubbed out the network?

What is the network anyway? It’s a set of protocols and APIs: HTTP, TCP/IP, URI, sockets, etc. HTTP (HTTP/1 to be specific) is a text protocol over TCP/IP. It’s trivial to create a stub HTTP server in Elixir/Erlang. Run this in IEx:

iex> {:ok, s} = :gen_tcp.listen(8080, [])
iex> {:ok, s} = :gen_tcp.accept(s)
iex> :gen_tcp.send(s, "HTTP/1.1 200 OK\r\ncontent-length: 5\r\n\r\nHello")

And in another terminal:

$ curl http://localhost:8080
Hello

That’s it! Swapping the implementation is trivial, we just need to swap the network address, the URL. There’s even a popular library, Bypass, which allows conveniently spawning Web servers on a test-by-test basis. In a nutshell, we get:

[MyApp]
 |
[MyApp.Weather]
 |
[Req]
 |
[Network (contract)]
 |\
 | [Weather Service]
  \
   [Custom Web Server]

This stub is useful when testing HTTP clients themselves because we can inspect and emit the exact bytes over the wire. However, it may be too low-level for most common Web application development needs.

Conclusion

Testing API clients is a pretty broad topic. In this article we went through these approaches:

1. “Explicit Contracts”
2. “Local Stubs” - pass Req :plug option throughout the system
3. “Ownership Stubs” - configure :plug on a test-by-test basis with Req.Test
4. “Network stubs” - introducing a separate web server that listens at a custom address

I believe “Explicit Contracts” is a great starting point as it makes us take a step back and think about the boundaries. (And we have Mox to conveniently stub things out!) We can also create ad-hoc “Local Stubs” and pass them down to Req :plug option. And if we can’t easily pass the options down, we can use “Global Stubs” using Req.Test

Happy hacking!