Goth redesign

Wojtek Mach
March 9th, 2021
http client, processes, performance

UPDATE: In the previous version of the article we were using :persistent_term but we’ve replaced it with an ETS table that is more suitable for data that periodically changes. Thanks to readers for pointing this out.

While working on Bytepack last year, we needed to authenticate our HTTP requests to the Google Cloud Storage and we chose the popular Goth library to generate the OAuth2 tokens. The library worked great out of the box, however we noticed a few potential areas of improvements that we were glad to contribute back to the library.

First, a quick introduction to Goth. This is how we used to use it:

Add it to your dependencies:
```
def deps do
  [{:goth, "~> 1.2.0"}]
end
```

Configure it:

# config/config.exs

config :goth,
  json: File.read!("path/to/google/json/creds.json")

And use it:

iex> Goth.Token.for_scope("https://www.googleapis.com/auth/cloud-platform.read-only")
{:ok, %Goth.Token{expires: 1614245694, token: "ya29.cAL...", ...}}

A given token is valid for one hour which led to two important features of the library:

While the user of the library could save the token off somewhere to be re-used later, the library conveniently provides a built-in cache so it’s not necessary. Only the first time you request a token it would actually be generated, subsequent calls would read from the cache.
The token is automatically refreshed before it goes stale.

For our project, we identified a few missing pieces in the library though, we needed some more customization. We wanted to use a different HTTP client as well as request token refresh earlier so that if we run into any network issues, there’s enough time to try again a few times before the token gets stale.

We also noticed that fetching from built-in cache was done through a single GenServer which means that process could easily become a bottleneck under heavy traffic. This wasn’t a big concern for us as we only needed a token for writes and our application was read-heavy. However, one of our Elixir Development Subscription customers was also using Goth and they were very performance cautious so removing the bottleneck was an important improvement for them.

Finally, for libraries we prefer explicit configuration over the application environment, so we worked on that too. Despite the improvements on Elixir v1.9 with config/releases.exs and Elixir v1.11 with config/runtime.exs, it is still a best practice to avoid global configuration, as there are better alternatives that we’ll show in this article.

New usage

We wanted to eventually contribute back all of these changes, however at that point we needed to change how the library works in a pretty fundamental way so instead we ended up writing a new library from scratch and trying that out in our project first. We also contacted Phil Burrows, the original author and maintainer of Goth, and came up with a plan how to backport our changes. We have deprecated the existing API, so the existing users can upgrade at their own pace, and came up with a new API.

To use it, the first step is to add it your dependencies:

def deps do
  [{:goth, "~> 1.3-rc"}]
end

Then, add the Goth child spec to your supervision tree:

defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    credentials =
      "GOOGLE_APPLICATION_CREDENTIALS_JSON"
      |> System.fetch_env!()
      |> Jason.decode!()

    source = {:service_account, credentials, []}

    children = [
      {Goth, name: MyApp.Goth, source: source}
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

You can now finally use it:

iex> Goth.fetch(MyApp.Goth)
{:ok, %Goth.Token{expires: 1614245694, token: "ya29.cAL...", ...}}

As you can see, we no longer rely on the :goth application starting it’s own supervision tree, but instead we explicitly add it to our own tree. This gives us more control when exactly it starts as well as we can trivially start multiple instances, each with different credentials and scopes. This is not something we needed ourselves, but it was a long-requested feature by the community.

Let’s dive a little bit deeper into two particular improvements we’ve made: switching HTTP clients and avoiding single-process bottleneck.

HTTP client contract

Goth depended on the HTTPoison HTTP client but we already picked Finch as our HTTP client of choice and it would be wasteful and potentially error-prone to use different clients for different parts of the system so we definitely wanted to standardise on just one. We need a way to tell Goth which HTTP client to use and we did that by introducing a Goth.HTTPClient contract, a default implementation for backwards-compatibility as well as nice out-of-the-box experience, and an option to switch.

Our Finch-based adapter roughly looked like this:

defmodule Bytepack.Extensions.Goth.FinchClient do
  @moduledoc """
  Finch-based HTTP client for Goth.

  ## Options

    * `:name` - the name of the `Finch` pool to use.

    * `:default_opts` - default options that will be used on each request,
      defaults to `[]`. See `Finch.request/3` for a list of supported options.
  """

  @behaviour Goth.HTTPClient

  defstruct [:name, default_opts: []]

  @impl true
  def init(opts) do
    struct!(__MODULE__, opts)
  end

  @impl true
  def request(method, url, headers, body, opts, initial_state) do
    opts = Keyword.merge(initial_state.default_opts, opts)

    Finch.build(method, url, headers, body)
    |> Finch.request(initial_state.name, opts)
  end
end

and this is how we’d use it in our supervision tree:

children = [
  {Finch, name: Bytepack.Finch, pools: pools},
  {Goth,
   name: Bytepack.Goth,
   source: source,
   http_client: {Bytepack.Extensions.Goth.FinchClient, name: Bytepack.Finch}}
]

The init/1 callback is an important extension point of the HTTP contract. While in the snippet above, it doesn’t do much, it just converts options keywords list into a struct (to make sure we didn’t make a typo in the key names so it’s pretty useful!), in the future the built-in Hackney-based adapter could be changed like this:

defmodule Goth.HTTPClient.Hackney do
  @behaviour Goth.HTTPClient

  @impl true
  def init(opts) do
    if Code.ensure_loaded?(:hackney) do
      # ...
    else
      raise "please add :hackney to your dependencies"
    end
  end
end

and then Goth could mark its dependency on Hackney as optional:

{:hackney, "~> 1.7", optional: true}

This means that if users intended to use Goth with a different HTTP client, they wouldn’t even download and compile hackney in the first place. A small but important win!

Taking a step back from Goth for a moment, in general we believe that libraries should have as least dependencies as possible, and the dependencies they have should be easily customisable. Customisation via explicit contract is one option, another one is adding extension points via being able to pass anonymous functions or a {module, function, args} tuple. As an example for the latter, here’s an excerpt from the docs of our Broadway connector for the Google Cloud Pub/Sub service:

 * `:token_generator` - Optional. An MFArgs tuple that will be called before
   each request to fetch an authentication token. It should return
   `{:ok, String.t()} | {:error, any()}`.
   Default generator uses `Goth.Token.for_scope/1` with
   `"https://www.googleapis.com/auth/pubsub"`.

This way, when users of broadway_cloud_pub_sub update to latest version of Goth, they’ll be able to easily use the new API:

token_generator: {Goth, :fetch, [MyApp.Goth]}

Last but not least, worth mentioning the extension points are not only useful for library users but for the library authors themselves. Being able to easily swap some implementation details is really useful for tests!

Removing single-process bottleneck

A given process can only handle one message at a time. This is typically fine, but if you send a lot of messages to that single process, it’s message queue will built-up and that can become a bottleneck. The common and preferred strategy to improve performance is to use ETS.

This is how our new Goth cache implementation looks like:

defmodule Goth do
  defdelegate fetch(server), to: Goth.Server
end

defmodule Goth.Server do
  @moduledoc false

  use GenServer

  def fetch(server) do
    %{config: config, token: token} = get(server)

    if token do
      {:ok, token}
    else
      Token.fetch(config)
    end
  end

  @impl true
  def init(opts) when is_list(opts) do
    opts =
      Keyword.update!(opts, :http_client, fn {module, opts} ->
        {module, module.init(opts)}
      end)

    state = struct!(__MODULE__, opts)
    :ets.new(state.name, [:named_table, read_concurrency: true])

    # given calculating JWT for each request is expensive, we do it once
    # on system boot to hopefully fill in the cache.
    case Token.fetch(state) do
      {:ok, token} ->
        store_and_schedule_refresh(state, token)

      {:error, _} ->
        put(state, nil)
        send(self(), :refresh)
    end

    {:ok, state}
  end

  @impl true
  def handle_info(:refresh, state) do
    case Token.fetch(state) do
      {:ok, token} ->
        store_and_schedule_refresh(state, token)
        {:noreply, state}

      {:error, exception} ->
        ...
    end
  end

  defp store_and_schedule_refresh(state, token) do
    put(state, token)
    time_in_seconds = ...
    Process.send_after(self(), :refresh, time_in_seconds * 1000)
  end

  defp get(name) do
    :ets.lookup_element(name, :data, 2)
  end

  defp put(state, token) do
    config = Map.take(state, [:source, :http_client])
    :ets.insert(state.name, {:data, %{config: config, token: token}})
  end
end

We still built it as a GenServer because we want to periodically refresh the token, but notice fetching the token isn’t done via message passing but by fetching it from the ETS table using the name of our GenServer!

Conclusion

In this article we discussed our efforts to redesign the Goth library to be more flexible and performant. In particular, we introduced a HTTP client contract to easily swap clients out and we’ve removed a single-process bottleneck. We are very glad to have contributed these changes upstream and we hope library authors and users would perfom similar changes wherever they make sense!

For reference, here’s our Goth redesign proposal and please give Goth v1.3.0-rc a go!

Special thanks to Phil Burrows for writing Goth in the first place, helping with the transition, and reviewing the draft of this post. Thanks to Michael Crumm for helping with backporting some of the functionality into the new design too!

Goth redesign

New usage

HTTP client contract

Removing single-process bottleneck

Conclusion

Subscribe to Dashbit Updates

Follow us

Recent Posts

Tags