How we verify webhooks

• Philip Sampaio
• October 27th, 2020
• plug, auth, phoenix

Webhooks are widely used as a tool to communicate changes of state, to notify events and to trigger actions between HTTP services. But there is gap in the security side: if there is a proxy between the server and the client, it is possible that this proxy gets the request and changes it somehow before reaching the client. This is a type of Man-in-the-middle attack. It is unlikely that this will happen because TLS/SSL is very important and required nowadays, but if you are adept of the “zero trust” model it’s worth implementing a HTTP signature method for your webhooks in order to increase the security level of your communications.

We added this layer of security to our webhooks in one of our projects following Stripe’s specification, and it works well! The idea is to sign the entire body of the request using a secret shared by the emitter (the server) and the client of the webhook. The signature contains the timestamp and the hash of the body. This hash is generated with a HMAC using the SHA256 algorithm to ensure a balance between performance and security.

UPDATE #1: Thanks to the feedback from Pawel Szafran we fixed the order of “secret” and “payload” arguments when using :crypto.mac/4 in this blog post.

Signing the request - the server side

The signature is a header sent on each request and it looks like this:

t=1492774577,
v1=6ffbb59b2300aae63f272406069a9788598b792a944a07aba816edb039989a39

Where t is the timestamp, which is the time in seconds that the event was signed, and the v1 is the version of this signature. Note that multiple versions may be present.

The value of v1 is the calculated hash of the body, along with the timestamp and a shared secret between the server and the client. The timestamp is important to prevent replay attacks. To calculate the HMAC of a string in Elixir, you can use the :crypto module, normally available from Erlang:

:crypto.mac(:hmac, :sha256, "your-shared-secret", "your-entire-body")

We will be using the following body as an example:

{
  "data": "hello world"
}

And the secret will be just secret. Let’s freeze the time in 1603136520, which is 2020-10-19 19:42:00 in UTC.

Following the Stripe’s specification, the signature will be the HMAC of:

• the timestamp;
• a . (dot) character;
• the request body.

This is something like:

signed_payload = "1603136520.{\n  \"data\":\"hello world\"\n}"
hmac = :crypto.mac(:hmac, :sha256, "secret", signed_payload)

You may notice that this code will generate a binary, which is not what we want. Instead, we need to encode the signature as a base 16 string:

Base.encode16(hmac, case: :lower)

The result will be the string 47f795dce546e011e7da48824b1ccaccd3b667a455d6f8cee47499cadaf6427a. Awesome! Now we need to put the hash and the timestamp in the request headers. For the request, you will need a HTTP client. For my example I will be using Finch, which is a new and robust HTTP client.

body = "{\n  \"data\":\"hello world\"\n}"
now = System.system_time(:second)
sig = "t=#{now},v1=#{signature(body, now)}"

Finch.build(:post, "https://httpbin.org/post", [{"signature", sig}], body)
|> Finch.request(MyFinch)

The signature/2 function looks like this:

def signature(body, timestamp) do
  signed_payload = "#{timestamp}.#{body}"
  hmac = :crypto.mac(:hmac, :sha256, "secret", signed_payload)
  Base.encode16(hmac, case: :lower)
end

For the server that is it. Now each client will receive the signature in a signature header.

Verifying the webhook - the client side

The clients that receives the webhook requests can now verify the integrity of the data. If you are receiving webhooks from Stripe, you can use this exact approach to validate them.

The algorithm is similar to what is needed to sign, but there are some details regarding reading the request body from Plug.Conn, and the comparison between the two signatures.

In order to read the original request body from Plug.Conn, you will need to write a custom body_reader that caches the body from webhook requests. This is because Plug will replace the body by the parsed version when you have something like a JSON request. Here is how this custom body reader looks like:

defmodule BodyReader do
  def cache_raw_body(conn, opts) do
    with {:ok, body, conn} <- Plug.Conn.read_body(conn, opts) do
      conn = update_in(conn.assigns[:raw_body], &[body | &1 || []])

      {:ok, body, conn}
    end
  end
end

We have two options to configure that with our Phoenix or Plug application:

1. you just pass the body_reader: {BodyReader, :cache_raw_body, []} to Plug.Parsers and caches all the requests;
2. or you define two plugs and explicitly match the routes you want to keep the body.

We went with the second option because our application also had uploads and we didn’t want to load the uploads into memory. In our application we changed the Phoenix Endpoint to look like this:

defmodule MyApp.Endpoint do
  use Phoenix.Endpoint, otp_app: :my_app_web

  # This line replaces the "plug Plug.parsers" setup.
  plug :parse_body

  opts = [
    parsers: [:urlencoded, :multipart, :json],
    pass: ["*/*"],
    json_decoder: Phoenix.json_library()
  ]

  @parser_without_cache Plug.Parsers.init(opts)
  @parser_with_cache Plug.Parsers.init([body_reader: {BodyReader, :cache_raw_body, []}] ++ opts)

  # All endpoints that start with "webhooks" have their body cached.
  defp parse_body(%{path_info: ["webhooks" | _]} = conn, _),
    do: Plug.Parsers.call(conn, @parser_with_cache)

  defp parse_body(conn, _),
    do: Plug.Parsers.call(conn, @parser_without_cache)
end

Now every request to enpoints that starts with /webhooks will be cached at Plug.Conn under assigns.raw_body. We will be using this to check if the signature matches.

Calculating the hash and verifying

This is the last part in the steps to verify the webhook request. We now have the raw body cached into our Plug connection, and we need to read that and compare with what we have in the signature header that the server sent to us.

First of all, we need to parse the signature header. To do so, let’s write a new plug with a function called parse:

defmodule HTTPSignature do
  @behaviour Plug

  @impl true
  def init(opts), do: opts

  @impl true
  def call(conn, _) do
    # TODO
  end

  defp parse(signature, schema) do
    parsed =
      for pair <- String.split(signature, ","),
          destructure([key, value], String.split(pair, "=", parts: 2)),
          do: {key, value},
          into: %{}

    with %{"t" => timestamp, ^schema => hash} <- parsed,
         {timestamp, ""} <- Integer.parse(timestamp) do
      {:ok, timestamp, hash}
    else
      _ -> {:error, "signature is in a wrong format or is missing #{schema} schema"}
    end
  end
end

This function will receive a signature like we described in the beginning: t=timestamp,v1=signature-hash, and will transform into a tuple in case of success.

After that we need to actually fetch the raw_body from the connection, and verify against the signature header. To do that, we will introduce another private function to our plug module:

defp raw_body(conn) do
  case conn do
    %Plug.Conn{assigns: %{raw_body: raw_body}} ->
      # We cached as iodata, so we need to transform here.
      {:ok, IO.iodata_to_binary(raw_body)}

    _ ->
      # If we forget to use the plug or there is no content-type on the request
      raise "raw body is not present or request content-type is missing"
  end
end

And finally to verify, we need to get the header, parse it and compare. For the comparison, we cannot simply do a basic equality check. This is to avoid timing attacks. Luckily we have a function for that from Plug: Plug.Crypto.secure_compare/2. Here is how the verification looks like:

def verify(header, payload, secret, opts \\ []) do
  with {:ok, timestamp, hash} <- parse(header, @schema) do
    current_timestamp = System.system_time(:second)

    cond do
      timestamp + @valid_period_in_seconds < current_timestamp ->
        {:error, "signature is too old"}

      not Plug.Crypto.secure_compare(hash, hash(timestamp, payload, secret)) ->
        {:error, "signature is incorrect"}

      true ->
        :ok
    end
  end
end

Summing up, the plug’s call/2 function looks like this:

@impl true
def call(conn, opts) do
  with {:ok, header} <- signature_header(conn),
       {:ok, body} <- raw_body(conn),
       :ok <- verify(header, body, "secret", opts) do
    conn
  else
    {:error, error} ->
      conn
      |> put_status(400)
      |> json(%{
        "error" => %{"status" => "400", "title" => "HTTP Signature is invalid: #{error}"}
      })
      |> halt()
  end
end

Done! Now we can use this Plug in our webhook pipeline at Phoenix router. Every request that does not have a valid signature will return an error.

Wrapping up

Signing webhook requests can increase a lot the level of security of communication between services! Elixir helps us to implement this in a safe and easy way with its tools. We saw how to implement HTTP signatures for our webhook endpoints, and we introduced a Plug at the client side to verify the body of those webhooks requests.

Happy coding!