# Implementing a webhook handler

Upvest's webhooks are HTTP requests made from our servers to the endpoints you define.  We'll talk about how you inform us of the addresses of those endpoints in a later step of this tutorial.

Because we don't know what technology you're building your application with, we can't yet give you a step-by-step guide to the implementation. Instead, this documentation guides you through the things you have to consider when implementing an API endpoint to handle incoming webhook requests from the Upvest Investment API. Given this knowledge you should be able to successfully create the webhook handlers required.

## Implementation steps

### 1. Delivery and retry logic

This section describes how Upvest delivers its requests, and how failures in that process are handled via retries. You'll need to make sure your handlers take these behaviours into account:

- Upvest's webhook service sends **multiple events bundled into a single request**. This keeps the overhead of establishing a TCP / HTTPS connection per webhook as low as possible.
- Events within a request are **ordered by the `created_at` field**, in ascending order (from oldest to newest). There are no ordering guarantees across requests.
- If a webhook delivery is successful, your handler should return the HTTP `200` status code. If that code is not received, the webhook service retries the delivery of the entire request using an exponential back off mechanism. This means the service will back off and retry within an increasingly large interval (1,2,3,5,8...600 sec) with no limit on the number or retry attempts.
- An **unsuccessful webhook delivery blocks all subsequent deliveries**, which will be queued indefinitely.
- A **successful delivery retry unblocks all queued deliveries**.


### Recommendations

To deal robustly with the webhook delivery logic, we highly recommend:

1. Idempotent webhook handlers based on the `event_id` as an idempotency ID.
2. Keeping an event log.


The reasons for those recommendations:

- The webhook service follows "at least once (or more)" logic. That means you're guaranteed to get every event at least once, but you may also see the same event multiple times.
- If multiple webhook subscriptions are set up for the same event type, the same event will be delivered to **each one of them**, causing duplication.
- If a webhook call fails, and an HTTP status other than `200` is returned from your endpoint, the **entire webhook call**, which may contain multiple events, will be retried.
- Once a webhook is delivered and an HTTP status of `200` is returned from your endpoint, the webhook service will never resend the same request.


### 2. Webhook request structure

The payload of a webhook request is a JSON structure with a single top-level key, `payload`.  This key contains a list of JSON objects, each containing the details of an event.

Each event contains the following fields:

| Field name | Type | Description |
|  --- | --- | --- |
| `id` | String (uuid) | The unique ID of the event. |
| `created_at` | String | An RFC 3339 timestamp indicating the time at which the event occurred. |
| `type` | String | The ENUM of the event type of the event.  See the specific event in the [API Reference](/api) for details. |
| `object` | JSON Object | The event-specific payload, structured as a JSON object.  See the documentation for the specific event  within our [API Reference](/api) for details. |
| `webhook_id` | String (uuid) | The UUID of the registered webhook, within the Upvest API, that triggered this message.  This can be used in the webhook API, for example to [retrieve a webhook subscription](/api/webhook-subscriptions/retrieve_webhook). |


**Example batch payload**

Including a single event related to a user object and emitted when a `USER.CREATED` event occurs:


```json
{
    "payload": [
        {
            "id": "fbecea50-2f35-4969-96af-342271da9eca",
            "created_at": "2021-07-21T14:10:00.00Z",
            "type": "USER.CREATED",
            "object": {
                "id": "83d83ec2-d2ca-49ff-bbea-b92b5c3be202",
                "type": "USER"
            },
            "webhook_id": "091915d8-7b4b-4d25-8208-f8e2d4b30070"
        }
    ]
}
```

### 3. Validating webhook signatures

To ensure that webhook requests originate from us and have not been modified without authorisation, you should validate the HTTP message signatures we include in these requests.

They follow the [same standard](https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-message-signatures-06) as the [API request signatures](/products/omnibus/concepts/api_concepts/http_signatures) we ask you to use.

**Example of signature validation**

A typical set of headers sent as part of a webhook request might look like this:

**Example headers**


```go
Accept-Encoding: gzip
Content-Length: 1177
Digest: SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=
Host: tenant.tld
Signature: sig1=:MIGIAkIB2MuP6GGvrOxZhvSROnBjMlofArjjGaazcyusqyGoiHiGVUuH2+qmwQR0GI6NrvkYUW0bRRuxO3f9Ng1Suh6SR4cCQgCQsyQvoNfeLs/PImskOvvjU9dhmRc0z4YshLKxB3krYYY9Ae6nP4Vq/4hJSrGHhTT6V46edzOen/9nxuCPtJ6SoQ==:
Signature-Input: sig1=("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333
```

Given these headers, the validation steps are as follows.

Parse signature-input header
Parse the `signature-input` header. It has metadata for one signature called `sig1`. It lists `content-length`, `@method`, `@path`, `digest` as signature components, identifies the signing key, provides two timestamps and one nonce value.


Check webhook time frame
Check that the time you received the webhook request lies between the `created` and `expires` timestamps. If it does not lie between these bounds, discard the request.


Check the body length
Check whether the length of the Webhook request body matches the number specified in the content-length header. If they do not match, discard the request.


The length is counted in bytes, not UTF-8 characters.

Check the content digest
Check if the digest of the Webhook request body matches the hash in the `digest` header. If they do not match, discard the request.

The calculation of the digest is described [here](/products/omnibus/concepts/api_concepts/http_signatures).


Obtain the public key
Check if you have cached a public key with ID `9f030355-3da5-4417-b3fe-4726f462b4b7`. If not, you can retrieve it with **GET** [`/auth/verify_keys`](/api/webhook-subscriptions/get_jwks).


Calculate component values
For the four listed components, calculate their values. (Details can be found [here](/products/omnibus/concepts/api_concepts/http_signatures)).

- For `content-length`, use the value of the `Content-Length` header, which in this example is `1177`.
- For `@method`, use `POST` as a constant value. We send all webhooks as `POST`.
- For `@path`, use the path part of your webhook URL.
- For `digest`, use the value of the `Digest` header, which in this example is `SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=`.


Calculate `@signature-params`
For the mandatory `@signature-params` component, use the metadata for signature `sig1` as the value.

**Example**


```
("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333
```

Calculate signature-input
Create the signature input string by concatenating the signature components (in the same order as listed, plus `@signature-params`) as follows:


```http
content-length: 1177
@method: POST
@path: /webhooks/users
digest: SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=
@signature-params: ("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333
```

Base64 decode the signature
Base64-decode the value between the colon (`:`) characters in the `signature` header to get the binary value of the signature.


Validate the signature
Validate the signature with the signature input string and the public key `9f030355-3da5-4417-b3fe-4726f462b4b7`.


Accept or discard
If the validation outcome is negative, discard the webhook request.


Now you're ready to implement webhooks. If you've read and understood this content, you should be ready to implement a handler endpoint for Upvest webhooks.

## Next Steps

We suggest you continue by returning to the "Implementing webhooks" tutorial and reading the ["Register a webhook"](/products/omnibus/getting_started/implementing_webhooks/webhooks_registration) section.