Go back to the summary

Webhooks

Webhooks enable you to make the most of the Investment API by enabling asynchronous interaction with your system whenever a state change occurs across all applicable business domains.

In this tutorial you will learn how to implement webhooks.

How webhooks work

The Upvest architecture was designed as an asynchronous, event-driven system. Events in this context are records of something that has happened, a change of state. Events are immutable and ordered in sequence in which they occur. In general, an event-driven architecture refers to a system of loosely coupled microservices that exchange information with each other through the creation and consumption of events. This architecture approach is commonly considered as good systems design, because:

• The event-driven architecture prevents a client from having to constantly polling to determine state changes.
• It allows for true decoupling of producers and consumers as microservices.
• A single source of truth with a log of immutable events.
• Eventing systems tend to be more resilient as they are only loosely coupled.

All information that is exposed through webhooks is also accessible through our HTTP API, however, much more powerful and convenient use cases are facilitated by leveraging webhooks.

Implementing webhooks

The Investment API allows you to create, read, update and delete (CRUD) webhooks and to test-trigger them to help you integrate with the Investment API.

Creating webhooks

To create a webhook, provide the following information to the Investment API:

• A URL under your control for receiving incoming webhook payloads sent by the Investment API.
  • The scheme of the URL is restricted to be the HTTPS protocol only.
  • The 'host' part of the URL is restricted to DNS names only, no IP addresses.
  • The called URL must not respond with HTTP redirects (3xx).
  • If you want to change the URL, you need to update the webhook subscription with an UPDATE request.
  • The server that processes the URL should support TLS version 1.2 or higher.
• A list of event categories you want to receive webhooks for.
  • The list of currently supported categories includes USER, ORDER and USER_CHECK, but this list will expand rapidly beyond the publication of this guide.
  • If you need all of the events, use the ALL category.
• To reduce the load of high event volumes, they are batched either by time or size. You can specify the maximum payload size and/or the maximum delay between payloads that you are willing to accept.
  • The delay must be a string in the format {number}{unit}, for example 1s, 5s, 1m. The smallest possible delay is 1s. The Investment API supports the units s for seconds and m for minutes.
  • The package size is in bytes. For example 10240 is for 10 KB. KiB. If you specify a small size here, and a single event payload exceeds it, then we will send a webhook payload bigger than the configured maximum.
• You can add a title to a webhook to help distinguish it from other webhooks.

You can find the Open API specification for creating, reading, updating and deleting webhooks here.

Example for creating a webhook

In this example, we create a webhook with the title User webhook, the URL is https://tenant.tld/webhooks/users, and only for events in the USER category. Payload batches will be sent every 5 seconds or less and the payload size will not be greater than 10240 bytes.

Send POST /webhooks including the following body:

Example body

{
  "title": "User webhook",
  "url": "https://tenant.tld/webhooks/users",
  "type": [
    "USER"
  ],
  "config": {
    "delay": "5s",
    "max_package_size": 10240
  }
}

Example response

{
  "id": "a8eb3540-5a84-40f9-b2bb-7f99f282fc5a",
  "created_at": "2021-07-21T14:10:00.00Z",
  "updated_at": "2021-07-21T14:10:00.00Z",
  "title": "User webhook",
  "url": "https://tenant.tld/webhooks/users",
  "type": [
    "USER"
  ],
  "enabled": false,
  "config": {
    "delay": "5s",
    "max_package_size": 10240
  }
}

Testing webhooks

To make sure that you have set up your webhook correctly, you can send GET /webhooks/{id}/test. This will trigger test webhook data to be sent to the URL you specified in the previous step.

The response to the trigger request contains the body, status and headers that we received in response to the POST request to https://tenant.tld/webhooks/users.

{
    "url": "https://tenant.tld/webhooks/users",
    "response": {
        "status": 200,
        "headers": {
            "Access-Control-Allow-Origin": "*",
            "Content-Length": "19",
            "Content-Type": "application/json; charset=utf-8",
            "Date": "Fri, 15 Oct 2021 16:03:17 GMT",
            "X-Powered-By": "Express"
        },
        "body": "{ \"success\": true }"
    }
}

Listing webhooks

To get a list of webhooks send GET /webhooks.

The resulting list could look like this:

Example response

{
  "meta": {
    "offset": 0,
    "limit": 100,
    "count": 1,
    "total_count": 1,
    "sort": "id",
    "order": "ASC"
  },
  "data": [
    {
      "id": "a8eb3540-5a84-40f9-b2bb-7f99f282fc5a",
      "created_at": "2021-07-21T14:10:00.00Z",
      "updated_at": "2021-07-21T14:10:00.00Z",
      "title": "User webhook",
      "url": "https://tenant.tld/webhooks/users",
      "type": [
        "USER"
      ],
      "enabled": false,
      "config": {
        "delay": "5s",
        "max_package_size": 10240
      }
    }
  ]
}

Updating webhooks

To update a webhook object send PATCH /webhooks{id} including the following request body:

Example request body

{
  "id": "a8eb3540-5a84-40f9-b2bb-7f99f282fc5a",
  "created_at": "2021-07-21T14:10:00.00Z",
  "updated_at": "2021-07-21T14:10:00.00Z",
  "title": "User webhook",
  "url": "https://tenant.tld/webhooks/users",
  "type": [
    "USER"
  ],
  "enabled": true,
  "config": {
    "delay": "5s",
    "max_package_size": 10240
  }
}

Deleting webhook by ID

To delete a webhook object specified by its ID, send DELETE /webhooks/{id}.

Accepting webhooks

Delivering and retry logic

1. The Webhook service sends events as a request which can contain multiple events. This is done to keep the overhead of establishing a TCP / HTTPS connection per webhook as low as possible.
2. Events in one request are ordered by field created_at in ascending order (from oldest to newest). There are no ordering guarantees across requests.
3. If a webhook delivery is not successful (not 200 HTTP status), the Webhook service retries the delivery of the entire request using an exponential backoff mechanism (1,2,3,5,8...600 sec) with no limit.
4. An unsuccessful webhook delivery blocks all subsequent deliveries, which will be queued indefinitely.
5. A successful delivery retry unblocks all queued deliveries.

Implementation recommendations

To deal robustly with the webhook delivery logic, we highly recommend

1. Idempotent webhook handlers based on the event_id as an idempotent id.
2. Keeping an event log.

The reasons for those recommendations:

1. The Webhook service follows "at least once (or more)" logic.
2. If multiple Webhook subscriptions are set up for the same event type, the same event will be delivered to each one of them, causing duplication.
3. If a Webhook calls fails (not 200 HTTP status returned from the client's endpoint) the whole Webhook call (which may contain multiple events) will be retried.
4. Once delivered (200 HTTP status returned from the client's endpoint), Webhook service will never resend the same request.

Structure of a webhook payload

Each time an event occurs that corresponds to a webhook, we generate a webhook message and send it in batches.

Example batch payload Including a single event related to a user object and emitted when a USER.CREATED event occurs:

{
    "payload": [
        {
            "id": "fbecea50-2f35-4969-96af-342271da9eca",
            "created_at": "2021-07-21T14:10:00.00Z",
            "event_type": "USER.CREATED",
            "object": {
                "id": "83d83ec2-d2ca-49ff-bbea-b92b5c3be202",
                "type": "USER"
            },
            "webhook_id": "091915d8-7b4b-4d25-8208-f8e2d4b30070"
        }
    ]
}

Events will be batched according to the configuration of your webhook. For example if the event rates are high, you will most likely get more than one event in the webhook payload.

Validating webhook signatures

To ensure that webhook requests originate from us and have not been modified without authorisation, you should validate the HTTP message signatures we include in these requests.

They follow the same standard as the API request signatures we ask you to use.

Example of signature validation

A typical set of headers sent as part of a webhook request might looks like this:

Example headers

Accept-Encoding: gzip
Content-Length: 1177
Digest: SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=
Host: tenant.tld
Signature: sig1=:MIGIAkIB2MuP6GGvrOxZhvSROnBjMlofArjjGaazcyusqyGoiHiGVUuH2+qmwQR0GI6NrvkYUW0bRRuxO3f9Ng1Suh6SR4cCQgCQsyQvoNfeLs/PImskOvvjU9dhmRc0z4YshLKxB3krYYY9Ae6nP4Vq/4hJSrGHhTT6V46edzOen/9nxuCPtJ6SoQ==:
Signature-input: sig1=("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333

Given these headers, the next validation steps should be:

1. Parse the Signature-input header. It has metadata for one signature called sig1. It lists content-length, @method, @path, digest as signature components, identifies the signing key, provides two timestamps and one nonce value.
2. Check that the time you received the webhook request lies between the created and expires timestamps. If it does not lie between these bounds, discard the request.
3. Check whether the length of the webhook request body (counted in bytes, not UTF-8 characters!) matches the number specified in the content-length header. If they do not match, discard the request.
4. Check if the digest of the webhook request body (calculated as described here) matches the hash in the Digest header. If they do not match, discard the request.
5. Check if you have cached a public key with ID 9f030355-3da5-4417-b3fe-4726f462b4b7. If not, you can retrieve it with GET /auth/verify_keys.
6. For the four listed components, calculate their values. (Details can be found here).
   • For content-length, use the value of the Content-Length header, which in this example is 1177.
   • For @method, use POST as a constant value. We send all webhooks as POST.
   • For @path, use the path part of your webhook URL.
   • For digest, use the value of the Digest header, which in this example is SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=.
7. For the mandatory @signature-params component, use the metadata for signature sig1 as the value, i.e. ("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333
8. Create the signature input string by concatenating the signature components (in the same order as listed, plus @signature-params) as follows:

content-length: 1177
@method: POST
@path: /webhooks/users
digest: SHA-256=2f0qZGzgUQ/jIS9v0aHA1z6sOAnL1trgSuhTVxdAzSY=
@signature-params: ("content-length" "@method" "@path" "digest");keyid="9f030355-3da5-4417-b3fe-4726f462b4b7";created=1635425273;nonce="0343692993";expires=1635425333

1. Base64-decode the value between the colon (:) characters in the Signature header to get the binary value of the signature.
2. Validate the signature with the signature input string and the public key 9f030355-3da5-4417-b3fe-4726f462b4b7.
3. If the validation outcome is negative, discard the webhook request.