Retrieving data via the Investment API

This chapter describes the process of data retrieval. We also show you what steps are required to retrieve various files via the Investment API.

How data retrieval works

Downloading a file is a two-step process and starts with requesting a signed URL from the Investment API.

1. Call the /files API to request a signed URL and file metadata.
2. In the second step, use the returned signed URL to fetch data.

This diagram shows the whole process:

The first API call triggers the creation of a one-time download link within Upvest’s google cloud infrastructure utilising Google’s secure content delivery network (CDN). Those links are designed to be very hard to guess by Google and contain a relatively short-lived expiration period, 15 minutes by default. Given that expiration period, Upvest expects that the real download of the file happens briefly after it has been requested by the /files API.

Files encryption

For security reasons, all shared files can be encrypted. To do this, the public PGP key must be provided to Upvest.

Note, for encrypting files, you must provide us with a separate PGP key (this is not the PGP key used for the client credentials described in the 'Getting Started' guide).

If you have not yet activated encryption for data exchange, please get in touch with Upvest.

To learn how to generate a PGP key pair in order to provide it to us, go to the 'PGP Keys' guide. Here, we show you step by step how to do this.

Once an encryption key is provided and Upvest has this option enabled, every file in the /files API will be encrypted, including billing reports, MiFIR files and any files that are created in the future.

Retrieving files

To fetch a file with the data you need, you must proceed in 2 steps:

1. Requesting a signed URL
2. Fetching the file

Prerequisites

To be able to fetch files via the /files endpoint, the following requirements must be met:

• The client must be linked to the tenant to which the requested file belongs.
• The access token must be created with the files:read scope.
• Each request must be authorised.

1. Requesting the signed URL

In the first step, call the Upvest /files endpoint.

GET /files/{folder}/{file_name}

Example response

{
    "id": "d588e071-6e02-4ae6-a2b6-a5f08499633b",
    "signed_url": "https://storage.googleapis.com/upvest-tooling-datasharing-service-ia-unstable-7263/477ed9c4943c47f235712a1e80c1ef8491768f0a31ed2d81f244dfebe418e909/tats-tests-encrypted-text.txt?Expires=1710415438&GoogleAccessId=tooling-datasharing-service%40ia-unstable-7263.iam.gserviceaccount.com&Signature=cxb7JQXJVfO6ltBK%2FCWJkzoQcONakNiVsitn0nwtLDJmBj4cznBbMnT0yKtIO4v7BjhR92JmUd9wm7MurZYcuxdBPTMk%2BLTl0KN3QMEJ7%2FVjM1zSerIOxQ8OLFWCswt16Fj9%2BpwD7ZMplIrSEBGijPD48gLlZF%2FTW4yPqkLbE6sYn5Kn3K9A%2B3iF7%2BUaljQ2jLTxG9vCapp8WKjGjDyDXRtDDYY7OXJ6aDrjxDEUxO6kkOGzxMEBCALoHjD2lMpkEy5Gp1BkUbPka7hYVrbQrDM1xpBRsq%2F%2FCT%2F%2Bt196e8JP9YGr%2FxeA%2Bqt6ZTTryzGZhHlvHJFjqIqj2Ip7HioFjw%3D%3D",
    "created_at": "2024-03-04T08:24:18Z",
    "updated_at": "2024-03-14T11:18:39Z",
    "file_name": "tats-tests-encrypted-text.txt",
    "content_length": 919,
    "checksum": "709b190a108791217f62d839f85a7a997c2039cc29a737d59fd50454b85b2956"
}

The response contains the following files metadata:

Please note that the signed URL is only valid for 15 minutes and only allows reads for the bucket.

2. Fetching the file

Now you can download the desired file via the signed URL that was obtained in the first step.

GET <PROVIDED signed_url PARAMETER>

Status code 400

If the signed url has already expired and the file has been deleted for security reasons, you will receive the status code 400 and an error "expired token":

Example error token

<?xml version='1.0' encoding='UTF-8'?><Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Details>Request signature expired at: 2023-03-30T22:04:46+00:00</Details></Error>

In this case, please request the URL from Upvest’s /files API again (see step 1).

HTTP redirect option

To simplify the file sharing process, you can use the redirect function. To enable it, you need to add a query parameter to the file access request that specifies HTTP redirection.

GET /files/{folder}/{file_name}?redirect=1

After you have sent the call with this parameter, our API returns an HTTP redirect, the 3xx status code response standard for redirects, telling you to request the file at a different URL that points to the Google CDN storage.

Depending on which HTTP library you use, the redirection is performed automatically or the HTTP library must be configured accordingly. Here is an example of such a configuration for GoLang.

In case of a correct configuration, no further action is required from you.