INFO

Diff for /documentation/guides/files/retrieving_data.md

On this page, you can preview the modified document with the changes.
Note that the links within this diff are most likely broken because they are

  • relative to the original document or

  • point to another document that is not part of the changeset.


Go back to the summary


Retrieving data via the Investments API

This chapter describes the process of data retrieval. We also show you what steps are required to retrieve various files via the Investments API.

How data retrieval works

Downloading a file is a two-step process and starts with requesting a signed URL from the Investments API.

  1. Call one of the /files API to request a signed URL and file metadata.
  2. In the second step, use the returned signed URL to fetch data.

This diagram shows the whole process: Process of data retrieval

The first API call triggers the creation of a one-time download link within Upvest’s google cloud infrastructure utilising Google’s secure content delivery network (CDN). Those links are designed to be very hard to guess by Google and contain a relatively short-lived expiration period, 15 minutes by default. Given that expiration period, Upvest expects that the real download of the file happens briefly after it has been requested by the /files API.

Files encryption

Some clients may wish to encrypt all shared files for security reasons. For this purpose, the public PGP key must be provided to Upvest.

Once an encryption key is provided and Upvest has this option enabled, every file in the files API will be encrypted, including billing reports, MiFIR files and any files that are created in the future.

Retrieving files

To fetch a file with the data you need, you must proceed in 2 steps:

  1. Requesting a signed URL
  2. Fetching the file

Prerequisites

To be able to fetch files via the /files endpoint, the following requirements must be met:

  • The client must be linked to the tenant to which the requested file belongs.
  • The access token must be created with the files:read scope.
  • Each request must be authorised.

1. Requesting the signed URL

In the first step, call the Upvest /files endpoint.

GET https://[upvestApiUrl]/files/[folder]/[fileName]

Example response

{
    <ins>"id": "d588e071-6e02-4ae6-a2b6-a5f08499633b",</ins>
    "signed_url": <del>"https://storage.googleapis.com/upvest-tooling-datasharing-service-ia-unstable-7263/48785b65bb0459547f9940db7678ef7601dc8826/test.txt?Expires=1680268944&GoogleAccessId=tooling-datasharing-service%40ia-unstable-7263.iam.gserviceaccount.com&Signature=AhbTgjMpKo%2BRB4T5pDEKI3Rw0kCWqyOPvkgKJHpvRKV%2FtSmWi2WvGPShBqc%2FvewnBSnFPeE%2BDrx5fdYSFbZ9WUo3n2BFUtFo%2FRIuh8zNyX5K7og58qO67PRlXKl%2B3HbHDyAx%2FgiOzmAbcIy0Au4LT%2F%2BYe2QMCVxxL7iM%2BPj5toANCyjZflTxM%2BdyZENhgiJ2j6q9uPsjBCP68PFHv%2BIwIIpsJhp9mTOIqH%2Bsbi%2FBckKHtYrrzCV1VzOpKi527y0%2FYUnmWDoV4ADXUmLuH73LlcoQ%2FOjBPvjR7PZCFT2yL4f9CBITfMzFacBDlX4jMZGZq%2BEbd04wxjMWQ%2B3N06zPeg%3D%3D",</del> <ins>"https://storage.googleapis.com/upvest-tooling-datasharing-service-ia-unstable-7263/477ed9c4943c47f235712a1e80c1ef8491768f0a31ed2d81f244dfebe418e909/tats-tests-encrypted-text.txt?Expires=1710415438&GoogleAccessId=tooling-datasharing-service%40ia-unstable-7263.iam.gserviceaccount.com&Signature=cxb7JQXJVfO6ltBK%2FCWJkzoQcONakNiVsitn0nwtLDJmBj4cznBbMnT0yKtIO4v7BjhR92JmUd9wm7MurZYcuxdBPTMk%2BLTl0KN3QMEJ7%2FVjM1zSerIOxQ8OLFWCswt16Fj9%2BpwD7ZMplIrSEBGijPD48gLlZF%2FTW4yPqkLbE6sYn5Kn3K9A%2B3iF7%2BUaljQ2jLTxG9vCapp8WKjGjDyDXRtDDYY7OXJ6aDrjxDEUxO6kkOGzxMEBCALoHjD2lMpkEy5Gp1BkUbPka7hYVrbQrDM1xpBRsq%2F%2FCT%2F%2Bt196e8JP9YGr%2FxeA%2Bqt6ZTTryzGZhHlvHJFjqIqj2Ip7HioFjw%3D%3D",
    "created_at": "2024-03-04T08:24:18Z",</ins>
    "updated_at": <del>"2023-03-31T12:30:13Z"</del> <ins>"2024-03-14T11:18:39Z",
    "file_name": "tats-tests-encrypted-text.txt",
    "content_length": 919,
    "checksum": "709b190a108791217f62d839f85a7a997c2039cc29a737d59fd50454b85b2956"</ins>
}

The response contains the following files metadata:

ParameterDescription
signed_URL idUnique ID of the file.
signed_url The signed one-time download link with expiration period of 15 minutes.
updated_atTha The date when the file was last updated.
created_atThe date when the file was created.
file_nameThe name of the file.
content_lengthThe length of the file.
checksum
WARNING

Please note that the signed URL is only valid for 15 minutes and only allows reads for the bucket.


2. Fetching the file

Now you can download the desired file via the signed URL that was obtained in the first step.

GET <PROVIDED signed_url PARAMETER>

Status code 400

If the signed url has already expired and the file has been deleted for security reasons, you will receive the status code 400 and an error "expired token":

Example error token

<?xml version='1.0' encoding='UTF-8'?><Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Details>Request signature expired at: 2023-03-30T22:04:46+00:00</Details></Error>

In this case, please request the URL from Upvest’s /files API again (see step 1).

HTTP redirect option

To simplify the file sharing process, you can use the redirect function. To enable it, you need to add a query parameter to the file access request that specifies HTTP redirection.

GEThttps://[upvestApiUrl]/files/[folder]/[fileName]?redirect=1

ParameterDescription
redirectOptional query parameter. To enable the redirect option, set redirect=1.

After you have sent the call with this parameter, our API returns an HTTP redirect, the 3xx status code response standard for redirects, telling you to request the file at a different URL that points to the Google CDN storage.

INFO

Depending on which HTTP library you use, the redirection is performed automatically or the HTTP library must be configured accordingly. Here is an example of such a configuration for GoLang.


In case of a correct configuration, no further action is required from you.

The file metadata is returned as the HTTP header of the downloaded file. This is a custom HTTP header for each metadata property.

Example

x-goog-meta-file-name: mifir.csv

Was this page helpful?