Setting up cloud storage buckets

You can generate a keyset with:

tinkey create-keyset --key-template=AES256_GCM_HKDF_4KB

Prepare to send the Tink keyset to Upvest as a cleartext JSON object in the correct format. It must be an unwrapped keyset (the tinkey create-keyset output), containing a symmetric streaming-AEAD key.

Example of the Tink keyset JSON object

{
  "primaryKeyId": 1234567890,
  "key": [
    {
      "keyData": {
        "typeUrl": "type.googleapis.com/google.crypto.tink.AesGcmHkdfStreamingKey",
        "value": "<base64 key material>",
        "keyMaterialType": "SYMMETRIC"
      },
      "status": "ENABLED",
      "keyId": 1234567890,
      "outputPrefixType": "RAW"
    }
  ]
}

A freely chosen, pre-agreed plain string, passed as associated data on encryption. It must match byte-for-byte on both sides; any mismatch causes decryption to fail. An empty AAD is accepted but not recommended.

Hand-over. Because the keyset is shared in cleartext, it must be transferred over a channel agreed with Upvest for secret material and never sent by email, chat, or a ticket attachment. Upvest stores the keyset and AAD in a managed secret store, accessible only to the download service.

Rotation. To rotate, add a new key to the existing keyset, make it the primary key, and share the updated keyset with Upvest. Keep previously used keys in the keyset (status ENABLED) for as long as files encrypted under them may still be requested. Upvest must receive the updated keyset before the first file encrypted with the new key is referenced.

Each encrypted object must carry the custom metadata entry encrypted = true.

Upvest uses this flag to decide whether to decrypt. An encrypted object without the flag is treated as plaintext and the resulting download is unusable.

The format of the metadata depends on your cloud storage provider:

• Google Cloud Storage: metadata x-goog-meta-encrypted: true. This value may also be set after upload (gsutil setmeta).
• AWS S3: user metadata encrypted: true. This value must be set at upload time. AWS S3 does not allow adding metadata to an existing object without rewriting it.
• NetApp / S3-compatible: user metadata encrypted: true. This value must be set at upload time.
• Azure Blob Storage: metadata x-ms-meta-encrypted: true

The value is parsed as a boolean; true is expected. Surrounding whitespace is tolerated.

As it can sometimes take some time to work through the following stages, we offer you use of our "Default Bucket" in the Sandbox environment. This will allow you to get started exchanging files in the Sandbox environment for testing purposes.

If you choose this option, you can omit the cloud storage credentials from the initial setup form that you'll fill out later in the "Getting Started" tutorial, please remember to indicate that this is your wish during the completion of the "Investment API credentials request".

You will need to identify your organisation's Public Cloud of choice and arrange for a Bucket. As this may require working with other departments in your organisation, we strongly recommend that you set up download buckets for your organisation as early as possible, as you will want to test these mechanisms prior to going live with your product.

We currently support the following cloud storage providers:

• Amazon AWS
• Microsoft Azure
• Google Cloud

Access to storage objects is usually facilitated via service accounts. Upvest requires a shared service account that has the correct permissions to read file metadata and content. Make sure the service account has the following permissions before you provide us with its credentials:

• for GCP it is storage.objects.get, more details can be found here.
• for AWS GetObject and GetObjectAttributes, more details can be found here and here.
  

Once you've established which compatible cloud storage provider you'd like to use, you will have to create the bucket. The exact process required to do this will depend on the cloud you choose and the policies of your own cloud platform team. At the end of this process you will need a file containing access credentials for that bucket that you can share securely with Upvest.

To ensure you have the right data, please check that you have a JSON file with the structure below, that matches the cloud provider you chose:

{
  "bucket": "...",
  "region": "...",
  "access_key_id": "...",
  "secret_access_key": "...",
}

For the secure exchange of bucket credentials we use public-private PGP keys as we do for all other Upvest secret exchanges.

Please read our Secure communication with Upvest document. There you will find our public PGP key, which you should import into your PGP encryption tool. Once you have done that, you can use it to encrypt your credentials file ready for transfer to Upvest.

In order for us to test connectivity and correct integration of your bucket with the Investment API, we will require a dummy file to be uploaded to the bucket that we can try to access.

Please upload a file to your bucket and keep a copy of the URL for accessing this file in the bucket so that you can submit it to us later in the process.