# Setting up cloud storage buckets To allow you to share documents with Upvest, we require a cloud storage bucket that is securely accessible by both you and us. We currently support the following cloud storage providers: - [Amazon Web Services (AWS S3)](https://aws.amazon.com/pm/serv-s3/) - [Microsoft Azure](https://azure.microsoft.com/) - [Google Cloud Storage (GCP GCS)](https://cloud.google.com/storage) - [NetApp](https://www.netapp.com/) ## Prerequisites ### Know your cloud provider and policies If you do not know what options are available within your organisation, and what policies exist around these platforms, please involve a member of your cloud platform and/or IT security teams in this process. ### Prepare your file encryption scheme Files must be encrypted using [Google Tink's Streaming AEAD primitive](https://developers.google.com/tink/streaming-aead). Upvest supports using the following key template: ``` AES256_GCM_HKDF_4KB (AES-256-GCM, HKDF-derived per-segment keys, 4 KB segments). ``` The ciphertext must be in Tink's streaming wire format. Upvest's decryption relies on the header and segment structure provided by Tink. Tink is available in a variety of languages including Java, Go, Python, C++, and others. You can generate a keyset with: ``` tinkey create-keyset --key-template=AES256_GCM_HKDF_4KB ``` Non-streaming Tink primitives (AEAD, HybridEncrypt, DeterministicAEAD) are not supported. ### Enable access to Upvest IP addresses Whitelist the following IP addresses by adding them to your allow list: **Sandbox** ```go 34.90.198.99 34.91.134.98 34.141.35.115 34.147.108.129 34.159.54.216 35.198.99.106 35.204.209.125 35.246.238.18 ``` **Live** ```go 34.32.131.178 34.89.219.22 34.141.83.243 34.141.214.141 34.159.127.47 34.159.198.202 35.204.0.90 35.234.163.52 ``` Add these IP addresses to your allow list for webhook delivery and cloud storage file transfers. ## Prepare your file encryption scheme Encrypting the files you place in your bucket is optional. If you choose to encrypt them, they must follow the scheme below exactly. Upvest can only decrypt files that match it. Files must be encrypted using [Google Tink's Streaming AEAD primitive](https://developers.google.com/tink/streaming-aead) with the key template `AES256_GCM_HKDF_4KB` (AES-256-GCM, HKDF-derived per-segment keys, 4 KB segments). The ciphertext must be in Tink's streaming wire format. Encryption performed with generic tooling (for example `openssl enc -aes-256-gcm`, application-level AES-GCM, or any custom framing) is not supported, even if the underlying cipher is the same. Tink writes its own header and segment structure, and Upvest's decryption relies on it. Tink is available for Java, Go, Python, C++, and other languages. Use the `streamingaead` primitive of the respective library, or the `tinkey` CLI. Other Tink streaming templates (`AES128_GCM_HKDF_4KB`, `AES256_GCM_HKDF_1MB`, `AES256_CTR_HMAC_SHA256_*`) are technically interoperable, but `AES256_GCM_HKDF_4KB` is the supported and tested configuration. Agree any deviation with Upvest in advance. Non-streaming Tink primitives (AEAD, HybridEncrypt, DeterministicAEAD) are not supported. Generate a keyset You can generate a keyset with: ``` tinkey create-keyset --key-template=AES256_GCM_HKDF_4KB ``` Provide Upvest the Tink keyset Prepare to send the Tink keyset to Upvest as a cleartext JSON object in the correct format. It must be an unwrapped keyset (the `tinkey create-keyset` output), containing a symmetric streaming-AEAD key. Keysets that are wrapped in a Key Management Service (KMS) (e.g., `encryptedKeyset`) cannot be processed. **Example of the Tink keyset JSON object** ```json { "primaryKeyId": 1234567890, "key": [ { "keyData": { "typeUrl": "type.googleapis.com/google.crypto.tink.AesGcmHkdfStreamingKey", "value": "", "keyMaterialType": "SYMMETRIC" }, "status": "ENABLED", "keyId": 1234567890, "outputPrefixType": "RAW" } ] } ``` Provide Upvest the Additional Authenticated Data (AAD) string A freely chosen, pre-agreed plain string, passed as associated data on encryption. It must match byte-for-byte on both sides; any mismatch causes decryption to fail. An empty AAD is accepted but not recommended. **Hand-over.** Because the keyset is shared in cleartext, it must be transferred over a channel agreed with Upvest for secret material and never sent by email, chat, or a ticket attachment. Upvest stores the keyset and AAD in a managed secret store, accessible only to the download service. **Rotation.** To rotate, add a new key to the existing keyset, make it the primary key, and share the updated keyset with Upvest. Keep previously used keys in the keyset (status `ENABLED`) for as long as files encrypted under them may still be requested. Upvest must receive the updated keyset **before** the first file encrypted with the new key is referenced. Prepare to use the correct metadata Each encrypted object must carry the custom metadata entry **`encrypted` = `true`**. Upvest uses this flag to decide whether to decrypt. An encrypted object **without** the flag is treated as plaintext and the resulting download is unusable. The format of the metadata depends on your cloud storage provider: - **Google Cloud Storage**: metadata `x-goog-meta-encrypted: true`. This value may also be set after upload (`gsutil setmeta`). - **AWS S3**: user metadata `encrypted: true`. This value **must be set at upload time**. AWS S3 does not allow adding metadata to an existing object without rewriting it. - **NetApp / S3-compatible**: user metadata `encrypted: true`. This value **must be set at upload time**. - **Azure Blob Storage**: metadata `x-ms-meta-encrypted: true` The value is parsed as a boolean; `true` is expected. Surrounding whitespace is tolerated. ### Encryption behaviour and constraints - **Mixed content is supported.** Once decryption is configured for you, you can provide both encrypted (flagged) and unencrypted objects in the same bucket. This lets you migrate gradually. - **Encrypted files are rejected if decryption is not configured.** If an object is flagged `encrypted: true` but Upvest holds no decryption configuration for you, the download fails by design. Upvest does not accept encrypted files it cannot decrypt. - **A key or AAD mismatch fails the download.** A wrong key, a wrong AAD, or a file that is flagged as encrypted but is in fact plaintext results in a failed download (`no matching key found for the ciphertext in the stream`). There is no silent fallback. - **The file size limit applies to the ciphertext** as stored in the bucket. The default maximum is 200 MB. Tink's streaming format adds a small constant overhead plus a per-segment tag, so a plaintext file near the limit may exceed it once encrypted. - **Encryption does not replace access control.** The existing bucket access configuration (the credentials and permissions Upvest uses to read the objects) is still required and unchanged. Once you have the encryption scheme in place, you can complete the following steps to test cloud storage file transfers. ## Implementation steps Optional: Choose to use our default bucket in Sandbox As it can sometimes take some time to work through the following stages, we offer you use of our "Default Bucket" in the Sandbox environment. This will allow you to get started exchanging files in the Sandbox environment for testing purposes. If you choose this option, you can omit the cloud storage credentials from the initial setup form that you'll fill out later in the "Getting Started" tutorial, please remember to indicate that this is your wish during the completion of the "Investment API credentials request". Identify the cloud provider You will need to identify your organisation's Public Cloud of choice and arrange for a Bucket. As this may require working with other departments in your organisation, we strongly recommend that you set up download buckets for your organisation as early as possible, as you will want to test these mechanisms prior to going live with your product. We currently support the following cloud storage providers: - Amazon AWS - Microsoft Azure - Google Cloud Access to storage objects is usually facilitated via service accounts. Upvest requires a shared service account that has the correct permissions to read file metadata and content. Make sure the service account has the following permissions before you provide us with its credentials: - for GCP it is `storage.objects.get`, more details can be found [here](https://cloud.google.com/storage/docs/access-control/iam-permissions#objects). - for AWS `GetObject` and `GetObjectAttributes`, more details can be found [here](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html) and [here](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAttributes.html). Gather bucket credentials Once you've established which compatible cloud storage provider you'd like to use, you will have to create the bucket. The exact process required to do this will depend on the cloud you choose and the policies of your own cloud platform team. At the end of this process you will need a file containing access credentials for that bucket that you can share securely with Upvest. To ensure you have the right data, please check that you have a JSON file with the structure below, that matches the cloud provider you chose: Amazon AWS ```json { "bucket": "...", "region": "...", "access_key_id": "...", "secret_access_key": "...", } ``` Microsoft Azure ```json { "query_sign": "..." } ``` Google Cloud ```json { "bucket": "...", "json_key": "..." } ``` Encrypt credentials For the secure exchange of bucket credentials we use public-private PGP keys as we do for all other Upvest secret exchanges. Please read our [Secure communication with Upvest](/products/tol/concepts/secure_communication) document. There you will find our [public PGP key](/products/tol/concepts/secure_communication#upvest-public-pgp-key), which you should import into your PGP encryption tool. Once you have done that, you can use it to encrypt your credentials file ready for transfer to Upvest. Provide a dummy file for testing In order for us to test connectivity and correct integration of your bucket with the Investment API, we will require a dummy file to be uploaded to the bucket that we can try to access. Please upload a file to your bucket and keep a copy of the URL for accessing this file in the bucket so that you can submit it to us later in the process. You should now have encrypted credentials for a public cloud storage bucket, ready to be transferred to Upvest later. ## Next steps Return to the "Getting Started" tutorial and continue at [Submit Credentials Request](/products/tol/getting_started/preparation_and_start_up/credentials_request).